A Multi-Strategy Caching and Automatic Data Generation Method and System Based on Node Environment

Abstract

This invention provides a multi-strategy caching and automatic data generation method and system based on a Node environment, relating to the field of data processing technology. The method includes receiving client requests, obtaining data generation templates, constructing a data dependency graph to analyze field dependencies, using a recursive generation strategy and depth-first search to determine the field generation order, performing data validity verification, creating a data generation thread pool based on the data dependency graph using an adaptive load balancing mechanism, and implementing asynchronous processing through a double-buffered queue. This improves data generation efficiency, ensures data logical integrity, and reduces system resource consumption.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multi-strategy caching and automatic data generation method and system based on a Node environment. Background Technology

[0002] With the rapid development of information technology, data simulation and test data generation have become crucial aspects of software development and testing. In the Node.js environment, developers require a large amount of simulated data for application testing, demonstration, and development. Currently, various data generation tools exist on the market, typically generating test data based on predefined templates or random algorithms. These tools are widely used in scenarios such as web application development, interface testing, and performance evaluation.

[0003] In real-world development environments, especially when building large-scale applications or microservice architectures, data generation tools need to handle complex data structures, dependencies between fields, and high-performance data generation requirements. Developers often need to simulate data interactions and data flows in real-world business scenarios, which requires data generation tools to not only provide basic random data but also support complex data models and efficient caching mechanisms.

[0004] However, existing data generation technologies based on the Node environment have the following defects and shortcomings: Existing technologies lack a systematic approach to handling complex data structures and dependencies between fields, typically relying on hard coding or simple rules to handle these relationships. This fails to effectively address deep data dependencies, leading to logical consistency issues in the generated data and making it difficult to meet the testing requirements of complex business scenarios. Existing data generation tools do not adequately consider performance optimization, especially in scenarios involving large data volumes. They lack effective concurrent processing mechanisms and load balancing strategies, resulting in uneven resource utilization and low generation efficiency, failing to meet the needs of high-performance application development. Existing technologies have relatively simplistic data caching strategies, typically employing fixed caching mechanisms and lacking flexible multi-level caching strategies and cache management mechanisms. They cannot dynamically adjust caching strategies based on different data request characteristics and business scenarios, leading to wasted system resources or low cache hit rates. Summary of the Invention

[0005] This invention provides a method and system for multi-strategy caching and automatic data generation based on a Node environment, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a multi-strategy caching and automatic data generation method based on a Node environment, comprising:

[0007] Receive a data request sent by the client, the data request containing a data type identifier, data volume parameters, and caching strategy parameters;

[0008] According to the data type identifier, the corresponding data generation template is obtained from the preset data template library. According to the data structure definition and field constraint rules in the data generation template, the type of each data field is validated and the value range is limited.

[0009] A data dependency graph is constructed based on the data generation template and the verified field constraint rules. The dependency relationships between data fields are analyzed using a graph traversal algorithm. A recursive generation strategy is adopted for data fields with dependency relationships. During the recursive generation process, the field generation order is determined by depth-first search. In each level of recursion, data validity verification is performed according to the constraint rules defined in the data generation template to ensure the logical integrity of the generated target data. The generated target data is then subjected to structural matching verification with the data generation template to ensure that the target data meets the preset data structure requirements.

[0010] Based on the field generation order of the data dependency graph and the data volume parameter, an adaptive load balancing mechanism is adopted to dynamically create a data generation thread pool in the Node environment; each thread in the thread pool executes the target data generation task according to the field generation order, and the asynchronous processing of target data generation and cache writing is realized through a double buffer queue mechanism.

[0011] The target data is stored according to the caching strategy parameters and then returned to the client.

[0012] According to the data type identifier, the corresponding data generation template is obtained from the preset data template library, including:

[0013] Multi-dimensional feature information is extracted based on the data type identifier. The multi-dimensional feature information is then subjected to dimensionality reduction to obtain the main feature components. The information importance is calculated based on the uniqueness and discriminability of the main feature components. The main feature components with information importance higher than a preset importance threshold are combined to generate initial semantic features.

[0014] Based on the initial semantic features, a hierarchical index structure for the data template library is constructed. The conditional probability between each index node and its upper-level nodes is calculated, and the node importance score is obtained by combining the index depth weight value. A dual-channel retrieval network is established in the hierarchical index structure according to the node importance score.

[0015] The search request is input into the dual-channel search network, and the transmission probability of the search request in the direct matching channel and the associated matching channel of the dual-channel search network is calculated respectively; the data generation template is initially screened based on the transmission probability.

[0016] The feature similarity of the selected candidate templates is calculated, and the feature similarity is weighted and fused with the historical matching accuracy of the templates to obtain a comprehensive score. Finally, the template with the highest comprehensive score is selected from the data template library to generate the search result output.

[0017] A data dependency graph is constructed based on the data generation template and validated field constraint rules. The dependencies between data fields are analyzed using a graph traversal algorithm, including:

[0018] The data fields in the verified field constraint rules are set as graph nodes, the dependencies between fields are set as graph edges, the dependency feature values ​​between each pair of nodes are calculated, and the dependency feature values ​​are attenuated and assigned to the corresponding graph edges as weights, thereby constructing a data dependency graph.

[0019] The data dependency graph is divided into source node layer, intermediate node layer and target node layer according to the ratio of node in-degree to out-degree. Within each layer, the nodes are sorted according to the total connection weight. Starting from the source node of the highest layer, the adjacent nodes are traversed in each layer using a breadth-first strategy and all nodes whose distance from the current node is less than a preset distance threshold are recorded to obtain the local dependency node set of that layer.

[0020] Starting from the nodes in the set of locally dependent nodes, and under the constraints that the path length is less than a preset length threshold and the edge weights on the path are all greater than the preset minimum weight requirement, the node traverses to the lower level through a depth-first strategy to obtain a complete set of cross-level dependent paths. The dependency strength is calculated cumulatively along the forward and reverse dependency paths respectively. The dependency strengths in the two directions are multiplied to obtain the dependency reliability metric. The dependency relationship with the dependency reliability metric value higher than the preset dependency threshold is output as the final verification result.

[0021] The order in which fields are generated is determined by depth-first search during the recursive generation process, including:

[0022] During the recursive generation process, the number of dependent fields of the current field node to be processed is obtained; the level depth of the field node in the current recursive path is calculated; and the level value of the field node is obtained by normalizing the number of dependent fields and the level depth and then weighting the calculation.

[0023] Entering the depth-first field generation stage, for each field on the current search path, obtain the penalty factor corresponding to the level of the field; combine the level value of the field node with the cumulative result of the penalty factors of each level before the current level to obtain the path score of the field;

[0024] During the search process, the number of visited fields and the number of fields to be visited are counted in real time. The depth limit coefficient is dynamically adjusted according to the ratio of the number of visited fields to the total number of fields. The maximum depth limit for the current search stage is obtained by combining the pre-set baseline search depth with the depth limit coefficient.

[0025] When the search reaches the maximum depth limit, a backtracking mechanism is triggered to return to the nearest branch point, select a new search direction to continue exploring, until a field generation order that satisfies the dependency constraint is found.

[0026] The generated target data is subjected to structural matching verification with the data generation template to ensure that the target data meets the preset data structure requirements, including:

[0027] The target data is constructed into a target data tree according to the node relationships, and the data generation template is constructed into a template data tree according to the node relationships. For each node in the target data tree, the cosine distance of the corresponding node in the template data tree is used as the structure matching score at the node level. The structure matching scores of all nodes are accumulated to obtain the basic structure matching score.

[0028] Structural layer features are extracted based on the hierarchical distribution relationship of the target data tree, semantic layer features are extracted based on the correlation strength between nodes in the target data tree, and constraint layer features are obtained by constraining the target data tree. The three features are weighted and combined with their corresponding weight coefficients to obtain a comprehensive feature score that reflects the multidimensional characteristics of the data structure.

[0029] When the comprehensive feature score is greater than the preset score threshold, it is confirmed that the target data meets the preset data structure requirements.

[0030] An adaptive load balancing mechanism is used to dynamically create a data generation thread pool in the Node environment, including:

[0031] Obtain the system baseline thread count, the current system load, and the maximum system load. Multiply the ratio of the current system load to the maximum system load by the system baseline thread count to obtain the initial thread count. Create a data generation thread pool based on the initial thread count.

[0032] Obtain the load variance and response time of the data generation thread pool; calculate the load weight coefficient and response weight coefficient based on historical running data; obtain the load score by multiplying the load variance and the load weight coefficient, obtain the response score by multiplying the response time by the response weight coefficient, and obtain the load balance score by adding the load score and the response score.

[0033] When the load balancing score exceeds the preset load threshold, the load balancing operation of the data generation task is triggered; when the load increases, data generation threads are dynamically added to the data generation thread pool; otherwise, idle threads are removed; the adjusted running data is used as new historical running data for the next round of data generation and resource adjustment.

[0034] A second aspect of this invention provides a multi-strategy caching and automatic data generation system based on a Node environment, comprising:

[0035] The first unit is used to receive data requests sent by the client, wherein the data requests include data type identifiers, data volume parameters, and caching strategy parameters;

[0036] The second unit is used to obtain the corresponding data generation template from the preset data template library according to the data type identifier, and to perform type verification and value range limitation on each data field according to the data structure definition and field constraint rules in the data generation template.

[0037] The third unit is used to construct a data dependency graph based on the data generation template and the verified field constraint rules. It analyzes the dependencies between data fields using a graph traversal algorithm, and adopts a recursive generation strategy for data fields with dependencies. During the recursive generation process, the field generation order is determined by depth-first search. In each level of recursion, data validity verification is performed according to the constraint rules defined in the data generation template to ensure the logical integrity of the generated target data. The generated target data is then subjected to structural matching verification with the data generation template to ensure that the target data meets the preset data structure requirements.

[0038] The fourth unit is used to dynamically create a data generation thread pool in the Node environment based on the field generation order of the data dependency graph and the data volume parameter, using an adaptive load balancing mechanism; each thread in the thread pool executes the target data generation task according to the field generation order, and realizes asynchronous processing of target data generation and cache writing through a double buffer queue mechanism;

[0039] The fifth unit is used to store the target data according to the caching strategy parameters and return it to the client.

[0040] A third aspect of the present invention,

[0041] An electronic device is provided, comprising:

[0042] processor;

[0043] Memory used to store processor-executable instructions;

[0044] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0045] Fourth aspect of the embodiments of the present invention,

[0046] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0047] The beneficial effects of this application are as follows:

[0048] This invention implements a multi-strategy caching and automatic data generation method based on the Node environment. It can flexibly generate data that conforms to specific structures and constraints according to the parameters in the client request, which greatly improves the automation and adaptability of data generation, reduces manual operations by developers, and significantly improves development and testing efficiency.

[0049] The data dependency graph and depth-first search algorithm constructed in this invention ensure the logical integrity and consistency of the generated data. By adopting a recursive generation strategy for data fields with dependencies, the invention achieves the accurate construction of complex data structures, effectively solving the technical problem of handling complex data dependencies in traditional data generation methods and improving data quality.

[0050] This invention employs an adaptive load balancing mechanism and a double-buffered queue technology to achieve efficient asynchronous processing of data generation and cache writing. It fully utilizes the asynchronous and non-blocking characteristics of the Node environment, significantly improving the system's concurrent processing capability and response speed. It is particularly suitable for large-scale data generation scenarios. At the same time, it optimizes data storage and access efficiency through a multi-strategy caching mechanism. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the multi-strategy caching and automatic data generation method based on the Node environment according to an embodiment of the present invention;

[0052] Figure 2 A flowchart for selecting data templates based on hierarchical dual-channel retrieval. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes will not be repeated in some embodiments.

[0055] Figure 1 This is a flowchart illustrating the multi-strategy caching and automatic data generation method based on a Node environment according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0056] Receive a data request sent by the client, the data request containing a data type identifier, data volume parameters, and caching strategy parameters;

[0057] According to the data type identifier, the corresponding data generation template is obtained from the preset data template library. According to the data structure definition and field constraint rules in the data generation template, the type of each data field is validated and the value range is limited.

[0058] A data dependency graph is constructed based on the data generation template and the verified field constraint rules. The dependency relationships between data fields are analyzed using a graph traversal algorithm. A recursive generation strategy is adopted for data fields with dependency relationships. During the recursive generation process, the field generation order is determined by depth-first search. In each level of recursion, data validity verification is performed according to the constraint rules defined in the data generation template to ensure the logical integrity of the generated target data. The generated target data is then subjected to structural matching verification with the data generation template to ensure that the target data meets the preset data structure requirements.

[0059] Based on the field generation order of the data dependency graph and the data volume parameter, an adaptive load balancing mechanism is adopted to dynamically create a data generation thread pool in the Node environment; each thread in the thread pool executes the target data generation task according to the field generation order, and the asynchronous processing of target data generation and cache writing is realized through a double buffer queue mechanism.

[0060] The target data is stored according to the caching strategy parameters and then returned to the client.

[0061] In one optional implementation, obtaining a corresponding data generation template from a preset data template library based on the data type identifier includes:

[0062] Multi-dimensional feature information is extracted based on the data type identifier. The multi-dimensional feature information is then subjected to dimensionality reduction to obtain the main feature components. The information importance is calculated based on the uniqueness and discriminability of the main feature components. The main feature components with information importance higher than a preset importance threshold are combined to generate initial semantic features.

[0063] Based on the initial semantic features, a hierarchical index structure for the data template library is constructed. The conditional probability between each index node and its upper-level nodes is calculated, and the node importance score is obtained by combining the index depth weight value. The node importance score is used to measure the degree of influence of the index node on the retrieval results. According to the node importance score, a dual-channel retrieval network is established in the hierarchical index structure, wherein the direct matching channel is constructed based on the direct association between index nodes, and the association matching channel is constructed based on the indirect association between nodes.

[0064] The search request is input into the dual-channel search network, and the transmission probability of the search request in the direct matching channel and the associated matching channel is calculated respectively. The transmission probability decreases exponentially as the path extends. The data generation template is initially screened based on the transmission probability.

[0065] The feature similarity of the selected candidate templates is calculated, and the feature similarity is weighted and fused with the historical matching accuracy of the templates to obtain a comprehensive score. Finally, the template with the highest comprehensive score is selected from the data template library to generate the search result output.

[0066] For the process of extracting multi-dimensional feature information based on data type identifiers, the system first parses the semantic structure of the data type identifier and extracts feature information including dimensions such as data format, data granularity, business domain, and usage scenario. For example, for the "transactional financial data" type identifier, the system will extract feature dimensions such as "financial domain," "transaction type," "numerical data," and "time series characteristics." To reduce computational complexity, the system uses principal component analysis (PCA) to perform dimensionality reduction on these multi-dimensional features, selecting the feature components with the highest distinguishability to data types. In specific implementation, the system calculates the contribution of each feature component to distinguishing data types and quantifies the contribution as an information importance value. Assuming the system sets the importance threshold to 0.75, the "financial domain" feature with an importance of 0.83 and the "transaction type" feature with an importance of 0.79 will be retained and combined into the initial semantic feature of "financial transaction," while the "time series characteristics" feature with an importance of 0.62 will be temporarily filtered out.

[0067] When constructing the hierarchical index structure of the data template library based on initial semantic features, the system uses a tree structure to organize the data templates. The top-level nodes represent the most basic feature classifications, such as "industry sector," the second level is "data type," and the third level is "specific application scenario," etc. In this index structure, the system calculates the conditional probability between each child node and its parent node, representing the likelihood of that child node appearing given its parent node. Simultaneously, an index depth weight factor is introduced to appropriately increase the importance of deeper nodes, preventing the retrieval process from being overly biased towards top-level features. For example, under the parent node "finance," the conditional probability of the child node "transaction data" is 0.45. Combined with a depth weight of 1.2, the importance score of this node is calculated to be 0.54. Based on these node importance scores, the system constructs a dual-channel retrieval network: the direct matching channel is built based on direct parent-child or sibling relationships between nodes, while the association matching channel is built based on indirect semantic relationships between nodes, capable of capturing cross-level feature relevance.

[0068] During the retrieval process, the system converts data type identifiers into retrieval request vectors and inputs them into a dual-channel retrieval network. In the direct matching channel, the system calculates the propagation probability along the hierarchical structure of the index tree. For example, when retrieving "bank transaction records," the propagation probability of the retrieval information from the "finance" node to the "transaction data" node is 0.85, and then to the "banking business" node is 0.72. As the path extends, the propagation probability exhibits exponential decay, achieved by multiplying by a decay factor less than 1, such as 0.9, at each node propagation. In the associated matching channel, the system considers the indirect associations between nodes, such as the semantic association between "bank transaction records" and "payment system logs," calculating an indirect propagation probability of 0.63. The system combines the propagation probabilities from both channels and selects the template with the highest overall propagation probability as the candidate set.

[0069] For the candidate template set, the system further calculates the feature vector similarity between the search request and the templates, commonly using the cosine similarity method to quantify this similarity relationship. For the search request "bank transaction records," the feature similarity of template A is 0.88, and that of template B is 0.76. The system also considers the historical matching accuracy of the templates; for example, template A has a historical accuracy of 92%, and template B has 87%. Finally, the system performs weighted fusion by setting weight coefficients (e.g., feature similarity weight 0.7, historical accuracy weight 0.3) to calculate a comprehensive score. For template A, its comprehensive score is 0.88×0.7+0.92×0.3=0.892; for template B, its comprehensive score is 0.76×0.7+0.87×0.3=0.793. Therefore, the system selects template A, with the higher score, as the final search result and returns it to the user.

[0070] In practical applications, this method can adapt to template retrieval needs for different types of data. For example, when a user submits a "mobile phone call record" data type identifier, the system first extracts feature dimensions such as "communication domain," "user behavior," and "time series," and retains the core feature combination after importance filtering. In the dual-channel retrieval network, the system can simultaneously match two candidate templates: "call detail template" and "user behavior analysis template." By calculating feature similarity (0.91 and 0.83 respectively) and combining it with historical accuracy (89% and 94% respectively), a weighted fusion is performed, and finally, the "call detail template" with a comprehensive score of 0.91×0.7+0.89×0.3=0.904 is selected as the best matching result.

[0071] The above method achieves accurate retrieval of data generation templates through multi-dimensional feature extraction, hierarchical index construction, dual-channel retrieval, and comprehensive scoring. It can quickly locate the most suitable data template based on the data type identifier provided by the user, thereby improving the efficiency and accuracy of data generation.

[0072] In one optional implementation, a data dependency graph is constructed based on the data generation template and validated field constraint rules. A graph traversal algorithm is then used to analyze the dependencies between data fields, including:

[0073] The data fields in the verified field constraint rules are set as graph nodes, the dependencies between fields are set as graph edges, the dependency feature values ​​between each pair of nodes are calculated, and the dependency feature values ​​are attenuated and assigned to the corresponding graph edges as weights, thereby constructing a data dependency graph.

[0074] The data dependency graph is divided into source node layer, intermediate node layer and target node layer according to the ratio of node in-degree to out-degree. Within each layer, the nodes are sorted according to the total connection weight. Starting from the source node of the highest layer, the adjacent nodes are traversed in each layer using a breadth-first strategy and all nodes whose distance from the current node is less than a preset distance threshold are recorded to obtain the local dependency node set of that layer.

[0075] Starting from the nodes in the set of locally dependent nodes, and under the constraints that the path length is less than a preset length threshold and the edge weights on the path are all greater than the preset minimum weight requirement, the node traverses to the lower level through a depth-first strategy to obtain a complete set of cross-level dependent paths. The dependency strength is calculated cumulatively along the forward and reverse dependency paths respectively. The dependency strengths in the two directions are multiplied to obtain the dependency reliability metric. The dependency relationship with the dependency reliability metric value higher than the preset dependency threshold is output as the final verification result.

[0076] When constructing the data dependency graph, the system first sets the data fields in the validated field constraint rules as graph nodes. For example, in an e-commerce order system, fields such as "User ID," "Order Amount," and "Payment Status" are all set as independent graph nodes. The system sets the dependencies between fields as graph edges. For example, "Order Amount" depends on "Unit Price" and "Quantity," so directed edges are established between the "Unit Price" node and the "Order Amount" node, as well as between the "Quantity" node and the "Order Amount" node.

[0077] For each pair of nodes with a dependency relationship, the system calculates a dependency feature value. The calculation of the dependency feature value is based on multiple indicators, including data correlation coefficient, co-occurrence frequency, and conditional probability. Taking "product unit price" and "order amount" as an example, the system calculates their correlation coefficient as 0.85 and co-occurrence frequency as 0.92 based on historical data, resulting in a combined dependency feature value of 0.88. Considering that dependencies gradually weaken as they are passed down through levels, the system introduces a decay mechanism to exponentially decay the dependency feature value. For example, if the decay coefficient is set to 0.9, the dependency feature value between nodes with a distance of 2 will become 0.81 times its original value. The decayed dependency feature value is assigned as weights to the corresponding graph edges, completing the construction of the data dependency graph.

[0078] After construction, the system divides the data dependency graph into different levels based on the in-degree to out-degree ratio of nodes. In-degree is the number of edges a node points to, and out-degree is the number of edges a node points to other nodes. Nodes with an in-degree to out-degree ratio less than 0.3 are classified as source nodes, such as "User ID" and "Product ID"; nodes with an in-degree to out-degree ratio greater than 3 are classified as target nodes, such as "Order Status" and "Logistics Status"; the remaining nodes are classified as intermediate nodes, such as "Payment Amount" and "Delivery Address". Within each level, the system sorts nodes according to their total connection weight (i.e., the sum of the weights of all edges connected to that node), with nodes having higher weight values ​​appearing earlier in the ranking.

[0079] The graph traversal begins from the source node at the highest level. The system employs a breadth-first search strategy, starting with the node with the highest weight in the source node layer (e.g., "User ID") and traversing its adjacent nodes. If the distance between a node and the current node is less than a preset distance threshold (e.g., set to 2), it is added to the set of locally dependent nodes. For example, starting from the "User ID" node, it can find adjacent nodes with a distance of 1 such as "User Level" and "Number of Historical Orders," as well as nodes that can be reached further through these nodes, such as "Discount Rate" (via "User Level"). These nodes together constitute the set of locally dependent nodes in the source node layer.

[0080] Starting with nodes within the local dependency node set, the system performs a depth-first traversal to construct a complete set of dependency paths across levels. During the traversal, the system imposes three constraints: the path length must be less than a preset length threshold (e.g., 4), the weight of each edge on the path must be greater than a preset minimum weight requirement (e.g., 0.5), and the traversal direction must be from higher to lower levels. Starting with "User ID," the system will find the following dependency path: "User ID" → "User Level" → "Discount Rate" → "Actual Payment Amount." Each path represents a specific data flow and dependency chain.

[0081] For each identified dependency path, the system assesses its reliability by cumulatively calculating dependency strength. Specifically, it calculates the forward and reverse dependency strengths separately, then multiplies the two strengths to obtain a dependency reliability metric. The forward dependency strength is the cumulative product of all edge weights on the path, reflecting the influence of the source node on the target node; the reverse dependency strength is the cumulative product of all edge weights on the reverse path, reflecting the target node's dependence on the source node. For example, the path "User ID → User Level → Discount Rate → Actual Payment Amount" has a forward dependency strength of 0.75 and a reverse dependency strength of 0.68, resulting in a dependency reliability metric of 0.51. The system ultimately outputs dependencies with a dependency reliability metric higher than a preset dependency threshold (e.g., 0.5) as the final verification result.

[0082] Through the above implementation methods, the present invention can comprehensively and accurately analyze the dependencies between data fields, effectively support application scenarios such as data quality monitoring, anomaly detection, and data lineage analysis, and improve the efficiency and accuracy of data governance.

[0083] In one optional implementation, the field generation order is determined by depth-first search during the recursive generation process, including:

[0084] During the recursive generation process, the number of dependent fields of the current field node to be processed is obtained, and the number of dependent fields represents the correlation strength between the field and other fields; the level depth of the field node in the current recursive path is calculated, and the level depth reflects the recursive dependency relationship between fields; the level value of the field node is obtained by normalizing the number of dependent fields and the level depth and then weighting them.

[0085] Entering the depth-first field generation stage, for each field on the current search path, obtain the penalty factor corresponding to the level of the field; combine the level value of the field node with the cumulative result of the penalty factors of each level before the current level to obtain the path score of the field, which is used to guide the selection of search direction;

[0086] During the search process, the number of visited fields and the number of fields to be visited are counted in real time. The depth limit coefficient is dynamically adjusted according to the ratio of the number of visited fields to the total number of fields. The maximum depth limit for the current search stage is obtained by combining the pre-set baseline search depth with the depth limit coefficient.

[0087] When the search reaches the maximum depth limit, a backtracking mechanism is triggered to return to the nearest branch point, select a new search direction to continue exploring, until a field generation order that satisfies the dependency constraint is found.

[0088] In the implementation process, the system first constructs a field dependency graph, where each node represents a field to be generated, and the edges between nodes represent the dependencies between fields. Assume there is a set of fields F = {f1, f2, ..., fn}, where each field depends on other fields. For example, in a product information system, the "discount price" field depends on the "original price" and "discount rate" fields.

[0089] When the system initiates the recursive generation process, it starts from the currently pending field node and obtains the number of dependent fields for that node. The number of dependent fields is directly extracted from the dependency graph, representing the number of other fields directly related to that field. For example, if field f3 depends on f1 and f2, then f3 has 2 dependent fields. This value characterizes the strength of the association between that field and other fields; the more dependent fields, the higher the degree of association.

[0090] Simultaneously, the system calculates the level depth of the field node within the current recursive path. Level depth refers to the path length from the search starting point to the current node, reflecting the recursive dependency relationship between fields. For example, if the current search path is f1→f2→f5, then the level depth of f5 is 3. The larger the level depth, the further back in the dependency chain the field is.

[0091] To make these two metrics comparable, the system normalizes the number of dependent fields and the hierarchy depth. The normalized value for the number of dependent fields is equal to the number of dependencies for that field divided by the maximum number of dependencies in the field set. The normalized value for the hierarchy depth is equal to the current depth divided by the preset maximum search depth limit. For example, if a field has 3 dependencies and a maximum number of dependencies is 5, the normalized number of dependent fields is 0.6; if the hierarchy depth for that field is 2 and the maximum search depth is 10, the normalized hierarchy depth is 0.2.

[0092] The system calculates the hierarchical value of a field node by weighting the normalized number of dependent fields and the hierarchical depth. Specifically, the hierarchical value equals the normalized number of dependent fields multiplied by the weight α, plus the normalized hierarchical depth multiplied by the weight β, where α and β are preset weight parameters, and α + β = 1. In practical applications, these two parameters can be adjusted according to business needs; for example, α = 0.7 and β = 0.3 indicates a greater emphasis on the dependencies between fields.

[0093] After entering the depth-first field generation phase, the system obtains the penalty factor corresponding to the level of each field in the current search path. The penalty factor is used to control the balance between the breadth and depth of the search, and usually increases with the increase of the level depth. For example, the penalty factor of the first level can be set to 0.1, the second level to 0.2, and so on.

[0094] The system combines the hierarchical value of a field node with the cumulative result of the penalty factors from all levels preceding the current level to calculate the field's path score. Specifically, the path score equals the field's hierarchical value multiplied by (1 minus the cumulative value of the penalty factors from all levels). For example, if a field has a hierarchical value of 0.8, is currently at the third level, and the cumulative penalty factor from the first two levels is 0.3, then the path score for that field is 0.8 × (1 - 0.3) = 0.56. A higher path score indicates a higher priority for that field in the current search path.

[0095] During the search process, the system continuously monitors the number of fields visited and the number of fields yet to be visited. Assuming a total of 100 fields and 30 have been visited so far, the percentage of visited fields is 0.3. The system dynamically adjusts the depth restriction coefficient based on this percentage. The depth restriction coefficient can be calculated using a decreasing function; for example, the depth restriction coefficient equals 1 minus the square of the percentage of visited fields, which in the above example is 1 - (0.3). 2 =0.91.

[0096] The system combines a pre-set baseline search depth with a depth constraint coefficient to obtain the maximum depth limit for the current search stage. If the baseline search depth is 15 and the depth constraint coefficient is 0.91, then the current maximum depth limit is 15 × 0.91 ≈ 13. This dynamic adjustment mechanism ensures that deeper exploration is allowed in the early stages of the search, while the search depth gradually decreases as the number of visited fields increases, prompting the algorithm to backtrack and explore other paths in a timely manner.

[0097] When the search reaches the maximum depth limit, the system triggers a backtracking mechanism, returning to the nearest branch point and choosing a new search direction to continue exploring. Specifically, the system will backtrack to the previous level and start a new search from the next unvisited neighbor of the current node. If all neighbors have been visited, the backtracking continues until a node containing unvisited neighbors is found.

[0098] The system selects the next field to access based on the path score. At each branch point, the system calculates the path score for all available fields and selects the field with the highest score as the next access target. If multiple fields have the same score, they can be sorted based on additional information such as the field's identifier or creation time.

[0099] Through the depth-first search process described above, the system ultimately obtains a field generation order that satisfies all dependency constraints. For example, for a system containing 10 fields, the final generation order could be f1→f3→f2→f5→f4→f7→f6→f9→f8→f10, ensuring that each field is generated after its dependent fields, thereby guaranteeing data consistency and integrity.

[0100] In one optional implementation, the generated target data is subjected to a structure matching verification with the data generation template to ensure that the target data meets the preset data structure requirements, including:

[0101] The target data is constructed into a target data tree according to the node relationships, and the data generation template is constructed into a template data tree according to the node relationships. For each node in the target data tree, the cosine distance of the corresponding node in the template data tree is used as the structure matching score at the node level. The structure matching scores of all nodes are accumulated to obtain the basic structure matching score.

[0102] Structural layer features are extracted based on the hierarchical distribution relationship of the target data tree, semantic layer features are extracted based on the correlation strength between nodes in the target data tree, and constraint layer features are obtained by constraining the target data tree. The three features are weighted and combined with their corresponding weight coefficients to obtain a comprehensive feature score that reflects the multidimensional characteristics of the data structure.

[0103] When the comprehensive feature score is greater than the preset score threshold, it is confirmed that the target data meets the preset data structure requirements.

[0104] In this implementation, the target data and the data generation template are first acquired; both are datasets with a specific structural relationship. The target data can be JSON-formatted product description information output by the intelligent generation system, while the data generation template is a predefined standard data structure template.

[0105] When constructing a target data tree from target data according to node relationships, the system parses the hierarchical, referential, and logical relationships between data items in the target data to build a tree structure. For example, for electronic device description data, there are main nodes such as "Product Name," "Technical Parameters," and "Features," and "Technical Parameters" contains child nodes such as "Processor," "Memory," and "Storage." In this case, the system organizes the nodes into a tree structure according to their hierarchical relationships, where "Product Name," "Technical Parameters," and "Features" are first-level nodes, and "Processor," "Memory," and "Storage" are second-level nodes. Similarly, data generation templates are constructed into template data trees in the same way.

[0106] For each node in the target data tree, the system searches for the corresponding node in the template data tree and calculates the cosine distance between the two nodes as the node-level structure matching score. Specifically, the node's attributes, values, and type are converted into vector representations, and then the cosine similarity between the vectors is calculated. For example, the "Processor" node in the target data tree contains three attributes: "Model," "Frequency," and "Number of Cores," with values ​​of "A12," "2.4GHz," and "8," respectively. The corresponding node in the template data tree also contains the same three attributes but with values ​​of "B11," "2.2GHz," and "6." The system converts this information into vectors and calculates the cosine distance, resulting in a matching score of 0.85 for that node. The same operation is performed on all nodes in the target data tree, and the scores are accumulated to obtain the basic structure matching score; for example, the accumulated score might be 45.6.

[0107] Next, the system extracts features from three dimensions and performs a comprehensive score. When extracting structural layer features, the hierarchical distribution of the target data tree is analyzed, including the tree depth, the distribution of the number of nodes in each layer, and branching factors. For example, for the aforementioned electronic device description data, its tree depth is 3, the first layer has 3 nodes, the second layer has 8 nodes, and the third layer has 15 nodes. The calculated structural layer feature score is 0.92.

[0108] When extracting semantic layer features, the system analyzes the association strength between nodes in the target data tree. Node association strength is calculated based on factors such as inter-node reference relationships and content relevance. For example, the "memory" node and the "performance" node have a reference relationship and high content relevance, with an association strength of 0.88; while "product color" and "processor model" have a low association strength, with a strength of 0.12. Combining all node association strengths, the semantic layer feature score is calculated to be 0.85.

[0109] When extracting constraint layer features, the system performs constraint analysis on the target data tree, checking whether the data meets constraints such as business rules, data integrity, and value range. For example, it checks whether the "price" node value is positive and whether the "production date" conforms to the time format. By verifying various constraints, the constraint layer feature score is determined to be 0.95.

[0110] The system calculates a comprehensive feature score by weighting each of the three features with its corresponding weight coefficients. In this embodiment, the structural layer feature weight is 0.4, the semantic layer feature weight is 0.3, and the constraint layer feature weight is 0.3, resulting in a comprehensive feature score of 0.91, calculated as 0.92×0.4+0.85×0.3+0.95×0.3=0.91.

[0111] Assuming the preset scoring threshold is 0.8, since the comprehensive feature score of 0.91 is greater than the preset scoring threshold of 0.8, the system confirms that the target data meets the preset data structure requirements and can proceed to the subsequent processing flow.

[0112] This method allows for flexible adjustment of weight coefficients and scoring thresholds to suit different application scenarios. For example, in the financial field, where data integrity is strictly required, the feature weights of the constraint layer can be increased to 0.5, and the scoring threshold can be increased to 0.9; in the content creation field, the requirements of the constraint layer can be appropriately reduced, while the weights of the semantic layer can be increased.

[0113] This method can also provide optimization suggestions for data that does not meet the requirements. For example, when the overall score of a target data is 0.75, which does not reach the threshold of 0.8, the system will identify the dimensions and nodes with low scores, such as the "Technical Parameters" node which lacks the necessary "Compatibility" information, and provide specific correction suggestions.

[0114] Through the above technical solutions, this invention achieves structured verification of generated data, ensuring data quality and improving the reliability of subsequent data processing and applications. This method is particularly suitable for large-scale data generation scenarios, such as intelligent content creation, automatic product information generation, and knowledge graph construction.

[0115] In one optional implementation, an adaptive load balancing mechanism is employed to dynamically create a data generation thread pool in the Node environment, including:

[0116] Obtain the system baseline thread count, the current system load, and the maximum system load. Multiply the ratio of the current system load to the maximum system load by the system baseline thread count to obtain the initial thread count. Create a data generation thread pool based on the initial thread count.

[0117] Obtain the load variance and response time of the data generation thread pool; calculate the load weight coefficient and response weight coefficient based on historical running data; obtain the load score by multiplying the load variance and the load weight coefficient, obtain the response score by multiplying the response time by the response weight coefficient, and obtain the load balance score by adding the load score and the response score.

[0118] When the load balancing score exceeds the preset load threshold, the load balancing operation of the data generation task is triggered; when the load increases, data generation threads are dynamically added to the data generation thread pool; otherwise, idle threads are removed; the adjusted running data is used as new historical running data for the next round of data generation and resource adjustment.

[0119] The system first needs to obtain the baseline number of threads, the current system load, and the maximum system load. The baseline number of threads can be determined by obtaining the number of CPU cores in the Node.js environment, for example, by using the `os.cpus().length` method to get the physical number of CPU cores. For example, on an 8-core processor server, the baseline number of threads is 8. The current system load can be obtained using the `os.loadavg()` method, which retrieves the average load values ​​over the past 1 minute, 5 minutes, and 15 minutes, taking the 1-minute average load as the current load metric. For example, the currently obtained 1-minute average load is 3.5. The maximum system load is usually related to the number of CPU cores and can be set to twice the number of CPU cores; in this example, it is 16.

[0120] The system calculates the initial number of threads by multiplying the ratio of the current load to the maximum load by the system's baseline thread count. In the example above, the current load is 3.5, the maximum load is 16, the ratio is 0.21875, and the baseline thread count is 8. Therefore, the initial thread count is 8 × 0.21875 = 1.75, rounded up to 2. Based on the calculated initial thread count, the system creates a data generation thread pool. In a Node.js environment, the `worker_threads` module can be used to implement multithreaded processing, creating a thread pool containing two worker threads. Each worker thread is responsible for an independent data generation task and communicates with the main thread through a message passing mechanism.

[0121] After the thread pool is created, the system continuously monitors its running status, obtaining load variance and response time. Load variance reflects the balance of load distribution among threads and can be obtained by periodically sampling the CPU utilization of each thread and calculating its variance. For example, if two threads have CPU utilizations of 65% and 55% respectively, with an average of 60%, the load variance is 25. Response time represents the average processing time of a data generation task and can be obtained by recording the time interval between task submission and completion; for example, the current task's average response time is 120 milliseconds.

[0122] The system calculates load weighting and response weighting coefficients based on historical operational data. Historical operational data includes records of load variance and response time over a past period. The load weighting coefficient represents the impact of load balancing on system performance and can be determined based on the correlation between load variance and overall system performance in historical data. In this embodiment, analysis of historical data shows that for every 10-point decrease in load variance, system throughput increases by approximately 8%, thus the load weighting coefficient can be set to 0.8. The response weighting coefficient represents the impact of response time on system performance and can be determined based on the correlation between response time and user experience satisfaction in historical data. Analysis shows that for every 20 milliseconds reduction in response time, user satisfaction increases by approximately 5%, thus the response weighting coefficient is set to 0.25.

[0123] The system calculates the load score by multiplying the load variance and the load weighting coefficient. In the example above, the load variance is 25, the load weighting coefficient is 0.8, and the load score is 25 × 0.8 = 20. The response time is multiplied by the response weighting coefficient to obtain the response score. The response time is 120 milliseconds, the response weighting coefficient is 0.25, and the response score is 120 × 0.25 = 30. Adding the load score and the response score gives the load balancing score: 20 + 30 = 50.

[0124] The system pre-sets a load balancing threshold, for example, 45. When the calculated load balancing score is 50, exceeding the preset threshold of 45, the system triggers a load balancing operation for the data generation task. The system determines the current load trend by comparing multiple consecutive sampled load values. If an upward trend is detected, such as the load values ​​for the three most recent samples being 3.2, 3.5, and 3.8, showing a significant upward trend, the system dynamically adds data generation threads to the data generation thread pool. The number of threads added depends on the degree to which the load balancing score exceeds the threshold: 1 thread is added if the score exceeds the threshold by 5-15 points, 2 threads are added if it exceeds by 15-30 points, and 3 threads are added if it exceeds by more than 30 points. In this example, the score exceeds the threshold by 5 points, so the system adds 1 thread to the thread pool, increasing the total number of threads to 3.

[0125] Conversely, if the load shows a downward trend, such as the load values ​​for the three most recent samples being 3.8, 3.5, and 3.2 respectively, indicating a significant decrease, the system will remove idle threads from the thread pool. The system identifies threads that have been idle for a period of time by monitoring the activity status of each thread. The idle criterion is that the thread's CPU utilization is below 20% for multiple consecutive samples. When removing a thread, it is necessary to ensure that at least one active thread remains in the thread pool to ensure the normal operation of the system.

[0126] After each load balancing operation, the system records the adjusted operational data, including new load variance, response time, and thread count, as new historical operational data for use in the next round of data generation and resource adjustment. Historical operational data is stored using a sliding window approach, retaining data from the most recent period (e.g., the past 24 hours) to ensure the system can make accurate resource adjustment decisions based on the latest operational conditions.

[0127] Through the aforementioned adaptive load balancing mechanism, the system can dynamically adjust the resource configuration of the data generation thread pool according to the real-time load conditions, thereby optimizing system resource utilization and improving overall performance while ensuring data generation efficiency.

[0128] A second aspect of this invention provides a multi-strategy caching and automatic data generation system based on a Node environment, comprising:

[0129] The first unit is used to receive data requests sent by the client, wherein the data requests include data type identifiers, data volume parameters, and caching strategy parameters;

[0130] The second unit is used to obtain the corresponding data generation template from the preset data template library according to the data type identifier, and to perform type verification and value range limitation on each data field according to the data structure definition and field constraint rules in the data generation template.

[0131] The third unit is used to construct a data dependency graph based on the data generation template and the verified field constraint rules. It analyzes the dependencies between data fields using a graph traversal algorithm, and adopts a recursive generation strategy for data fields with dependencies. During the recursive generation process, the field generation order is determined by depth-first search. In each level of recursion, data validity verification is performed according to the constraint rules defined in the data generation template to ensure the logical integrity of the generated target data. The generated target data is then subjected to structural matching verification with the data generation template to ensure that the target data meets the preset data structure requirements.

[0132] The fourth unit is used to dynamically create a data generation thread pool in the Node environment based on the field generation order of the data dependency graph and the data volume parameter, using an adaptive load balancing mechanism; each thread in the thread pool executes the target data generation task according to the field generation order, and realizes asynchronous processing of target data generation and cache writing through a double buffer queue mechanism;

[0133] The fifth unit is used to store the target data according to the caching strategy parameters and return it to the client.

[0134] A third aspect of the present invention provides an electronic device, comprising:

[0135] processor;

[0136] Memory used to store processor-executable instructions;

[0137] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0138] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0139] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-strategy caching and automatic data generation method based on a Node environment, characterized in that, include: Receive a data request sent by the client, the data request containing a data type identifier, data volume parameters, and caching strategy parameters; According to the data type identifier, the corresponding data generation template is obtained from the preset data template library. According to the data structure definition and field constraint rules in the data generation template, the type of each data field is validated and the value range is limited. A data dependency graph is constructed based on the data generation template and the verified field constraint rules. The dependency relationship between data fields is analyzed by the graph traversal algorithm. A recursive generation strategy is adopted for data fields with dependency relationships. In the recursive generation process, the field generation order is determined by depth-first search. In each level of recursion, data validity verification is performed according to the constraint rules defined in the data generation template to ensure the logical integrity of the generated target data. The generated target data is structurally matched with the data generation template to ensure that the target data meets the preset data structure requirements. Based on the field generation order of the data dependency graph and the data volume parameter, an adaptive load balancing mechanism is adopted to dynamically create a data generation thread pool in the Node environment. Each thread in the thread pool executes the target data generation task according to the generation order of the fields, and the asynchronous processing of target data generation and cache writing is realized through a double-buffered queue mechanism; The target data is stored according to the caching strategy parameters and then returned to the client.

2. The method according to claim 1, characterized in that, According to the data type identifier, the corresponding data generation template is obtained from the preset data template library, including: Multi-dimensional feature information is extracted based on the data type identifier. The multi-dimensional feature information is then subjected to dimensionality reduction to obtain the main feature components. The information importance is calculated based on the uniqueness and discriminability of the main feature components. The main feature components with information importance higher than a preset importance threshold are combined to generate initial semantic features. Based on the initial semantic features, a hierarchical index structure for the data template library is constructed. The conditional probability between each index node and its upper-level nodes is calculated, and the node importance score is obtained by combining the index depth weight value. A dual-channel retrieval network is established in the hierarchical index structure according to the node importance score. The search request is input into the dual-channel search network, and the transmission probability of the search request in the direct matching channel and the associated matching channel of the dual-channel search network is calculated respectively; the data generation template is initially screened based on the transmission probability. The feature similarity of the selected candidate templates is calculated, and the feature similarity is weighted and fused with the historical matching accuracy of the templates to obtain a comprehensive score. Finally, the template with the highest comprehensive score is selected from the data template library to generate the search result output.

3. The method according to claim 1, characterized in that, A data dependency graph is constructed based on the data generation template and validated field constraint rules. The dependencies between data fields are analyzed using a graph traversal algorithm, including: The data fields in the verified field constraint rules are set as graph nodes, the dependencies between fields are set as graph edges, the dependency feature values ​​between each pair of nodes are calculated, and the dependency feature values ​​are attenuated and assigned to the corresponding graph edges as weights, thereby constructing a data dependency graph. The data dependency graph is divided into source node layer, intermediate node layer and target node layer according to the ratio of node in-degree to out-degree. Within each layer, the nodes are sorted according to the total connection weight. Starting from the source node of the highest layer, the adjacent nodes are traversed in each layer using a breadth-first strategy and all nodes whose distance from the current node is less than a preset distance threshold are recorded to obtain the local dependency node set of that layer. Starting from the nodes in the set of locally dependent nodes, and under the constraints that the path length is less than a preset length threshold and the edge weights on the path are all greater than the preset minimum weight requirement, the node traverses to the lower level through a depth-first strategy to obtain a complete set of cross-level dependent paths. The dependency strength is calculated cumulatively along the forward and reverse dependency paths respectively. The dependency strengths in the two directions are multiplied to obtain the dependency reliability metric. The dependency relationship with the dependency reliability metric value higher than the preset dependency threshold is output as the final verification result.

4. The method according to claim 1, characterized in that, The order in which fields are generated is determined by depth-first search during the recursive generation process, including: During the recursive generation process, the number of dependent fields of the current field node to be processed is obtained; the level depth of the field node in the current recursive path is calculated; and the level value of the field node is obtained by normalizing the number of dependent fields and the level depth and then weighting the calculation. Entering the depth-first field generation stage, for each field on the current search path, obtain the penalty factor corresponding to the level of the field; combine the level value of the field node with the cumulative result of the penalty factors of each level before the current level to obtain the path score of the field; During the search process, the number of visited fields and the number of fields to be visited are counted in real time. The depth limit coefficient is dynamically adjusted according to the ratio of the number of visited fields to the total number of fields. The maximum depth limit for the current search stage is obtained by combining the pre-set baseline search depth with the depth limit coefficient. When the search reaches the maximum depth limit, a backtracking mechanism is triggered to return to the nearest branch point, select a new search direction to continue exploring, until a field generation order that satisfies the dependency constraint is found.

5. The method according to claim 1, characterized in that, The generated target data is subjected to structural matching verification with the data generation template to ensure that the target data meets the preset data structure requirements, including: The target data is constructed into a target data tree according to the node relationships, and the data generation template is constructed into a template data tree according to the node relationships. For each node in the target data tree, the cosine distance of the corresponding node in the template data tree is used as the structure matching score at the node level. The structure matching scores of all nodes are accumulated to obtain the basic structure matching score. Structural layer features are extracted based on the hierarchical distribution relationship of the target data tree, semantic layer features are extracted based on the correlation strength between nodes in the target data tree, and constraint layer features are obtained by constraining the target data tree. The three features are weighted and combined with their corresponding weight coefficients to obtain a comprehensive feature score that reflects the multidimensional characteristics of the data structure. When the comprehensive feature score is greater than the preset score threshold, it is confirmed that the target data meets the preset data structure requirements.

6. The method according to claim 1, characterized in that, An adaptive load balancing mechanism is used to dynamically create a data generation thread pool in the Node environment, including: Obtain the system baseline thread count, the current system load, and the maximum system load. Multiply the ratio of the current system load to the maximum system load by the system baseline thread count to obtain the initial thread count. Create a data generation thread pool based on the initial thread count. Obtain the load variance and response time of the data generation thread pool; calculate the load weight coefficient and response weight coefficient based on historical running data; obtain the load score by multiplying the load variance and the load weight coefficient, obtain the response score by multiplying the response time by the response weight coefficient, and obtain the load balance score by adding the load score and the response score. When the load balancing score exceeds the preset load threshold, the load balancing operation of the data generation task is triggered; when the load increases, data generation threads are dynamically added to the data generation thread pool; otherwise, idle threads are removed; the adjusted running data is used as new historical running data for the next round of data generation and resource adjustment.

7. A multi-strategy caching and automatic data generation system based on a Node environment, used to implement the method as described in any one of claims 1-6, characterized in that, include: The first unit is used to receive data requests sent by the client, wherein the data requests include data type identifiers, data volume parameters, and caching strategy parameters; The second unit is used to obtain the corresponding data generation template from the preset data template library according to the data type identifier, and to perform type verification and value range limitation on each data field according to the data structure definition and field constraint rules in the data generation template. The third unit is used to construct a data dependency graph based on the data generation template and the verified field constraint rules. It analyzes the dependency relationship between data fields through a graph traversal algorithm, adopts a recursive generation strategy for data fields with dependency relationships, determines the field generation order through depth-first search during the recursive generation process, and performs data validity verification according to the constraint rules defined in the data generation template in each recursive level to ensure the logical integrity of the generated target data. The generated target data is structurally matched with the data generation template to ensure that the target data meets the preset data structure requirements. The fourth unit is used to dynamically create a data generation thread pool in the Node environment based on the field generation order of the data dependency graph and the data volume parameter, using an adaptive load balancing mechanism. Each thread in the thread pool executes the target data generation task according to the generation order of the fields, and the asynchronous processing of target data generation and cache writing is realized through a double-buffered queue mechanism; The fifth unit is used to store the target data according to the caching strategy parameters and return it to the client.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images