Dynamic data packet configuration method and system based on source node graph model and field tree structure

Abstract

This application relates to the field of data processing technology and provides a dynamic data packet configuration method based on a source node graph model and a field tree structure. The method includes: loading the field tree, source nodes, and source relationship configuration according to the interface encoding; establishing a mapping table in memory and locating the root node; recursively retrieving data from the root node according to the parent-child structure, dynamically constructing SQL queries and caching the results, with child nodes using batch IN queries based on the parent row set; using the field tree configuration as the driver, distributing processing according to field type and extracting data from the cache, recursively assembling the data into a nested JSON structure; and persisting the assembled nested JSON structure as a delivery data packet. This application achieves zero-code configuration of the delivery interface, effectively avoiding the N+1 performance problem during recursive data retrieval, and improving configuration efficiency and system maintainability.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a dynamic data packet configuration method and system based on a source node graph model and a field tree structure. Background Technology

[0002] In fields such as digital engineering delivery, it is often necessary to assemble and deliver multi-source heterogeneous business data according to a specific format. Existing technologies, such as Chinese patent CN112035091B which discloses an agile delivery management method for shale gas fields, allow for the generation of interfaces through visual point-and-click configuration for single objects. However, for the delivery of multiple related objects, technicians still need to write SQL statements to define the relationships and query logic between objects. While this approach lowers the barrier to entry to some extent, it still fundamentally relies on code or scripts. When the delivery data structure changes or is reused across projects, the SQL statements still need to be modified and redeployed, resulting in high maintenance costs. Furthermore, when dealing with complex nested data with hierarchical relationships, traditional solutions often use a loop-based, row-by-row query approach for sub-tables, which easily leads to N+1 query performance degradation and fails to meet the delivery timeliness requirements of large data volume scenarios. Therefore, there is an urgent need for a dynamic data package configuration method based on a source node graph model and field tree structure that can completely decouple data structure definition from data retrieval logic and automatically optimize the performance of related queries. Summary of the Invention

[0003] To address the issues of high coupling in data delivery interface configuration and poor performance in multi-table join queries in existing technologies, this application aims to provide a dynamic data package configuration method and system based on a source node graph model and field tree structure, achieving zero-code configuration of delivery interfaces and high-performance dynamic assembly of nested data.

[0004] To achieve the above objectives, this application adopts the following technical solution: A dynamic data packet configuration method based on a source node graph model and a field tree structure includes: Load the field tree configuration, source node configuration, and source relationship configuration of the corresponding interface based on the interface code; Establish source node mapping table, source relation mapping table and field binding mapping table in memory, and locate the root node; Starting from the root node, the data retrieval operation is recursively executed according to the parent-child structure defined by the source relationship configuration. Each source node dynamically constructs an SQL query based on its own query conditions and the association parameters passed by the parent node. The query results are stored in the result set cache with the source node code as the key. Among them, the child nodes use batch IN queries based on the parent row set. Driven by the field tree configuration, processing is distributed according to the field type of each field, and corresponding data is extracted from the result set cache and recursively assembled into a nested JSON structure; The assembled nested JSON structure is persisted as a delivery data package.

[0005] The above solution separates the output structure, data retrieval unit, and association relationship into three independent configuration entities, and achieves complete decoupling of delivery logic and business code by recursively retrieving data based on the source node graph and driving assembly based on the field tree at runtime. At the same time, it avoids the N+1 performance problem in the recursive process by using a batch IN query mechanism based on the parent row set.

[0006] As an optional implementation, the field tree configuration is used to define the output JSON structure and the data source of each field; the source node configuration is used to define the database table and query conditions corresponding to each data retrieval unit; and the source relationship configuration is used to define the parent-child relationship between source nodes.

[0007] As an optional implementation, each source node dynamically constructs an SQL query based on its own query conditions and the association parameters passed in by the parent node, including: collecting the database columns that the current node needs to query and automatically supplementing the join key columns required for the association query; parsing the whereConfig configuration of the current source node and replacing the placeholders in it with the actual project information and interface information.

[0008] As an optional implementation, the child node employs a batch IN query based on the parent row set, including: extracting all non-empty, non-repeating values ​​of the associated fields from the parent node query result set, constructing IN parameterized conditions, and appending them, along with the additional conditions in the source relation configuration, to the WHERE clause of the child node.

[0009] As an optional implementation, the distribution processing based on the field type of each field includes: if the field type is string, retrieving the corresponding database value from the result set cache; if the field type is object, taking the first row as the current context and recursively assembling the subfield; if the field type is array, traversing the data row set and recursively assembling the subfield row by row.

[0010] As an optional implementation, a source node pooling mechanism is also included: the source node and the interface are decoupled into an independent resource pool, and a many-to-many relationship is established through an intermediate association table. The same source node is referenced by multiple interfaces, and each interface uses its own source relationship configuration to define the association method.

[0011] As an optional implementation, the source node configuration also includes a whereConfig configuration, which is in JSON format as an array object and supports ten operators; the placeholders in the values ​​of the whereConfig configuration are replaced with actual values ​​at runtime.

[0012] As an optional implementation, an SQL injection protection mechanism is also included: regular expression validation is performed on all table names and column names used to dynamically construct SQL queries, allowing only identifiers composed of letters, numbers, and underscores; values ​​are passed through parameterized queries.

[0013] As an optional implementation, a three-level value fallback mechanism for leaf fields is also included: the leaf field value follows the actual value in the database, the default value configured in the field table, and the null value, which are all three levels of priority fallback.

[0014] This application dynamically constructs SQL placeholder replacements, replacing placeholders such as {projectCode}, {projectEname}, and {interfaceCode} with actual values ​​at runtime. The whereConfig uses an array object in JSON format, where object properties are fixed and support ten operators: =, !=, >, >=, <, <=, like, in, between, is null, and is not null.

[0015] Furthermore, this application also provides a dynamic data packet configuration system based on a source node graph model and a field tree structure, including a memory and a processor. When the processor executes a computer program stored in the memory, it implements the dynamic data packet configuration method based on the source node graph model and field tree structure as described above.

[0016] Beneficial Effects: This application separates the field tree configuration, source node configuration, and source relationship configuration into three independent configuration entities, decoupling the output structure definition, data retrieval unit definition, and data association relationship definition. Adding or modifying delivery interfaces only requires adjusting configuration data without writing code, significantly shortening the development and maintenance cycle of delivery interfaces. By constructing a source node graph and employing a batch IN query mechanism based on parent row sets, non-empty, unique values ​​of associated fields can be extracted from the parent node result set during recursive data retrieval, allowing for a one-time query of all child row data. This effectively avoids the N+1 performance degradation problem caused by traditional loop queries, ensuring query efficiency in complex nested data scenarios. By using field tree configuration as the driving force and distributing processing according to field type, it can automatically adapt to different data types such as string, object, and array, recursively assembling JSON delivery packages with arbitrary nesting depths, meeting the needs of diverse delivery formats. Furthermore, the source node pooling mechanism enables cross-interface reuse of data sources, and combined with placeholder replacement and SQL injection protection mechanisms, further enhances the system's versatility, flexibility, and security. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the overall process of the dynamic data packet configuration method based on the source node graph model and field tree structure according to an embodiment of this application. Figure 2 This is a flowchart illustrating the execution of the field tree-driven JSON assembly stage in an embodiment of this application. Figure 3 This is a flowchart of the recursive data retrieval stage execution in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0020] Example 1

[0021] like Figure 1 As shown, this embodiment provides a dynamic data packet configuration method based on a source node graph model and a field tree structure. This method achieves zero-code configuration of data delivery logic through a three-layer configuration decoupling and runtime dynamic assembly mechanism. The dynamic data packet configuration method based on the source node graph model and field tree structure includes the following steps: Step S1, Configuration Loading. Receive the interface code and project code. Based on the interface code, load the complete configuration information for that interface from the database, including: a field tree configuration list (defining the structure of the output JSON and the data source for each field), a source node configuration list (defining the database table and query conditions corresponding to each data retrieval unit), and a source relationship configuration list (defining the parent-child relationships between source nodes). Based on the project code, load project information from the database for subsequent dynamic condition placeholder replacement.

[0022] This step loads the corresponding interface's field tree configuration, source node configuration, and source relationship configuration based on the interface code. The system receives the interface code from the external request as a unique index key and reads the three independent configuration entities associated with that interface from persistent storage. The field tree configuration defines the hierarchical structure and field metadata of the final output data; the source node configuration defines the query units and filtering conditions of the underlying physical data table; and the source relationship configuration defines the parent-child relationship paths between source nodes. This design, which separates the output structure, data source, and relationship logic, ensures that changes to any one component will not affect other configurations, thus laying the architectural foundation for subsequent zero-code maintenance. It should be understood that although this embodiment uses the interface code as the loading trigger condition, in other implementations, configuration file paths, memory cache keys, or other identifiers can also be used to locate and load these three types of configuration entities.

[0023] Step S2, Source Node Graph Construction. Three mapping structures are constructed in memory: The source node mapping table (sourceCode → source node entity) is used to quickly locate the data retrieval unit based on the encoding; The source relationship mapping table (parentSourceCode → list of child relationships) is used to determine the tree-like relationship structure between source nodes; The field binding mapping table (sourceCode → field list) is used to determine the database columns that each source node needs to query.

[0024] This step establishes a source node mapping table, a source relationship mapping table, and a field binding mapping table in memory, and locates the root node. To improve runtime processing efficiency, the system converts the loaded configuration data into a hash mapping structure in memory. The source node mapping table uses the source node code as the key to achieve O(1) complexity access to the attributes of the data retrieval unit; the source relationship mapping table uses the parent node code as the key to quickly retrieve the association information of all its subordinate child nodes, thereby constructing a complete source node topology graph in memory; the field binding mapping table establishes a reverse index between the source node and the output field, which facilitates the rapid location of the data source in the subsequent assembly stage. At the same time, the system traverses the source node configuration to identify entities of type root node, and uses them as the starting anchor point for subsequent recursive data retrieval. By constructing the above memory mapping structure, frequent database I / O or linear lookups during the recursive process are avoided, significantly improving the performance of configuration parsing and graph traversal.

[0025] Step S3, recursively retrieve data. Locate the root node (the source node with nodeType set to ROOT) in the source node configuration list. Starting from the root node, recursively perform the data retrieval operation according to the parent-child relationship defined in the source relationship mapping. The data retrieval process for each source node includes: (a) Collect the database columns that the current node needs to query based on the field binding mapping table, and automatically supplement the join key columns required for the related query; (b) Dynamically construct parameterized SQL statements based on the whereConfig configuration of the current source node and the association parameters passed in by the parent node; (c) Execute the SQL query and cache the results in the result set mapping table with sourceCode as the key; (d) Traverse the list of child relationships of the current source node, and recursively execute the data retrieval of child nodes with the current query result as the parent row data to realize batch IN query based on the parent row value and avoid the N+1 query problem.

[0026] This step starts from the root node and recursively executes data retrieval operations according to the parent-child structure defined in the source relationship configuration. Each source node dynamically constructs an SQL query based on its own query conditions and the association parameters passed from the parent node. The query results are stored in the result set cache with the source node code as the key. Child nodes use batch IN queries based on the parent row set. This step completely decouples data acquisition from structure generation. The system only focuses on efficiently fetching all relevant data into memory, without concern for the final nested JSON structure. During the recursive process, after the current node executes a query, its result set is cached for subsequent reuse. For child node queries, the system does not initiate a separate query for each row in the parent result set, but rather collects all values ​​of the associated fields in the parent result set and constructs an IN condition with multiple parameters for a one-time batch query. This batch IN query mechanism based on the parent row set fundamentally eliminates the N+1 performance degradation problem caused by traditional loop queries, ensuring fast response times during the data retrieval phase under large data volumes. It should be noted that the specific implementation details of dynamic SQL construction and batch query will be further elaborated in subsequent embodiments. This embodiment only establishes its functional positioning as an independent data retrieval stage in the overall process.

[0027] Step S4, Field Tree JSON Assembly. Driven by the field tree configuration list, recursively construct the output JSON starting from the root field (the top-level field with an empty parentId). Processing is then performed based on the fieldType of each field node: If fieldType is string (leaf field): Based on the sourceCode and sourceField bound to the field, retrieve the corresponding database value from the result set mapping cached in step S3. If the database value is empty, fall back to defaultValue. If defaultValue is also empty, output null. If fieldType is object (object field): Locate the corresponding source node data row (take the first row) based on the sourceCode bound to the field, recursively assemble its subfields, and output the nested object; If fieldType is array (array field): Locate the corresponding source node data row set based on the sourceCode bound to the field, match and filter according to parent-child relationship, recursively assemble its child fields row by row, and output object array.

[0028] This step is driven by the field tree configuration, distributing processing based on the field type of each field, retrieving corresponding data from the result set cache, and recursively assembling it into a nested JSON structure. Specifically, once all necessary data exists in the result set cache, the system then assembles the data according to the topology described by the field tree configuration. This process is entirely configuration-driven and does not rely on any hard-coded business logic. The system traverses the field tree nodes, executing the corresponding value retrieval or recursive assembly strategy based on the declared type of the node (such as string, object, array, etc.). For example, when encountering an object type, it enters the next level of recursion; when encountering an array type, it traverses the corresponding data row set for batch assembly. Since the batch loading and caching of data have been pre-completed in the data retrieval stage, the assembly operation here is only a pure in-memory computation, without needing to access the database again, thus ensuring the real-time generation of complex nested structures.

[0029] Step S5, Data Packet Output. The nested JSON structure assembled in Step S4 is persisted as a standard JSON string, either as the final delivery data packet output or persisted to the delivery data table.

[0030] Furthermore, the process of dynamically constructing parameterized SQL statements in step S3 includes: Collect the database column names that the current source node is bound to in all field trees; Iterate through all child relationships of the current source node, extract the parentField from the joinConfig of each child relationship, and automatically add the parentField if the column is not already in the collection list; Parse the whereConfig (JSON array format) of the current source node and replace the placeholders ({projectCode}, {projectEname}, {interfaceCode}, {businessKey}) with the actual project information and interface information; If the current node has a related parameter passed from the parent node, a batch related condition in the form of "childField IN (?, ?, ...)" is generated, and all unique values ​​of parentField in the parent result set are collected as IN parameters to avoid querying row by row.

[0031] This step persists the assembled nested JSON structure as a delivery package. Specifically, the system converts the nested object graph built in memory into standard JSON text format, outputting it as the final deliverable to the caller or for persistent storage. The structure of this delivery package strictly follows the field tree configuration definition, and its content is consistent with the data source specified in the source node configuration.

[0032] Through steps S1 to S5 above, this embodiment constructs a dynamic data packet generation framework that is configuration-driven, decouples data retrieval and assembly, and has batch query optimization capabilities. This framework sinks business logic from the code level to the configuration level, which not only solves the problem of high interface development and maintenance costs in traditional solutions, but also ensures system performance through architecture-level query optimization, providing a solid technical foundation for the subsequent development of specific implementation details.

[0033] Example 2

[0034] Building upon Example 1, this example further details the core data retrieval and assembly mechanism in the dynamic data packet configuration method based on the source node graph model and field tree structure. In this example, the field tree configuration defines the output JSON structure and the data source for each field; the source node configuration defines the database table and query conditions corresponding to each data retrieval unit; and the source relationship configuration defines the parent-child relationships between source nodes. Specifically, these three configuration entities remain independent in both physical storage and logical processing. For example, the field tree configuration can use a tree-structured data structure, including attributes such as field name, field type, bound source node code, and source field name, specifically responsible for describing "what the output looks like"; the source node configuration includes attributes such as physical table name, node type, and filtering conditions, specifically responsible for describing "where the data is"; and the source relationship configuration includes attributes such as parent source node code, child source node code, and associated field mapping, specifically responsible for describing "how the data is connected". This tripartite separation of powers configuration architecture gives the system extremely high flexibility. When business requirements change and only the output format needs to be adjusted, only the field tree configuration needs to be modified without touching the underlying data retrieval logic. Conversely, when the underlying database table structure is migrated, only the source node configuration needs to be updated, and the upper-level output structure definition is completely unaffected, thus truly achieving zero-code maintenance of the delivery interface.

[0035] In the recursive number retrieval phase, such as Figure 3As shown, each source node dynamically constructs an SQL query based on its own query conditions and the association parameters passed from the parent node. This process specifically includes: collecting the database columns that the current node needs to query and automatically supplementing the join key columns required for the association query; parsing the whereConfig configuration of the current source node and replacing the placeholders with the actual project information and interface information. To illustrate this mechanism more clearly, a project delivery scenario is used as an example for demonstration. Assuming that the source node "Contract List" is currently being processed, the system first traverses all leaf fields bound to the source node in the field tree and collects their corresponding database column names (such as contract number and contract amount). Next, the system checks whether the source node has child nodes (such as "Supplier Information"). If it does, it reads the association fields defined in the source relationship configuration (such as supplier code) and determines whether the association field is already in the aforementioned collection list. If it is not included, the system automatically supplements it into the query column. This step is crucial, as it ensures that even if the supplier code is not needed in the output field, the correct parent-child relationship can still be established through this field when assembling child nodes later, avoiding data gaps caused by missing association keys. Meanwhile, when constructing the WHERE clause, the system parses the conditional expressions in the source node configuration and replaces the preset placeholders (such as project code, interface code, etc.) with the actual values ​​in the current request context in real time, so that the same set of configurations can adapt to the dynamic filtering requirements of different projects or different business scenarios.

[0036] To address common performance bottlenecks in multi-level data queries, this embodiment employs a batch IN query based on the parent row set for child nodes. Specifically, this involves extracting all non-null, unique values ​​of the associated fields from the parent node's query result set, constructing parameterized IN conditions, and appending these, along with any additional conditions configured in the source relation, to the child node's WHERE clause. Continuing with the example above, when the "Contract List" query returns 100 records involving 20 different suppliers, the system, when querying the child node "Supplier Information," will not initiate 100 single-row queries in a loop as in traditional ORM frameworks. Instead, it extracts 20 unique supplier codes from these 100 parent row records and constructs a parameterized query statement like "supplier_code IN (?, ?, ... ?)" in one go. This approach compresses what could have been up to N database interactions into a constant-order batch query, fundamentally eliminating the performance degradation problem of N+1 queries. It should be understood that although this embodiment uses IN query as the preferred implementation method, in certain specific database dialects or scenarios with extremely large data volumes, equivalent alternatives such as temporary table association or EXISTS subqueries can also be used to achieve batch retrieval. As long as the core idea is to retrieve sub-data based on the parent result set as a whole rather than a single row, it should be covered within the scope of protection of this application.

[0037] After the data acquisition is complete, proceed to... Figure 2The JSON assembly stage is shown below. This stage is driven by the field tree configuration and distributes processing according to the field type of each field. Specifically, if the field type is string, the corresponding database value is retrieved from the result set cache; if the field type is object, the first row is used as the current context to recursively assemble subfields; if the field type is array, the data row set is traversed and subfields are recursively assembled row by row. Specifically, when encountering a string type leaf field, the system directly locates the specific column value of the corresponding data row in the result set cache in memory based on the bound source node code and source field name, and fills it in. When encountering an object type field, the system recognizes that the data bound to its source node is usually a one-to-one relationship, so it directly selects the first row of data in the result set as the current scope and recursively enters its subfield list for assembly, finally generating a nested JSON object. When encountering an array type field, the system obtains the complete data row set corresponding to the source node, creates a new JSON object context for each row of data in the set, and recursively assembles its subfields, finally aggregating all generated objects into a JSON array. Through this type-based recursive assembly strategy, the system can automatically adapt to nested structures of arbitrary depth. Whether it's a simple, flat list or a complex, multi-level tree-like data, it can be accurately generated through pure configuration-driven methods without the need to write any hard-coded persistence logic. It should be noted that the field types in the above example are only typical examples; in practical applications, custom types such as map and enum can also be supported, and their distribution processing logic can be adapted accordingly, referring to the approach in this embodiment.

[0038] Example 3

[0039] Building upon Embodiment 1 or Embodiment 2, this embodiment further details the source node reuse architecture and the security protection mechanism for dynamic queries. This application also includes a source node pooling mechanism: source nodes and interfaces are decoupled into independent resource pools, establishing many-to-many relationships through an intermediate association table. The same source node is referenced by multiple interfaces, and each interface defines its association method using its own source relationship configuration. Specifically, the system extracts the source node configuration from the specific interface definition, forming a globally shared resource pool. At the physical storage level, a separate interface-source node association table maintains the many-to-many mapping relationship between them. The core consideration of this architecture design is that although multiple delivery interfaces may need to query the same business table (e.g., a project information table or a supplier table), their association paths and filtering conditions for that table often differ. For example, interface A may need to associate the project table with the project number to obtain basic information, while interface B may need to associate the same project table with the contract number to obtain financial data. If the source node is strongly bound to the interface, the same project table query unit must be repeatedly defined for each interface, resulting in configuration redundancy and difficulty in unified maintenance. By introducing an intermediate association table and sinking the association logic down to the source relationship configuration level, the system implements the single-source principle for data retrieval unit definitions. When the underlying database table structure changes, only the configuration of the corresponding unique source node in the resource pool needs to be modified, and all interfaces referencing that node can automatically inherit the change. This significantly reduces the configuration and maintenance costs across interfaces and effectively prevents the risk of data inconsistency caused by multiple duplicate definitions. It should be understood that although this embodiment uses a database association table as the preferred method for implementing many-to-many associations, other embodiments may also use equivalent means such as memory object graphs, nested configuration files, or graph databases to achieve decoupling management of source nodes and interfaces. As long as the essence is to abstract the data retrieval unit into a reusable resource and support interface-level differentiated associations, they should all be covered within the scope of protection of this application.

[0040] To enhance the flexibility and security of dynamic query conditions, the source node configuration in this application also includes a `whereConfig` configuration. This `whereConfig` configuration uses an array object in JSON format and supports ten operators. Placeholders within the values ​​of this `whereConfig` configuration are replaced with actual values ​​at runtime. Specifically, `whereConfig` is designed as a standardized JSON array, where each element represents an independent filtering condition item, containing three core attributes: field name, operator, and value. The system supports ten commonly used logical operators: equal to, not equal to, greater than, greater than or equal to, less than, less than or equal to, fuzzy matching (like), contain (in), range (between), and null value judgment (is null, is not null), sufficient to cover most business filtering scenarios. More importantly, the condition value field supports embedding business context placeholders such as project code, project English name, and interface code. These placeholders are not fixed when the configuration is saved, but are parsed and replaced with actual values ​​in real time by the system based on the current request context during the runtime phase of each request to perform data retrieval operations and dynamically construct SQL statements. For example, the condition value specified in the configuration might be the string {projectCode}. When processing a request with project number NUC-P001, the system will automatically replace it with NUC-P001 before generating the SQL. This delayed binding mechanism allows the same source node configuration to seamlessly adapt to delivery requests from different projects, batches, or tenants, without having to copy a separate configuration for each project, greatly improving the versatility and reusability of configuration assets.

[0041] Furthermore, since dynamic SQL construction inherently carries the risk of injection attacks, this application also includes an SQL injection protection mechanism: regular expression validation is performed on all table and column names used for dynamically constructed SQL queries, allowing only identifiers composed of letters, numbers, and underscores; values ​​are passed through parameterized queries. Specifically, this protection mechanism employs a dual defense strategy combining identifier whitelist validation and parameterized value passing. When concatenating the structural parts of the SQL statement (i.e., the identifier positions such as table names, column names, and aliases), the system performs strict regular expression matching on each configured identifier string, allowing only strings composed entirely of English letters, Arabic numerals, and underscores. Any input containing spaces, quotation marks, semicolons, comment symbols, or other special characters is directly rejected and an exception is thrown. This measure eliminates the possibility of injecting malicious SQL code by tampering with table or column names at the source. Simultaneously, for the data value part of the SQL statement, the system enforces the use of pre-compiled parameterized queries, binding all dynamic values ​​as independent parameters to the SQL template, rather than directly concatenating them into the SQL text. Even if an attacker bypasses front-end validation and injects malicious fragments into the value field, the database engine will treat it as pure literal data rather than executable instructions, thus completely eliminating the risk of value injection. Through this defense-in-depth system that separates structure from value and combines validation and binding, this application ensures the operational security of dynamic SQL in an open configuration environment while maintaining its high flexibility, resolving the technical contradiction between security and flexibility in traditional dynamic concatenation schemes.

[0042] Example 4

[0043] Building upon Examples 1 to 3, this example further details the fault-tolerance logic for leaf field values ​​during the field tree assembly stage. This application also includes a three-level value fallback mechanism for leaf fields: leaf field values ​​follow a three-level priority fallback mechanism based on the actual database value, the default value configured in the field table, and null. Specifically, this mechanism is specifically designed for leaf fields of type string, providing a deterministic fallback strategy when retrieving data from the result set cache to ensure the integrity and predictability of the output data structure. During execution, the system first attempts to retrieve the actual value found in the database. If the actual value exists and is not null, it is directly used as the field output value; this is the highest priority data source, ensuring the authenticity and timeliness of the data. When the actual database value is NULL, or when the current source node does not have a corresponding record in the result set cache due to a mismatch in the association conditions, the system does not immediately output a null value. Instead, it automatically enters the second-level fallback, reading the default_value attribute value preset for the field in the field tree configuration to fill it. The system will only execute the third-level fallback and ultimately return null as the field value when the actual value in the database is missing and the default value configured in the field table is not set or is an empty string. For example, in a project delivery scenario, suppose a string-type leaf field named "Remarks" is defined in the field tree. The source field it is bound to has a large number of NULL values ​​in the database due to historical data migration, but the business specifications require that this field in the delivery data package must have clear semantics. In this case, the configuration personnel can set the default_value of this field to "No Remarks" in the field tree configuration. When the system processes a database record where this field is NULL, it will automatically fill it with "No Remarks", thereby ensuring that this field always has a value in the output JSON structure and avoiding the downstream receiver triggering an exception due to parsing unexpected null values ​​or missing keys. It should be understood that although this embodiment uses string type fields as the main application object of the three-level value fallback mechanism, in other implementations, this mechanism can also be extended to other leaf field types such as numeric and boolean types. It is only necessary to adjust the parsing and conversion logic of the default value according to the characteristics of the data type. By adopting this tiered fallback design, the system shifts data integrity assurance from relying on the quality of underlying data to proactive defense at the configuration level. This effectively isolates the impact of dirty and messy source data on the stability of the delivery interface, significantly improving the system's robustness and delivery success rate in complex and heterogeneous data environments.

[0044] Example 5

[0045] Digital delivery requirements for nuclear power engineering projects necessitate the delivery of heterogeneous business data from multiple sources, such as ERP, CRM, SRM, and MES, according to the structured format specified by the client during project construction. Different clients and projects have varying requirements for the content, format, and field naming conventions of the delivered data, leading to the following prominent challenges in the development and maintenance of the delivery interface. Further assuming that the digital delivery interface for a nuclear power project needs to output the following JSON data packets: ads_project (project table), ads_project_contact (project contact table), ads_project_contract (contract table), and ads_supplier (supplier table).

[0046] This invention is implemented in the backend service of a nuclear power digital delivery platform, and its core components include: Delivery Data Service (TDeliveryDataService): Receives delivery requests and coordinates the entire process of configuration loading, data retrieval, and JSON assembly; Interface Configuration Service (TInterfaceConfigService): Manages complete configuration information for interfaces, including field trees, source nodes, and source relationships; Project Service (TProjectService): Manages basic project information and is used for placeholder replacement; Database access component (JdbcTemplate): Executes dynamically constructed parameterized SQL queries; Three core configuration tables: field tree configuration table (t_interface_field), source node configuration table (t_interface_source), and source relationship configuration table (t_interface_source_relation); A single intermediate table: the interface-source node association table (t_interface_source_mapping), which supports source node pooling and reuse; A delivery data table (t_delivery_data): stores the final generated delivery data packets.

[0047] Table 1 is the field tree configuration table (t_interface_field). This table defines the output JSON structure and value mapping relationship of the delivery data packet.

[0048] Table 1. Field Tree Configuration Table

[0049] Table 2 is the source node configuration table (t_interface_source). This table defines the data retrieval unit, and each source node corresponds to a query definition of a physical database table.

[0050] Table 2. Source Node Configuration Table

[0051] The `where_config` directive uses a JSON array format, where each condition item contains a field (column name), an op (operator), and a value. Supported operators include: =, !=, >, >=, <, <=, like, in, between, is null, and is not null. Values ​​can use placeholders such as `{projectCode}`, `{projectEname}`, and `{interfaceCode}`, which are automatically replaced at runtime.

[0052] Table 3 is the source relationship configuration table (t_interface_source_relation), which defines the parent-child relationship between source nodes.

[0053] Table 3. Source Relationship Configuration Table

[0054] Based on the above configuration list, the implementation process of this embodiment is as follows: Step 1: Load Configuration The system receives input parameters interfaceCode = "NUC_DELIVERY_V1" and projectCode = "NUC-P001".

[0055] First, query the interface configuration entity to obtain the list of associated field trees, source nodes, and source relationships. Then, query the project information to obtain projectCode = "NUC-P001" and projectEname = "SampleNucProject".

[0056] Construct three mapping structures in memory: (a) Source node mapping (sourceCode → source node entity): "rootProject" → {tableName:"ads_project", nodeType:"ROOT",whereConfig:'[{"field":"project_code","op":"=","value":"{projectCode}"}]'} "contactInfo" → {tableName:"ads_project_contact", nodeType:"OBJECT",whereConfig:null} "contractList" → {tableName:"ads_project_contract", nodeType:"ARRAY", whereConfig:null} "supplierInfo" → {tableName:"ads_supplier", nodeType:"OBJECT",whereConfig:null} (b) Field binding mapping (sourceCode → field list): "rootProject" → [{fieldName:"projectName", sourceField:"project_name"}, {fieldName:"projectCode", sourceField:"project_code"}] "contactInfo" → [{fieldName:"contactName", sourceField:"contact_name"}, {fieldName:"phone", sourceField:"phone"}] "contractList" → [{fieldName:"contractCode", sourceField:"contract_code"}, {fieldName:"contractAmount", sourceField:"contract_amount"}] "supplierInfo" → [{fieldName:"supplierName", sourceField:"supplier_name"}, {fieldName:"supplierCode", sourceField:"supplier_code"}] (c) Source relationship mapping (parentSourceCode → child relationship list): "rootProject" → [ {childSourceCode:"contactInfo", joinConfig:'{"parentField":"project_code","childField":"project_code"}'},{childSourceCode:"contractList", joinConfig:'{"parentField":"project_code","childField":"project_code"}'} ] "contractList" → [ {childSourceCode:"supplierInfo", joinConfig:'{"parentField":"supplier_code","childField":"supplier_code"}'} ] Simultaneously locate the root node: traverse the source node list to find the node "rootProject" with nodeType set to ROOT.

[0057] Step 2: Recursively retrieve numbers

[0058] 2.1 Execute the root node rootProject

[0059] Collect the columns to be queried: project_name (field tree binding) and project_code (field tree binding). Traverse the sub-relationships and find that the parentField in joinConfig of contactInfo and contractList is project_code, but project_code is already in the collected list and does not need to be added.

[0060] Parse the whereConfig configuration, replace the placeholder {projectCode} with "NUC-P001", and generate the condition: "project_code = 'NUC-P001'".

[0061] Dynamically construct SQL: SELECT project_name, project_code FROM nuc_biz.ads_project WHERE project_code = ?, with parameter ["NUC-P001"].

[0062] Execute the query and get the result row: [{project_name:"Example Nuclear Power Project", project_code:"NUC-P001"}], which is cached in the result set mapping with "rootProject" as the key.

[0063] 2.2 Recursively execute the child node contactInfo

[0064] The parent node result set is: [{project_name:"Example Nuclear Power Project", project_code:"NUC-P001"}].

[0065] Collection columns: contact_name, phone. Navigate to the relationship contactInfo → rootProject; the parentField is project_code, which is already in the collection list.

[0066] Construct association condition: Extract the value of project_code "NUC-P001" from the parent result set, generate condition: project_code IN (?), with parameter ["NUC-P001"].

[0067] Dynamically construct and execute the SQL, and get the result: [{contact_name:"Zhang San", phone:"13800000000"}], which is then cached in the result set mapping.

[0068] 2.3 Recursively execute child nodes contractList

[0069] Similarly, construct the SQL to obtain two rows of result set: [{contract_code:"CT-001", contract_amount:5000000.00, supplier_code:"SUP-001"}, {contract_code:"CT-002", contract_amount:3000000.00, supplier_code:"SUP-002"}], which can then be cached.

[0070] 2.4 Recursively execute the child node supplierInfo

[0071] The parent node `contractList` has two rows of results. Extract the value of `parentField(supplier_code)`: ["SUP-001", "SUP-002"].

[0072] Construct the associated condition: supplier_code IN (?, ?), with parameters ["SUP-001", "SUP-002"].

[0073] Executing the query yields two rows of results: [{supplier_name:"Supplier A", supplier_code:"SUP-001"}, {supplier_name:"Supplier B", supplier_code:"SUP-002"}], which are then cached.

[0074] At this point, the data retrieval phase is complete, and the result set mapping contains the complete data of the four source nodes.

[0075] Step 3: Field Tree-Driven JSON Assembly

[0076] Construct a parent-child mapping for the field tree list by parentId, and locate the root field (the field where parentId is null): projectName (string), projectCode (string), contactInfo (object), contractList (array).

[0077] 3.1 Processing the leaf field projectName

[0078] `fieldType` is string, `sourceCode` is "rootProject", and `sourceField` is "project_name". Retrieve the first row of data from `rootProject` in the result set mapping, and obtain the value of `project_name`: "Example Nuclear Power Project". Output: `"projectName": "Example Nuclear Power Project"`.

[0079] 3.2 Processing the contactInfo field of the object

[0080] The fieldType is object, and the sourceCode is "contactInfo". Parse the source node data of contactInfo: [{contact_name:"Zhang San", phone:"13800000000"}], and take the first line as the current context.

[0081] Recursively assemble the subfields of contactInfo: contactName(string, sourceField=contact_name) → Retrieves contact_name from the current context = "Zhang San" phone(string, sourceField=phone) → Retrieves phone = "13800000000" from the current context. Output: "contactInfo": {"contactName": "Zhang San", "phone": "13800000000"}.

[0082] 3.3 Processing the array field contractList

[0083] The fieldType is array, and the sourceCode is "contractList". Parse the source node data of contractList to obtain two rows.

[0084] Iterate through each row and recursively assemble: Line 0 → {contractCode:"CT-001", contractAmount:5000000.00}, simultaneously parsing the supplierInfo child object, locating the supplierInfo line {supplier_name:"Supplier A", supplier_code:"SUP-001"} by matching parent and child (supplier_code=SUP-001), and outputting the nested object. Line 1 → {contractCode:"CT-002", contractAmount:3000000.00}, similarly matching supplierInfo. Output: "contractList": [{"contractCode":"CT-001","contractAmount":5000000.00,"supplierInfo":{...}}, ...].

[0085] Step 4: Output delivery data package

[0086] The assembled nested Map structure is persisted as a JSON string, which can then be further persisted to the delivery data table or returned to the caller via API.

[0087] Example 6

[0088] This embodiment provides a dynamic data packet configuration device, including a memory and a processor. When the processor executes a computer program stored in the memory, it implements a dynamic data packet configuration method based on a source node graph model and a field tree structure as described in any one of Embodiments 1 to 5.

[0089] Specifically, the memory is used to store computer program instructions that implement the above methods, as well as related configuration data and runtime cache data. In actual deployment, the memory may include high-speed random access memory, or non-volatile memory such as hard disks or flash memory, for persistently storing basic metadata such as field tree configuration, source node configuration, and source relationship configuration. The processor, as the core of the device's computation and control, reads and executes program instructions from the memory to complete the entire process from configuration loading, memory mapping construction, recursive data retrieval to JSON assembly and persistence. It should be understood that the processor here can be a general-purpose processor, digital signal processor, application-specific integrated circuit, or other programmable logic device; this application does not impose specific limitations on this.

[0090] To facilitate understanding of the internal functional divisions of the device, in one optional implementation, the process of the processor executing the program can be abstracted into multiple functional modules. For example, the configuration loading module corresponds to step S1 in embodiment 1, which is responsible for reading three independent configuration entities from memory or an external database according to the interface encoding; the memory mapping construction module corresponds to step S2, which is used to establish a source node mapping table, a source relationship mapping table, and a field binding mapping table in memory and locate the root node; the recursive data retrieval module corresponds to step S3, which is responsible for recursively performing data retrieval operations according to the parent-child structure defined by the source relationship, and storing the results in the cache using a batch IN query mechanism based on the parent row set; the JSON assembly module corresponds to step S4, which is driven by the field tree configuration, distributes processing according to the field type, and recursively assembles the nested structure by extracting data from the cache; and the persistent output module corresponds to step S5, which converts the assembled nested JSON structure into a delivery data package. It should be noted that the above module division is only a logical functional decoupling. In actual physical implementation, these functions can be integrated and executed by the same processor, or they can be completed by multiple processors working together. Some functions can even be implemented by dedicated hardware circuits, as long as the whole can realize the dynamic data packet configuration method based on the source node graph model and field tree structure described in this application.

[0091] Furthermore, this dynamic data packet configuration device can be deployed in various computing environments. For example, it can be a standalone database server, application server, a computing node in a distributed cloud computing platform, or an edge computing device. When the device runs as a server, it can receive delivery requests from clients through a network interface, obtain configuration information locally or through remote calls, and finally return the generated delivery data packet to the caller via the network. Through this hardware and software combined implementation method, the technical solution of this application can be implemented at the product level, providing clear physical evidence for subsequent rights protection.

[0092] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dynamic data packet configuration method based on a source node graph model and a field tree structure, characterized in that, include: Load the field tree configuration, source node configuration, and source relationship configuration of the corresponding interface based on the interface code; Establish source node mapping table, source relation mapping table and field binding mapping table in memory, and locate the root node; Starting from the root node, the data retrieval operation is recursively executed according to the parent-child structure defined by the source relationship configuration. Each source node dynamically constructs an SQL query based on its own query conditions and the association parameters passed by the parent node. The query results are stored in the result set cache with the source node code as the key. Among them, the child nodes use batch IN queries based on the parent row set. Driven by the field tree configuration, processing is distributed according to the field type of each field, and corresponding data is extracted from the result set cache and recursively assembled into a nested JSON structure; The assembled nested JSON structure is persisted as a delivery data package.

2. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 1, characterized in that: The field tree configuration is used to define the output JSON structure and the data source of each field; the source node configuration is used to define the database table and query conditions corresponding to each data retrieval unit; and the source relationship configuration is used to define the parent-child relationship between source nodes.

3. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 2, characterized in that, Each source node dynamically constructs an SQL query based on its own query conditions and the association parameters passed in by the parent node, including: collecting the database columns that the current node needs to query and automatically supplementing the join key columns required for the association query; parsing the whereConfig configuration of the current source node and replacing the placeholders in it with the actual project information and interface information.

4. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 3, characterized in that, The child node uses a batch IN query based on the parent row set, which includes: extracting all non-empty and non-repeating values ​​of the associated fields from the parent node query result set, constructing IN parameterized conditions, and attaching them, along with the additional conditions in the source relation configuration, to the WHERE clause of the child node.

5. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 4, characterized in that, The process of distributing data based on the field type of each field includes: If the field type is string, retrieve the corresponding database value from the result set cache; If the field type is object, take the first row as the current context and recursively assemble the subfields; If the field type is array, iterate through the data row set and recursively assemble the subfields row by row.

6. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 1, characterized in that, It also includes a source node pooling mechanism: source nodes and interfaces are decoupled into independent resource pools, and many-to-many relationships are established through an intermediate association table. The same source node is referenced by multiple interfaces, and each interface uses its own source relationship configuration to define the association method.

7. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 6, characterized in that: The source node configuration also includes the whereConfig configuration, which uses an array object in JSON format and supports ten operators; the placeholders in the values ​​of the whereConfig configuration are replaced with the actual values ​​at runtime.

8. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 7, characterized in that, It also includes SQL injection protection mechanisms: regular expression validation is performed on all table and column names used to dynamically construct SQL queries, allowing only identifiers composed of letters, numbers, and underscores; values ​​are passed through parameterized queries.

9. The dynamic data packet configuration method based on source node graph model and field tree structure according to claim 1, characterized in that, The method also includes constructing a three-level value fallback mechanism for leaf fields: the leaf field value follows the actual value in the database, the default value configured in the field table, and the null value, which are three levels of priority fallback mechanism.

10. A dynamic data packet configuration system based on a source node graph model and a field tree structure, comprising a memory and a processor, characterized in that, When the processor executes the computer program stored in the memory, it implements the dynamic data packet configuration method based on the source node graph model and field tree structure as described in any one of claims 1 to 9.

Images