A labor data structure dynamic reconstruction and fine-grained change tracing method

Abstract

The application discloses a labor data structure dynamic reconstruction and fine-grained change tracing method, and decouples data structure and business logic from fixed program code, so that the system can dynamically construct data descriptors according to metadata at runtime, and realizes dynamic expansion of labor data structure. When business attributes are newly added, modified or deleted, data model adjustment can be completed without modifying entity classes and database table structures, and in combination with a mixed storage mode of fixed columns and JSON columns, flexible storage of dynamic attributes is realized, the adaptive capacity of the system to business changes is improved, and the system upgrade and maintenance cost is reduced.

Description

Technical Field

[0001] This invention belongs to the field of change tracing methods, specifically, it relates to a method for dynamic reconstruction of labor force data structure and fine-grained change tracing. Background Technology

[0002] With the continuous improvement of digital management, labor information management systems are widely used in business scenarios such as labor resource collection, personnel management, employment supervision, and statistical analysis. Existing labor information management systems typically adopt a data management model that binds strongly typed entity objects to fixed database tables. That is, developers predefine entity classes and establish a correspondence between the attributes in the entity classes and fixed fields in the database tables. At the same time, the front-end page input items, data transmission objects, and business processing logic are all developed around the predefined data structure. The back-end completes data validation and business processing through hard-coded programs or annotations. Statistical analysis functions usually rely on pre-written database query statements or mapping files to achieve fixed-dimensional data statistics.

[0003] However, in practical applications, as workforce management operations are constantly adjusted, the data collection fields, regulatory requirements, and statistical standards frequently change. When adding, modifying, or deleting business attributes, existing systems typically require simultaneous modifications to entity classes, database table structures, data transmission objects, and business processing programs, as well as recompiling and redeploying the entire system. This not only results in long development cycles but also high system maintenance costs, making it difficult to meet the business needs of high-frequency dynamic adjustments to workforce data.

[0004] Furthermore, existing data validation rules are typically scattered across various business processes or annotation configurations, with strong coupling between different business rules. Dynamic configuration is difficult for complex correlation validations that rely on context information, leading to difficulties in maintaining business rules and poor system scalability. Simultaneously, the auditing functions of existing systems usually only record user operation logs, failing to perform fine-grained comparisons and recording of changes before and after updates to individual attributes within data objects. This makes it difficult to form a complete attribute-level change trajectory, hindering data change tracing and audit management.

[0005] On the other hand, existing statistical analysis functions are generally based on pre-built query statements using fixed database fields. When new dynamic business attributes are added, they cannot be automatically identified and corresponding aggregated statistical queries can be built during operation. This requires the redevelopment of statistical programs, resulting in fixed statistical dimensions and insufficient flexibility, making it difficult to meet the needs of multi-dimensional and dynamic statistical analysis of labor force data.

[0006] In view of this, the present invention is proposed. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method for dynamic reconstruction of labor data structure and fine-grained change tracing, which solves the problems mentioned in the background art.

[0008] To solve the above-mentioned technical problems, the basic concept of the technical solution adopted by the present invention is as follows: A method for dynamic reconstruction of labor force data structure and fine-grained change tracing includes the following steps: S1. Obtain the metadata corresponding to the labor force data, and construct a dynamic data descriptor based on the metadata; S2. Receive labor force data, parse the labor force data according to the dynamic data descriptor, and complete the dynamic storage of the labor force data; S3. Construct a rule chain based on the verification rules corresponding to the metadata, dynamically verify the parsed labor force data, and perform data updates based on the verification results; S4. Obtain the labor force data before and after the update, compare the differences between the labor force data before and after the update according to the dynamic data descriptor, and generate attribute-level change records. S5. Receive a statistics request, determine the statistical attributes based on the metadata, and generate the corresponding dynamic statistical results.

[0009] Optionally, the step of obtaining the metadata corresponding to the labor force data and constructing a dynamic data descriptor based on the metadata is as follows: Obtain the metadata configured by the administrator. The metadata includes at least the unique identifier of the attribute, data type, tree path, display rules and validation rules, and write the metadata into the metadata configuration table. The dynamic model parsing engine reads the metadata from the metadata configuration table and uses reflection to parse the metadata, generating the corresponding dynamic attribute description object; The generated multiple dynamic attribute description objects are hierarchically associated according to a tree path to form dynamic data descriptors, and a mapping relationship is established between the attribute unique identifier and the dynamic data descriptors.

[0010] Optionally, after obtaining the dynamic data descriptor, the following steps also need to be performed: The dynamic data descriptor is loaded into the dynamic model parsing engine so that the dynamic model parsing engine can parse the labor force data based on the dynamic data descriptor; During the process of parsing labor data using the dynamic data descriptor, the change log corresponding to the metadata configuration table is monitored, and it is determined whether the metadata corresponding to the dynamic data descriptor has changed. When it is determined that the corresponding metadata has been added, modified or deleted, the changed metadata is reread, the corresponding dynamic attribute description object is regenerated, and the dynamic data descriptor is updated using the regenerated dynamic attribute description object, so that the updated dynamic data descriptor maintains the correspondence with the metadata configuration table.

[0011] Optionally, the steps of receiving labor force data, parsing the labor force data according to the dynamic data descriptor, and completing the dynamic storage of the labor force data include: Receive labor force data and call the corresponding dynamic data descriptor; The attribute definition information corresponding to the labor force data is obtained based on the dynamic data descriptor, each attribute in the labor force data is identified, and data parsing and format conversion are completed according to the data type corresponding to each attribute. Based on the parsed attribute information, determine the storage location corresponding to each attribute, map fixed attributes to corresponding fixed data columns, and organize dynamic attributes into extended attribute data; Perform JSON serialization on the extended attribute data to generate the corresponding extended data object; Data from fixed data columns is associated with extended data objects and then written together into the labor force data storage table to complete the dynamic storage of labor force data.

[0012] Optionally, the extended data object is encapsulated using a JSON object, the fixed data column is used to store predefined fixed attributes, and the extended data object is used to store dynamically added attributes, so that the addition of attributes can be completed without modifying the table structure of the labor data storage table.

[0013] Optionally, the step of dynamically validating the parsed labor force data according to the validation rules corresponding to the metadata, and performing data updates based on the validation results, includes: Obtain the verification rules corresponding to the metadata, and construct the corresponding rule chain based on each verification rule; The dynamic data descriptor is invoked to determine the target attributes to be verified based on the rule chain, and the attribute values ​​corresponding to each target attribute are obtained. The validation rules in the rule chain are executed in the order of priority corresponding to each validation rule, and each target attribute is dynamically validated and the corresponding validation result is generated. Summarize the verification results corresponding to each verification rule to form a verification result set of labor force data; Based on the verification result set, determine whether the labor force data meets the update conditions. If the update conditions are met, perform the labor force data update; if the update conditions are not met, output the corresponding verification exception information.

[0014] Optionally, constructing the corresponding rule chain according to each verification rule includes: Obtain each validation rule in the rule chain and parse each validation rule into a corresponding abstract syntax tree, wherein the abstract syntax tree includes attribute nodes, operation nodes, judgment nodes and logical connection nodes; Traverse each attribute node in the abstract syntax tree, and call the dynamic data descriptor according to the attribute identifier corresponding to each attribute node to determine the data type and attribute value of the corresponding attribute; Replace the node data of the corresponding attribute node in the abstract syntax tree with the obtained attribute value, while maintaining the logical relationship between the nodes; Based on the node hierarchy and logical connection relationship of the abstract syntax tree, the logical operations corresponding to each operation node and judgment node are executed from bottom to top to obtain the verification results corresponding to each verification rule. The system summarizes the verification results corresponding to each verification rule in the rule chain and generates the rule execution results for dynamic verification of labor force data.

[0015] Optionally, the step of obtaining labor force data before and after the update, comparing the differences between the labor force data before and after the update based on the dynamic data descriptor, and generating attribute-level change records includes: Obtain the labor force data before and after the update, and call the dynamic data descriptor corresponding to the labor force data; Based on the dynamic data descriptor, traverse the attributes in the labor force data before and after the update to determine the target attributes for difference comparison; Based on the attribute identifier corresponding to each target attribute, obtain the attribute values ​​before and after the update, and compare the corresponding attribute values ​​item by item; The target attribute that has changed is identified based on the attribute value comparison results, and the change type of the corresponding target attribute is determined. The labor force data before and after the update includes the labor force data corresponding to newly added records, modified records, and deleted records. Based on the identified target attributes and corresponding change types, attribute-level change records are generated, and the association between the attribute-level change records and the corresponding labor force data is established.

[0016] Optionally, the step of identifying the target attribute that has changed based on the attribute value comparison result and determining the change type of the corresponding target attribute includes: Based on the attribute value comparison results, obtain the attribute state of the target attribute before and after the update, and determine whether the attribute value has changed before and after the update; When the corresponding attribute value is empty before the update and the corresponding attribute value is not empty after the update, the target attribute is determined to be in a new state, and the change type corresponding to the target attribute is determined to be a new attribute. When both the corresponding attribute value before and after the update exist, a consistency comparison is performed on the attribute values ​​before and after the update. If the comparison result shows that the two are inconsistent, the target attribute is determined to be in a modified state, and the change type corresponding to the target attribute is determined to be a modified attribute. When the corresponding attribute value is not empty before the update and the corresponding attribute value is empty after the update, the target attribute is determined to be in a deleted state, and the change type corresponding to the target attribute is determined to be a deleted attribute. The identified new, modified, and deleted attributes are categorized and summarized, and the change type corresponding to each target attribute is passed to the attribute-level change record generation step to generate the corresponding attribute-level change record.

[0017] Optionally, the steps of receiving a statistical request, determining statistical attributes based on the metadata, and generating corresponding dynamic statistical results include: Receive a statistics request and parse the statistical dimensions and conditions in the statistics request; Based on the statistical dimensions, query the corresponding metadata to obtain the attribute identifier, data type and storage location information of the statistical attributes, and determine the data source of the statistical attributes based on the storage location information; Based on the determined data source and statistical conditions, a corresponding dynamic aggregation query statement is generated. Specifically, when the statistical attribute corresponds to a fixed data column, an aggregation query statement for the fixed data column is generated; when the statistical attribute corresponds to extended data, an attribute extraction query statement for the extended data is generated. The dynamic aggregation query statement is executed to perform aggregation calculations on the data corresponding to the statistical attributes and obtain the statistical results; The statistical results are organized and encapsulated to generate dynamic statistical results corresponding to the statistical request and output them.

[0018] Optionally, it can be determined whether the changed target attribute is a sensitive attribute based on a preset sensitive monitoring strategy; When the target attribute is a sensitive attribute, a corresponding monitoring event is generated; The monitoring event is sent to the message queue to trigger the corresponding alarm notification.

[0019] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all of the following advantages at the same time: This invention decouples data structure and business logic from fixed program code, enabling the system to dynamically construct data descriptors at runtime based on metadata, thus achieving dynamic expansion of the workforce data structure. When business attributes are added, modified, or deleted, the data model can be adjusted without modifying the entity class and database table structure. Furthermore, by combining fixed columns with JSON columns for hybrid storage, flexible storage of dynamic attributes is achieved, improving the system's adaptability to business changes and reducing system upgrade and maintenance costs.

[0020] Meanwhile, this invention dynamically parses the configured verification rules into an abstract syntax tree and uses a rule chain executor to dynamically execute each verification rule according to the business context, realizing the configurable management of complex business rules and dynamic verification of the relationships between attributes. Compared with the method of embedding the verification logic in the program code, this invention can complete the adjustment of business rules without modifying the program, improving the efficiency of rule maintenance and the scalability of the system, and can output all verification exception information at once, improving the integrity of data verification and the efficiency of user interaction.

[0021] Furthermore, this invention utilizes a reflection mechanism to perform attribute-level deep comparisons of labor force data before and after updates, automatically identifying attribute changes such as additions, modifications, and deletions, generating fine-grained attribute-level change records, and establishing a complete change trajectory. Simultaneously, combined with a sensitive attribute monitoring mechanism, it automatically triggers alarms for changes to key attributes, achieving full-process change auditing and real-time monitoring of labor force data, thus improving the traceability of data changes, data security, and audit management capabilities.

[0022] Furthermore, this invention dynamically generates aggregation query plans based on the physical storage location of attributes defined in the metadata. For fixed attributes, it directly generates fixed column query statements, and for dynamic attributes, it automatically generates JSON attribute extraction statements. Combined with dynamic SQL, it completes statistical calculations, enabling the system to perform real-time statistical analysis of different business attributes without the need to pre-design statistical dimensions or modify statistical programs. This improves the flexibility, real-time performance, and adaptability of dynamic statistical calculations to constantly changing business needs.

[0023] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0024] The accompanying drawings described below are merely some embodiments. Those skilled in the art can obtain other drawings based on these drawings without any creative effort. In the drawings: Figure 1 This is a schematic diagram of the method flow of the present invention.

[0025] It should be noted that these accompanying drawings and textual descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0026] The invention will now be described in further detail with reference to the accompanying drawings.

[0027] This embodiment provides a method for dynamic reconstruction of labor force data structure and fine-grained change tracing. The method is deployed in a labor force data management platform and mainly includes a metadata configuration center, a dynamic model parsing engine, a rule chain executor, a differential audit module, a dynamic aggregation calculator, and a data storage layer. The modules work together to achieve dynamic management of labor force data through data interaction.

[0028] The metadata configuration center receives the workforce data attribute information configured by the administrator and saves metadata such as attribute unique identifiers, data types, tree paths, explicit / implicit rules, verification rules, and monitoring strategies to the metadata configuration table, providing a unified data configuration foundation for system operation.

[0029] The dynamic model parsing engine reads metadata from the metadata configuration table and uses reflection to construct dynamic data descriptors at runtime, enabling dynamic parsing of the workforce data structure. When metadata is added, modified, or deleted, the dynamic model parsing engine reloads the corresponding metadata and updates the dynamic data descriptors, allowing the system to adapt to changes in business attributes without modifying program code or database table structure.

[0030] The rule chain executor is used to load the validation rules configured in the metadata, parse the rule expressions into an abstract syntax tree, obtain the corresponding attribute values ​​based on the dynamic data descriptor, perform dynamic validation on the labor force data, and generate the corresponding validation results.

[0031] The differential audit module is used to compare attribute-level differences between labor force data before and after the update during the process of adding, modifying or deleting labor force data, generate corresponding attribute-level change records, and record the change trajectory of labor force data to achieve fine-grained change tracing.

[0032] The dynamic aggregation calculator receives statistical requests, determines the storage location of statistical attributes based on metadata, dynamically generates aggregation query statements, and completes real-time statistical analysis of labor force data.

[0033] The data storage layer adopts a hybrid storage structure that combines fixed data columns and JSON extended columns. The fixed data columns store basic attributes, while the JSON extended columns store dynamically extended attributes. This decouples the workforce data structure from the database storage structure and improves the system's adaptability to business changes.

[0034] Please see Figure 1 As shown, this embodiment provides a method for dynamic reconstruction of labor force data structure and fine-grained change tracing, including the following steps: S1. Obtain the metadata corresponding to the labor force data, and construct a dynamic data descriptor based on the metadata; In this embodiment, the steps of obtaining metadata corresponding to labor force data and constructing a dynamic data descriptor based on the metadata are as follows: Obtain the metadata configured by the administrator. The metadata includes at least the unique identifier of the attribute, data type, tree path, display rules and validation rules, and write the metadata into the metadata configuration table. When the engine receives JSON data submitted from the front end, it does not perform traditional object mapping. Instead, it extracts the dynamic attributes based on the descriptor, serializes them into binary JSON format, and directly stores them in the extended column of the database table. This process does not require modification of the entity class or database table modification.

[0035] The dynamic model parsing engine reads metadata from the metadata configuration table and uses reflection to parse the metadata, generating corresponding dynamic attribute description objects. The reloading of the dynamic model parsing engine is triggered as follows after metadata configuration changes: Passive listener refresh: The dynamic model parsing engine, as an independent component, subscribes to the change log of the database configuration table (such as based on MySQL Binlog). When it detects the addition, deletion, or modification of metadata records, it automatically triggers incremental or full reload, updating only the affected attribute descriptors and avoiding the performance loss caused by full reconstruction.

[0036] The generated multiple dynamic attribute description objects are hierarchically associated according to a tree path to form dynamic data descriptors, and a mapping relationship is established between the attribute unique identifier and the dynamic data descriptors.

[0037] In this embodiment, after obtaining the dynamic data descriptor, the following steps also need to be performed: The dynamic data descriptor is loaded into the dynamic model parsing engine so that the dynamic model parsing engine can parse the labor force data based on the dynamic data descriptor; During the process of parsing labor data using the dynamic data descriptor, the change log corresponding to the metadata configuration table is monitored, and it is determined whether the metadata corresponding to the dynamic data descriptor has changed. When it is determined that the corresponding metadata has been added, modified or deleted, the changed metadata is reread, the corresponding dynamic attribute description object is regenerated, and the dynamic data descriptor is updated using the regenerated dynamic attribute description object, so that the updated dynamic data descriptor maintains the correspondence with the metadata configuration table.

[0038] In this embodiment, the administrator first configures the metadata corresponding to the workforce data through the metadata configuration center. The metadata includes at least the following information: unique attribute identifier, data type, tree path, display / concealment rules, and validation rules. The unique attribute identifier uniquely identifies each business attribute; the data type describes the data format of the corresponding attribute; the tree path establishes the hierarchical relationship between attributes; the display / concealment rules control the display status of attributes; and the validation rules define the data validation logic corresponding to the attributes. After configuration, the metadata configuration center saves all metadata to the metadata configuration table.

[0039] Subsequently, the dynamic model parsing engine reads the metadata from the metadata configuration table and uses reflection to parse the metadata information corresponding to each attribute. Based on the attribute's unique identifier, data type, and tree path, it constructs corresponding dynamic attribute description objects. Then, based on the hierarchical relationship between these dynamic attribute description objects, it establishes a tree-like association, forming a dynamic data descriptor (MetaDescriptor). Simultaneously, it establishes a mapping relationship between the attribute's unique identifier and the dynamic data descriptor. This dynamic data descriptor describes the dynamic data structure of the labor force data and serves as a unified data model for subsequent labor force data parsing, dynamic storage, and business processing.

[0040] Furthermore, dynamic rule chain validation based on Abstract Syntax Tree (AST) Physical process: The acquisition end submits a data request to the backend controller.

[0041] Logical judgment: The rule chain executor reads the set of validation rules configured for this business scenario. The system parses and compiles the rule expression (in text form) into an abstract syntax tree node object in memory.

[0042] Execution steps: 1. Iterate through the data objects to be verified.

[0043] 2. Prune the tree path, activating and validating only child nodes that meet the current context conditions.

[0044] 3. Call the built-in script engine or a custom interpreter to perform logical operations on the AST nodes.

[0045] 4. If the verification result is false, throw a custom business exception, block the transaction commit, and return the specific error attribute path and prompt information.

[0046] Execution order and interruption logic of multiple verification rules: The rule chain executor internally maintains a priority-ordered queue of validation rules. Each validation rule includes a priority field in its metadata definition. When constructing the validation chain, the executor loads and executes rules sequentially according to their priority values, from smallest to largest; the smaller the value, the higher the priority. For handling validation failures, the system uses a full-collection mode. Even if the current rule fails to validate, the executor will continue executing all remaining rules in the queue until all rules have been executed. Finally, it summarizes all failed attribute paths and error messages into a complete error list and returns it, allowing the front-end application to display all form items requiring correction to the user at once.

[0047] In this embodiment, the dynamic model parsing engine runs independently and continuously monitors the change logs corresponding to the metadata configuration table. When it detects that metadata has been added, modified, or deleted, the dynamic model parsing engine rereads the changed metadata and regenerates the corresponding dynamic attribute description objects. It only updates the affected attribute nodes in the dynamic data descriptor, ensuring that the dynamic data descriptor always remains consistent with the metadata configuration. This allows for the dynamic adjustment of the labor force data structure without modifying the program code or database table structure, providing a unified data description foundation for the subsequent dynamic parsing and storage of labor force data.

[0048] S2. Receive labor force data, parse the labor force data according to the dynamic data descriptor, and complete the dynamic storage of the labor force data; In this embodiment, the steps of receiving labor force data, parsing the labor force data according to the dynamic data descriptor, and completing the dynamic storage of the labor force data include: Receive labor force data and call the corresponding dynamic data descriptor; The attribute definition information corresponding to the labor force data is obtained based on the dynamic data descriptor, each attribute in the labor force data is identified, and data parsing and format conversion are completed according to the data type corresponding to each attribute. Based on the parsed attribute information, determine the storage location corresponding to each attribute, map fixed attributes to corresponding fixed data columns, and organize dynamic attributes into extended attribute data; Perform JSON serialization on the extended attribute data to generate the corresponding extended data object; Data from fixed data columns is associated with extended data objects and then jointly written into the labor force data storage table to achieve dynamic storage of labor force data. In this embodiment, the extended data object is encapsulated using a JSON object, the fixed data columns are used to store predefined fixed attributes, and the extended data object is used to store dynamically added attributes, so that adding new attributes can be stored without modifying the table structure of the labor force data storage table.

[0049] After constructing the dynamic data descriptor, the system receives the labor force data submitted by the front end and calls the corresponding dynamic data descriptor to dynamically parse the labor force data. The dynamic model parsing engine first obtains the unique attribute identifier, data type, tree path, and storage rules corresponding to each business attribute based on the dynamic data descriptor, identifies each attribute in the labor force data item by item, and completes the corresponding data parsing and format conversion according to different data types, so that the input data is converted into a data object that the system can recognize.

[0050] After data parsing, the dynamic model parsing engine further determines the physical storage location of each attribute based on the dynamic data descriptor. For predefined basic attributes, the parsed attribute values ​​are mapped to corresponding fixed data columns. For extended attributes dynamically configured in metadata, extended attribute data is organized according to the attribute's unique identifier and uniformly encapsulated and serialized using JSON objects to generate corresponding extended data objects. Subsequently, the data in the fixed data columns and the extended data objects are associated and jointly written into the labor force data storage table, achieving dynamic storage of labor force data. Because dynamic attributes are stored using JSON extended columns, when adding or adjusting business attributes, only the metadata configuration needs to be modified to complete the attribute extension; there is no need to modify the entity class or database table structure, improving the flexibility of labor force data management and the system's scalability.

[0051] After the labor force data storage is completed, the rule chain executor reads the verification rules configured in the metadata according to the current business scenario, and constructs the corresponding rule chain based on each verification rule. The rule chain executor first parses the text-based rule expression into an Abstract Syntax Tree (AST). The AST includes attribute nodes, operation nodes, judgment nodes, and logical connection nodes. The nodes establish corresponding logical relationships according to the rule expression, thereby forming an executable rule model.

[0052] Furthermore, the dynamic model parsing engine does not require pre-defining fixed entity objects; instead, it directly completes JSON data parsing and attribute mapping based on dynamic data descriptors.

[0053] S3. Construct a rule chain based on the verification rules corresponding to the metadata, dynamically verify the parsed labor force data, and perform data updates based on the verification results; In this embodiment, the verification rules corresponding to the metadata are obtained, and a corresponding rule chain is constructed based on each verification rule; The dynamic data descriptor is invoked to determine the target attributes to be verified based on the rule chain, and the attribute values ​​corresponding to each target attribute are obtained. The validation rules in the rule chain are executed in the order of priority corresponding to each validation rule, and each target attribute is dynamically validated and the corresponding validation result is generated. Summarize the verification results corresponding to each verification rule to form a verification result set of labor force data; Based on the verification result set, determine whether the labor force data meets the update conditions. If the update conditions are met, perform the labor force data update; if the update conditions are not met, output the corresponding verification exception information.

[0054] In this embodiment, the step of constructing the corresponding rule chain according to each verification rule includes: Obtain each validation rule in the rule chain and parse each validation rule into a corresponding abstract syntax tree, wherein the abstract syntax tree includes attribute nodes, operation nodes, judgment nodes and logical connection nodes; Traverse each attribute node in the abstract syntax tree, and call the dynamic data descriptor according to the attribute identifier corresponding to each attribute node to determine the data type and attribute value of the corresponding attribute; Replace the node data of the corresponding attribute node in the abstract syntax tree with the obtained attribute value, while maintaining the logical relationship between the nodes; Based on the node hierarchy and logical connection relationship of the abstract syntax tree, the logical operations corresponding to each operation node and judgment node are executed from bottom to top to obtain the verification results corresponding to each verification rule. The system summarizes the verification results corresponding to each verification rule in the rule chain and generates the rule execution results for dynamic verification of labor force data.

[0055] The rule chain executor traverses each attribute node in the abstract syntax tree (AST), and obtains the data type and attribute value of the corresponding attribute node based on the dynamic data descriptor. It then replaces the data content of the corresponding node in the AST with the obtained attribute value, while maintaining the original logical relationships between nodes. During execution, the target attribute for validation in the current business scenario is determined based on the tree path in the dynamic data descriptor. Only attribute nodes that meet the context conditions are activated to participate in logical operations, thereby reducing invalid rule execution and improving rule validation efficiency.

[0056] After loading the attribute values, the rule chain executor constructs a rule execution queue according to the pre-configured priorities of each validation rule, and executes each validation rule in the rule chain sequentially according to priority. For each validation rule, the script engine or interpreter is invoked to perform logical calculations on the operation nodes and judgment nodes in the abstract syntax tree, generating the corresponding validation result. To ensure that users can obtain all validation information at once, this embodiment uses a full collection method to execute the rule chain. Even if a validation rule fails to execute, the system continues to execute subsequent validation rules until all rules have been executed, and then all validation results are aggregated.

[0057] After all validation rules are executed, the rule chain executor generates a corresponding validation result object (ValidationResult). This result object includes a validation status and an error message set. The validation status determines whether the current workforce data meets the update conditions, and the error message set records the field paths, error codes, and error descriptions for validation failures. When the validation result meets the update conditions, the system updates the workforce data. When validation failures occur, the system terminates the current data update operation and returns all validation exception information to the front end. The front end then prompts the user to modify the corresponding workforce data in one go, thereby improving data validation efficiency and business processing efficiency.

[0058] It should be noted that this invention prunes the abstract syntax tree based on a tree-like path, activating only attribute nodes that meet the current business context conditions for validation. Furthermore, even if the current validation rule fails, subsequent validation rules continue to be executed until all rules are completed, at which point all error information is aggregated. A dynamic data descriptor (MetaDescriptor) is used to describe dynamic attributes and their behavior. Validation results are encapsulated as ValidationResult objects. Attribute-level change records are encapsulated as ChangeItem objects.

[0059] S4. Obtain the labor force data before and after the update, compare the differences between the labor force data before and after the update according to the dynamic data descriptor, and generate attribute-level change records. In this embodiment, the steps of obtaining labor force data before and after the update, comparing the differences between the labor force data before and after the update based on the dynamic data descriptor, and generating attribute-level change records include: Obtain the labor force data before and after the update, and call the dynamic data descriptor corresponding to the labor force data; Based on the dynamic data descriptor, traverse the attributes in the labor force data before and after the update to determine the target attributes for difference comparison; Based on the attribute identifier corresponding to each target attribute, obtain the attribute values ​​before and after the update, and compare the corresponding attribute values ​​item by item; The target attribute that has changed is identified based on the attribute value comparison results, and the change type of the corresponding target attribute is determined. The labor force data before and after the update includes the labor force data corresponding to newly added records, modified records, and deleted records. Based on the identified target attributes and corresponding change types, attribute-level change records are generated, and the association between the attribute-level change records and the corresponding labor force data is established.

[0060] In this embodiment, the step of identifying the changed target attribute based on the attribute value comparison result and determining the change type of the corresponding target attribute includes: Based on the attribute value comparison results, obtain the attribute state of the target attribute before and after the update, and determine whether the attribute value has changed before and after the update; When the corresponding attribute value is empty before the update and the corresponding attribute value is not empty after the update, the target attribute is determined to be in a new state, and the change type corresponding to the target attribute is determined to be a new attribute. When both the corresponding attribute value before and after the update exist, a consistency comparison is performed on the attribute values ​​before and after the update. If the comparison result shows that the two are inconsistent, the target attribute is determined to be in a modified state, and the change type corresponding to the target attribute is determined to be a modified attribute. When the corresponding attribute value is not empty before the update and the corresponding attribute value is empty after the update, the target attribute is determined to be in a deleted state, and the change type corresponding to the target attribute is determined to be a deleted attribute. The identified new, modified, and deleted attributes are categorized and summarized, and the change type corresponding to each target attribute is passed to the attribute-level change record generation step to generate the corresponding attribute-level change record.

[0061] Once the labor force data has been verified and meets the update conditions, the system enters the data update phase. In this embodiment, the differential audit module is triggered by the data update transaction in the service layer and obtains the labor force data before and after the update before the database transaction is committed. Based on the dynamic data descriptor, it performs fine-grained difference comparison on each attribute and generates corresponding attribute-level change records, thereby realizing the traceability of changes in labor force data.

[0062] Specifically, the differential audit module first calls the dynamic data descriptor corresponding to the labor force data to obtain all attribute information corresponding to the current labor force data, and then traverses each business attribute based on the unique attribute identifier in the dynamic data descriptor. Subsequently, it obtains the attribute values ​​before and after the update through a reflection mechanism, compares each attribute value item by item, identifies the target attribute that has changed, and determines the corresponding change type based on the attribute status before and after the update.

[0063] Specifically, when performing a new operation, since there is no data object before the update, the differential audit module identifies all non-empty attributes in the current labor force data as new attributes and records the initial value of the corresponding attribute. When performing a modification operation, the labor force data before and after the update are obtained respectively, and the attribute values ​​corresponding to the same attribute are compared for consistency. When there is a difference between the attribute values ​​before and after the update, the corresponding attribute is identified as a modified attribute, and the attribute values ​​before and after the update are recorded. When performing a deletion operation, the corresponding historical data is read before deleting the labor force data, the original attributes are identified as deleted attributes, and the corresponding attribute values ​​before deletion are recorded to preserve a complete data change trajectory.

[0064] After completing the attribute value comparison, the differential audit module generates corresponding attribute-level change records based on the comparison results. Each attribute-level change record includes at least the attribute path, the attribute value before the update, the attribute value after the update, and the change type, which includes addition, modification, and deletion. Subsequently, each attribute-level change record is associated with the corresponding labor force data and encapsulated into a change event object according to a unified data format. The corresponding operation time and operator information are also recorded and written to the audit log, achieving fine-grained change recording of the entire labor force data process.

[0065] Furthermore, this embodiment can also identify sensitive attributes in the generated attribute-level change records according to a pre-configured monitoring strategy. When it is determined that the changed target attribute belongs to a preset sensitive monitoring set, the system generates a corresponding monitoring event and publishes alarm information through a message queue so that relevant business systems can obtain the change status of sensitive attributes in a timely manner, thereby realizing real-time monitoring of workforce data and full-process change traceability.

[0066] S5. Receive a statistics request, determine the statistical attributes based on the metadata, and generate the corresponding dynamic statistical results.

[0067] In this embodiment, the steps of receiving the statistical request, determining the statistical attributes based on the metadata, and generating the corresponding dynamic statistical results include: Receive a statistics request and parse the statistical dimensions and conditions in the statistics request; Based on the statistical dimensions, query the corresponding metadata to obtain the attribute identifier, data type and storage location information of the statistical attributes, and determine the data source of the statistical attributes based on the storage location information; Based on the determined data source and statistical conditions, a corresponding dynamic aggregation query statement is generated. Specifically, when the statistical attribute corresponds to a fixed data column, an aggregation query statement for the fixed data column is generated; when the statistical attribute corresponds to extended data, an attribute extraction query statement for the extended data is generated. The dynamic aggregation query statement is executed to perform aggregation calculations on the data corresponding to the statistical attributes and obtain the statistical results; The statistical results are organized and encapsulated to generate dynamic statistical results corresponding to the statistical request and output them.

[0068] Upon receiving a statistics request, the dynamic aggregation calculator first parses the statistical dimensions and conditions in the request. Then, it queries the corresponding metadata based on the statistical dimensions to obtain the attribute identifier, data type, and storage location information for the statistical attributes. Subsequently, it determines the corresponding data source based on the physical storage location of the statistical attributes. When the statistical attribute corresponds to a fixed data column, it directly generates an aggregation query statement for that fixed data column. When the statistical attribute corresponds to a dynamic attribute in a JSON extended column, it dynamically generates the corresponding JSON attribute extraction statement based on the metadata and constructs the corresponding dynamic aggregation query statement in conjunction with the statistical conditions.

[0069] After the query statement is constructed, the dynamic aggregation calculator calls the database to execute the corresponding aggregation calculations. It performs statistical analysis on labor force data that meets the statistical conditions, and performs aggregation operations such as record quantity statistics, numerical summarization, average calculation, maximum value, and minimum value calculation according to business needs, obtaining the corresponding statistical results. Subsequently, the statistical results are organized and encapsulated to generate dynamic statistical results corresponding to the statistical request and returned to the front end, realizing real-time statistical analysis of labor force data. Because the statistical process dynamically determines statistical attributes based on metadata, adding or adjusting business attributes does not require modification of the statistical program to participate in statistical calculations, improving the flexibility of statistical analysis and the system's scalability.

[0070] In this embodiment, the functional modules collaborate to complete the labor force data processing flow in a metadata-driven manner. First, the metadata configuration center completes the configuration of labor force data metadata, and the dynamic model parsing engine constructs dynamic data descriptors. Subsequently, based on the dynamic data descriptors, the dynamic parsing and dynamic storage of labor force data are completed, and the labor force data is dynamically verified through the rule chain executor. When labor force data is added, modified, or deleted, the differential audit module generates corresponding attribute-level change records to achieve fine-grained change tracing of labor force data. Finally, the dynamic aggregation calculator dynamically generates aggregation query statements based on statistical requests and completes statistical analysis, thereby forming a complete processing flow covering data modeling, data storage, rule verification, differential auditing, and dynamic statistics. This enables dynamic reconstruction of the labor force data structure, dynamic execution of business rules, full-process traceability of data changes, and real-time generation of statistical results.

[0071] Perform aggregation operations such as counting, summing, averaging, maximum or minimum values ​​according to different statistical needs.

[0072] In this embodiment, after the rule chain executor completes the execution of all validation rules, it generates a corresponding ValidationResult object. The ValidationResult object encapsulates the overall execution status of the rule chain, including the validation status and a set of error messages. Specifically, the isValid() method returns the validation status, and the getErrorList() method returns a complete set of error messages. The error message object includes fieldName (field name), errorCode (error code), and errorMessage (error description). When the ValidationResult object satisfies the condition that isValid() returns true and getErrorList() returns an empty set of error messages, it is determined that the current workforce data meets all validation rules, and subsequent data update operations are allowed; otherwise, validation is deemed a failure, the transaction is terminated, and the corresponding error message is returned.

[0073] After rule validation, the system enters the reflection-based object differential algorithm and fine-grained trajectory monitoring phase. When executing workforce data update transactions at the service layer, the differential audit module is triggered via an aspect-oriented programming interceptor, and the differential audit process is performed before the database transaction is committed. If an exception occurs during the differential audit process, the current transaction is synchronously rolled back to ensure consistency between workforce data and audit logs. The differential audit module is applicable to three data operation types: adding, modifying, and deleting records. The audit scope includes fixed column attributes and dynamic attributes in JSON extended columns.

[0074] When performing a new record operation, since the corresponding historical object does not exist in the database, the system retrieves the labor force data from the current request and instantiates a new object, Obj_new. Subsequently, the differential auditing module calls the getFieldNames() method provided by MetaDescriptor to obtain all attribute names defined in the dynamic data descriptor and uses reflection to iterate through each attribute name. When the corresponding attribute value newVal is obtained, if newVal is not null, a corresponding ChangeItem object is created, the change type changeType is set to "new", oldValue is assigned null, and newValue is assigned the current attribute value. The generated ChangeItem object is then added to the changeList difference collection to record the initial state of all valid attributes of the new record.

[0075] When performing a record modification operation, the system first reads historical labor force data from the database based on the primary key and instantiates an old object, Obj_old; simultaneously, it retrieves the labor force data in the current request and instantiates a new object, Obj_new. Subsequently, the differential auditing module calls the getFieldNames() method provided by MetaDescriptor to traverse all attribute names and uses reflection to obtain the attribute values ​​of the corresponding attributes in both the old and new objects, Obj_old and Obj_new. By comparing the attribute values ​​before and after the update item by item, the system identifies the target attributes that have changed and saves the identified differences to the changeList difference collection, thus achieving a deep attribute-level difference comparison of the labor force data.

[0076] When performing a record deletion operation, before physically deleting the record in the database, the system first reads the historical workforce data to be deleted based on the primary key and instantiates an old object, Obj_old. Then, the differential auditing module calls the getFieldNames() method provided by MetaDescriptor to iterate through all attribute names and uses reflection to obtain the corresponding attribute value, oldVal. When oldVal is not null, a corresponding ChangeItem object is created, with changeType set to deletion, oldValue assigned the attribute value before deletion, and newValue assigned null. The generated ChangeItem object is then added to the changeList difference collection to save a complete snapshot of all attributes before deletion.

[0077] Furthermore, the differential auditing module calls the `diff(Object oldObj, Object newObj, MetaDescriptor descriptor)` method to perform difference identification, where `oldObj` represents the object before the update, `newObj` represents the object after the update, and `descriptor` represents the dynamic data descriptor. The system iterates through the entire set of attribute names returned by `descriptor.getFieldNames()`, retrieving `oldVal` and `newVal` for each `fieldName`. When `oldVal` is empty and `newVal` is not empty, the change type `changeType` is determined to be "addition"; when `oldVal` is not empty and `newVal` is empty, the change type `changeType` is determined to be "deletion"; when both `oldVal` and `newVal` exist and are not equal, the change type `changeType` is determined to be "modification". When any of the above conditions are met, the corresponding `ChangeItem` object is created and added to the `changeList` difference collection. After completing the traversal of all attributes, the current number of differences can be obtained through `changeList.size()`, and all change records can be traversed through through `changeList.iterator()`.

[0078] After completing the difference identification, the system constructs a corresponding standardized change event object for each ChangeItem object. The change event object includes at least the attribute path fieldPath, old value oldValue, new value newValue, change type changeType, timestamp, and operator identifier. The generated change event object is serialized and asynchronously written to the audit log table to realize the traceability of changes in the entire process of labor data.

[0079] Furthermore, the system uses the `isSensitiveField(fieldName)` method provided by `monitorConfig` to determine whether the changed attribute belongs to a preset sensitive monitoring set. When the return result is true, the system publishes the corresponding domain event through a message queue, triggering a monitoring alarm to achieve real-time monitoring of sensitive attribute changes.

[0080] Upon entering the dynamic aggregation and statistics phase, the dynamic aggregation calculator receives statistical query requests and parses the dynamic grouping and measurement attributes within them. Then, it queries the MetaDescriptor dynamic data descriptor based on the attribute identifier to determine the physical storage location of the corresponding attribute. When the statistical attribute corresponds to a fixed data column, it directly generates an aggregation query statement with the corresponding column name; when the statistical attribute corresponds to a dynamic attribute in a JSON extended column, it dynamically generates the corresponding JSON extraction function from the database and uses a dynamic SQL builder to assemble the complete aggregation query statement. After executing the aggregation query, it performs aggregation calculations such as COUNT, COUNT(DISTINCT), SUM, AVG, MAX, MIN, STDDEV_POP, STDDEV_SAMP, VAR_POP, and VAR_SAMP according to business requirements, ultimately generating real-time dynamic statistical results and returning them to the calling client.

[0081] This invention is not limited to the embodiments described above. Anyone should understand that structural changes made under the guidance of this invention, and any technical solutions that are the same as or similar to this invention, fall within the protection scope of this invention. Technical aspects, shapes, and structures not described in detail in this invention are all publicly known technologies.

Claims

1. A method for dynamic reconstruction of labor force data structure and fine-grained change tracing, characterized in that, Includes the following steps: S1. Obtain the metadata corresponding to the labor force data, and construct a dynamic data descriptor based on the metadata; S2. Receive labor force data, parse the labor force data according to the dynamic data descriptor, and complete the dynamic storage of the labor force data; S3. Construct a rule chain based on the verification rules corresponding to the metadata, dynamically verify the parsed labor force data, and perform data updates based on the verification results; S4. Obtain the labor force data before and after the update, compare the differences between the labor force data before and after the update according to the dynamic data descriptor, and generate attribute-level change records. S5. Receive a statistics request, determine the statistical attributes based on the metadata, and generate the corresponding dynamic statistical results.

2. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 1, characterized in that, The steps for obtaining the metadata corresponding to the labor force data and constructing a dynamic data descriptor based on the metadata are as follows: Obtain the metadata configured by the administrator. The metadata includes at least the unique identifier of the attribute, data type, tree path, display rules and validation rules, and write the metadata into the metadata configuration table. The dynamic model parsing engine reads the metadata from the metadata configuration table and uses reflection to parse the metadata, generating the corresponding dynamic attribute description object; The generated multiple dynamic attribute description objects are hierarchically associated according to a tree path to form dynamic data descriptors, and a mapping relationship is established between the attribute unique identifier and the dynamic data descriptors.

3. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 2, characterized in that, After obtaining the dynamic data descriptor, the following steps also need to be performed: The dynamic data descriptor is loaded into the dynamic model parsing engine so that the dynamic model parsing engine can parse the labor force data based on the dynamic data descriptor; During the process of parsing labor data using the dynamic data descriptor, the change log corresponding to the metadata configuration table is monitored, and it is determined whether the metadata corresponding to the dynamic data descriptor has changed. When it is determined that the corresponding metadata has been added, modified or deleted, the changed metadata is reread, the corresponding dynamic attribute description object is regenerated, and the dynamic data descriptor is updated using the regenerated dynamic attribute description object, so that the updated dynamic data descriptor maintains the correspondence with the metadata configuration table.

4. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 3, characterized in that, The steps of receiving labor force data, parsing the labor force data according to the dynamic data descriptor, and completing the dynamic storage of the labor force data include: Receive labor force data and call the corresponding dynamic data descriptor; The attribute definition information corresponding to the labor force data is obtained based on the dynamic data descriptor, each attribute in the labor force data is identified, and data parsing and format conversion are completed according to the data type corresponding to each attribute. Based on the parsed attribute information, determine the storage location corresponding to each attribute, map fixed attributes to corresponding fixed data columns, and organize dynamic attributes into extended attribute data; Perform JSON serialization on the extended attribute data to generate the corresponding extended data object; Data from fixed data columns is associated with extended data objects and then written together into the labor force data storage table to complete the dynamic storage of labor force data.

5. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 4, characterized in that, The extended data object is encapsulated using a JSON object. The fixed data column is used to store predefined fixed attributes, and the extended data object is used to store dynamically added attributes, so that the addition of attributes can be completed without modifying the table structure of the labor data storage table.

6. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 5, characterized in that, The step of dynamically validating the parsed labor force data according to the validation rules corresponding to the metadata, and updating the data based on the validation results, includes: Obtain the verification rules corresponding to the metadata, and construct the corresponding rule chain based on each verification rule; The dynamic data descriptor is invoked to determine the target attributes to be verified based on the rule chain, and the attribute values ​​corresponding to each target attribute are obtained. The validation rules in the rule chain are executed in the order of priority corresponding to each validation rule, and each target attribute is dynamically validated and the corresponding validation result is generated. Summarize the verification results corresponding to each verification rule to form a verification result set of labor force data; Based on the verification result set, determine whether the labor force data meets the update conditions. If the update conditions are met, perform the labor force data update; if the update conditions are not met, output the corresponding verification exception information.

7. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 6, characterized in that, The step of constructing the corresponding rule chain based on each verification rule includes: Obtain each validation rule in the rule chain and parse each validation rule into a corresponding abstract syntax tree, wherein the abstract syntax tree includes attribute nodes, operation nodes, judgment nodes and logical connection nodes; Traverse each attribute node in the abstract syntax tree, and call the dynamic data descriptor according to the attribute identifier corresponding to each attribute node to determine the data type and attribute value of the corresponding attribute; Replace the node data of the corresponding attribute node in the abstract syntax tree with the obtained attribute value, while maintaining the logical relationship between the nodes; Based on the node hierarchy and logical connection relationship of the abstract syntax tree, the logical operations corresponding to each operation node and judgment node are executed from bottom to top to obtain the verification results corresponding to each verification rule. The system summarizes the verification results corresponding to each verification rule in the rule chain and generates the rule execution results for dynamic verification of labor force data.

8. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 7, characterized in that, The steps of obtaining labor force data before and after the update, comparing the differences between the labor force data before and after the update based on the dynamic data descriptor, and generating attribute-level change records include: Obtain the labor force data before and after the update, and call the dynamic data descriptor corresponding to the labor force data; Based on the dynamic data descriptor, traverse the attributes in the labor force data before and after the update to determine the target attributes for difference comparison; Based on the attribute identifier corresponding to each target attribute, obtain the attribute values ​​before and after the update, and compare the corresponding attribute values ​​item by item; The target attribute that has changed is identified based on the attribute value comparison results, and the change type of the corresponding target attribute is determined. The labor force data before and after the update includes the labor force data corresponding to newly added records, modified records, and deleted records. Based on the identified target attributes and corresponding change types, attribute-level change records are generated, and the association between the attribute-level change records and the corresponding labor force data is established.

9. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 8, characterized in that, The step of identifying the target attribute that has changed based on the attribute value comparison result and determining the change type of the corresponding target attribute includes: Based on the attribute value comparison results, obtain the attribute state of the target attribute before and after the update, and determine whether the attribute value has changed before and after the update; When the corresponding attribute value is empty before the update and the corresponding attribute value is not empty after the update, the target attribute is determined to be in a new state, and the change type corresponding to the target attribute is determined to be a new attribute. When both the corresponding attribute value before and after the update exist, a consistency comparison is performed on the attribute values ​​before and after the update. If the comparison result shows that the two are inconsistent, the target attribute is determined to be in a modified state, and the change type corresponding to the target attribute is determined to be a modified attribute. When the corresponding attribute value is not empty before the update and the corresponding attribute value is empty after the update, the target attribute is determined to be in a deleted state, and the change type corresponding to the target attribute is determined to be a deleted attribute. The identified new, modified, and deleted attributes are categorized and summarized, and the change type corresponding to each target attribute is passed to the attribute-level change record generation step to generate the corresponding attribute-level change record.

10. The method for dynamic reconstruction and fine-grained change tracing of labor force data structure according to claim 9, characterized in that, The steps of receiving a statistical request, determining statistical attributes based on the metadata, and generating corresponding dynamic statistical results include: Receive a statistics request and parse the statistical dimensions and conditions in the statistics request; Based on the statistical dimensions, query the corresponding metadata to obtain the attribute identifier, data type and storage location information of the statistical attributes, and determine the data source of the statistical attributes based on the storage location information; Based on the determined data source and statistical conditions, a corresponding dynamic aggregation query statement is generated. Specifically, when the statistical attribute corresponds to a fixed data column, an aggregation query statement for the fixed data column is generated; when the statistical attribute corresponds to extended data, an attribute extraction query statement for the extended data is generated. The dynamic aggregation query statement is executed to perform aggregation calculations on the data corresponding to the statistical attributes and obtain the statistical results; The statistical results are organized and encapsulated to generate dynamic statistical results corresponding to the statistical request and output them.

Images