Multi-round increment rule matching method and device for master-slave object data and medium

Abstract

The invention provides a multi-round incremental rule matching method and device for master-slave object data and a medium, and relates to the technical field of database data processing. The method comprises the steps that master and slave object data in a database are monitored, when one of the master and slave object data is changed, a structured change message is generated, and the structured change message comprises an identifier used for indicating change of the master and slave object data and a change range; determining a processing rule used for matching the change data from a rule base in a database based on the structured change message, and positioning a row where the change data is located; performing increment re-calculation on the master-slave object data by using a processing rule based on the change identifier of the master-slave object data, the change range and the line where the change data is located; through the data structure of the last matched data and Redis persistent storage, a core state memory function is provided for multi-round incremental rule matching, full-amount recalculation is avoided, and the calculation efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of database data processing, and in particular to a multi-round incremental rule matching method, device and medium for master-slave object data. Background Art

[0002] In enterprise-level CRM / ERP systems, the rules underlying functions such as pricing, adjustment factors, and risk control often involve multi-dimensional judgments (object attributes, time windows, historical cumulative indicators, etc.) and require millisecond-level responses in high-concurrency scenarios. The main drawbacks of existing technologies are: Full recalculation solution: State memory is lost. Traditional solutions perform independent one-time matching for each request and cannot save and reuse the state information of previous matches, resulting in repeated calculations in multiple rounds of interactions; Incomplete incremental matching: Existing incremental calculations are only based on change detection for the current request, lacking continuity across requests and unable to achieve true multi-round incremental matching; Inadequate adaptation to master-slave object scenarios. Traditional solutions mainly target single objects or simple object relationships, and lack targeted optimization for handling complex relationships between master and slave objects; Aggregation statistics require repeated traversal of historical data tables, which increases DB pressure sharply and has poor scalability.

[0003] Simple incremental calculation solution: When processing multi-dimensional rules and active object relationships, it lacks an accurate change detection mechanism and cannot identify which specific fields have changed. It also lacks cross-request state memory and cannot support true multi-round incremental matching. Window aggregation still requires full-window statistics, making incremental aggregation calculation impossible. There is a lack of effective means for rule conflict and priority processing, which easily leads to data inconsistencies.

[0004] Windowed pre-computation solution: Quasi-real-time performance is insufficient, and the batch processing granularity (minutes to hours) cannot meet the needs of second-level services. Data expansion, the combination of multiple windows and multiple dimensions leads to storage expansion. Parameter adjustment is difficult, and window or function adjustments require processing historical data. Summary of the Invention

[0005] The present invention aims to solve one or more technical deficiencies in the above-mentioned prior art and proposes the following technical solutions.

[0006] A multi-round incremental rule matching method for master-slave object data, the method comprising: a change detection step of monitoring the master and slave object data in the database, and generating a structured change message when a change occurs to one of the master and slave object data, wherein the structured change message includes an identifier indicating the change to the master and slave object data and a change scope; a multidimensional rule engine screening step, determining, based on the structured change message, a processing rule for matching the changed data from a rule base in the database, and locating a row where the changed data is located; Incremental recalculation step: using the processing rule to incrementally recalculate the master and slave object data based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located; The state memory step is used to save a snapshot of the key field status of each round of matching to achieve cross-request state memory capabilities. Through the data structure of the last matching data and Redis persistent storage, it provides the core state memory function for multiple rounds of incremental rule matching.

[0007] Furthermore, the operation of the change detection step is: parsing the action code of the master-slave object data, and judging the change type as addition, editing or deletion based on the parsing result; when the change type is editing, triggering the combination key cascade detection and filtering processing, retaining only the changed data rows and identifying the aggregation rule type, and outputting the modified slave object index list; performing multiple types of security comparisons on the rows covered by the modified slave object index list based on field comparison to obtain a list of fields affected by the change range; performing change analysis on the affected field list, determining the impact range and performing field weight sorting and field-rule dependency analysis to generate a data index-rule identifier; sending the data index-rule identifier to the multidimensional rule engine for incremental calculation.

[0008] Furthermore, in the multidimensional rule engine screening, the built-in rule expression DSL parser of the multidimensional rule engine is used to convert the JSON-formatted business rules into an abstract syntax tree, which is then handed over to the extended expression engine for immediate compilation into bytecode. The initial compilation result is cached in the local cache, and subsequent hits are directly executed. After receiving the structured change message, batch filtering is first performed according to the macro group dimension consisting of the rule combination ID + the main object. The batch filtering includes a two-stage pipeline: Stage 1 - Condition Matching: Boolean expressions are executed in parallel to match the data attributes of the main and subordinate objects to obtain a set of rule candidates. Stage 2 - Row-level Judgment: The execution conditions of the candidate rules are evaluated row by row, and each data row and rule is traversed through a double loop to execute expression judgment and collect matching results. Only the truly affected data indexes are marked, and it is determined whether aggregation calculation is required. If so, the batch aggregation value acquisition interface is uniformly called to inject the returned array of aggregation rule IDs and value Map combinations into the execution context for direct value retrieval by subsequent formulas. A single call can complete the calculation of N aggregation indicators.

[0009] Furthermore, in the incremental recalculation step: the modified slave object index list obtained based on change detection is input, and the fields in the modified slave object index list are analyzed. If the master object field is changed, all data indexes are directly included in the recalculation to avoid full table scanning; based on the data index-rule identifier mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors; if the user manually specifies a rule combination, its priority is set to the minimum integer value during the recalculation phase to ensure consistent results.

[0010] Furthermore, during recalculation, a rule conflict resolution algorithm is used to provide a unique and predictable result when facing multiple executable rules on the same data row. The rule conflict resolution algorithm is as follows: the conflict set is stably sorted in descending order of priority; if there is any rule tuple with a user priority identifier equal to 1, the entire set is directly placed in the front, and the other elements remain in the original order; if the head element still has the same priority and user priority identifier value, the one with the smallest creation time wins; a lock-free concurrency and consistency strategy is used to implement the following operations: the decision logic runs on a copy of the conflict set cached locally in the thread, and does not involve shared write operations; the results are written to the row-level decision tracking view dictionary through write-time copy to avoid lock contention, where the user priority identifier is 1 only when explicitly selected by the user, otherwise it is 0.

[0011] Furthermore, sparse bitmap change positioning is used for change detection: the snapshot field is mapped to a sparse bitmap, which is compared with the current request bitmap to obtain the set of changed rows. The operation is optimized based on the master-slave object data scenario: a multi-level dependency graph such as the master-slave object data combination is constructed to realize that a single point change automatically affects other data points. The impact radius is trimmed when the breadth-first traversal reaches the weight threshold. After deleting a row or disabling a policy, the row or policy is blacklisted for a certain period of time, and subsequent requests are directly skipped. The data index-rule identification is optimized using a bitmap dependency trimming algorithm: the field bitmap is precompiled, and a 256-bit bitmap is generated when the rule is published to identify its field dependency set. Only one operation is required during its runtime to determine "possible hits". Double-layer trimming is used to enable bitmap trimming to eliminate ≥50% of useless rules.

[0012] The present invention also proposes a multi-round incremental rule matching device for master-slave object data, the device comprising: a change detection unit, configured to monitor the master and slave object data in the database and generate a structured change message when a change occurs to one of the master and slave object data, wherein the structured change message includes an identifier indicating the change to the master and slave object data and a change range; a multidimensional rule engine screening unit, which determines, based on the structured change message, a processing rule for matching the changed data from a rule base in the database, and locates a row where the changed data is located; Incremental recalculation unit: using the processing rule to perform incremental recalculation on the master and slave object data based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located; The state memory unit is used to save the key field status snapshot of each round of matching to achieve cross-request state memory capabilities. Through the data structure of the last matching data and Redis persistent storage, it provides the core state memory function for multiple rounds of incremental rule matching.

[0013] Furthermore, the operation of the change detection unit is: parsing the action code of the master-slave object data, and judging the change type as addition, editing or deletion based on the parsing result; when the change type is editing, triggering the combination key cascade detection and filtering processing, retaining only the changed data rows and identifying the aggregation rule type, and outputting the modified slave object index list; performing multiple types of security comparisons on the rows covered by the modified slave object index list based on field comparison to obtain a list of fields affected by the change range; performing change analysis on the affected field list, determining the impact range and performing field weight sorting and field-rule dependency analysis to generate a data index-rule identifier; sending the data index-rule identifier to the multidimensional rule engine for incremental calculation.

[0014] Furthermore, in the multidimensional rule engine screening, the built-in rule expression DSL parser of the multidimensional rule engine is used to convert the JSON-formatted business rules into an abstract syntax tree, which is then handed over to the extended expression engine for immediate compilation into bytecode. The initial compilation result is cached in the local cache, and subsequent hits are directly executed. After receiving the structured change message, batch filtering is first performed according to the macro group dimension consisting of the rule combination ID + the main object. The batch filtering includes a two-stage pipeline: Stage 1 - Condition Matching: Boolean expressions are executed in parallel to match the data attributes of the main and subordinate objects to obtain a set of rule candidates. Stage 2 - Row-level Judgment: The execution conditions of the candidate rules are evaluated row by row, and each data row and rule is traversed through a double loop to execute expression judgment and collect matching results. Only the truly affected data indexes are marked, and it is determined whether aggregation calculation is required. If so, the batch aggregation value acquisition interface is uniformly called to inject the returned array of aggregation rule IDs and value Map combinations into the execution context for direct value retrieval by subsequent formulas. A single call can complete the calculation of N aggregation indicators.

[0015] Furthermore, in the incremental recalculation unit: the modified slave object index list obtained based on change detection is input, and the fields in the modified slave object index list are analyzed. If the master object field is changed, all data indexes are directly included in the recalculation to avoid full table scanning; based on the data index-rule identifier mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors; if the user manually specifies a rule combination, its priority is set to the minimum integer value during the recalculation phase to ensure consistent results.

[0016] Furthermore, during recalculation, a rule conflict resolution algorithm is used to provide a unique and predictable result when facing multiple executable rules on the same data row. The rule conflict resolution algorithm is as follows: the conflict set is stably sorted in descending order of priority; if there is any rule tuple with a user priority identifier equal to 1, the entire set is directly placed in the front, and the other elements remain in the original order; if the head element still has the same priority and user priority identifier value, the one with the smallest creation time wins; a lock-free concurrency and consistency strategy is used to implement the following operations: the decision logic runs on a copy of the conflict set cached locally in the thread, and does not involve shared write operations; the results are written to the row-level decision tracking dictionary through write-time copy to avoid lock contention, where the user priority identifier is 1 only when explicitly selected by the user, otherwise it is 0.

[0017] Furthermore, sparse bitmap change positioning is used for change detection: the snapshot field is mapped to a sparse bitmap, and the set of changed rows is obtained by comparing it with the current request bitmap. The operation is optimized based on the master-slave object data scenario: a multi-level dependency graph such as the master-slave object data combination is constructed to realize that a single point change automatically affects other data points. The impact radius is trimmed when the breadth-first traversal reaches the weight threshold. After deleting a row or disabling a policy, the row or policy is blacklisted for a certain period of time, and subsequent requests are directly skipped. For data index-rule identification, a bitmap dependency trimming algorithm is used for optimization: pre-compiled field bitmap: a 256-bit bitmap is generated when the rule is published to identify its field dependency set. Only one operation is required during its runtime to determine "possible hits"; double-layer trimming is used to enable bitmap trimming to eliminate ≥50% of useless rules.

[0018] The present invention further provides a computer-readable storage medium, wherein the storage medium stores computer program code, and when the computer program code is executed by a computer, any one of the above methods is executed.

[0019] The technical effect of the present invention is: a multi-round incremental rule matching method, device and storage medium for master-slave object data of the present invention, the method comprising: a change detection step S101, monitoring the master-slave object data in the database, and generating a structured change message when one of the master-slave object data changes, wherein the structured change message includes an identifier and a change range for indicating the change of the master-slave object data; a multi-dimensional rule engine screening step S102, determining a processing rule for matching the changed data from a rule base in the database based on the structured change message, and locating the row where the changed data is located; an incremental recalculation step S103: using the processing rule to incrementally recalculate the master-slave object data based on the identifier, change range and row where the changed data is located of the master-slave object data; a state memory step S104, used to save a snapshot of the key field status of each round of matching to achieve cross-request state memory capability, and provide a core state memory function for multi-round incremental rule matching through the data structure of the last matching data and Redis persistent storage. The beneficial effects of the present invention are: multi-round state memory processing: based on the last matching data state memory mechanism, multi-round interaction across requests is realized, and each round can identify changed data and perform incremental recalculation; master-slave object optimization: specially designed for master-slave object relationships, through a dynamic priority adjustment algorithm, it supports conflict handling and user preference priority during concurrent matching of multiple rules; incremental recalculation framework: incremental calculation based on state memory avoids full recalculation and significantly improves computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0021] Figure 1 This is a flowchart of a multi-round incremental rule matching method for master-slave object data based on state memory according to an embodiment of the present invention.

[0022] Figure 2 4 is a structural diagram of a multi-round incremental rule matching device for master-slave object data based on state memory according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0024] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] Figure 1 A multi-round incremental rule matching method for master-slave object data based on state memory of the present invention is shown, and the method includes: Change detection step S101, monitoring the master and slave object data in the database, and generating a structured change message when one of the master and slave object data changes, wherein the structured change message includes an identifier indicating the change of the master and slave object data and a change range; A multi-dimensional rule engine screening step S102 is to determine, based on the structured change message, a processing rule for matching the changed data from a rule base in the database, and locate the row where the changed data is located; Incremental recalculation step S103: using the processing rule to perform incremental recalculation on the master and slave object data based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located; The state memory step S104 is used to save the key field state snapshot of each matching round to achieve cross-request state memory capability. Through the LastMatchData (last matching data) data structure and Redis persistent storage, it provides the core state memory function for multiple rounds of incremental rule matching; The state memory step S104 is operated as follows: Status snapshot saving: After each round of rule matching is completed, the status snapshot of key fields is automatically saved; the processing flow is as follows: a) Extract key fields of the primary object (such as account_id (customer ID), total_amount (total), etc.); b) Extract all key fields from the object (such as quantity, product_id, unit); c) Construct a LastMatchData object containing the complete data status; d) Use Redis cache storage, the key format is "p_lastMatchData:{tenantId(tenant ID)}:{objectId(object ID)}", and the expiration time is 3 hours; State snapshot retrieval: Before a new round of matching begins, the state snapshot of the last match is retrieved from the distributed cache; State comparison analysis: Change detection algorithm, the core implementation is as follows: a) Snapshot data acquisition: call getOldMatchData (last match data) to retrieve the LastMatchData (last match data) snapshot from Redis; b) New row processing: When oldDetailDoc (the last matched slave object data - single record) = null, directly add it to modifiedDataIndexList (modified slave object index list); c) Changed row detection: perform exact comparison on key fields; d) Change mark: When the fields are different, add dataIndex (from object index) to the modifiedDataIndexList (modified from object index list) collection.

[0026] The present invention targets dynamic multi-dimensional rule matching scenarios of master and slave objects, uses a state memory mechanism to perform multiple rounds of incremental calculations, and accurately detects and classifies data changes. The present invention optimizes and processes multi-dimensional complex scenarios through state memory, multi-dimensional rule engine multi-round matching, master-slave object processing, etc., wherein the present invention first monitors the master and slave object data in the database. When there is a change in one of the master and slave object data, a structured change message is generated, wherein the structured change message includes an identifier for indicating the change in the master and slave object data (for example, using identifier 1 to indicate a change in the master object data and 0 to indicate a change in the slave data object) and a change range. Based on the structured change message, a processing rule for matching the changed data is determined from the rule library in the database, and the row where the changed data is located is located. Multiple rounds of state memory processing are performed, and the master and slave object data are incrementally recalculated based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located using the processing rule. Based on the LastMatchData (last matching data) state memory mechanism, multiple rounds of interaction across requests are implemented, and each round can identify the changed data and perform incremental recalculation, thereby achieving accurate detection and classification processing of data changes in the database. This is the important inventive concept of the present invention.

[0027] In one embodiment, the change detection step S101 operates as follows: parsing the action code of the master-slave object data, and determining whether the change type is Add, Edit, or Delete based on the parsing result; when the change type is Edit, triggering a combination key cascade detection filter (Combo-Filter) process to retain only the changed data rows and identify the aggregation rule type, and outputting a modified data object list modifiedDataIndexList; performing multiple type safety comparisons on the rows covered by the modifiedDataIndexList based on field comparison to obtain a change range changedFields; performing change analysis on the changedFields to determine the impact scope and perform field weight sorting and field-rule dependency analysis to generate a data index-rule identifier {dataIndex→ruleIds}; and sending the {dataIndex→ruleIds} to a multidimensional rule engine for incremental calculation.

[0028] In the present invention, when the change type is Edit, the combination key cascade detection filter (Combo-Filter combination filter) processing is triggered to retain only the changed data rows and identify the aggregation rule type. Based on field comparison, multiple types of safety comparisons are performed on the rows covered by modifiedDataIndexList (modified slave object index list) to obtain the change range changedFields (affected field list); the changed Fields (affected field list) is subjected to change analysis to determine the impact range and perform field weight sorting and field-rule dependency analysis, which is then used for incremental calculation, thereby achieving accurate processing of the changed data rows, avoiding the operation of using processing rules to process all data, and improving system performance. This is one of the important inventive concepts of the present invention.

[0029] In one embodiment, in the multidimensional rule engine screening step S102: the JSON-formatted business rules are converted into an abstract syntax tree (AST) using the built-in rule expression DSL parser of the multidimensional rule engine, and then handed over to the extended expression engine for immediate compilation into bytecode; the first compilation result is cached in the local cache, and subsequent hits are directly executed; after receiving the structured change message, batch filtering is first performed according to the macro group dimension consisting of "rule combination ID + main object", and the batch filtering includes a two-stage pipeline: Stage 1 - Condition matching: Boolean expressions are executed in parallel to match the data attributes of the main and subordinate objects to obtain rule candidates Phase 2 – Row-level determination: Evaluate the execution conditions of candidate rules row by row, traverse each data row and rule through a double loop, execute expression judgment and collect matching results, mark only the truly affected dataIndex (from the object index), and determine whether aggregation calculation is required. If so, call the computeAggregateValues ​​(batch obtain aggregate values) interface uniformly and inject the returned aggregation rule ID → value map (i.e., an array of aggregation rule ID and value map) into the execution context for direct value retrieval by subsequent formulas. A single call can complete the calculation of N aggregation indicators, avoiding multiple rounds of IO.

[0030] In the present invention, batch filtering is first performed according to the macro group dimension composed of "rule combination ID + main object", and data processing is performed in a two-level pipeline manner. Each data row and rule is traversed through a double loop, expression judgment is executed and matching results are collected, and only the truly affected dataIndex (from object index) is marked. If aggregation calculation is required, the computeAggregateValues ​​(batch obtain aggregate values) interface is uniformly called, and the returned aggregation rule ID→value Map is injected into the execution context for direct value retrieval by subsequent formulas, thereby improving the efficiency, accuracy and timeliness of change data processing. This is one of the important inventive concepts of the present invention.

[0031] In one embodiment, in the incremental recalculation step S103: the modifiedDataIndexList (modified slave object index list) obtained based on change detection is input, and the fields in the modifiedDataIndexList (modified slave object index list) are analyzed. If the master object field is changed, all dataIndex (slave object index) are directly included in the recalculation to avoid full table scanning; based on the {dataIndex→ruleIds} mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors; if the user manually specifies a rule combination, its priority is set to Integer.MIN_VALUE (integer minimum value) during the recalculation phase to ensure consistent results.

[0032] In the present invention, by analyzing the fields in modifiedDataIndexList (modified slave object index list), if the main object fields are changed, all dataIndex (slave object index) are directly included in the recalculation to avoid full table scan; based on the {dataIndex→ruleIds} mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors and omissions. Production data practice has shown that the amount of rule calculation can be reduced by ≥50%. During row-level recalculation, the rule engine is called row by row, using local compiled-bytecode and aggregate value cache, and the single-row time is less than ms. The result is merged, and the unchanged rows reuse the previous round of ExecutionData (to be executed), and the changed rows only replace the difference fields and write them back. The CPU drops significantly during peak hours. The key optimization is the blacklist cache: deleted rows are added to the blacklist, and subsequent requests are directly skipped. This greatly improves the incremental data processing performance in the database, which is another important inventive concept of the present invention.

[0033] In one embodiment, during recalculation, a rule conflict decision algorithm is used to provide a unique and predictable result when facing multiple executable rules on the same data row. The rule conflict decision algorithm is as follows: the ConflictSet conflict set is stably sorted in descending order of priority; if there is any RuleTuple with userBoost (user priority identifier) ​​equal to 1, the entire set is directly placed in the front, and the other elements remain in the original order; if the head element still has the same priority (priority) & userBoost (user priority identifier) ​​value, the one with the smallest createTime (creation time) wins; a lock-free concurrency and consistency strategy is used to implement the following operations: the decision logic runs on the ConflictSet copy cached locally in the thread, and does not involve shared write operations; the results are written to the row-level DecisionTrace Map (decision tracking dictionary) through Copy-On-Write to avoid lock contention, where userBoost (user priority identifier) ​​is 1 only when explicitly selected by the user, otherwise it is 0. This invention introduces ruleHash (rule ID + version number + policy fingerprint) as idempotent key in RuleTuple (rule tuple), which can safely cover repeated calculations and maintain consistency across nodes: the decision output is serialized as<dataIndex,ruleHash> , written to Redis; downstream consumers use the last-write-wins strategy, combined with createTime to ensure weak consistency. This is the key inventive concept of this invention.

[0034] In one embodiment, sparse bitmap change positioning is used for change detection: the snapshot field is mapped into a sparse bitmap (bit-vector), and the changed row set Δ-Index is obtained by comparing it with the current request bitmap O(Δ). The operation is optimized based on the master-slave object data scenario: a multi-level dependency graph such as the master-slave object data combination is constructed to realize that a single point change automatically affects other data points. The impact radius is trimmed when the breadth-first traversal reaches the weight threshold. After deleting a row or disabling a policy, the row or policy is blacklisted for a certain period of time, for example, several hours, during which subsequent requests are directly skipped. A bitmap dependency pruning algorithm is used for optimization of {dataIndex→ruleIds}: a pre-compiled field bitmap is generated when the rule is published to identify its field dependency set. Only one operation is required during its runtime to determine "possible hit". Double-layer pruning is used to enable bitmap pruning to eliminate ≥50% of useless rules.

[0035] The present invention provides a foundation for incremental computation based on state memory, a master-slave object optimization algorithm helps reduce computational complexity, a lock-free conflict resolution mechanism ensures the uniqueness and interpretability of results, and asynchronous aggregation improves the real-time performance of indicator statistics, thus supporting application requirements in high-concurrency, multi-dimensional rule-based scenarios. These are all key inventive concepts of the present invention.

[0036] Figure 2 The present invention shows a multi-round incremental rule matching device for master-slave object data based on state memory, which includes: The change detection unit 201 monitors the master and slave object data in the database and generates a structured change message when one of the master and slave object data changes, wherein the structured change message includes an identifier indicating the change of the master and slave object data and a change range; The multidimensional rule engine screening unit 202 determines, based on the structured change message, a processing rule for matching the changed data from a rule base in the database, and locates a row where the changed data is located; Incremental recalculation unit 203: performs incremental recalculation on the master and slave object data based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located, using the processing rule; The state memory unit 204 is used to save the key field state snapshot of each round of matching to achieve cross-request state memory capability. Through the LastMatchData (last matching data) data structure and Redis persistent storage, it provides the core state memory function for multiple rounds of incremental rule matching.

[0037] The present invention targets dynamic multi-dimensional rule matching scenarios of master and slave objects, uses a state memory mechanism to perform multiple rounds of incremental calculations, and accurately detects and classifies data changes. The present invention optimizes and processes multi-dimensional complex scenarios through state memory, multi-dimensional rule engine multi-round matching, master-slave object processing, etc., wherein the present invention first monitors the master and slave object data in the database. When there is a change in one of the master and slave object data, a structured change message is generated, wherein the structured change message includes an identifier for indicating the change in the master and slave object data (for example, using identifier 1 to indicate a change in the master object data and 0 to indicate a change in the slave data object) and a change range. Based on the structured change message, a processing rule for matching the changed data is determined from the rule library in the database, and the row where the changed data is located is located. Multiple rounds of state memory processing are performed, and the master and slave object data are incrementally recalculated based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located using the processing rule. Based on the LastMatchData (last matching data) state memory mechanism, multiple rounds of interaction across requests are implemented, and each round can identify the changed data and perform incremental recalculation, thereby achieving accurate detection and classification processing of data changes in the database. This is the important inventive concept of the present invention.

[0038] In one embodiment, the change detection unit 201 operates as follows: parsing the actionCode of the master-slave object data, and determining whether the change type is Add, Edit, or Delete based on the parsed result; when the change type is Edit, triggering a combination key cascade detection filter (Combo-Filter) process to retain only the changed data rows and identify the aggregation rule type, and outputting a modified data object list modifiedDataIndexList; performing multiple type safety comparisons on the rows covered by the modifiedDataIndexList based on field comparison to obtain a change range changedFields; performing change analysis on the changedFields to determine the impact scope and perform field weight sorting and field-rule dependency analysis to generate a data index-rule identifier {dataIndex→ruleIds}; and sending the {dataIndex→ruleIds} to the multidimensional rule engine for incremental calculation.

[0039] In the present invention, when the change type is Edit, the combination key cascade detection filter (Combo-Filter combination filter) processing is triggered to retain only the changed data rows and identify the aggregation rule type. Based on field comparison, multiple types of safety comparisons are performed on the rows covered by modifiedDataIndexList (modified slave object index list) to obtain the change range changedFields (affected field list); the changed Fields (affected field list) is subjected to change analysis to determine the impact range and perform field weight sorting and field-rule dependency analysis, which is then used for incremental calculation, thereby achieving accurate processing of the changed data rows, avoiding the operation of using processing rules to process all data, and improving system performance. This is one of the important inventive concepts of the present invention.

[0040] In one embodiment, in the multidimensional rule engine screening step S102, the multidimensional rule engine's built-in rule expression DSL parser is used to convert JSON-formatted business rules into an abstract syntax tree (AST), which is then compiled into bytecode by the extended expression engine in real-time. The initial compilation result is cached in the local cache, and subsequent hits are directly executed. After receiving the structured change message, batch filtering is first performed according to the macro group dimension consisting of "rule combination ID + master object". The batch filtering includes a two-stage pipeline: Stage 1 - Condition Matching: Boolean expressions are executed in parallel to match the data attributes of the master and slave objects to obtain a set of rule candidates. Stage 2 - Row-Level Decision: The execution conditions of the candidate rules are evaluated row by row. A double loop is used to traverse each data row and rule, execute expression judgment, and collect matching results. Only the truly affected dataIndex (slave object index) is marked, and it is determined whether aggregation calculation is required. If so, the computeAggregateValues ​​(batch obtain aggregate values) interface is uniformly called to inject the returned aggregation rule ID → value map into the execution context for direct value retrieval by subsequent formulas. A single call can complete the calculation of N aggregate indicators, avoiding multiple rounds of IO.

[0041] In the present invention, batch filtering is first performed according to the macro group dimension composed of "rule combination ID + main object", and data processing is performed in a two-level pipeline manner. Each data row and rule is traversed through a double loop, expression judgment is executed and matching results are collected, and only the truly affected dataIndex (from object index) is marked. If aggregation calculation is required, the computeAggregateValues ​​(batch obtain aggregate values) interface is uniformly called, and the returned aggregation rule ID→value Map is injected into the execution context for direct value retrieval by subsequent formulas, thereby improving the efficiency, accuracy and timeliness of change data processing. This is one of the important inventive concepts of the present invention.

[0042] In one embodiment, in the incremental recalculation unit 203: the modifiedDataIndexList (modified slave object index list) obtained based on change detection is input, and the fields in the modifiedDataIndexList (modified slave object index list) are analyzed. If the master object field is changed, all dataIndex (slave object index) are directly included in the recalculation to avoid full table scanning; based on the {dataIndex→ruleIds} mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors; if the user manually specifies a rule combination, its priority is set to Integer.MIN_VALUE (integer minimum value) during the recalculation phase to ensure consistent results.

[0043] In the present invention, by analyzing the fields in modifiedDataIndexList (modified slave object index list), if the main object fields are changed, all dataIndex (slave object index) are directly included in the recalculation to avoid full table scan; based on the {dataIndex→ruleIds} mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors and omissions. Production data practice has shown that the amount of rule calculation can be reduced by ≥50%. During row-level recalculation, the rule engine is called row by row, using local compiled-bytecode and aggregate value cache, and the single-row time is less than ms. The result is merged, and the unchanged rows reuse the previous round of ExecutionData (to be executed), and the changed rows only replace the difference fields and write them back. The CPU drops significantly during peak hours. The key optimization is the blacklist cache: deleted rows are added to the blacklist, and subsequent requests are directly skipped. This greatly improves the incremental data processing performance in the database, which is another important inventive concept of the present invention.

[0044] In one embodiment, during recalculation, a rule conflict decision algorithm is used to provide a unique and predictable result when facing multiple executable rules on the same data row. The rule conflict decision algorithm is as follows: the ConflictSet is stably sorted in descending order of priority; if there is any RuleTuple with userBoost equal to 1, the entire set is directly placed in the front, and the other elements remain in the original order; if the head element still has the same priority & userBoost value, the one with the smallest createTime wins; a lock-free concurrency and consistency strategy is used to implement the following operations: the decision logic runs on a copy of the ConflictSet cached locally in the thread, and does not involve shared write operations; the results are written to the row-level DecisionTrace Map through Copy-On-Write to avoid lock contention, where userBoost is 1 only when explicitly selected by the user, otherwise it is 0. This invention introduces ruleHash (rule ID + version number + policy fingerprint) as idempotent key in RuleTuple (rule tuple), which can safely cover repeated calculations and maintain consistency across nodes: the decision output is serialized as<dataIndex,ruleHash> , written to Redis; downstream consumers use the last-write-wins strategy, combined with createTime to ensure weak consistency. This is the key inventive concept of this invention.

[0045] In one embodiment, sparse bitmap change positioning is used for change detection: the snapshot field is mapped to a sparse bitmap (bit-vector), and compared with the current request bitmap O(Δ) to obtain the changed row set Δ-Index, and the operation is optimized based on the master-slave object data scenario: a multi-level dependency graph such as the master-slave object data combination is constructed to realize that a single point change automatically affects other data points, and the impact radius is trimmed when the breadth-first traversal reaches the weight threshold. After deleting a row or disabling a policy, the row or policy is blacklisted for a certain period of time, and subsequent requests during this period are directly skipped; for {dataIndex→ruleIds}, a bitmap dependency trimming algorithm is used for optimization: pre-compiled field bitmap: a 256-bit bitmap is generated when the rule is published to identify its field dependency set, and only one operation is required during its runtime to determine "possible hit"; double-layer trimming is used so that bitmap trimming can eliminate ≥50% of useless rules.

[0046] The present invention provides a foundation for incremental computation based on state memory, a master-slave object optimization algorithm helps reduce computational complexity, a lock-free conflict resolution mechanism ensures the uniqueness and interpretability of results, and asynchronous aggregation improves the real-time performance of indicator statistics, thus supporting application requirements in high-concurrency, multi-dimensional rule-based scenarios. These are all key inventive concepts of the present invention.

[0047] The terms used in this invention are as follows:

[0048] In one embodiment of the present invention, a computer storage medium is provided, on which a computer program is stored. When the computer program on the computer storage medium is executed by a processor, the above-mentioned method is implemented. The computer storage medium can be a hard disk, DVD, CD, flash memory or other memory.

[0049] For the convenience of description, the present invention divides the above device into various units and describes them separately according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0050] From the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the devices described in various embodiments of the present application or certain parts of the embodiments.

[0051] Finally, it should be noted that the above embodiments are only intended to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A multi-round incremental rule matching method for master-slave object data, characterized in that: The method includes: a change detection step of monitoring the master and slave object data in the database, and generating a structured change message when a change occurs to one of the master and slave object data, wherein the structured change message includes an identifier indicating the change to the master and slave object data and a change scope; a multidimensional rule engine screening step, determining, based on the structured change message, a processing rule for matching the changed data from a rule base in the database, and locating a row where the changed data is located; Incremental recalculation step: using the processing rule to incrementally recalculate the master and slave object data based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located; The state memory step is used to save a snapshot of the key field status of each round of matching to achieve cross-request state memory capabilities. Through the data structure of the last matching data and Redis persistent storage, it provides the core state memory function for multiple rounds of incremental rule matching.

2. The method according to claim 1, characterized in that The change detection step comprises: parsing the action code of the master-slave object data, and determining whether the change type is addition, editing, or deletion based on the parsed result; if the change type is editing, triggering a combination key cascade detection and filtering process to retain only the changed data rows, identify the aggregation rule type, and output a modified slave object index list; Perform multiple type-safe comparisons on the modified rows covered by the object index list based on field comparison to obtain a list of fields affected by the change range; Perform change analysis on the affected field list, determine the impact scope, and perform field weight sorting and field-rule dependency analysis to generate a data index-rule identifier; The data index-rule identifier is sent to a multidimensional rule engine for incremental calculation.

3. The method according to claim 2, characterized in that In the multidimensional rule engine screening: the built-in rule expression DSL parser of the multidimensional rule engine is used to convert the JSON-formatted business rules into an abstract syntax tree, which is then handed over to the extended expression engine for immediate compilation into bytecode; the first compilation result is cached in the local high-speed cache, and subsequent hits are directly executed; after receiving the structured change message, batch filtering is first performed according to the macro group dimension consisting of the rule combination ID + the main object. The batch filtering includes a two-stage pipeline: Stage 1 - condition matching: Boolean expressions are executed in parallel to match the data attributes of the main and subordinate objects to obtain a set of rule candidates; Stage 2 - row-level judgment: The execution conditions of the candidate rules are evaluated row by row, and each data row and rule is traversed through a double loop to execute expression judgment and collect matching results. Only the truly affected data indexes are marked, and it is determined whether aggregation calculation is required. If so, the batch aggregation value acquisition interface is uniformly called to inject the returned array of aggregation rule IDs and value maps into the execution context for direct value retrieval by subsequent formulas; A single call can complete the calculation of N aggregate indicators.

4. The method according to claim 3, characterized in that In the incremental recalculation step: the modified slave object index list obtained based on change detection is input, and the fields in the modified slave object index list are analyzed. If the master object field is changed, all data indexes are directly included in the recalculation to avoid full table scanning; based on the data index-rule identifier mapping, only the rules related to the changed data rows are retained; if a row of the combined slave object is changed, other rows in the same group are automatically included in the recalculation to avoid errors; if the user manually specifies a rule combination, its priority is set to the minimum integer value during the recalculation phase to ensure consistent results.

5. The method according to claim 4, characterized in that During recalculation, a rule conflict resolution algorithm is used to provide a unique and predictable result when multiple executable rules are encountered on the same data row. The algorithm performs the following steps: The conflict set is stably sorted in descending order of priority. If any rule tuple with a user priority identifier equal to 1 exists, the entire set is moved to the front, while the remaining elements retain their original order. If the same priority and user priority identifier values ​​still appear at the head of the rule, the one with the smallest creation time wins. A lock-free concurrency and consistency strategy is used to implement the following operations: the decision logic runs on a copy of the conflict set cached locally in the thread, and does not involve shared write operations; the results are written to the row-level decision tracking dictionary via copy-on-write to avoid lock contention. The user priority flag is 1 only when the user explicitly selects it, and 0 otherwise.

6. The method according to claim 5, characterized in that Use sparse bitmap change location for change detection: Map snapshot fields to sparse bitmaps, compare them with the current request bitmap to obtain the set of changed rows, and optimize operations based on master-slave object data scenarios: Construct multi-level dependency graphs such as master-slave object data combinations to ensure that single-point changes automatically affect other data points. Use breadth-first traversal to trim the impact radius when the weight threshold is reached. After deleting a row or disabling a policy, the row or policy is blacklisted for a certain period of time and is skipped directly by subsequent requests. For data index-rule identification, use a bitmap dependency pruning algorithm for optimization: precompile field bitmaps, generate a 256-bit bitmap when the rule is published, and identify its field dependency set.

7. A multi-round incremental rule matching device for master-slave object data, characterized in that: The device includes: a change detection unit, configured to monitor the master and slave object data in the database and generate a structured change message when a change occurs to one of the master and slave object data, wherein the structured change message includes an identifier indicating the change to the master and slave object data and a change range; a multidimensional rule engine screening unit, which determines, based on the structured change message, a processing rule for matching the changed data from a rule base in the database, and locates a row where the changed data is located; Incremental recalculation unit: using the processing rule to perform incremental recalculation on the master and slave object data based on the identifier of the master and slave object data change, the change range, and the row where the changed data is located; The state memory unit is used to save the key field status snapshot of each round of matching to achieve cross-request state memory capabilities. Through the data structure of the last matching data and Redis persistent storage, it provides the core state memory function for multiple rounds of incremental rule matching.

8. The device according to claim 7, characterized in that The change detection unit operates as follows: parsing the action code of the master-slave object data, and determining whether the change type is addition, editing, or deletion based on the parsed result; if the change type is editing, triggering a combination key cascade detection and filtering process to retain only the changed data rows, identify the aggregation rule type, and output a modified slave object index list; Perform multiple type-safe comparisons on the modified rows covered by the object index list based on field comparison to obtain a list of fields affected by the change range; Perform change analysis on the affected field list, determine the impact scope, and perform field weight sorting and field-rule dependency analysis to generate a data index-rule identifier; The data index-rule identifier is sent to a multidimensional rule engine for incremental calculation.

9. The device according to claim 8, characterized in that In the multidimensional rule engine screening: the built-in rule expression DSL parser of the multidimensional rule engine is used to convert the JSON-formatted business rules into an abstract syntax tree, which is then handed over to the extended expression engine for immediate compilation into bytecode; the first compilation result is cached in the local high-speed cache, and subsequent hits are directly executed; after receiving the structured change message, batch filtering is first performed according to the macro group dimension consisting of the rule combination ID + the main object. The batch filtering includes a two-stage pipeline: Stage 1 - condition matching: Boolean expressions are executed in parallel to match the data attributes of the main and subordinate objects to obtain a set of rule candidates; Stage 2 - row-level judgment: The execution conditions of the candidate rules are evaluated row by row, and each data row and rule is traversed through a double loop to execute expression judgment and collect matching results. Only the truly affected data indexes are marked, and it is determined whether aggregation calculation is required. If so, the batch aggregation value acquisition interface is uniformly called to inject the returned array of aggregation rule IDs and value maps into the execution context for direct value retrieval by subsequent formulas; A single call can complete the calculation of N aggregate indicators.

10. A computer storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program on the computer storage medium is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images