Marketing inspection work quality intelligent evaluation closed-loop management method

Abstract

This invention discloses a closed-loop management method for intelligent evaluation of marketing audit work quality, relating to the field of power marketing audit technology. The method includes the following steps: S1, constructing a boundary map of all audit rules; S2, calculating the rule gap exploration degree; S3, constructing a causal relationship model between gap exploration and risk emergence; S4, implementing a double-helix dynamic evaluation mechanism; and S5, establishing a self-evolving closed loop of evaluation, rules, and risks. This invention utilizes the technical means of constructing a boundary map of all audit rules to quantify rule gap areas, calculating the rule gap exploration degree, establishing a causal relationship model between gap exploration and risk emergence, implementing a double-helix dynamic evaluation mechanism, and constructing a self-evolving closed loop of evaluation, rules, and risks. This solves the vicious cycle problem where existing evaluation systems become increasingly closed as they become more comprehensive, and unknown risks are systematically shielded. It achieves a dynamic balance between known risk control and unknown risk exploration, constructing a continuously evolving, comprehensive risk control system.

Description

Technical Field

[0001] This invention relates to the field of power marketing inspection technology, specifically to a closed-loop management method for intelligent evaluation of the quality of marketing inspection work. Background Technology

[0002] With the deepening of the construction of new power systems and the reform of the power system, the business scenarios of electricity marketing are becoming increasingly complex. Violations such as incorrect electricity pricing, illegal electricity transfer, and electricity theft occur frequently. Marketing auditing, as a core link in risk management, directly determines the economic losses and service levels of power companies. China Southern Power Grid Company has constructed a three-tiered marketing defense system, and the evaluation of the quality of marketing auditing work has become a key management tool to ensure the effective operation of these three lines of defense.

[0003] Currently, the commonly used evaluation method in this field is to establish three types of evaluation templates: indicators, business processes, and entities. Audit data is then compiled monthly or on a custom basis, with points deducted based on error severity and added based on contribution level. Finally, evaluation results and reports are generated. The industry generally improves the accuracy and comprehensiveness of evaluations by increasing evaluation dimensions, refining scoring rules, and optimizing statistical methods.

[0004] However, all indicators in existing evaluation systems are based on known risk definitions, and the essence of the evaluation logic is to guide inspectors to complete the task of controlling known risks. The more complete the evaluation system and the more refined the indicators, the stronger the constraint on inspection behavior. This leads frontline staff to devote all their energy to completing known indicators, thus systematically blocking the discovery of unknown risks. This creates a vicious cycle that cannot be broken by existing technology: higher evaluation scores indicate better control of known risks, but also indicate that unknown risks are hidden more deeply, resulting in a weaker overall risk prevention and control capability of the entire system. All optimization directions of existing technologies reinforce this closed logic, never realizing that its underlying logic itself is the source of risk. Therefore, this paper proposes an intelligent closed-loop management method for evaluating the quality of marketing inspection work to overcome the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a closed-loop management method for intelligent evaluation of the quality of marketing audit work, so as to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides a closed-loop management method for intelligent evaluation of marketing audit work quality. This method evaluates the quality of audit work based on data from a metrology automation system and a marketing management system, and includes the following steps: S1. Construct a full-scale audit rule boundary map, quantify the coverage boundaries and blank areas of existing audit rules, identify rule gaps, and classify them according to the historical risk occurrence frequency of the corresponding business. S2. Calculate the rule gap exploration degree. Based on the degree of matching between the operator's operation behavior and the rule gap, quantify their ability to discover unknown risks and classify the types of inspection behavior. S3. Construct a causal relationship model of gap exploration-risk emergence to establish a causal mapping relationship between exploration behavior and new risk discovery, and provide an objective basis for evaluation and incentives; S4. Implement a double-helix dynamic evaluation mechanism to calculate the known risk management baseline score and the unknown risk exploration contribution score. Dynamically adjust the coupling coefficient of the two dimensions based on the known risk management quality and associate rule gap exploration permissions. S5. Establish a self-evolving closed loop of evaluation, rules, and risks, automatically generate new inspection rules and update the rule boundary map and evaluation model, while tracking and predicting the risk migration between rules and dynamically adjusting the exploration direction.

[0007] Furthermore, the process of constructing the boundary map of the full audit rules in step S1 is as follows: Each effective audit rule is broken down into five dimensions: applicable business type, triggering conditions, audit scope, exclusion scenarios, and output results, forming a standardized boundary description of the five-tuple. Based on set theory, the intersection, union, and complement of any two rule boundaries are calculated to identify three types of rule gaps: single rule edge gaps, multi-rule intersection gaps, and global blank gaps. The historical risk occurrence frequency of each rule gap corresponding to the business area is statistically analyzed, and the rule gaps are divided into three levels: Level 1, Level 2, and Level 3, and a corresponding level coefficient is assigned.

[0008] Furthermore, the process of calculating the rule gap exploration degree in step S2 is as follows: Each operation is mapped to a corresponding rule gap, distinguishing between operations within the rule coverage and operations in the three types of rule gaps; For operations that match the rule gaps, calculate their operation depth and operation novelty. The operation depth is calculated by weighting the operation duration, the number of cross-system data calls, and the length of the evidence chain, while the operation novelty is calculated by the global occurrence frequency of similar operations. The comprehensive rule gap exploration degree is obtained by weighting and summing the level coefficient, operation depth, and operation novelty of all operations within the evaluation period. Based on the degree of exploration in the comprehensive rules gaps, inspection behaviors are divided into three categories: execution-oriented, exploratory, and innovative.

[0009] Furthermore, the process of constructing the gap exploration-risk emergence causal relationship model in step S3 is as follows: Extract all exploratory and innovative operations from the past 12 months, mark whether new risks have been discovered, and construct a sample set containing 12-dimensional operational features; The causal forest algorithm was used to train the model, with 100 trees, a maximum depth of 10 for each tree, and a minimum number of leaf node samples of 5. The loss function was the squared error of causal effects. The model outputs the causal effect value corresponding to each operational feature, and assigns causal weights to each feature based on the causal effect value. When the model discovers 10 new risks, it automatically adds the new samples to the sample set and retrains the model, updating the feature causal weights.

[0010] Furthermore, the process of calculating the known risk management baseline score in step S4 is as follows: The number of inspection samples, problem detection rate, rectification completion rate, and error rate are statistically analyzed according to the indicator evaluation template, and the weighted calculation is used to obtain the indicator evaluation score. The completion status of audits for seven major business categories is statistically analyzed based on the business evaluation template, and the business evaluation score is calculated by weighting the scores according to the business risk level. The main evaluation score is calculated by weighting the work order processing timeliness rate, user satisfaction, and peer evaluation scores according to the main evaluation template. The known risk management baseline score is obtained by weighted summing of the indicator evaluation score, business evaluation score, and entity evaluation score. When the baseline score is below 60, the entity's first-level and second-level rule gap exploration permissions are frozen.

[0011] Furthermore, the process of double-helix dynamic evaluation in step S4 is as follows: The contribution score for exploring unknown risks is calculated based on the exploration degree of rule gaps and the causal matching degree. The causal matching degree is the weighted sum of the causal weights of all operational features of the subject. When the known risk management baseline score is greater than or equal to 60 points and less than 80 points, the comprehensive score is calculated as 0.7 times the baseline score plus 0.3 times the exploration contribution score; When the known risk management baseline score is greater than or equal to 80 points and less than 90 points, the comprehensive score is calculated as 0.5 times the baseline score plus 0.5 times the exploration contribution score; When the known risk management baseline score is greater than or equal to 90 points, the comprehensive score is calculated as 0.3 times the baseline score plus 0.7 times the exploration contribution score; For every 10 points increase in contribution score, access to explore higher-level rule gaps will be automatically unlocked.

[0012] Furthermore, the process of automatically generating new audit rules in step S5 is as follows: When the cumulative number of new risks discovered within the same rule gap reaches a preset trigger threshold, the operational and risk characteristics of all new risks within that gap are extracted. Based on the existing rule structure template, generate the applicable business types, triggering conditions, verification scope, exclusion scenarios and output results of the new rules; Generate the execution script for the new rule and test its running efficiency and false alarm rate. The new rule that passes the test will be pushed to the provincial marketing audit expert database for review. New rules that have been approved are added to the rule base, and the rule boundary map is updated synchronously.

[0013] Furthermore, the process of tracking and predicting the migration of rule gap risks in step S5 is as follows: A topological correlation matrix of regular gaps is constructed based on the regular boundary map, and the ratio of the boundary intersection length of any two regular gaps to the total length is calculated as the topological correlation degree. When a new rule fills a gap in a rule, the rate of change in the exploration degree of adjacent gaps, the rate of change in the number of anomaly annotations, and the rate of change in the new risk discovery rate are extracted to form a migration feature vector. A risk migration path model is established using a first-order Markov chain to calculate the probability of risk migrating from a filled gap to an adjacent gap. A risk migration warning list is generated based on migration probability. The potential risk value of adjacent gaps with a migration probability greater than 0.5 is increased by 50%, and exploration suggestions are pushed out first.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. By constructing a boundary map of full-scale audit rules and a double-helix dynamic evaluation mechanism, the evaluation system is transformed from a closed system that constrains behavior into an open system that expands boundaries. The higher the evaluation score, the more solid the control of known risks and the deeper the exploration of unknown risks. This completely solves the problem that the higher the evaluation score of the existing system, the more hidden the unknown risks.

[0015] 2. By quantifying the gaps between rules and the degree of exploration of these gaps, we can accurately locate all behavioral spaces not covered by the rules, transforming the discovery of unknown risks from a chance event into an inevitable process; the longer the system runs, the more comprehensive the rule coverage, and the smaller the remaining space for unknown risks.

[0016] 3. By establishing a self-evolving closed loop of evaluation, rules, and risk, the evaluation process automatically generates new inspection rules, which enhance control capabilities. The enhanced control capabilities release more exploration resources, forming a continuously accelerating positive cycle. This effect is completely unattainable by existing technologies.

[0017] 4. The system fully adopts the calculation logic and process of the existing evaluation system, including three types of templates: indicator evaluation, business evaluation, and subject evaluation. Existing staff do not need to change their original work habits. The system automatically completes the exploration guidance and rule evolution in the background, resulting in low implementation costs.

[0018] 5. By monitoring changes in the exploration degree of rule gaps, when the exploration degree of a certain type of rule gap suddenly increases, the system can predict in advance that new risks may appear in the area and issue an early warning before the risks cause losses, thus achieving true proactive prevention and control.

[0019] 6. By using a pre-defined rule gap dynamic migration tracking and prediction mechanism, the topological association of rule gaps is quantified, a risk migration path model is established, the risk migration direction is predicted in advance and the exploration direction is dynamically adjusted, avoiding the repetitive nature of risk management and achieving full coverage and continuous iteration of risk management. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the closed-loop management method for intelligent evaluation of marketing audit work quality according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 The present invention provides a technical solution: See Figure 1 The following is an example of an intelligent closed-loop management method for evaluating the quality of marketing audit work: S1. Construct a boundary map of full-scale audit rules: The system quantifies the coverage boundaries and gaps of existing audit rules, rather than directly using the rules themselves for control. By drawing precise boundaries of the rules, it clarifies all behavioral spaces not covered by the rules, i.e., rule gaps, providing precise coordinates for the discovery of unknown risks. The rule boundary map output in this step is the foundation for all subsequent steps, and its accuracy directly determines the accuracy of unknown risk discovery.

[0023] Extract all effective audit rules from the marketing management system and construct a rule boundary graph. The specific process is as follows: 1. Rule Metadata Parsing: Each rule is broken down into five dimensions: applicable business type, triggering conditions, verification scope, exclusion scenarios, and output results, forming a standardized boundary description. Applicable business type is extracted from the business classification dictionary of the marketing management system, corresponding to seven categories: business expansion, meter reading and verification, metering management, electricity consumption inspection, line loss management, demand-side management, and distributed photovoltaic power generation. Triggering conditions are parsed from the rule's execution script, extracting all numerical thresholds, time conditions, spatial conditions, and logical judgment relationships. Verification scope is extracted from the rule's data source table, field filtering conditions, and data time window. Exclusion scenarios are extracted from the rule's exception clauses and whitelist configuration. Output results are extracted from the rule's alarm level, work order type, and processing flow. The standardized boundary description of each rule is represented by a five-tuple. ,in For rule numbering, , This represents the total number of rules in effect.

[0024] 2. Boundary Topology Calculation: Based on set theory, calculate the intersection, union, and complement of any two rule boundaries to identify three types of rule gaps. Define the entire business behavior space as the complete set. , No. The coverage space of the rule is ,but: Single regular edge gap , indicating rules The ring-shaped area formed by extending the boundary outward by 10% of the threshold range corresponds to the uncovered area near the rule threshold, where 10% is a preset threshold that can be adjusted according to actual business needs. Multiple regular cross gaps , indicating the first Article and Section The overlapping area outside the coverage of a single rule corresponds to the blank area between multiple rules; Global blank space This indicates a business scenario that is not covered by any rules.

[0025] All regular gaps constitute a set of regular gaps: .

[0026] 3. Gap Level Classification: Based on the historical risk occurrence frequency of the corresponding business, the rule gaps are divided into three levels. The number of risk occurrences in the business area corresponding to each rule gap over the past 12 months is statistically analyzed. Calculate risk frequency : Primary gap: This corresponds to a gap in high-risk business operations; Secondary gap: This corresponds to a gap in medium-risk business operations; Level 3 gap: This corresponds to a gap in low-risk business.

[0027] The grade coefficient for each rule gap The corresponding values ​​are 3, 2, and 1. This coefficient is used for subsequent calculation of the rule gap exploration degree.

[0028] S2, Quantitative Calculation of Rule Gap Exploration: A novel quantitative indicator, the "Rule Gap Index," is proposed to accurately measure an inspector's ability to detect unknown risks by analyzing the degree to which their operational behavior matches the rule gaps. This indicator is completely independent of known risk outcomes, distinguishing between standardized execution and exploratory verification based on the nature of the behavior. This avoids the selective inspection problem caused by existing evaluation systems that rely solely on known risk control results as the only evaluation criterion.

[0029] Collect all operation logs from the inspectors. Each log entry includes fields such as operation time, operator, operation type, data range, rule number, operation duration, number of data calls, evidence chain length, and operation result. Calculate the rule gap exploration degree for each inspector. The specific process is as follows: 1. Operation Gap Matching: Map each operation to a corresponding rule gap. For the first... This operation extracts the range of its operational data. and calling rule number ,calculate The percentage of intersection with the space covered by all rules. If With a certain rule If the intersection percentage is greater than 90%, the operation is considered an operation within the rule coverage area; if the intersection percentage is between 10% and 90%, the operation is considered a single rule edge gap operation; if the intersection percentage is less than 10% and the intersection percentage with the union of any two rules is greater than 90%, the operation is considered a multi-rule intersection gap operation; if the intersection percentage is less than 10% and the intersection percentage with the union of all rules is less than 90%, the operation is considered a global blank gap operation.

[0030] 2. Single-operation quantization calculation: For matching rule gaps The The depth of the operation is calculated for each subsequent operation. and operational novelty : Operational depth: , in The duration of this operation. This refers to the average duration of similar operations, which are audit operations under the same business type and the same rule interval. The number of cross-system data calls for this operation. This represents the average number of cross-system data calls for similar operations. The length of the chain of evidence generated by this operation. This represents the average chain of evidence length for similar operations. The value range is from 0.1 to 1. When the calculation result is greater than 1, it is taken as 1, and when it is less than 0.1, it is taken as 0.1.

[0031] Operational novelty: , in This represents the number of times all audit personnel across the bureau performed similar operations in the past 30 days. This represents the total number of all audit operations conducted globally over the past 30 days. The value of is between 0 and 1, and the rarer the operation, the higher the novelty.

[0032] 3. Calculation of Comprehensive Rule Gap Exploration Degree: The comprehensive rule gap exploration degree is obtained by weighted summing of the quantitative values ​​of all operations performed by the inspector within the evaluation period. , in: Indicates the degree of exploration of rule gaps; This indicates the total number of operations performed by the inspector during the evaluation period; Indicates the first The rule gap level coefficient corresponding to this operation; Indicates the first The depth of operation of each step; Indicates the first Operational novelty of the second operation.

[0033] 4. Classification of Inspection Behavior Types: Based on the degree of exploration of gaps in comprehensive rules, inspection behaviors are classified into three categories: Execution type: It fully follows known rules and only deals with standardized known risks; Exploratory: Appropriately deviate from known rules and proactively check the edges and intersections of rules; Innovative: It significantly breaks through known rules and delves into the global blank areas.

[0034] S3, Gap Exploration - Risk Emergence Causal Relationship Modeling: This step establishes a causal mapping between exploratory behavior and the discovery of unknown risks, rather than a simple statistical association. Using a causal forest algorithm, it accurately identifies which exploratory behaviors truly uncover new risks, eliminating spurious associations and chance factors, thus providing an objective and impartial basis for evaluation and incentives. The causal relationship model constructed in this step is the core support of the double-helix dynamic evaluation mechanism, ensuring that the evaluation results truly reflect the risk discovery capabilities of inspectors.

[0035] A causal relationship model for gap exploration and risk emergence is constructed based on historical data. The specific process is as follows: 1. Sample Construction: Extract all exploratory and innovative operations from the past 12 months to form an initial sample set. For each operation sample, mark whether it ultimately reveals a new risk. A new risk is defined as: a risk that is not covered by any existing audit rules and has not been discovered by other auditors in the past 12 months. The final sample set contains... There are n samples, each sample is represented as... ,in To operate on the feature vector, For binary labels, This indicates the discovery of new risks. This indicates that no new risks have been identified.

[0036] 2. Construction of Operational Feature Vectors: Extract a 12-dimensional feature vector for each operation sample. , Specifically, this includes: operation time distribution characteristics (proportion of weekdays / weekends, proportion of working hours / non-working hours), data combination characteristics (number of cross-system data combinations, number of data combinations at different time granularities), anomaly labeling characteristics (number of custom anomaly labels, number of multi-dimensional anomaly labels), operation sequence characteristics (operation step sequence entropy, operation step repetition rate), and gap type characteristics (proportion of single-rule edge gaps, proportion of multi-rule intersection gaps, and proportion of global blank gaps).

[0037] 3. Causal Forest Model Training: A causal association model is constructed using the causal forest algorithm. The model parameters are set as follows: Number of trees: 100; Maximum depth per tree: 10; Minimum number of leaf node samples per tree: 5; Splitting criterion: mean square error; Loss function: Squared error of causal effect, i.e.: , in For true causal effect, The causal effects predicted by the model; Initial parameters: The initial split thresholds for all nodes are randomly generated, ranging from the minimum to the maximum value of the corresponding feature.

[0038] The model training employs a bootstrap sampling method, where 80% of the samples are randomly selected with replacement from the dataset each time as the training set, and the remaining 20% ​​is used as the validation set. After training, the model outputs the causal effect value corresponding to each operational feature. This indicates the increase in the probability of discovering new risks for each additional unit of this feature.

[0039] 4. Assigning Causal Weights: Assigning causal weights to each operational feature. The weight is directly proportional to the causal effect value of the feature, that is: ; The sum of the causal weights of all features is 1.

[0040] 5. Automatic Model Iteration: When the model discovers 10 new risks, an iteration is automatically triggered. The operation samples corresponding to the newly discovered risks are added to the sample set, the causal forest model is retrained, and the feature causal weights are updated. During the iteration process, the parameters of the original model are retained as initial values ​​to accelerate training convergence.

[0041] S4. Double Helix Dynamic Evaluation Mechanism: Instead of adopting the existing multi-dimensional weighted static evaluation model, a double-helix evaluation structure is constructed, encompassing known risk management and unknown risk exploration. The two dimensions are not independent but intertwined and mutually reinforcing: the quality of known risk management determines exploration authority, while the results of unknown risk exploration upgrade management capabilities, forming a natural dynamic balance. This mechanism ensures that basic risk management is not relaxed while guiding inspectors to proactively explore unknown risks through dynamic authority adjustments, thus completely resolving the closed nature of the original system.

[0042] The specific implementation process of the double helix dynamic evaluation mechanism is as follows: S4.1 Calculation of known risk management baseline: Using the entire calculation logic of the existing evaluation system, including three types of templates: indicator evaluation, business evaluation, and entity evaluation, the known risk management baseline score is calculated. The maximum score is 100 points.

[0043] 1. Indicator Evaluation Score Based on the indicator evaluation template, the number of inspection samples, problem discovery rate, rectification completion rate, error rate, and other indicators are statistically analyzed and calculated according to preset weights of 0.3, 0.3, 0.2, and 0.2, respectively.

[0044] 2. Business Evaluation Score Based on the business evaluation template, the audit completion status of seven major business categories, including business expansion, meter reading and collection, and measurement management, is statistically analyzed. The results are calculated by weighting the risk levels of each business category, with high-risk businesses having a weight of 0.2, medium-risk businesses having a weight of 0.15, and low-risk businesses having a weight of 0.1.

[0045] 3. Subject Evaluation Score Based on the main evaluation template, the timeliness of work order processing, user satisfaction, and peer evaluation scores of the inspectors are statistically analyzed and calculated according to preset weights of 0.4, 0.3, and 0.3.

[0046] 4. Based on the known risk management baseline score: .

[0047] 5. Permission Freeze Rules: When During the time period, all level 1 and level 2 rule gap exploration permissions for this entity are frozen, while level 3 rule gap exploration permissions are retained until the next evaluation cycle. The freeze will be automatically lifted at certain times.

[0048] S4.2 Calculation of contribution to the exploration of unknown risks: Based on the rule gap exploration degree and causal relationship model, the contribution score for exploring unknown risks is calculated. Full marks: 100 points ; in: This indicates the highest rule gap exploration level among all inspectors within the entire evaluation period; This represents the causal matching degree, with a value between 0 and 1. It is the weighted sum of the causal weights of all operational features of the subject, i.e.: , in For the subject The standardized value of each operational feature.

[0049] S4.3 Evaluation of Double Helix Coupling: The coupling method between the two dimensions is dynamically adjusted based on the known risk management baseline score to achieve an adaptive shift in the evaluation focus: when At that time, the overall score The approach should prioritize control and supplement it with exploration, ensuring that basic risk management is in place. when At that time, the overall score We should give equal importance to control and exploration, and encourage exploration while ensuring the quality of control. when At that time, the overall score The approach should prioritize exploration and supplement it with control measures, freeing up more resources for the discovery of unknown risks.

[0050] Simultaneously, for every 10 points increase in contribution score, a higher level of rule gap exploration permission is automatically unlocked: when Time-sharing, grant exploration permissions for the gaps in secondary rules; when During certain time periods, access to explore level-one rule gaps will be granted. Once granted, this access is permanent and will remain in effect until the next evaluation cycle. The time slot was frozen.

[0051] S5. Evaluation-Rules-Risk Self-Evolutionary Closed Loop: This step is used for the deep integration and synchronous evolution of the evaluation process and risk management system. Evaluation is no longer a summary at the end of a cycle, but rather the core driving force for the continuous upgrading of the entire risk management system. Each evaluation cycle automatically fills rule gaps, expands management boundaries, and tracks the dynamic migration of rule gaps, driving the next round of more precise exploration. This step pre-sets a dynamic migration tracking and prediction mechanism for rule gaps, solving the risk migration effect of risks moving to adjacent gaps after the addition of new rules—a problem that existing technologies have never discovered or solved.

[0052] The specific implementation process of the evaluation-rule-risk self-evolutionary closed loop is as follows: S5.1, Transformation of New Risk Rules: When the cumulative number of new risks discovered through exploratory activities within the same rule gap reaches three, the system automatically generates a corresponding new audit rule. Three risks is a preset trigger threshold, which can be adjusted according to the required risk control precision. The new rule generation process is as follows: 1. Extract all operational and risk characteristics of newly discovered risks within the gaps in this rule; 2. Based on the existing rule structure template, generate the applicable business types, triggering conditions, verification scope, exclusion scenarios, and output results of the new rules; 3. Generate execution scripts for new rules and automatically test the script's running efficiency and false alarm rate; 4. Submit the new rules to the provincial marketing audit expert database for review, and add them to the rule library after the review is approved.

[0053] S5.2, Update of the regular boundary map: After a new rule is added, the boundaries of all rules are automatically recalculated, and the position and level of rule gaps are updated. 1. Add the quintuple description of the new rule to the rule set; 2. Recalculate the intersection, union, and complement of the entire business behavior space, and update the range of all rule gaps; 3. Recalculate the historical risk occurrence frequency for each rule gap, and update the gap level and level coefficient; 4. Generate an updated rule boundary map and push it to the business processing interface of all inspectors.

[0054] S5.3 Evaluation Model Iteration: Automatically update the causal relationship model and double-helix coupling parameters based on the new rules and new risk data: 1. Add the operational samples corresponding to the newly discovered risks to the sample set of the causal association model; 2. Retrain the causal forest model and update the feature causal weights and causal weight vectors; 3. Adjust the weight of each business type in the business evaluation score according to the business types covered by the new rules; 4. Based on the global new risk discovery rate, fine-tune the baseline value of the double helix coupling coefficient, with an adjustment range not exceeding 0.05.

[0055] S5.4 Exploration-guided optimization: Based on the updated rule boundary map, the latest suggestions for exploring high-value rule gaps are pushed to inspectors: 1. Calculate the potential risk value for each rule gap, which is the gap level coefficient multiplied by the exploration growth rate of that gap over the past 30 days; 2. Sort by potential risk value from highest to lowest to generate a Top 10 list of high-value rule gaps; 3. Personalize exploration suggestions based on each inspector's historical exploration behavior characteristics and scope of authority; 4. Exploration suggestions include the business areas to be explored, data scope, operation methods, and reference cases.

[0056] S5.5, Dynamic migration tracking and prediction of rule gaps: By quantifying the topological relationships between rule gaps, a dynamic model of risk migration is established to predict the direction of risk migration in advance and achieve dynamic adjustment of the exploration direction.

[0057] 1. Construction of the gap topological correlation matrix: Based on the regular boundary map, construct the regular gap topological correlation matrix. ,in Indicates the interval between rules and rule gaps The topological correlation between the two gaps is calculated by dividing the intersection length of the two gap boundaries by the total length of the two gap boundaries. The value ranges from 0 to 1, and the larger the value, the closer the spatial relationship between the two gaps.

[0058] 2. Transfer Feature Extraction: When new rules fill the gaps between rules Then, the rate of change in exploration degree, the rate of change in the number of anomaly annotations, and the rate of change in the rate of new risk discovery for all adjacent gaps over the past 30 days are extracted to form a migration feature vector. .

[0059] 3. Migration Path Modeling: Based on historical migration data, a risk migration path model is established using a first-order Markov chain. The state is defined as the risk level of each rule gap, and the state transition matrix is... elements Indicates risk from gaps Migration to gap The probability is calculated based on historical gaps. After being filled, the gap The number of times the risk level is raised divided by the interval The total number of times it was filled.

[0060] 4. Migration probability prediction: When new rules fill the gaps Then, combining the topological correlation matrix and the state transition matrix, the probability of risk migrating to each adjacent gap within the next 30 days is predicted: ; A risk migration warning list is generated by sorting migration probabilities from high to low.

[0061] 5. Dynamic adjustment of exploration direction: Based on the risk migration warning list, dynamically adjust the sorting of high-value rule gaps, increase the potential risk value of adjacent gaps with a migration probability greater than 0.5 by 50%, and prioritize pushing exploration suggestions for these gaps to auditors.

[0062] Summarize: The evaluation system has been transformed from a closed system that constrains behavior to an open system that expands its boundaries. The higher the evaluation score, the more solid the management of known risks and the deeper the exploration of unknown risks, thus completely resolving the fundamental contradiction of high scores but low capabilities.

[0063] By precisely identifying and guiding the gaps in the rules, the discovery of unknown risks is transformed from a chance event into an inevitable process. The longer the system runs and the more comprehensive the rule coverage, the smaller the remaining space for unknown risks.

[0064] The evaluation process automatically generates new rules, which improve control capabilities. Enhanced control capabilities then release more exploration resources, creating a continuously accelerating positive cycle. This effect is completely unattainable with current technology.

[0065] The system retains all the functions and processes of the existing evaluation system, and existing staff do not need to change their original work habits. The system automatically completes exploration guidance and rule evolution in the background.

[0066] When the exploration degree of a certain type of rule gap suddenly increases, the system can predict in advance that new risks may appear in that area and issue an early warning before the risks cause losses, thus achieving true proactive prevention and control.

[0067] By dynamically tracking the migration path and predicting the migration direction of rule gaps, the risk migration effect of risk management is avoided, and full coverage and continuous iteration of risk management are achieved.

Claims

1. A closed-loop management method for intelligent evaluation of marketing audit work quality, which evaluates the quality of audit work based on data from a metrological automation system and a marketing management system, characterized by: Includes the following steps: S1. Construct a full-scale audit rule boundary map, quantify the coverage boundaries and blank areas of existing audit rules, identify rule gaps, and classify them according to the historical risk occurrence frequency of the corresponding business. S2. Calculate the rule gap exploration degree. Based on the degree of matching between the operator's operation behavior and the rule gap, quantify their ability to discover unknown risks and classify the types of inspection behavior. S3. Construct a causal relationship model of gap exploration-risk emergence to establish a causal mapping relationship between exploration behavior and new risk discovery, and provide an objective basis for evaluation and incentives; S4. Implement a double-helix dynamic evaluation mechanism to calculate the known risk management baseline score and the unknown risk exploration contribution score. Dynamically adjust the coupling coefficient of the two dimensions based on the known risk management quality and associate rule gap exploration permissions. S5. Establish a self-evolving closed loop of evaluation, rules, and risks, automatically generate new inspection rules and update the rule boundary map and evaluation model, while tracking and predicting the risk migration between rules and dynamically adjusting the exploration direction.

2. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of constructing the boundary map of the full audit rules in step S1 is as follows: Each effective audit rule is broken down into five dimensions: applicable business type, triggering conditions, audit scope, exclusion scenarios, and output results, forming a standardized boundary description of the five-tuple. Based on set theory, the intersection, union, and complement of any two rule boundaries are calculated to identify three types of rule gaps: single rule edge gaps, multi-rule intersection gaps, and global blank gaps. The historical risk occurrence frequency of each rule gap corresponding to the business area is statistically analyzed, and the rule gaps are divided into three levels: Level 1, Level 2, and Level 3, and a corresponding level coefficient is assigned.

3. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of calculating the rule gap exploration degree in step S2 is as follows: Each operation is mapped to a corresponding rule gap, distinguishing between operations within the rule coverage and operations in the three types of rule gaps; For operations that match the rule gaps, calculate their operation depth and operation novelty. The operation depth is calculated by weighting the operation duration, the number of cross-system data calls, and the length of the evidence chain, while the operation novelty is calculated by the global occurrence frequency of similar operations. The comprehensive rule gap exploration degree is obtained by weighting and summing the level coefficient, operation depth, and operation novelty of all operations within the evaluation period. Based on the degree of exploration in the comprehensive rules gaps, inspection behaviors are divided into three categories: execution-oriented, exploratory, and innovative.

4. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of constructing the gap exploration-risk emergence causal relationship model in step S3 is as follows: Extract all exploratory and innovative operations from the past 12 months, mark whether new risks have been discovered, and construct a sample set containing 12-dimensional operational features; The causal forest algorithm was used to train the model, with 100 trees, a maximum depth of 10 for each tree, and a minimum number of leaf node samples of 5. The loss function was the squared error of causal effects. The model outputs the causal effect value corresponding to each operational feature, and assigns causal weights to each feature based on the causal effect value. When the model discovers 10 new risks, it automatically adds the new samples to the sample set and retrains the model, updating the feature causal weights.

5. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of calculating the known risk management baseline score in step S4 is as follows: The number of inspection samples, problem detection rate, rectification completion rate, and error rate are statistically analyzed according to the indicator evaluation template, and the weighted calculation is used to obtain the indicator evaluation score. The completion status of audits for seven major business categories is statistically analyzed based on the business evaluation template, and the business evaluation score is calculated by weighting the scores according to the business risk level. The main evaluation score is calculated by weighting the work order processing timeliness rate, user satisfaction, and peer evaluation scores according to the main evaluation template. The known risk management baseline score is obtained by weighted summing of the indicator evaluation score, business evaluation score, and entity evaluation score. When the baseline score is below 60, the entity's first-level and second-level rule gap exploration permissions are frozen.

6. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of double-helix dynamic evaluation in step S4 is as follows: The contribution score for exploring unknown risks is calculated based on the exploration degree of rule gaps and the causal matching degree. The causal matching degree is the weighted sum of the causal weights of all operational features of the subject. When the known risk management baseline score is greater than or equal to 60 points and less than 80 points, the comprehensive score is calculated as 0.7 times the baseline score plus 0.3 times the exploration contribution score; When the known risk management baseline score is greater than or equal to 80 points and less than 90 points, the comprehensive score is calculated as 0.5 times the baseline score plus 0.5 times the exploration contribution score; When the known risk management baseline score is greater than or equal to 90 points, the comprehensive score is calculated as 0.3 times the baseline score plus 0.7 times the exploration contribution score; For every 10 points increase in contribution score, access to explore higher-level rule gaps will be automatically unlocked.

7. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of automatically generating new audit rules in step S5 is as follows: When the cumulative number of new risks discovered within the same rule gap reaches a preset trigger threshold, the operational and risk characteristics of all new risks within that gap are extracted. Based on the existing rule structure template, generate the applicable business types, triggering conditions, verification scope, exclusion scenarios and output results of the new rules; Generate the execution script for the new rule and test its running efficiency and false alarm rate. The new rule that passes the test will be pushed to the provincial marketing audit expert database for review. New rules that have been approved are added to the rule base, and the rule boundary map is updated synchronously.

8. The intelligent evaluation closed-loop management method for marketing audit work quality as described in claim 1, characterized in that: The process of tracking and predicting the migration of rule gap risks in step S5 is as follows: A topological correlation matrix of regular gaps is constructed based on the regular boundary map, and the ratio of the boundary intersection length of any two regular gaps to the total length is calculated as the topological correlation degree. When a new rule fills a gap in a rule, the rate of change in the exploration degree of adjacent gaps, the rate of change in the number of anomaly annotations, and the rate of change in the new risk discovery rate are extracted to form a migration feature vector. A risk migration path model is established using a first-order Markov chain to calculate the probability of risk migrating from a filled gap to an adjacent gap. A risk migration warning list is generated based on migration probability. The potential risk value of adjacent gaps with a migration probability greater than 0.5 is increased by 50%, and exploration suggestions are pushed out first.

Images