Government affair information conversion method and system based on multi-source data

Abstract

The invention discloses a government affair information conversion method and system based on multi-source data, and relates to the technical field of heterogeneous government affair data management. The government affair information conversion method based on the multi-source data comprises the following steps of performing bidirectional calibration delay evaluation; evaluating rule matching delay; and evaluating data association delay. According to the method, the bidirectional calibration delay evaluation is carried out on the heterogeneous government affair data based on the bidirectional calibration delay data to judge whether the bidirectional calibration data volume optimization is carried out, and then the matching delay evaluation is carried out on the heterogeneous government affair data based on the rule matching delay data to judge whether the matching delay optimization is carried out. And finally, performing association delay evaluation on the heterogeneous government affair data based on the matching result cache replacement frequency to judge whether cache block parameter optimization is performed or not, thereby achieving the effect of improving the timeliness of corresponding rule matching and data association of the government affair heterogeneous data. The problem that government affair information updating is not synchronous due to the fact that information conversion of multi-source government affair heterogeneous data is not timely in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of heterogeneous government data management technology, and in particular to a method and system for converting government information based on multi-source data. Background Technology

[0002] To break down the silos of heterogeneous structured government data, achieve data sharing and circulation, and improve the efficiency and quality of government services, the government service center collects data from various heterogeneous government data source providers, such as the population information database of the public security department and the tax collection and management database of the tax department. Data is acquired through API interfaces, file transfer, message queues, and other methods to achieve seamless access to different data sources. The data is then transmitted to the front-end database of the government cloud platform. The received data undergoes cleaning. For structured data that still contains conflicts after cleaning, rule matching is performed using established data verification rules and business logic checks to identify and correct errors in the data. Data differences between different data sources are analyzed, and multi-source data of the same entity are associated through association key matching (e.g., individuals use "ID number," enterprises use "unified social credit code") to ensure the accuracy of the fusion object. For conflicting data after association, data fusion algorithms, such as weighted average, median, and time decay methods, are used to fuse related data from multiple data sources, ultimately generating unified structured data. This process transforms and integrates heterogeneous government data, generating more accurate and comprehensive government information to support government decision-making, public services, regulatory enforcement, and other government activities.

[0003] For example, the Chinese invention patent with announcement number CN110070279B discloses a method, system, and storage medium for quantitative evaluation of land use efficiency based on multi-source data, which includes: acquiring road network information; dividing the area to be evaluated into fine-grained regional units based on the road network information; acquiring government geographic information; generating geocodes based on the government geographic information; converting the geocodes into coordinate information; performing nearest-neighbor spatial matching based on the fine-grained regional units and coordinate information to obtain optimized coordinate information; calculating efficiency indicators based on the land parcels corresponding to the optimized coordinate information to obtain the calculation results; and displaying the calculation results in a spatialized form.

[0004] For example, Chinese invention patent CN119294851A discloses a method, system, and medium for processing government data. This method includes: acquiring multi-source government data based on big data to obtain a government data set; preprocessing the government data set to obtain an optimized data set; inputting the optimized data set into a government governance model trained using deep learning technology to generate governance decisions; analyzing the quality of government services based on the governance decisions to determine whether the quality of government services meets preset quality requirements; if it meets the preset quality requirements, then governing the government based on the governance decisions to generate governance data; if it does not meet the preset quality requirements, then generating correction prompts to suggest adjustments to the government service process. This method constructs a large model containing a large amount of government data and, combined with advanced technologies such as artificial intelligence and big data analysis, improves the efficiency of processing government data.

[0005] The above-mentioned technology has at least the following technical problems:

[0006] In existing technologies, the transformation of heterogeneous government structured data relies on cross-source association rules. However, the matching logic of many-to-many associations involves a more complex computational process. When the data volume reaches millions, this bidirectional verification requires nested loop operations: for each entity in the source dataset (millions), it traverses the candidate entities in the target dataset (potentially thousands), and then iterates in reverse to confirm. Essentially, this involves a computational scale of "millions × thousands," with the computational load increasing geometrically with the data volume. This can result in a single association calculation taking several minutes, failing to meet real-time requirements. Furthermore, after the association transformation, multi-dimensional verification (such as format verification, business rule verification, and cross-source consistency verification) is required. However, the verification rules package multiple independent verification logics into a multi-level, multi-branch nested dependency with granular rules. The extended time of a single rule directly translates into an increase in the total time. Moreover, it is necessary to wait for all preceding rules stored in the cache block to complete, making it impossible to skip intermediate steps and execute them in parallel for speed-up. Therefore, the total time of the entire process equals the sum of the time of each rule, increasing processing time and difficulty. This also leads to the problem of untimely information transformation of heterogeneous government data from multiple sources, resulting in asynchronous updates of government information. Summary of the Invention

[0007] To address the problem of asynchronous government information updates caused by untimely information conversion of heterogeneous multi-source government data in existing technologies, this invention provides a method and system for converting government information based on multi-source data. The technical solution is as follows:

[0008] On the one hand, a method for transforming government information based on multi-source data is provided, which includes the following steps: Step 1, based on the acquired two-way calibration delay data, a two-way calibration delay assessment is performed on the data cleaning process of heterogeneous government data to determine whether two-way calibration data volume optimization is necessary. If the two-way calibration efficiency meets the standard, the heterogeneous government data transformation period is obtained. Two-way calibration data volume optimization means reducing the two-way calibration delay of heterogeneous government data during the data cleaning process by adjusting the amount of two-way calibration data in a single cleaning process. The heterogeneous government data transformation period includes a rule matching period and a data association period; Step 2, based on the rule matching period, the rule... Then, for the matching latency data, the matching latency evaluation of the rule matching process of heterogeneous government data is performed to determine whether to optimize the matching latency. Matching latency optimization means reducing the rule matching latency of heterogeneous government data in the rule matching process by adjusting the number of parallel matching rules. Step 3: If the rule matching rate meets the standard, the association latency evaluation of the data association process of heterogeneous government data is performed based on the cache replacement frequency of the matching results obtained during the data association period to determine whether to optimize the cache block parameters. Cache block parameter optimization means reducing the data association latency of heterogeneous government data in the data association process by adjusting the number of cache blocks and the cache block capacity.

[0009] On the other hand, a government information conversion system based on multi-source data is provided. This system includes: a two-way calibration delay evaluation module, a matching delay evaluation module, and an association delay evaluation module. The two-way calibration delay evaluation module evaluates the data cleaning process of heterogeneous government data based on acquired two-way calibration delay data to determine whether two-way calibration data volume optimization is needed. If the two-way calibration efficiency meets the standard, the heterogeneous government data conversion period is obtained. The matching delay evaluation module evaluates the rule matching process of heterogeneous government data based on rule matching delay data obtained during the rule matching period to determine whether matching delay optimization is needed. The association delay evaluation module evaluates the association process of heterogeneous government data based on the matching result cache replacement frequency obtained during the data association period, if the rule matching rate meets the standard, to determine whether cache block parameter optimization is needed.

[0010] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0011] 1. Based on bidirectional calibration delay data, a bidirectional calibration delay assessment is performed on heterogeneous government data to determine whether bidirectional calibration data volume optimization is needed. This reduces computational resource constraints and processing wait times caused by processing data all at once, while improving resource utilization. Then, a matching delay assessment is performed based on rule matching delay data to determine whether matching delay optimization is needed. This reduces waiting time in the queue, improves system resource utilization, and enhances overall processing capacity. Finally, based on the cache replacement frequency of matching results, a correlation delay assessment is performed on heterogeneous government data to determine whether cache block parameters need to be optimized. This reduces cache access latency, improves cache hit rate, and reduces invalid replacements to lower replacement overhead. Ultimately, this improves the timeliness of rule matching and data correlation for heterogeneous government data.

[0012] 2. By obtaining the deviation between the number of fill value query interactions, outlier verifications, and I / O interactions at the end of the bidirectional calibration period and the preset bidirectional calibration delay data in the database, this method simultaneously uses the bidirectional calibration delay data correction coefficient in the database to correct each deviation. The corrected results are then coupled to obtain the bidirectional calibration delay index. Compared to existing technologies that rely on only a single indicator or a few simple indicators to evaluate bidirectional calibration delay, which cannot fully reflect the complexities of the calibration process, this method considers data from three dimensions simultaneously. This allows for a more comprehensive and accurate assessment of bidirectional calibration delay. Furthermore, by using the correction coefficient in the database to correct each deviation, the influence of interference factors on the deviation is reduced, making the corrected results closer to the actual situation. This reduces the impact of bidirectional calibration delay on the heterogeneous government data cleaning process.

[0013] 3. By performing a proportional calculation on the acquired matching rule loading time, matching rule traversal count, and rule version change frequency at the end of the rule matching period, and comparing them with the preset rule matching delay data in the database, and simultaneously calling the rule matching delay data correction coefficient in the database to correct each proportional calculation result, and then coupling the corrected results to obtain the rule matching delay index, this method first performs a proportional calculation, proportionalizing the data of different dimensions with the preset rule matching delay data in the database, eliminating the difference in dimensions and making different indicators comparable. Then, it calls the rule matching delay data correction coefficient in the database to correct each proportional calculation result, providing timely and effective support for system optimization, thereby reducing the impact of rule matching delay on the heterogeneous government data conversion process. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating a method for converting government information based on multi-source data, provided in this application embodiment;

[0016] Figure 2 A logical framework diagram of a government information conversion method based on multi-source data provided in this application embodiment;

[0017] Figure 3 A flowchart for bidirectional calibration delay evaluation and optimization provided in the embodiments of this application;

[0018] Figure 4 A flowchart for rule matching delay evaluation and optimization provided in the embodiments of this application;

[0019] Figure 5 A flowchart for data association delay evaluation and optimization provided in the embodiments of this application;

[0020] Figure 6 A schematic diagram of the structure of a government information conversion system based on multi-source data provided in this application embodiment;

[0021] Figure 7 This invention provides a main interface for a government information conversion system based on multi-source data, as provided in an embodiment of the invention.

[0022] Figure 8 This invention provides a bidirectional calibration display interface in a government information conversion system based on multi-source data, as provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0024] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0025] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0026] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0027] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0028] like Figure 1 The flowchart shown is a method for transforming government information based on multi-source data. The processing flow of this method may include the following steps:

[0029] Step 1: Based on the acquired bidirectional calibration delay data, a bidirectional calibration delay assessment is performed on the data cleaning process of heterogeneous government data to determine whether bidirectional calibration data volume optimization is necessary. If the bidirectional calibration efficiency meets the standard, the heterogeneous government data conversion period is obtained. The bidirectional calibration delay assessment is used to measure the impact of bidirectional calibration delay data on the dual-effect calibration delay of heterogeneous government data during the data cleaning process. The bidirectional calibration data volume optimization means reducing the bidirectional calibration delay of heterogeneous government data during the data cleaning process by adjusting the bidirectional calibration data volume in a single cleaning process. The heterogeneous government data conversion period includes the rule matching period and the data association period.

[0030] Step 2: Based on the rule matching delay data obtained during the rule matching period, perform a matching delay assessment on the rule matching process of heterogeneous government data to determine whether to optimize the matching delay. The matching delay assessment is used to measure the impact of the rule matching delay data on the rule matching delay of heterogeneous government data during the rule matching process. Matching delay optimization means reducing the rule matching delay of heterogeneous government data in the rule matching process by adjusting the number of parallel matching rules.

[0031] Step 3: If the rule matching rate meets the standard, then based on the cache replacement frequency of the matching results obtained during the data association period, the association latency assessment of the data association process of heterogeneous government data is carried out to determine whether to optimize the cache block parameters. The association latency assessment is used to measure the impact of the cache replacement frequency of the matching results on the association latency of heterogeneous government data during the data association process. Cache block parameter optimization means reducing the data association latency of heterogeneous government data in the data association process by adjusting the number of cache blocks and the cache block capacity.

[0032] like Figure 2 The diagram illustrates a logical framework for a government information transformation method based on multi-source data. This process design method clarifies the sequence and dependencies between each step, focusing on latency assessment and optimization for heterogeneous government data transformation. It includes three main parts: two-way calibration latency assessment, rule matching latency assessment, and data association latency assessment. If the two-way calibration efficiency fails to meet the standard after the two-way calibration latency assessment, the latency is reduced by directly adjusting the data volume factor during the data cleaning process. This avoids problems such as excessive processing time or resource waste caused by excessively large or small data volumes, thus improving the efficiency of the two-way calibration step. For the rule matching step, the overall performance is effectively improved by adjusting the number of parallel matching rules and rationally allocating computing resources. In the data association step, reasonable caching configuration can reduce data reading time and improve the speed of data association. The optimization effects of each step are superimposed, shortening the data transformation time from input to output and enabling faster provision of accurate data support for government decision-making and business processing.

[0033] like Figure 3 The flowchart shown is for the evaluation and optimization of bidirectional calibration delay. The design logic of the flowchart is as follows: the process starts by collecting bidirectional calibration delay data from the data source. Based on the obtained bidirectional calibration delay data, a bidirectional calibration delay evaluation is performed to determine whether the bidirectional calibration delay evaluation result meets the standard. If it does, the heterogeneous government data conversion period is obtained; otherwise, the bidirectional calibration data volume is optimized.

[0034] Further understanding is needed regarding the bidirectional calibration delay assessment of the data cleaning process for heterogeneous government data based on the acquired bidirectional calibration delay data. The specific process is as follows: At the end of the bidirectional calibration period, the deviation between the acquired bidirectional calibration delay data and the preset bidirectional calibration delay data in the database is obtained. Simultaneously, the correction coefficients for each deviation are applied using the bidirectional calibration delay data correction coefficients in the database. The corrected results are then coupled to obtain the bidirectional calibration delay index. The bidirectional calibration delay data includes the number of fill value query interactions, the number of outlier checks, and the number of I / O (Input / Output) interactions, all of which can be monitored using counters. The number of fill value query interactions represents the number of interactions performed during the bidirectional calibration process to obtain the desired data. The number of interactions with external data sources to retrieve fill values; the number of outlier checks represents the total number of times outlier data was detected and the check logic was triggered in a single cleansing of heterogeneous government data; the number of I / O interactions represents the number of times raw data needs to be read, intermediate calibration results need to be written, and final corrected data needs to be written between the data source and the cleansing engine during the bidirectional calibration process; the preset bidirectional calibration delay data includes the preset number of fill value query interactions, outlier checks, and I / O interactions; the bidirectional calibration delay data correction coefficient includes the correction coefficient for fill value query interactions, outlier checks, and I / O interactions; and the bidirectional calibration delay index represents the quantitative data on the impact of the bidirectional calibration delay data on the bidirectional calibration efficiency of heterogeneous government data.

[0035] Specifically, the expression for limiting the number of query interactions and the number of results for fill-in value queries is as follows: In the formula, TW1 represents the number of fill value query interactions corresponding to the heterogeneous government data at the end of the two-way calibration period, a1 represents the fill value query interaction correction coefficient, C1 represents the fill value query interaction number corresponding to the heterogeneous government data at the end of the two-way calibration period, and C10 represents the preset fill value query interaction number, which is the result of summing and averaging the fill value query interaction number corresponding to the end of the historical two-way calibration period in the database.

[0036] The specific expression for limiting the number of outlier checks is: In the formula, TW2 represents the number of outlier verifications corresponding to the heterogeneous government data at the end of the two-way calibration period, a2 represents the outlier verification correction coefficient, C2 represents the number of outlier verifications corresponding to the heterogeneous government data at the end of the two-way calibration period, and C20 represents the preset number of outlier verifications, which is the result of summing and averaging the number of outlier verifications corresponding to the end of the historical two-way calibration period in the database.

[0037] The specific expression limiting the number of query interactions and result counts for populated values ​​is: In the formula, TW3 represents the number of I / O interactions corresponding to the heterogeneous government data at the end of the two-way calibration period, a3 represents the I / O interaction correction coefficient, C3 represents the number of I / O interactions corresponding to the heterogeneous government data at the end of the two-way calibration period, and C30 represents the preset number of I / O interactions, which is the result of summing and averaging the number of I / O interactions corresponding to the end of the historical two-way calibration period in the database.

[0038] The specific constraint expression for the bidirectional calibration delay index TW is: TW = TW1 + TW2 + TW3, where TW represents the bidirectional calibration delay index corresponding to the heterogeneous government data at the end of the bidirectional calibration period.

[0039] Pre-set correction coefficients closely related to the bidirectional calibration latency index have been configured in the database. These correction coefficients, along with the number of fill value query interactions, outlier checks, and I / O interactions, form a clear and orderly mapping relationship based on the specific requirements of the business scenario and the unique characteristics of the system architecture. This mapping system has two main association modes: the first is a direct correspondence between a single parameter and a correction coefficient; the second is a mode where multiple parameters are combined and mapped to correction coefficients. Taking the bidirectional calibration latency evaluation phase as an example, the system automatically matches the currently collected values ​​of fill value query interactions, outlier checks, and I / O interactions with the pre-set mapping relationship model to accurately extract the corresponding correction coefficients.

[0040] Most importantly, in order to ensure the consistency and comparability of the evaluation results, the range of values ​​for the fill value query interaction number correction coefficient, outlier check number correction coefficient, and I / O interaction number correction coefficient in this example are all limited to the closed interval [0,1], and the algebraic sum of these three coefficients is always kept at 1.

[0041] In this embodiment, the bidirectional calibration latency index increases with the increase of the number of fill value query interactions, the number of outlier checks, and the number of I / O interactions. Specifically, when the number of fill value query interactions increases, the diversity and complexity of data sources increase, and the selected fill values ​​may differ from the original data distribution, thereby increasing the frequency of outliers in the data. Therefore, more outlier checks are required, increasing the number of outlier checks. Furthermore, each fill value query may trigger one or more I / O interactions, leading to an increase in the number of I / O interactions. Outlier checks require data acquisition for judgment and analysis, and each outlier check may require reading relevant government data from the storage system. When the number of outlier checks increases, the number of I / O interactions also increases.

[0042] This example influences the real-time performance of multi-source heterogeneous government data cleaning through a linkage mechanism. By acquiring the deviation between the two-way calibration delay data and the preset data in the database, and by coupling and analyzing the interaction of these three parameters, it is possible to accurately understand the gap between the actual operation and the ideal state of the two-way calibration process from multiple dimensions. The data after correction is more accurate and reliable, achieving precise location, dynamic balancing, and predictive optimization of real-time bottlenecks. While ensuring the accuracy of government data cleaning, it reduces invalid operations and delays through collaborative optimization between parameters, enabling multi-source heterogeneous data to be transformed into usable information more quickly and supporting efficient collaboration in government services.

[0043] Furthermore, the process for determining whether to optimize the bidirectional calibration data volume is as follows: The numerical relationship between the obtained bidirectional calibration delay index and the preset bidirectional calibration delay index in the database is determined. If the obtained bidirectional calibration delay index is less than the preset bidirectional calibration delay index, it is considered that the bidirectional calibration efficiency meets the standard and a matching delay evaluation is performed. If the obtained bidirectional calibration delay index is not less than the preset bidirectional calibration delay index, it is considered that the bidirectional calibration efficiency does not meet the standard and bidirectional calibration data volume optimization is performed. The preset bidirectional calibration delay index is represented by the summation and averaging of the bidirectional calibration delay indices corresponding to the end of the historical bidirectional calibration period in the database.

[0044] The optimization of bidirectional calibration data volume involves the following steps: First, based on the acquired bidirectional calibration delay index deviation, a bidirectional calibration data volume adjustment value is mapped in the database. This value prompts distributed computing nodes to adjust the bidirectional calibration data volume accordingly, thereby improving bidirectional calibration efficiency. The bidirectional calibration delay index deviation represents the difference between the acquired bidirectional calibration delay index at the end of the bidirectional calibration period and the preset bidirectional calibration delay index. After optimizing the bidirectional calibration data volume, a bidirectional calibration efficiency verification command is sent, prompting designated personnel to verify the bidirectional calibration efficiency based on the optimized bidirectional calibration data volume. If the bidirectional calibration delay index re-acquired after the efficiency verification is less than the preset index, the bidirectional calibration data volume optimization is completed, and a matching delay assessment is performed. Otherwise, a bidirectional calibration delay alarm is issued. The bidirectional calibration efficiency verification measures the speed at which the system completes the calibration task after the bidirectional calibration data volume optimization.

[0045] In this embodiment, the adjustment value of the bidirectional calibration data volume is encoded as input using a genetic algorithm. The adjustment value is used as a chromosome to randomly generate an initial population. Then, a fitness function of the genetic algorithm is designed to consider the impact of the adjustment value on the bidirectional calibration delay exponential deviation. The population is continuously evolved through operations such as selection, crossover, and mutation. Finally, an optimized adjustment value of the bidirectional calibration data volume is output. This adjustment value can effectively balance the relationship between the bidirectional calibration delay and the bidirectional calibration data volume in the bidirectional calibration process, thereby improving the efficiency and quality of data cleaning.

[0046] This process accurately identifies efficiency bottlenecks by dynamically evaluating the relationship between the bidirectional calibration latency index and a preset threshold. When the latency exceeds the limit, a data volume adjustment strategy is intelligently mapped based on the deviation value, driving distributed nodes to optimize transmission load and reduce redundant interactions. After optimization, the effectiveness of the improvement is ensured through manual verification within a closed loop; if the performance still fails to meet the standard, an alarm is triggered. This mechanism achieves an adaptive balance between calibration efficiency and data volume, avoiding resource waste while ensuring system real-time performance and improving the stability and reliability of bidirectional calibration in industrial scenarios.

[0047] like Figure 4 The flowchart shown is for the rule matching delay evaluation and optimization process. The design logic of the flowchart is as follows: The process starts by acquiring the heterogeneous government data conversion period. First, it enters the rule matching period in the conversion period. Based on the acquired rule matching delay data, the matching delay is evaluated to determine whether the matching delay evaluation result meets the standard. If it does, it enters the data association period. Otherwise, it optimizes the number of parallel matching rules.

[0048] Further understanding is needed regarding the rule matching delay data obtained during the rule matching period. The process for evaluating the rule matching delay of heterogeneous government data involves the following steps: At the end of the rule matching period, a ratio is calculated between the obtained rule matching delay data and the preset rule matching delay data in the database. Simultaneously, a correction coefficient for the rule matching delay data in the database is used to correct each ratio calculation result. The corrected results are then coupled to obtain the rule matching delay index. The rule matching delay data includes the rule loading time, the number of rule traversals, and the rule version change frequency. The rule loading time represents the time required for the rule matching engine of heterogeneous government data to load all relevant format rules from the rule base when it starts. This is achieved through the engine's built-in... The loading event listener detects the number of times the matching rule traversal is performed, which represents the number of times the system performs a complete scan of the loaded matching rule set during the matching operation. This is detected by the built-in counter of the rule engine. The rule version change frequency represents the number of times the rule version is updated during the matching operation. This is detected by the version control system server. The preset rule matching delay data includes the preset matching rule loading time, the number of matching rule traversals, and the rule version change frequency. The rule matching delay data correction coefficient includes the matching rule loading time correction coefficient, the matching rule traversal number correction coefficient, and the rule version change frequency correction coefficient. The rule matching delay index represents the quantitative data on the impact of rule matching delay data on the rule matching delay in the heterogeneous government data transformation.

[0049] Specifically, the expression for limiting the number of results during the matching rule loading time is as follows: In the formula, RM1 represents the result of the matching rule loading time of heterogeneous government data at the end of the rule matching period, b1 represents the matching rule loading time correction coefficient, S1 represents the matching rule loading time of heterogeneous government data at the end of the rule matching period, and S10 represents the preset matching rule loading time, which is the result of summing and averaging the matching rule loading times corresponding to the end of the historical rule matching periods in the database.

[0050] The specific expression limiting the number of iterations and the result quantity of the matching rule is as follows: In the formula, RM2 represents the number of times the heterogeneous government data is traversed according to the matching rule at the end of the rule matching period, b2 represents the correction coefficient for the number of times the matching rule is traversed, S2 represents the number of times the heterogeneous government data is traversed according to the matching rule at the end of the rule matching period, and S20 represents the preset number of times the matching rule is traversed, which is the result of summing and averaging the number of times the matching rule is traversed according to the historical number of times the matching rule is traversed at the end of the rule matching period in the database.

[0051] The specific expression for limiting the result quantity of rule version change frequency is: In the formula, RM3 represents the result of the rule version change frequency corresponding to the heterogeneous government data at the end of the rule matching period, b3 represents the rule version change frequency correction coefficient, S3 represents the rule version change frequency corresponding to the heterogeneous government data at the end of the rule matching period, and S30 represents the preset rule version change frequency, which is the result of summing and averaging the rule version change frequencies corresponding to the end of the historical rule matching period in the database.

[0052] The specific constraint expression for the rule matching delay index RM is: RM = RM1 + RM2 + RM3, where RM represents the rule matching delay index corresponding to heterogeneous government data at the end of the rule matching period.

[0053] A series of preset correction coefficients closely related to the rule matching latency index are pre-configured in the database. These correction coefficients, along with the rule loading time, rule traversal count, and rule version change frequency, establish a clear and orderly mapping relationship based on the specific needs of the business scenario and the unique attributes of the system architecture. This mapping system can be a single parameter directly corresponding to a correction coefficient, or a combination of multiple parameters mapped to the correction coefficient. For example, during the match latency evaluation phase, the system collects the specific values ​​of the rule loading time, rule traversal count, and rule version change frequency in real time, and then automatically matches them with the preset mapping relationship model to accurately extract the corresponding correction coefficients.

[0054] Most importantly, to ensure that the evaluation results are highly consistent and comparable, the matching rule loading time correction coefficient, the matching rule traversal number correction coefficient, and the rule version change frequency correction coefficient in this example are all constrained to the range of values ​​within the closed interval [0,1]. At the same time, their numerical combination must follow the normalization criterion, that is, the algebraic sum of these three coefficients is always 1.

[0055] In this embodiment, the rule matching latency index increases with the increase of the rule loading time, the number of rule traversals, and the rule version change frequency. When the rule loading time increases, it means that the system needs to spend a lot of time preparing rule data before starting rule matching. Therefore, only some key rules can be traversed a limited number of times in order to output results before the deadline. When the number of rule traversals decreases, the system needs to parse, verify, and load the new rule data every time the rule version is changed, which requires additional time and resources. Therefore, when the rule version change frequency increases, the rule loading time increases. When the number of rule traversals increases, there will be more and more unreleased objects remaining in memory, which will occupy memory resources, reduce system performance, and lead to an increase in the rule loading time.

[0056] This example obtains the rule matching latency index by coupling these three parameters after proportional calculation and correction. This allows for the quantification of their impact on real-time performance. By incorporating these three parameters into the calculation of the rule matching latency index, the real-time performance of rule matching can be comprehensively evaluated. Furthermore, analyzing the interaction between the three parameters can help identify performance bottlenecks in the rule matching process, reduce unnecessary version changes, and thus improve the efficiency of rule matching.

[0057] Furthermore, the process for determining whether to perform matching delay optimization is as follows: The numerical relationship between the obtained rule matching delay index and the preset rule matching delay index in the database is determined. If the obtained rule matching delay index is less than the preset rule matching delay index, the rule matching efficiency is considered to be up to standard, and an association delay assessment is performed. If the obtained rule matching delay index is not less than the preset rule matching delay index, the rule matching efficiency is considered to be down to standard, and the number of parallel rules is optimized. The preset rule matching delay index is represented by the summation and averaging of the rule matching delay indices corresponding to the end of the historical rule matching period in the database.

[0058] The optimization of the number of parallel rules involves the following steps: First, based on the obtained rule matching latency index deviation, an adjustment value for the number of parallel matching rules is mapped in the database. This value prompts the rule matching engine to adjust the number of parallel matching rules based on the obtained adjustment value, thereby improving rule matching efficiency. The rule matching latency index deviation represents the difference between the rule matching latency index obtained at the end of the rule matching period and the preset rule matching latency index. After optimizing the number of parallel rules, a rule call time verification command is sent, prompting the designated personnel to verify the rule call time based on the adjusted rule call time after the optimization. If the rule matching latency index obtained again after the rule call time verification is less than the preset rule matching latency index, the optimization of the number of parallel rules is completed and an associated latency assessment is performed; otherwise, a rule matching latency alarm is issued. The rule call time verification measures the execution time required for the rule call to complete after the optimization of the number of parallel rules.

[0059] In this embodiment, the incremental PID (Proportional-Integral-Derivative) control algorithm deployed in the system scheduling engine calculates the proportional term based on the magnitude of the deviation using a proportional coefficient, performs an integral operation on the deviation, multiplies it by the integral coefficient to obtain the integral term, performs a differential operation on the rate of change of the deviation, multiplies it by the differential coefficient to obtain the differential term, and adds the proportional, integral, and differential terms together to finally output the adjustment value for the number of parallel matching rules. Through the synergistic effect of the three stages of proportional, integral, and differential operations, the input deviation is continuously corrected, enabling the system to respond quickly to load changes, accelerate the speed of rule matching and processing, avoid data backlog, shorten the overall data conversion time, eliminate static errors, and make the number of parallel matching rules more accurately close to the expected value, thereby further improving the system's processing efficiency and data conversion speed.

[0060] This example maps the rule matching latency index deviation to a specific parallel quantity adjustment value. The rule matching engine can obtain the information on the parallel quantity that needs to be changed, reducing the inefficiency caused by blind adjustments. In addition, the rule call time verification can actually evaluate the optimization effect and ensure that the adjustment of the parallel quantity is within a reasonable range. This improves the real-time performance of rule matching and reduces resource consumption.

[0061] like Figure 5 The flowchart shown is for data association latency evaluation and optimization. The design logic of the flowchart is as follows: the process starts by entering the data association period in the conversion period. Based on the obtained matching results and cache replacement frequency, the association latency is evaluated to determine whether the association latency evaluation result meets the standard. If it does, the process ends; otherwise, the cache block parameters are optimized.

[0062] Further understanding is needed regarding the specific process for determining whether to optimize cache block parameters: The process involves determining the numerical relationship between the obtained matching result cache replacement frequency and the preset matching result cache replacement frequency in the database. The matching result cache replacement frequency reflects the number of cache item replacements triggered by insufficient cache space during the data association process of heterogeneous government data. This frequency is obtained through monitoring by the indicator collector. The preset matching result cache replacement frequency is represented by the average of the matching result cache replacement frequencies corresponding to the end of the historical data association period in the database. If the obtained matching result cache replacement frequency is less than the preset matching result cache replacement frequency, it is recorded as data association efficiency meeting the standard, and a government information conversion monitoring instruction for the next government information conversion period is sent. The government information conversion period includes the two-way calibration period, the rule matching period, and the data association period. If the obtained matching result cache replacement frequency is not less than the preset matching result cache replacement frequency, it is recorded as data association efficiency not meeting the standard, and cache block parameter optimization is performed.

[0063] The cache block parameter optimization involves the following steps: First, based on the obtained matching result cache replacement frequency deviation, a cache block capacity adjustment value is mapped in the database. This value prompts the cache controller to adjust the cache block capacity based on the obtained value, thereby improving cache utilization and hit rate. Second, based on the obtained matching result cache replacement frequency deviation, a cache block quantity adjustment value is mapped in the database. This value prompts the cache controller to adjust the cache block quantity based on the obtained value, thereby increasing data storage space and reducing the waiting time for association operations. The matching result cache replacement frequency deviation represents the difference between the obtained matching result cache replacement frequency at the end of the data association period and the preset matching result cache replacement frequency.

[0064] After optimizing the cache block parameters, a data association time verification command is sent to prompt the preset personnel to verify the data association time based on the data association time after the optimization of the number of parallel rules. If the frequency of cache replacement of the matching result after the data association time verification is less than the preset frequency of cache replacement of the matching result, the optimization of the number of parallel rules is completed and the monitoring command for the next government information conversion period is sent. Otherwise, a rule matching delay alarm is issued. The rule call time verification is used to measure the execution time required for data association to be completed after the optimization of the number of parallel rules.

[0065] In this embodiment, the adaptive PID control algorithm in the cache controller performs integral calculations based on the cache replacement frequency deviation of the matching results, multiplies it by an integral coefficient to obtain the integral term, performs differential calculations on the rate of change of the deviation, multiplies it by a differential coefficient to obtain the differential term, and adds the proportional term, integral term, and differential term to finally output accurate cache block capacity adjustment values ​​and cache block number adjustment values ​​to improve cache hit rate. Reasonable cache block capacity and number reduce the time for data segmentation and splicing, while appropriate replacement frequency reduces frequent cache update operations, thereby shortening the average access time and improving the system response speed.

[0066] This example optimizes the number and capacity of cache blocks through dynamic adjustment of replacement frequency deviation. By adjusting cache block parameters, it increases data storage space, reduces resource waste caused by high-frequency replacement and capacity redundancy, and ensures that the data required for rule matching can be obtained quickly, thus improving real-time performance.

[0067] like Figure 6The diagram illustrates the structure of a government information conversion system based on multi-source data. The processing flow of this method may include the following steps: a two-way calibration delay evaluation module, a matching delay evaluation module, and an association delay evaluation module. The two-way calibration delay evaluation module evaluates the data cleaning process of heterogeneous government data based on the acquired two-way calibration delay data to determine whether to optimize the two-way calibration data volume. If the two-way calibration efficiency meets the standard, the heterogeneous government data conversion period is obtained. The matching delay evaluation module evaluates the matching delay of the rule matching process of heterogeneous government data based on the rule matching delay data obtained during the rule matching period to determine whether to optimize the matching delay. The association delay evaluation module evaluates the association delay of the data association process of heterogeneous government data based on the cache replacement frequency of the matching results obtained during the data association period, if the rule matching rate meets the standard, to determine whether to optimize the cache block parameters.

[0068] In this embodiment, by setting up a two-way calibration delay evaluation module, a matching delay evaluation module, and an association delay evaluation module, the delay evaluation and optimization of the government information conversion process are carried out from the three key links of data cleaning, rule matching, and data association, thereby further improving the real-time performance of government information conversion, achieving overall collaborative optimization, and realizing efficient and smooth government information conversion.

[0069] It should be added that, such as Figure 7 The image shows the main interface of a government information conversion system based on multi-source data provided in this embodiment of the invention. The left side of the interface is a navigation bar containing options such as Homepage, Business Management, Data Access, Data Status Monitoring, Data Cleaning, Two-Way Calibration, Data Association, Early Warning Processing Center, System Management, and User Services. The right side of the interface displays the main content of the system overview. The top displays the system status and indicates that five anomalies have been detected, along with the increase in calibration delay and matching delay indicators compared to yesterday. The anomaly details section lists information such as the anomaly time, calibration delay indicators, matching delay indicators, and cache replacement frequency, and provides one-click calibration and ignore operation buttons. The bottom displays key indicators such as average calibration delay, average matching delay, and optimization effect, as well as anomaly statistics, including the cumulative number of anomalies today, average processing time, and resolution rate. The overall interface layout is reasonable, and the information is clearly displayed, allowing users to quickly understand the system status and key indicators.

[0070] Specifically, Business Management is primarily responsible for coordinating various business-related data processing tasks; Data Access is responsible for introducing external data into the system; Data Status Monitoring is primarily responsible for tracking data status in real time and ensuring normal data flow; Data Cleaning is primarily responsible for purifying raw data; Two-way Calibration is primarily responsible for ensuring data consistency across different dimensions; Data Association is primarily responsible for uncovering the intrinsic relationships between data; Early Warning Processing Center is primarily responsible for responding to data anomaly alerts in a timely manner; System Management is primarily responsible for maintaining stable system operation; and User Service is primarily responsible for focusing on user-related matters. These modules collaborate to build a complete data processing system.

[0071] like Figure 8 As shown, this is a bidirectional calibration display interface in a government information conversion system based on multi-source data provided by an embodiment of the present invention. The left side of the interface is a navigation bar containing multiple functional modules such as homepage, business management, and data access. Currently, "bidirectional calibration" is selected. The top of the interface displays a prompt message indicating that the latency exceeds the standard and needs optimization, and lists the threshold ranges for the number of fill value queries, outlier checks, and I / O interactions. Below are two buttons: "Generate Scheme" and "Confirm Optimization". The middle part is the verification area, which includes verification results, verification description input boxes, and a progress bar for the current verification indicators, displaying the values ​​of fill value query efficiency, outlier check efficiency, and I / O interaction efficiency, respectively. Users can submit results or re-verify. The bottom is a historical optimization record table, which records information such as operation time, operator, operation type, latency before and after optimization, and operation results, and provides a function to view details. At the very bottom, the values ​​of three key indicators are also displayed.

[0072] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. It will be clearly understood by those skilled in the art that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0073] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for converting government information based on multi-source data, characterized in that, Includes the following steps: Step 1: Based on the acquired bidirectional calibration delay data, evaluate the bidirectional calibration delay of the data cleaning process for heterogeneous government data to determine whether to optimize the bidirectional calibration data volume. If the bidirectional calibration efficiency meets the standard, obtain the heterogeneous government data conversion period. The bidirectional calibration data volume optimization means reducing the bidirectional calibration delay of heterogeneous government data in the data cleaning process by adjusting the bidirectional calibration data volume in a single cleaning process. The heterogeneous government data conversion period includes the rule matching period and the data association period. Step 2: Based on the rule matching delay data obtained during the rule matching period, evaluate the rule matching delay of heterogeneous government data to determine whether to optimize the matching delay. The matching delay optimization means reducing the rule matching delay of heterogeneous government data in the rule matching process by adjusting the number of parallel matching rules. Step 3: If the rule matching rate meets the standard, the association delay of the heterogeneous government data association process is evaluated based on the cache replacement frequency of the matching results obtained during the data association period, so as to determine whether to optimize the cache block parameters. The cache block parameter optimization means reducing the data association delay of heterogeneous government data in the data association process by adjusting the number of cache blocks and the cache block capacity.

2. The method for converting government information based on multi-source data as described in claim 1, characterized in that, The process of evaluating the bidirectional calibration delay of heterogeneous government data cleaning based on the acquired bidirectional calibration delay data is as follows: At the end of the bidirectional calibration period, the deviation between the bidirectional calibration delay data and the preset bidirectional calibration delay data in the database is obtained. At the same time, the correction coefficient of the bidirectional calibration delay data in the database is called to correct each deviation. The results of the correction are coupled to obtain the bidirectional calibration delay index. The bidirectional calibration delay data includes the number of fill value query interactions, the number of outlier checks, and the number of I / O (Input / Output) interactions. The bidirectional calibration delay index represents the quantitative data of the impact of the bidirectional calibration delay data on the bidirectional calibration efficiency of heterogeneous government data.

3. The method for converting government information based on multi-source data as described in claim 2, characterized in that, The specific process for determining whether to perform bidirectional calibration data volume optimization is as follows: Determine the numerical relationship between the obtained bidirectional calibration delay index and the preset bidirectional calibration delay index in the database: If the obtained bidirectional calibration delay index is less than the preset bidirectional calibration delay index, it is recorded as bidirectional calibration efficiency meeting the standard and a matching delay evaluation is performed. If the obtained bidirectional calibration delay index is not less than the preset bidirectional calibration delay index, it is recorded as bidirectional calibration efficiency not meeting the standard and bidirectional calibration data volume optimization is performed.

4. The method for converting government information based on multi-source data as described in claim 3, characterized in that, The optimization of the bidirectional calibration data volume involves the following steps: The bidirectional calibration delay index deviation is mapped in the database to obtain the bidirectional calibration data volume adjustment value, which is used to prompt the distributed computing nodes to improve the bidirectional calibration efficiency based on the obtained bidirectional calibration data volume adjustment value. After optimizing the bidirectional calibration data volume, a bidirectional calibration efficiency verification command is sent, prompting the preset personnel to perform bidirectional calibration efficiency verification based on the optimized and adjusted bidirectional calibration data volume. If the bidirectional calibration delay index re-acquired after bidirectional calibration efficiency verification is less than the preset bidirectional calibration delay index, then the bidirectional calibration data volume optimization is completed and a matching delay evaluation is performed; otherwise, a bidirectional calibration delay alarm is issued.

5. The method for converting government information based on multi-source data as described in claim 1, characterized in that, The rule matching delay data obtained based on the rule matching time period is used to evaluate the matching delay of the rule matching process for heterogeneous government data. The specific process is as follows: At the end of the rule matching period, the acquired rule matching delay data is proportionally calculated with the preset rule matching delay data in the database. At the same time, the rule matching delay data correction coefficient in the database is called to correct each proportional calculation result. The corrected results are then coupled to obtain the rule matching delay index. The rule matching delay data includes the matching rule loading time, the number of matching rule traversals, and the rule version change frequency. The rule matching delay index represents the quantitative data of the degree of influence of the rule matching delay data on the rule matching delay in the heterogeneous government data conversion.

6. The method for converting government information based on multi-source data as described in claim 5, characterized in that, The specific process for determining whether to perform matching delay optimization is as follows: Determine the numerical relationship between the obtained rule matching delay index and the preset rule matching delay index in the database: If the obtained rule matching delay index is less than the preset rule matching delay index, it is recorded as the rule matching efficiency has met the standard and an association delay evaluation is performed. If the obtained rule matching delay index is not less than the preset rule matching delay index, it is recorded as rule matching efficiency not meeting the standard and rule parallel quantity optimization is performed.

7. The method for converting government information based on multi-source data as described in claim 6, characterized in that, The specific steps for optimizing the number of parallel operations in the rule are as follows: The obtained rule matching delay index deviation is mapped in the database to obtain the adjustment value of the number of parallel matching rules, which is used to prompt the rule matching engine to improve the rule matching efficiency based on the obtained adjustment value of the number of parallel matching rules. After optimizing the number of parallel rules, a rule call time verification command is sent to prompt the designated personnel to verify the rule call time based on the adjusted rule call time after optimizing the number of parallel rules. If the rule matching latency index re-obtained after rule call time verification is less than the preset rule matching latency index, then the parallel rule quantity optimization is completed and the association latency evaluation is performed; otherwise, a rule matching latency alarm is issued.

8. The method for converting government information based on multi-source data as described in claim 1, characterized in that, The specific process for determining whether to perform cache block parameter optimization is as follows: Determine the numerical relationship between the obtained matching result cache replacement frequency and the preset matching result cache replacement frequency in the database. The matching result cache replacement frequency is used to reflect the number of cache item replacements triggered by insufficient cache space during the data association process of heterogeneous government data. If the frequency of cache replacement of the obtained matching results is less than the preset frequency of cache replacement of matching results, it is recorded as the data association efficiency has met the standard and the government information conversion monitoring instruction for the next government information conversion period is sent. The government information conversion period includes the two-way calibration period, the rule matching period and the data association period. If the frequency of cache replacement of the obtained matching results is not less than the preset frequency of cache replacement of matching results, it is recorded as the data association efficiency is not up to standard and the cache block parameters are optimized.

9. The method for converting government information based on multi-source data as described in claim 8, characterized in that, The specific steps for optimizing the cache block parameters are as follows: Based on the obtained matching results, the cache replacement frequency deviation is mapped in the database to obtain a cache block capacity adjustment value, which is used to prompt the cache controller to improve cache utilization and hit rate based on the obtained cache block capacity adjustment value. Based on the obtained matching results, the cache replacement frequency deviation is mapped in the database to obtain a cache block number adjustment value, which is used to prompt the cache controller to increase data storage space and reduce the waiting time of associated operations based on the obtained cache block number adjustment value. After optimizing the cache block parameters, a data association time verification command is sent to prompt the preset personnel to verify the data association time based on the optimized and adjusted data association time according to the number of parallel rules. If the frequency of cache replacement of matching results obtained after data association verification is less than the preset frequency of cache replacement of matching results, then the optimization of the number of parallel rules is completed and the monitoring instruction for the next government information conversion period is sent; otherwise, a rule matching delay alarm is issued.

10. A system applying the government information conversion method based on multi-source data as described in any one of claims 1-9, characterized in that, include: Two-way calibration delay evaluation module, matching delay evaluation module, and correlation delay evaluation module; The bidirectional calibration delay evaluation module is used to evaluate the bidirectional calibration delay of the data cleaning process of heterogeneous government data based on the acquired bidirectional calibration delay data, so as to determine whether to optimize the bidirectional calibration data volume. If the bidirectional calibration efficiency meets the standard, the heterogeneous government data conversion period is obtained. The matching delay evaluation module is used to evaluate the matching delay of the rule matching process of heterogeneous government data based on the rule matching delay data obtained during the rule matching period, so as to determine whether to optimize the matching delay. The association delay assessment module is used to assess the association delay of heterogeneous government data based on the cache replacement frequency of the matching results obtained during the data association period if the rule matching rate meets the standard, so as to determine whether to optimize the cache block parameters.

Images