Excel data emergency reporting and processing method based on dynamic mapping
By using dynamic mapping technology to parse the Excel file structure, construct a field mapping rule base, and generate dynamic reporting templates, the accuracy and efficiency issues in emergency reporting and processing of Excel data are solved. This enables intelligent processing of multi-source heterogeneous files, improving the speed and accuracy of emergency response.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HEBEI HUAYE JIKE INFORMATION TECH CO LTD
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies suffer from low accuracy, low efficiency, and difficulty in tracing the source of Excel data in emergency reporting and processing. In particular, when processing multi-source heterogeneous Excel files, manual operation is prone to errors and lacks reusable standardized processes, resulting in delayed emergency response.
By employing dynamic mapping technology, the header and data areas are identified through structural parsing, a field mapping rule base is built, and dynamic data entry templates are generated. Combined with task distribution and data aggregation units, an automated Excel data processing workflow is achieved, including real-time validation and data aggregation.
It improves the accuracy and efficiency of emergency reporting and processing of Excel data, realizes intelligent parsing and standardized processing of multi-source heterogeneous Excel files, solves the problems of low efficiency and error-proneness in traditional manual processing methods, and improves the speed of emergency response.
Smart Images

Figure CN121580997B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to an emergency reporting and processing method for Excel data based on dynamic mapping. Background Technology
[0002] Within the framework of technical service contracts, technical engineers bear the critical responsibility of rapidly integrating client business needs with technical systems. When faced with sudden data reporting and analysis requests, technical engineers often need to manually process Excel spreadsheets from various sources to support leadership decision-making. Building an intelligent emergency data processing product can free technical engineers from tedious and repetitive data collection and organization tasks, allowing them to focus on critical tasks such as anomaly analysis. Through standardized and automated processes, the accuracy and consistency of data extraction, transformation, and aggregation are ensured, improving emergency response speed and service quality.
[0003] Currently, technical engineers rely mainly on manual operation to perform emergency Excel tasks. When faced with multi-source and heterogeneous Excel files, personnel need to manually open, copy, filter, and paste, which is prone to errors. Especially when the data volume is large and the time is tight, it is difficult to guarantee timely and accurate results.
[0004] While the above methods can achieve emergency reporting and processing of Excel data, traditional methods make it difficult to trace the source when errors occur during data processing. Furthermore, each time a similar but slightly different requirement arises, it is necessary to start from scratch. The lack of reusable standardized processes leads to delayed responses, which restricts the efficiency and professionalism of emergency services. Therefore, how to improve the accuracy and efficiency of emergency reporting and processing of Excel data has become an urgent problem to be solved. Summary of the Invention
[0005] This invention provides an emergency reporting and processing method for Excel data based on dynamic mapping and a computer-readable storage medium. Its main purpose is to improve the accuracy and efficiency of emergency reporting and processing of Excel data.
[0006] To achieve the above objectives, the present invention provides an emergency reporting and processing method for Excel data based on dynamic mapping, comprising:
[0007] Upon receiving an emergency reporting instruction, the system identifies the emergency processing system and the Excel file to be processed based on the instruction. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets.
[0008] The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one.
[0009] A field mapping rule base is constructed using dynamic mapping units and table header field sets. A dynamic filling template set is generated based on the field mapping rule base and data region set. The dynamic filling template set includes multiple dynamic filling templates.
[0010] Obtain organizational hierarchy data and reporting terminals, generate a distribution task set using the task distribution unit, organizational hierarchy data and dynamic reporting template set, and send the distribution task set to the reporting terminals.
[0011] The system acquires the backfilled dataset from the reporting terminal in real time, verifies the backfilled dataset using a field mapping rule base, obtains a verification result set, aggregates the data using a data aggregation unit based on the verification result set and the backfilled dataset, obtains a summary data table, and sends the summary data table to the initiator of the emergency reporting command to realize emergency reporting and processing of Excel data.
[0012] Optionally, the step of parsing the structure of the Excel file to be processed to obtain the header field set and data range set includes:
[0013] Perform the following operations on each worksheet in the Excel file to be processed:
[0014] Obtain the cell matrix of the worksheet, identify the set of non-empty cells in the cell matrix, where the set of non-empty cells contains multiple non-empty cells;
[0015] Identify the position coordinates of each non-empty cell in the set of non-empty cells to obtain multiple position coordinates, where each non-empty cell corresponds one-to-one with its position coordinate;
[0016] Perform connectivity analysis on multiple location coordinates to obtain one or more connected regions;
[0017] Perform the following operation on each of one or more connected regions:
[0018] The sequence of first-row cells is determined based on the connected components. This sequence contains multiple first-row cells. The following operation is performed on each first-row cell in the sequence:
[0019] Obtain the cell content and cell format characteristics of the first row cell, determine whether the cell content conforms to the preset field naming rules, and obtain the naming rule judgment result, wherein the naming rule judgment result is conforming or not conforming;
[0020] The table header confidence is calculated based on the cell format features and the preset feature score mapping rule library. The table header confidence is compared with the preset confidence threshold. If the naming rule judgment result is consistent and the table header confidence is greater than or equal to the confidence threshold, the cell content is confirmed as the table header field. The table header fields are summarized to obtain the table header field set.
[0021] Remove the first row of cells from the connected region to obtain the data range;
[0022] The data regions are aggregated to obtain a data region set.
[0023] Optionally, the step of constructing a field mapping rule base using dynamic mapping units and table header field sets includes:
[0024] Obtain the standard field library, which includes multiple standard field nodes, and each standard field node includes: standard field name, field data type, and field constraints;
[0025] The cell formulas in the scanned data area set are identified, and the formula function set is dynamically generated based on the formula function set. The verification rule node includes regular expression verification rules and data dictionary verification rules.
[0026] Perform the following operation on each header field in the header field set:
[0027] The semantic similarity between the header field and the standard field name corresponding to each standard field node in the standard field library is calculated using a pre-built semantic similarity model to obtain a similarity set. The maximum similarity is extracted from the similarity set to obtain the target similarity.
[0028] Compare the target similarity with the preset similarity threshold. If the target similarity is greater than or equal to the similarity threshold, then associate the header field, the validation rule node, and the standard field node corresponding to the target similarity to obtain the mapping rule node.
[0029] Otherwise, generate a custom field node based on the table header fields, associate the table header fields, validation rule nodes, and custom field nodes to obtain a mapping rule node;
[0030] Summarize the mapping rule nodes to obtain the field mapping rule library.
[0031] Optionally, generating a dynamic data entry template set based on the field mapping rule base and data region set includes:
[0032] Extract data regions sequentially from the data region set, and perform the following operations on each extracted data region:
[0033] Extract the data row set from the data area, and perform the following operation on each data row in the data row set:
[0034] The data types of cells in the data row are inferred by using the mapping rule nodes in the field mapping rule base to obtain the inferred type set. The number of each inferred type in the inferred type set is counted to obtain the type count set.
[0035] By summarizing the number of types, a number of regional types is obtained. Cluster analysis is then performed on the number of regional types to obtain one or more cluster type groups.
[0036] Identify the clustering group with the largest number of clustering groups in one or more clustering groups to obtain the target group. Generate a dynamic filling template based on the target group and the field mapping rule base. The dynamic filling template includes a filling range identifier.
[0037] The dynamic data entry templates are summarized to obtain a dynamic data entry template set.
[0038] Optionally, the step of generating a distribution task set using the task distribution unit, organizational hierarchy data, and dynamic reporting template set includes:
[0039] Parse the organizational hierarchy data to obtain a hierarchical node tree, which includes multiple hierarchical nodes, and each hierarchical node includes: node identifier, node level, and parent node identifier;
[0040] Perform the following operations on each dynamic data entry template in the dynamic data entry template set:
[0041] Extract the filling range identifier from the dynamic filling template, and use the filling range identifier to perform matching and retrieval in the hierarchical node tree to obtain the target hierarchical node set;
[0042] Perform the following operation on each target-level node in the target-level node set:
[0043] By associating the dynamic data entry template with the target level node, the task distribution node is obtained. The task distribution node includes the task deadline and reminder frequency.
[0044] The distribution task nodes are aggregated to obtain the distribution task set.
[0045] Optionally, the step of using a field mapping rule base to validate the backfill dataset and obtain a validation result set includes:
[0046] Perform the following operation on each backfilled data in the backfilled dataset:
[0047] Extract the backfill field value set from the backfill data, and perform the following operations on each backfill field value in the backfill field value set:
[0048] Using the header field corresponding to the backfilled field value, the corresponding mapping rule node is retrieved in the field mapping rule base. Based on the field data type in the mapping rule node, the backfilled field value is type-validated to obtain the type validation result, which is either pass or fail.
[0049] The backfill field values are constrained and validated based on the field constraints in the mapping rule node to obtain the constraint validation result, which is either pass or fail.
[0050] The backfilled field values are dynamically validated based on the validation rule nodes in the mapping rule nodes to obtain dynamic validation results, which are either pass or fail.
[0051] The field validation results are obtained by combining the association type validation results, constraint validation results, and dynamic validation results.
[0052] The data validation results are summarized to obtain the data validation result set.
[0053] Optionally, the step of aggregating data using a data aggregation unit based on the verification result set and the backfilled dataset to obtain a summary data table includes:
[0054] The valid backfill dataset is filtered out from the backfill dataset using the validation result set. The valid backfill dataset includes multiple valid backfill data, and each valid backfill data contains a data source identifier and a data timestamp. The validation results of the data corresponding to the valid backfill data are all passed.
[0055] For each valid backfill data point in the valid backfill dataset, perform the following operation:
[0056] Extract the reporting source identifier from the valid backfilled data, use the reporting source identifier to identify the source hierarchical node in the hierarchical node tree, obtain the node hierarchy and number of subordinate nodes of the source hierarchical node, and obtain the source hierarchy and number of subordinate nodes.
[0057] By associating valid backfill data, source hierarchy, and the number of source sub-nodes, hierarchical backfill nodes are obtained. By summarizing the hierarchical backfill nodes, a hierarchical backfill node set is obtained.
[0058] Sort the hierarchical backfill nodes in the hierarchical backfill node set in ascending order of source hierarchy to obtain the hierarchical backfill sequence;
[0059] The aggregate weight is calculated using a pre-constructed aggregate weight calculation formula, wherein the aggregate weight calculation formula is as follows:
[0060] ,
[0061] in, Indicates the first node in the hierarchical backfill node set. The aggregate weights corresponding to each level of backfill nodes. , These represent the preset first weighting coefficient and the preset second weighting coefficient, respectively. + =1, This represents the highest level in the hierarchical node tree. , These represent the nodes in the hierarchical backfill node set. The, the The source level corresponding to each level of backfill node. This indicates that the hierarchical backfill node set has a total of Each level of backfill node , They represent the first The, the The number of source subordinate nodes corresponding to each level of backfill node;
[0062] Summarize the aggregated weights to obtain multiple aggregated weights;
[0063] By using multiple aggregation weights to aggregate the hierarchical backfill sequence level by level, a summary data table is obtained.
[0064] Optionally, the step of using multiple aggregation weights to aggregate the hierarchical backfill sequence level by level to obtain a summary data table includes:
[0065] In the hierarchical backfill sequence, identify backfill nodes from the same source level to obtain a set of sibling backfill nodes. Perform the following operation on each sibling backfill node group in the set:
[0066] In the same-level backfill node group, identify the set of numeric field values, perform outlier detection on the set of numeric field values, and obtain the outlier identifier set;
[0067] Outlier identifiers are used to remove outlier values from the set of numeric field values to obtain a set of valid values. The mean of the valid values in the set of valid values is then calculated to obtain the mean of the same level.
[0068] Calculate the standard deviation of the effective values in the effective value set to obtain the standard deviation of the same level, and construct a confidence interval based on the mean and standard deviation of the same level.
[0069] The values of numeric fields in the same-level backfill node group are corrected using confidence intervals to obtain the corrected backfill node group;
[0070] The corrected backfill node groups are summarized to obtain the corrected backfill sequence, and a summary data table is generated based on the corrected backfill sequence and multiple aggregation weights.
[0071] Optionally, after generating the summary data table based on the corrected backfill sequence and multiple aggregation weights, the method further includes:
[0072] Task progress monitoring data is obtained based on the distributed task set and the backfilled dataset. The task progress monitoring data includes: the number of data filled, the number of data not filled, and the data filling completion rate.
[0073] The task progress monitoring data is updated in real time using a preset monitoring time interval to obtain the progress monitoring time sequence. A pre-built reporting risk prediction model is used to perform trend analysis on the progress monitoring time sequence to obtain the reporting trend prediction value and reporting risk level.
[0074] Compare the predicted trend value with the preset warning threshold. If the predicted trend value is less than the warning threshold, dynamically adjust the task deadline or reminder frequency of the corresponding task node in the task distribution set based on the risk level of the reported task. Generate a reminder message based on the task distribution set and send the reminder message to the reporting terminal that has not reported.
[0075] Data traceability records are generated based on the source identifier and timestamp of the valid backfilled data, and these records are then linked to the summary data table.
[0076] Statistical analysis reports are generated using summary data tables and hierarchical node trees, and then sent to the initiator of emergency reporting instructions.
[0077] To achieve the above objectives, the present invention also provides an emergency reporting and processing system for Excel data based on dynamic mapping, comprising:
[0078] The structure parsing module is used to receive emergency reporting instructions and, based on the emergency reporting instructions, identify the emergency processing system and the Excel file to be processed. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets.
[0079] The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one.
[0080] The mapping rule construction module is used to construct a field mapping rule library using dynamic mapping units and table header field sets, and to generate a dynamic filling template set based on the field mapping rule library and data region set, wherein the dynamic filling template set includes multiple dynamic filling templates;
[0081] The data backfilling module is used to acquire organizational-level data and filling terminals, generate a distribution task set using the task distribution unit, organizational-level data and dynamic filling template set, and send the distribution task set to the filling terminal.
[0082] The data verification module is used to acquire the backfilled dataset from the reporting terminal in real time, verify the backfilled dataset using the field mapping rule base, obtain a verification result set, and aggregate the data using the data aggregation unit based on the verification result set and the backfilled dataset to obtain a summary data table. The summary data table is then sent to the initiating end of the emergency reporting command to realize the emergency reporting and processing of Excel data.
[0083] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:
[0084] Memory, storing at least one instruction;
[0085] The processor executes the instructions stored in the memory to implement the above-described method for emergency reporting and processing of Excel data based on dynamic mapping.
[0086] To address the aforementioned issues, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned method for emergency reporting and processing of Excel data based on dynamic mapping.
[0087] To address the problems described in the background section, this invention receives an emergency reporting instruction and, based on this instruction, identifies an emergency processing system and an Excel file to be processed. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit. The Excel file to be processed contains multiple worksheets. The invention performs structural parsing on the Excel file to obtain a header field set and a data region set. The header field set includes multiple header fields, and the data region set includes multiple data regions, with each header field corresponding to a specific data region. Therefore, this invention employs a cell matrix combined with connected component analysis to automatically extract non-empty cells to form independent data. The system uses a block-based approach, combined with field naming rules to determine the table header, breaking the limitations of traditional fixed-format parsing. It adapts to Excel files from different sources, eliminating the need for manual pre-setting of header and data row positions, resulting in more accurate and efficient parsing. This provides a reliable structured data foundation for subsequent data processing. A field mapping rule library is constructed using dynamic mapping units and header field sets. Based on this rule library and data region sets, a dynamic data entry template set is generated. This dynamic data entry template set includes multiple dynamic data entry templates. Therefore, this invention achieves intelligent matching between the original header and standard fields based on a semantic similarity model, adapting to multi-source heterogeneous data. Combined with data region type clustering analysis, it automatically generates dynamic data entry that fits business characteristics. Templates eliminate the need for manual intervention in field alignment and template design, unify multi-source data field standards, improve data integration efficiency, and offer strong template adaptability. They acquire organizational-level data and reporting terminals, utilize task distribution units, organizational-level data, and dynamic reporting template sets to generate distribution task sets, which are then sent to the reporting terminals. This invention constructs a hierarchical node tree based on organizational-level data, combines it with the reporting range identifier of dynamic reporting templates, accurately matches target reporting units, automatically generates and distributes distribution task sets, and achieves intelligent association between templates and organizational levels. Task distribution is accurate and efficient, eliminating the need for manual distribution, adapts to multi-level organizational structures, and improves the task flow efficiency of emergency data reporting, enabling real-time data acquisition. The backfilled dataset from the reporting terminal is retrieved, and the dataset is validated using a field mapping rule base to obtain a validation result set. Based on the validation result set and the backfilled dataset, data aggregation is performed using a data aggregation unit to obtain a summary data table. This summary data table is then sent to the initiating end of the emergency reporting command, realizing the emergency reporting and processing of Excel data. It is evident that this invention, by introducing a dynamic mapping mechanism and a hierarchical data aggregation strategy, achieves intelligent parsing and standardized processing of multi-source heterogeneous Excel files, effectively solving the problems of low efficiency, error-proneness, and difficulty in traceability inherent in traditional manual processing methods, and improving the accuracy and response speed of emergency data reporting and processing. Therefore, this invention can improve the accuracy and efficiency of emergency reporting and processing of Excel data. Attached Figure Description
[0088] Figure 1 This is a flowchart illustrating an emergency reporting and processing method for Excel data based on dynamic mapping, provided in an embodiment of the present invention.
[0089] Figure 2 This is a functional module diagram of an Excel data emergency reporting and processing system based on dynamic mapping, provided in an embodiment of the present invention.
[0090] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the Excel data emergency reporting and processing method based on dynamic mapping, according to an embodiment of the present invention.
[0091] Explanation of reference numerals in the attached figures:
[0092] 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.
[0093] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0094] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0095] This application provides an emergency reporting and processing method for Excel data based on dynamic mapping. The executing entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the method provided in this application, such as a server or a terminal. In other words, the emergency reporting and processing method for Excel data based on dynamic mapping can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.
[0096] Reference Figure 1 The diagram shown is a flowchart illustrating an emergency reporting and processing method for Excel data based on dynamic mapping, according to an embodiment of the present invention. In this embodiment, the emergency reporting and processing method for Excel data based on dynamic mapping includes:
[0097] S1. Receive emergency reporting instructions, and confirm the emergency processing system and the Excel file to be processed based on the emergency reporting instructions. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets.
[0098] It should be explained that the emergency reporting instruction is issued by personnel who wish to report and process Excel data in an emergency. The emergency processing system is an intelligent emergency data processing product. The dynamic mapping unit is a software module used to establish the mapping relationship between the original Excel header fields and standard fields, capable of automatically identifying and matching the meaning of fields in Excel files from different sources. The task distribution unit is a software module used to allocate reporting tasks to corresponding reporting terminals according to the organizational hierarchy. The data aggregation unit is a software module used to verify, aggregate, and statistically analyze the collected reporting data. The Excel file to be processed is the original data file that needs to be processed for emergency reporting, typically originating from multiple data sources, containing multiple worksheets, each recording business data of different categories or time periods.
[0099] For example, during a special food safety inspection, the Municipal Market Supervision Administration needed to urgently collect inspection data on food production enterprises in various districts and counties. It imported an Excel file containing inspection data from 15 districts and counties into the emergency processing system. This Excel file included multiple worksheets such as a food production license statistics table, a sampling inspection results statistics table, and a violation handling statistics table. The emergency processing system then intelligently processed this Excel file. Similarly, during its annual fund audit, a medical insurance regulatory department needed to urgently compile medical insurance settlement data from various designated medical institutions. The audit department's data management terminal issued an emergency reporting command, importing an Excel file containing multiple worksheets such as a medical insurance expense details table, a treatment item statistics table, and a drug usage statistics table. Furthermore, the IT department of a large enterprise received an urgent data collection task from the group headquarters. Technical personnel needed to complete the compilation and reporting of IT asset inventory data from various branches within a specified time. By mapping the task template Excel file to the reporting system, the data collection work in the operation and maintenance service was completed in a timely, controllable, and accurate manner.
[0100] S2. Perform structural parsing on the Excel file to be processed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one.
[0101] It should be understood that the header field set is the collection of all column headers identified from the Excel file to be processed, used to describe the meaning of each column's data. The data area set is the collection of the actual data storage areas in the Excel file to be processed, excluding the headers. Because Excel files from different sources differ in format, header positions, data start rows, and blank cell distributions are all different, intelligent structural parsing of the Excel file is required to accurately identify the boundaries between the headers and the data area. Therefore, the structural parsing of the Excel file to be processed to obtain the header field set and the data area set includes:
[0102] Perform the following operations on each worksheet in the Excel file to be processed:
[0103] Obtain the cell matrix of the worksheet, identify the set of non-empty cells in the cell matrix, where the set of non-empty cells contains multiple non-empty cells;
[0104] Identify the position coordinates of each non-empty cell in the set of non-empty cells to obtain multiple position coordinates, where each non-empty cell corresponds one-to-one with its position coordinate;
[0105] Perform connectivity analysis on multiple location coordinates to obtain one or more connected regions;
[0106] Perform the following operation on each of one or more connected regions:
[0107] The sequence of first-row cells is determined based on the connected components. This sequence contains multiple first-row cells. The following operation is performed on each first-row cell in the sequence:
[0108] Obtain the cell content and cell format characteristics of the first row cell, determine whether the cell content conforms to the preset field naming rules, and obtain the naming rule judgment result, wherein the naming rule judgment result is conforming or not conforming;
[0109] The table header confidence is calculated based on the cell format features and the preset feature score mapping rule library. The table header confidence is compared with the preset confidence threshold. If the naming rule judgment result is consistent and the table header confidence is greater than or equal to the confidence threshold, the cell content is confirmed as the table header field. The table header fields are summarized to obtain the table header field set.
[0110] Remove the first row of cells from the connected region to obtain the data range;
[0111] The data regions are aggregated to obtain a data region set.
[0112] Understandably, the cell matrix is a two-dimensional array structure formed by arranging all cells in an Excel worksheet according to rows and columns. The set of non-empty cells is the collection of cells in the cell matrix that contain valid data. The position coordinates are the row and column indices used to identify the position of a cell in the worksheet, usually represented as a combination of row and column numbers. The connected component analysis is an algorithm from the field of image processing used to identify continuous data blocks formed by adjacent non-empty cells in an Excel worksheet. Optionally, four-connectivity or eight-connectivity algorithms can be used to implement connected component analysis. Specifically, the boundaries of the connected region are determined by traversing non-empty cells and checking whether their adjacent cells (up, down, left, and right) are also non-empty. The connected region is an independent data block formed by adjacent non-empty cells, identified after connected component analysis. The first row cell sequence is an ordered sequence of all cells in the top row of the connected region.
[0113] It should be understood that the cell formatting features are a set of visual features used to help determine whether a cell is a table header. These include: font style, whether it is bold, background color, border style, and cell merging status. These formatting features are typically used by the Excel file creator to highlight the header row. The feature score mapping rule base is a pre-built set of structured rules used to establish a one-to-one mapping relationship between various cell formatting features (such as font style, whether it is bold, background color, etc.) and their corresponding quantitative scores. For example, taking background color as an example: a background color without color is mapped to a lower feature score (e.g., 0.1), and a background color with color is mapped to a higher feature score (e.g., 1.0). Taking boldness as an example: boldness is mapped to a higher feature score (e.g., 0.9), and no boldness is mapped to a lower feature score (e.g., 0.2). The reason is that background colors with color and bolding are common methods for creating table headers, so boldness and background colors with color are given higher feature scores, and vice versa, thus quantifying based on visual effects. The principle for setting feature scores for other formatting features is the same as that for the formatting features "background color" and "boldness," and will not be repeated here. Optionally, the method for obtaining the feature score mapping rule base includes, but is not limited to, historical data statistical analysis and expert experience definition. The naming rule judgment result is obtained after determining whether the cell content conforms to the field naming rules, with a value of "conforms" or "does not conform." The header confidence score is a value calculated based on the cell formatting features, used to quantify the likelihood of the cell serving as a header. The calculation method is: weighted scoring of each formatting feature followed by summation, where the weight for bold is 0.3, the weight for background color is 0.25, the weight for border style is 0.2, the weight for font style is 0.15, and the weight for cell merge status is 0.1. The confidence threshold is a pre-set judgment standard used to determine whether to confirm the cell content as a header field, typically set between 0.6 and 0.8. The field naming rules are a set of rules pre-stored in the emergency handling system configuration file, used to determine whether the cell content is a valid header field. These rules include, but are not limited to, conditions such as: content being plain text, length between 2 and 20 characters, not containing only numbers, and not containing special calculation symbols. The data area is the cell area remaining after removing the first row of cells from the connected regions, containing the actual business data.
[0114] For example, suppose a market supervision administration's Excel file containing a food production license statistics worksheet has 120 rows and 60 columns. First, the cell matrix of this worksheet is obtained, identifying 458 non-empty cells. After performing connectivity analysis on the coordinates of these 458 non-empty cells, two connected regions are identified: the first connected region is located in the area from A1 to J35, and the second connected region is located in the area from L1 to P20. Taking the first connected region as an example, the first row of cells (A1 to J1, a total of 10 cells) is identified, containing the following information: company name, license number, production category, legal representative, contact number, inspection date, inspection result, rectification requirements, re-inspection date, and remarks. The cell content and formatting features of these 10 cells are obtained sequentially. The formatting features of cell A1 are: font is bold, bold is enabled, background color is colored, border style is solid line, and cell merging is disabled. Based on these formatting features and the examples above (bold feature weight 0.3, background color feature weight 0.25, border style feature weight 0.2, font style feature weight 0.15, cell merging status feature weight 0.1), the header confidence score is calculated to be 0.85. The specific calculation method is as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Each formatting feature corresponds to a feature score. For example, bold mode corresponds to a feature score of 0.9, colored background mode corresponds to a feature score of 1.0, solid border mode corresponds to a feature score of 0.7, bold font mode corresponds to a feature score of 0.9, and merged cell mode corresponds to a feature score of 0.55. Based on this example, the header confidence score is calculated to be 0.85 = (0.9 × 0.3 + 1.0 × 0.25 + 0.7 × 0.2 + 0.9 × 0.15 + 0.55 × 0.1 = 0.27 + 0.25 + 0.14 + 0.135 + 0.055). It should be clarified that the header confidence score essentially uses a numerical value to represent the probability that a cell resembles a header. In this system, the feature score corresponds to the degree to which a single formatting feature resembles a table header. For example, bold text ("Yes") scores 0.9 points, indicating that the formatting feature (bold) has a 90% probability of matching the characteristics of a table header. If the text is not bold, the score is lower, for example, 0.2 points, indicating that the formatting feature (bold) has a 20% probability of matching the characteristics of a table header. The weight represents the importance of each formatting feature. For example, setting the weight of bold text to 0.3 means that the importance of the formatting feature (bold) accounts for 30% when determining whether a cell is a table header. Multiplying the two scores converts the "probability of resembling a table header" of a single feature into an effective score according to its corresponding "importance." Then, the effective scores of all formatting features are added together to obtain the final comprehensive score. The comprehensive score represents the probability of resembling a table header that conforms to human visual judgment habits, and therefore can represent the confidence level of the table header.
[0115] Furthermore, the cell content, specifically the company name, is checked to see if it conforms to the field naming rules. Since it is plain text and has a length of 4 characters, it meets the requirements, and the naming rule judgment result is positive. Because the naming rule judgment result is positive and the header confidence score of 0.85 is greater than the confidence threshold of 0.7, the company name is confirmed as a header field and summarized into the header field set. Similarly, the other 9 first-row cells are processed to obtain the complete header field set. Subsequently, the first-row cell sequence is removed from the connected regions, resulting in the area from A2 to J35 as the data area, which is summarized into the data area set. This embodiment of the invention uses a cell matrix combined with connected component analysis to automatically extract non-empty cells to form independent data blocks. It then combines field naming rules and cell format features to dual-determine the header, improving the accuracy of header recognition in complex Excel files by integrating content and format features. This breaks the limitations of traditional fixed-format parsing, adapts to Excel files from different sources, eliminates the need for manual preset of header and data row positions, and provides a more accurate and efficient parsing process, offering a reliable structured data foundation for subsequent data processing.
[0116] S3. Construct a field mapping rule base using dynamic mapping units and table header field sets, and generate a dynamic filling template set based on the field mapping rule base and data region set, wherein the dynamic filling template set includes multiple dynamic filling templates.
[0117] It should be explained that the field mapping rule base is a set of rules used to characterize the correspondence between original header fields and standard fields, used to achieve unified processing of fields from different sources of Excel files. The dynamic data entry template set is a set of standardized data entry templates automatically generated based on the field mapping rule base and data area characteristics, used to distribute to various data entry terminals for data backfilling. It should be clarified that in multi-source heterogeneous Excel files, fields with the same meaning may have different naming conventions; therefore, it is necessary to establish a unified field correspondence through dynamic mapping. The construction of the field mapping rule base using the dynamic mapping unit and header field set includes:
[0118] Obtain the standard field library, which includes multiple standard field nodes, and each standard field node includes: standard field name, field data type, and field constraints;
[0119] The cell formulas in the scanned data area set are identified, and the formula function set is dynamically generated based on the formula function set. The verification rule node includes regular expression verification rules and data dictionary verification rules.
[0120] Perform the following operation on each header field in the header field set:
[0121] The semantic similarity between the header field and the standard field name corresponding to each standard field node in the standard field library is calculated using a pre-built semantic similarity model to obtain a similarity set. The maximum similarity is extracted from the similarity set to obtain the target similarity.
[0122] Compare the target similarity with the preset similarity threshold. If the target similarity is greater than or equal to the similarity threshold, then associate the header field, the validation rule node, and the standard field node corresponding to the target similarity to obtain the mapping rule node.
[0123] Otherwise, generate a custom field node based on the table header fields, associate the table header fields, validation rule nodes, and custom field nodes to obtain a mapping rule node;
[0124] Summarize the mapping rule nodes to obtain the field mapping rule library.
[0125] It should be understood that the standard field library is a set of standardized field definitions pre-stored in the emergency response system database, which can be pre-configured according to common business scenarios. The standard field node is the basic data unit in the standard field library, containing three attributes: a standard field name for unified naming, a field data type to specify the data format, and field constraints to limit the data value range. The formula function set is a set of Excel functions identified from cell formulas in the data range set, including but not limited to VLOOKUP, IF, SUMIF, and COUNTIF functions. The validation rule node is a data validation rule unit dynamically generated based on the formula function set, containing two types of validation rules: regular expression validation rules to verify whether the data format conforms to a specific pattern, and data dictionary validation rules to verify whether the data value is within a predefined valid value range. The regular expression validation rule is a regular expression automatically generated based on the data validation logic in the cell formula, for example, extracting the numerical range condition from the IF function and converting it into a regular expression. The data dictionary validation rule is a set of valid values automatically generated based on VLOOKUP or data validation lists in the cell formula, used to limit field values to a specified range. The semantic similarity model is a text matching model based on natural language processing technology. Optionally, a Word2Vec word vector model or a BERT pre-trained model can be used to calculate semantic similarity. The calculation principle is as follows: convert the text into vectors and calculate the cosine similarity to measure the semantic closeness between the header field and the standard field name. The similarity set is the collection of semantic similarity values between the header field and all standard field names in the standard field library. The target similarity is the maximum semantic similarity value in the similarity set. The similarity threshold is a pre-set judgment standard used to determine whether to establish a field mapping relationship, and is usually set between 0.7 and 0.9. The mapping rule node is a data structure that records the correspondence between the original header field, the validation rule node, and the standard field node or custom field node. Multi-dimensional validation of the backfilled data is achieved by associating validation rule nodes. The custom field node is a new field definition automatically generated when the header field cannot match a standard field. Its field data type is inferred from the actual data content of the corresponding column in the data area.
[0126] Specifically, the process of multi-dimensional verification of backfilled data through associated verification rule nodes is as follows (explained with an example): Assume that the standard field library of the emergency response system contains standard field nodes with the following characteristics: Standard field name: Emergency material name, Field data type: Text, Field constraints: Non-empty and length ≤ 20 characters. If the similarity threshold is 0.8, and the semantic similarity model uses the BERT pre-trained model, the header field of the Excel table submitted by a reporting unit is: Material name. The formula in the data area cell contains IF(A2="", "Null value error", A2) and VLOOKUP(A2, Material dictionary!A:A, 1, FALSE). IF(A2="", "Null value error", A2) means: If cell A2 is empty, prompt "Null value error"; otherwise, use the content in A2. VLOOKUP(A2, Material dictionary!A:A, 1, FALSE) means: Search for the content of A2 in column A of the "Material dictionary" table; if it is not found, prompt an error (FALSE). The data range cell formulas generate validation rule nodes (regular expression validation rule: ^.{1, 20}$, data dictionary validation rule: emergency supplies list). Here, ^.{1, 20}$ means: check the content length between 1 and 20 characters, and is used to check the content length. The data dictionary validation rule is used to check whether the content belongs to the supplies in the list. If the semantic similarity between the material name and the emergency supplies name is 0.85 (greater than the similarity threshold of 0.8), the mapping rule node is established as follows: Original header field [Material Name] - Validation rule node - Standard field node [Emergency Supplies Name]. When the data is filled in by the reporting terminal, if "Medical Disposable Mask" is entered, the mapping rule node is retrieved, and the content length is checked using regular expression validation rules (the content length is 7 characters, which meets the requirement of ≤20 characters), and the data dictionary validation (Medical Disposable Mask is in the emergency supplies list) is also performed. If both types of validation pass, the data is successfully filled in. If "" (empty value) is entered, the regular expression validation fails, and the feedback result is: the field cannot be empty and the length must be 1-20 characters. If "Office Computer" is entered, the data dictionary validation fails, and the feedback result is: (Office Computer) is not in the emergency supplies list. This achieves accurate validation in both format and value dimensions.
[0127] Furthermore, the specific process of automatically generating new field definitions and inferring the field data type based on the actual data content of the corresponding column in the data area is as follows: Assuming the similarity threshold is 0.8, the header field of an Excel table submitted by a reporting unit is "Material Specification Parameters". Its target similarity with all standard fields in the standard field library is 0.7 (lower than the similarity threshold of 0.8). First, "[Material Specification Parameters]" is directly used as the field name of this custom field node. Then, the data area content of the column corresponding to "[Material Specification Parameters]" in the header is retrieved: 175mm×100mm, 500mL / bottle, medical grade, etc., mixed characters that cannot be classified as integer / decimal numeric type, YYYY-MM-DD format date type, or (yes / no) Boolean type. Therefore, it is inferred that the data type of this field is: text type. If the maximum character length of the corresponding column data is 15, based on the maximum character length of 15 reserving redundant space, the basic constraint condition "text length ≤ 20 characters" is automatically generated. Finally, the field name, the inferred field data type, and the basic constraint condition are integrated to generate a complete custom field node.
[0128] Understandably, after constructing the field mapping rule base, it is necessary to generate suitable data entry templates for distribution based on the actual characteristics of the data region. Therefore, the generation of a dynamic data entry template set based on the field mapping rule base and the data region set includes:
[0129] Extract data regions sequentially from the data region set, and perform the following operations on each extracted data region:
[0130] Extract the data row set from the data area, and perform the following operation on each data row in the data row set:
[0131] The data types of cells in the data row are inferred by using the mapping rule nodes in the field mapping rule base to obtain the inferred type set. The number of each inferred type in the inferred type set is counted to obtain the type count set.
[0132] By summarizing the number of types, a number of regional types is obtained. Cluster analysis is then performed on the number of regional types to obtain one or more cluster type groups.
[0133] Identify the clustering group with the largest number of clustering groups in one or more clustering groups to obtain the target group. Generate a dynamic filling template based on the target group and the field mapping rule base. The dynamic filling template includes a filling range identifier.
[0134] The dynamic data entry templates are summarized to obtain a dynamic data entry template set.
[0135] Further, the data row set is the collection of all data rows in the data area, with each data row containing a complete business record. The type inference is the process of determining the data type of a cell based on its content characteristics. Inferred types include, but are not limited to, integer, floating-point, date, text, and boolean types. The inferred type set is the set of type identifiers obtained after performing type inference on all cells in the data row. The type quantity set is the statistical result obtained by counting the occurrences of each type in the inferred type set. The regional type quantity set is the overall type distribution statistics obtained by summarizing the type quantity sets of all data rows in the data area. The clustering analysis is an unsupervised machine learning method; optionally, the K-Means clustering algorithm can be used to achieve this purpose, grouping similar type quantity distributions into the same group by calculating the distance between data points. The clustered type group is the set of data rows with similar type distribution characteristics formed after clustering analysis. The target type group is the group containing the most data rows among all clustered type groups, representing the mainstream data structure of the data area. The dynamic data entry template is a standardized Excel template file generated based on the field type distribution and field mapping rule base of the target type group. It includes standardized table headers, data type constraints, data entry instructions, and data entry range identifiers. The data entry range identifier is a preset field in the dynamic data entry template, used to specify which levels or types of organizational units the dynamic data entry template should be distributed to.
[0136] For example, suppose a medical insurance regulatory department's standard field library contains the following standard field nodes: Medical Institution Name (text, non-empty), Number of Outpatient Visits (integer, greater than or equal to 0), Number of Inpatient Bed Days (integer, greater than or equal to 0), Name of Responsible Person (text, length 2 to 10), Contact Information (text, 11 digits), and Statistical Date (date, format yyyy-MM-dd). First, scan the cell formulas in the data range to identify the formula function set containing the IF function to determine if the number of outpatient visits is within a reasonable range and the VLOOKUP function to match the medical institution code. Based on the formula function set, dynamically generate validation rule nodes. The regular expression validation rule requires that the number of outpatient visits must be a positive integer less than or equal to 99999, and the data dictionary validation rule requires that the medical institution code must be in the predefined medical institution code table. Semantic similarity was calculated for the institution names in the header field set. The similarities with the standard field names in the standard field library were as follows: Medical Institution Name 0.91, Number of Outpatient Visits 0.12, Number of Inpatient Bed Days 0.08, Name of Responsible Person 0.06, Contact Information 0.04, and Statistical Date 0.03. The target similarity was 0.91, which is greater than the similarity threshold of 0.8. Therefore, the institution name, validation rule node, and medical institution name standard field node were associated to generate a mapping rule node. After processing other header fields in the same way, a complete field mapping rule library was obtained. Subsequently, the medical insurance treatment data region was extracted from the data region set. This region contains 36 rows of data. After type inference for each row, the number of region types was statistically analyzed and displayed.
[0137] There are 120 integer types, 200 text types, and 30 date types. After cluster analysis, target type groups are identified. Based on these target type groups and a field mapping rule base, a medical insurance treatment reporting template is generated. The template includes six standard field columns, corresponding data validation rules, and reporting range identifiers. This embodiment of the invention uses a semantic similarity model to achieve intelligent matching between the original table header and standard fields, adapting to multi-source heterogeneous data. It dynamically generates regular expression validation rules and data dictionary validation rules by scanning cell formulas, improving the accuracy of backfilled field data validation. Combined with data region type cluster analysis, it automatically generates dynamic reporting templates that fit business characteristics, eliminating the need for manual intervention in field alignment and template design. It unifies multi-source data field standards, improves data integration efficiency, and has strong template adaptability.
[0138] Understandably, the process of counting the number of each inference type in the statistical inference type set involves indiscriminately counting all data rows, resulting in the regional type set containing a large amount of data from non-core rows (such as table end total rows and intermediate comment rows). In subsequent cluster analysis, this non-core row data will affect the true type distribution pattern, causing the target type group to reflect the most frequently occurring types in the data, rather than the field types that need to be filled in for business purposes. This leads to the generated dynamic reporting template failing to correctly identify the range and type of core reporting fields, resulting in reduced accuracy in emergency reporting. Therefore, weights are introduced during cluster analysis to improve the reliability of the clustering results. Thus, the cluster analysis of the regional type set yields one or more cluster type groups, including:
[0139] Once the current emergency reporting scenario is identified, m template scenarios and m template nodes are extracted from the pre-built template scenario library. Each template scenario corresponds to a template node, and each template node includes multiple initial inference type coefficients, which correspond to inference types.
[0140] Calculate the similarity between the current emergency reporting scenario and m template scenarios to obtain m scenario similarity scores, where each scenario similarity score corresponds one-to-one with a template scenario;
[0141] Inference types are extracted sequentially from the inference type set. The inference types are used as the analysis inference types. Using the analysis inference types, the analysis inference type coefficients corresponding to the analysis inference types are retrieved in each of the m template nodes to obtain m analysis inference type coefficients. The analysis inference type coefficients correspond one-to-one with the scene similarity.
[0142] By associating m scene similarities and m analysis and inference type coefficients, m initial nodes are obtained, where each initial node contains an analysis and inference type coefficient and a scene similarity.
[0143] The type weight coefficients corresponding to the analysis and inference type are calculated using m initial nodes, wherein the type weight coefficients are as follows:
[0144] ,
[0145] in, This represents the type weighting coefficient. This indicates that there are a total of m initial nodes. This represents the analysis inference type coefficient corresponding to the p-th initial node out of m initial nodes. This represents the scene similarity corresponding to the p-th initial node among m initial nodes;
[0146] By summing the type weight coefficients, multiple type weight coefficients are obtained, where each type weight coefficient corresponds one-to-one with the inferred type.
[0147] The number of region types is extracted sequentially from the set of region type numbers, the inferred type corresponding to the number of region type numbers is identified, and the corresponding type weight coefficient is identified from multiple type weight coefficients using the inferred type, and the type weight coefficient is used as the allocation coefficient.
[0148] Multiplying the allocation coefficient by the number of region types yields the number of weight types;
[0149] Summarize the number of weight types to obtain the set of weight type counts;
[0150] Cluster analysis is performed on the set of weight types to obtain one or more cluster type groups.
[0151] Understandably, the current emergency reporting scenario refers to the specific business scenario in which emergency data reporting is initiated (such as flood disaster statistics or epidemic case reporting). The template scenario library is a database storing historically mature emergency reporting scenarios and corresponding configuration information, used to match the current scenario. Optionally, the template scenario library can be obtained based on historical business scenario data. The template scenario refers to historical emergency reporting business scenarios stored in the template scenario library. The initial inference type coefficient is a preset basic weight value in the template node corresponding to each data type (such as integer and text). The scenario similarity is the degree of matching between the current emergency reporting scenario and each template scenario in the template scenario library.
[0152] For example, assuming the current emergency reporting scenario is flood disaster statistics, the template scenario library contains 3 template scenarios (m=3): Template Scenario 1 (historical flood disaster statistics), Template Scenario 2 (epidemic case reporting), and Template Scenario 3 (work safety accident statistics). The 3 template scenarios correspond to 3 template nodes. First, integer types are extracted from the inference type set as the analysis inference type. The analysis inference type coefficients corresponding to the integer types are retrieved from the 3 template nodes, with B1=1.0, B2=0.9, and B3=0.8 respectively. At the same time, the similarity between the current flood disaster statistics scenario and the 3 template scenarios is calculated as R1=0.95, R2=0.4, and R3=0.3. Then, each BP and RP are associated to obtain 3 initial nodes: Initial Node 1 (B1=1.0, R1=0.95), Initial Node 2 (B2=0.9, R2=0.4), and Initial Node 3 (B3=0.8, R3=0.3). The type weight coefficient corresponding to the integer type is calculated to be approximately 0.939. Similarly, the calculation process for the type weight coefficients corresponding to other inference types is the same as that for the type weight coefficients corresponding to integer types, and will not be repeated here.
[0153] It should be understood that the type weight coefficients are weight values assigned to different inference types, used to adjust the importance of each inference type in cluster analysis. The number of weighted types is the result of weighting the number of region types with their corresponding type weight coefficients, used to more accurately reflect the field types that actually need attention in the data region.
[0154] For example, suppose the set of region types is: 120 integer types, 200 text types, and 30 date types. If the type weight coefficients are calculated as follows (e.g., 1.0 for integer types, 0.7 for text types, and 0.9 for date types), the set of weighted type numbers is: 120 integer types, 140 text types, and 27 date types. It is evident that the number of weighted type numbers minus 120 (integer types) and 140 (text types) is significantly greater than the number of weighted type numbers minus 27 (date types). Performing cluster analysis (e.g., K-Means) on the set of weighted type numbers yields two cluster groups: Cluster Group A contains high-weighted integer and text types, and Cluster Group B contains low-weighted date types.
[0155] S4. Obtain organizational hierarchy data and reporting terminal, generate a distribution task set using the task distribution unit, organizational hierarchy data and dynamic reporting template set, and send the distribution task set to the reporting terminal.
[0156] It should be explained that the organizational hierarchy data is structured data describing the hierarchical relationship between reporting units, typically obtained from the organizational management module of the emergency response system or an external organizational structure interface. The task distribution set is a set of tasks assigned by associating dynamic reporting templates with target reporting units. The reporting terminal is a client device or application used by each reporting unit to receive tasks, fill in data, and submit the results. Since emergency data reporting usually involves multiple levels of organizational units, it is necessary to accurately distribute reporting tasks to the corresponding reporting terminals according to the organizational structure. Therefore, the generation of the task distribution set using the task distribution unit, organizational hierarchy data, and dynamic reporting template set includes:
[0157] Parse the organizational hierarchy data to obtain a hierarchical node tree, which includes multiple hierarchical nodes, and each hierarchical node includes: node identifier, node level, and parent node identifier;
[0158] Perform the following operations on each dynamic data entry template in the dynamic data entry template set:
[0159] Extract the filling range identifier from the dynamic filling template, and use the filling range identifier to perform matching and retrieval in the hierarchical node tree to obtain the target hierarchical node set;
[0160] Perform the following operation on each target-level node in the target-level node set:
[0161] By associating the dynamic data entry template with the target level node, the task distribution node is obtained. The task distribution node includes the task deadline and reminder frequency.
[0162] The distribution task nodes are aggregated to obtain the distribution task set.
[0163] It should be understood that the hierarchical node tree is a tree-shaped data structure formed after parsing organizational hierarchy data, which can clearly express the hierarchical relationship between organizational units. The hierarchical node is the basic data unit in the hierarchical node tree, containing three attributes: a node identifier to uniquely identify the organizational unit, a node level to indicate the organizational unit's hierarchical depth in the organizational structure, and a parent node identifier to point to the direct parent organization of the organizational unit. The reporting scope identifier is a preset field in the dynamic reporting template, used to specify which levels or types of organizational units the dynamic reporting template should be distributed to. The target hierarchical node set is the collection of all hierarchical nodes in the hierarchical node tree that match the reporting scope identifier. The distribution task node is a data structure that records the association between the dynamic reporting template and the target hierarchical nodes, containing attributes such as task number, template identifier, target node identifier, distribution time, task deadline, and reminder frequency. The task deadline is a preset reporting deadline point in the distribution task node, used to require the reporting terminal to complete data submission before this time. The reminder frequency is a preset reminder interval in the task distribution node, used to control the frequency of sending reminder messages to terminals that have not submitted reports.
[0164] For example, suppose the organizational hierarchy data of a large enterprise group describes the organizational structure of the group's information management system. After parsing, a hierarchical node tree is obtained, which includes the following hierarchical nodes: Group Information Center (node identifier 001, node level 1, parent node identifier empty), North China IT Department (node identifier 002, node level 2, parent node identifier 001), South China IT Department (node identifier 003, node level 2, parent node identifier 001), Beijing Branch Operation and Maintenance Group (node identifier 004, node level 3, parent node identifier 002), Tianjin Branch Operation and Maintenance Group (node identifier 005, node level 3, parent node identifier 002), etc., totaling 42 hierarchical nodes. Technical engineers received an urgent task from headquarters to collect IT asset inventory data. They extracted the reporting scope identifier (Level 3) from the IT asset inventory reporting template, and after matching and searching in the hierarchical node tree, obtained the target hierarchical node set, containing 28 branch office operations and maintenance group hierarchical nodes. They then sequentially associated the IT asset inventory reporting template with each target hierarchical node, generating 28 distribution task nodes. Each distribution task node had a deadline of 48 hours and a reminder frequency of once every 4 hours. After aggregation, the distribution task set was obtained and subsequently sent to the corresponding reporting terminals of the 28 branch office operations and maintenance groups through the task distribution unit, realizing the function of mapping the task template Excel to the reporting system. This embodiment of the invention constructs a hierarchical node tree based on organizational hierarchical data, combined with the reporting scope identifier of the dynamic reporting template, accurately matches the target reporting unit, automatically generates and distributes the distribution task set, and achieves intelligent association between the template and organizational hierarchy. Task distribution is accurate and efficient, eliminating the need for manual distribution, adapting to multi-level organizational structures, and improving the task flow efficiency of emergency data reporting.
[0165] S5. Real-time acquisition of the backfilled dataset from the reporting terminal, validation of the backfilled dataset using the field mapping rule base, obtaining a validation result set, and data aggregation using the data aggregation unit based on the validation result set and the backfilled dataset to obtain a summary data table. The summary data table is then sent to the initiating end of the emergency reporting command to realize emergency reporting and processing of Excel data.
[0166] Understandably, the backfilled dataset is the data set filled in and submitted by each reporting terminal according to the dynamic reporting template. The verification result set is the verification result set obtained after verifying the backfilled dataset, used to determine whether the backfilled data conforms to the data types and constraints defined in the field mapping rule base. The summary data table is a statistical result table generated after aggregating and calculating all valid backfilled data according to organizational hierarchy. The initiating end is the terminal device or user account that issues the emergency reporting command. Because the backfilled data may have problems such as format errors, missing data, or logical contradictions, it needs to be strictly verified before aggregation. Therefore, the verification of the backfilled dataset using the field mapping rule base to obtain the verification result set includes:
[0167] Perform the following operation on each backfilled data in the backfilled dataset:
[0168] Extract the backfill field value set from the backfill data, and perform the following operations on each backfill field value in the backfill field value set:
[0169] Using the header field corresponding to the backfilled field value, the corresponding mapping rule node is retrieved in the field mapping rule base. Based on the field data type in the mapping rule node, the backfilled field value is type-validated to obtain the type validation result, which is either pass or fail.
[0170] The backfill field values are constrained and validated based on the field constraints in the mapping rule node to obtain the constraint validation result, which is either pass or fail.
[0171] The backfilled field values are dynamically validated based on the validation rule nodes in the mapping rule nodes to obtain dynamic validation results, which are either pass or fail.
[0172] The field validation results are obtained by combining the association type validation results, constraint validation results, and dynamic validation results.
[0173] The data validation results are summarized to obtain the data validation result set.
[0174] It should be explained that the backfilled data refers to one or more business data records submitted by a single reporting terminal. The backfilled field value set is the collection of all field values in the backfilled data. The backfilled field value is the specific value of a certain field in the backfilled data. The type validation is the process of determining whether the backfilled field value conforms to the field data type defined in the mapping rule node, such as determining whether a numeric field is a valid number, or whether a date field is a valid date format. The type validation result is the judgment result of the type validation, which is either pass or fail. The constraint validation is the process of determining whether the backfilled field value meets the field constraint conditions defined in the mapping rule node, such as determining whether the value is within the allowed range, whether the text length meets the requirements, or whether it is a required field. The constraint validation result is the judgment result of the constraint validation, which is either pass or fail. The dynamic validation is a multi-dimensional validation process based on the validation rule node in the mapping rule node for the backfilled field value, including using regular expression validation rules to verify whether the data format conforms to a specific pattern, and using data dictionary validation rules to verify whether the data value is within a predefined valid value range. The dynamic validation result is the judgment result of the dynamic validation, which is either pass or fail. The field validation result is the final validation result of a single field obtained by combining the type validation result, constraint validation result, and dynamic validation result. The field validation result is considered passed only if all three validations (type validation, constraint validation, and dynamic validation) pass. The data validation result is the summary of all field validation results in a single backfilled data entry. The data validation result is considered passed only if all field validation results pass.
[0175] Furthermore, after data validation is completed, the valid data needs to be aggregated. Based on the validation result set and the backfilled dataset, the data aggregation unit is used to aggregate the data to obtain a summary data table, including:
[0176] The valid backfill dataset is filtered out from the backfill dataset using the validation result set. The valid backfill dataset includes multiple valid backfill data, and each valid backfill data contains a data source identifier and a data timestamp. The validation results of the data corresponding to the valid backfill data are all passed.
[0177] For each valid backfill data point in the valid backfill dataset, perform the following operation:
[0178] Extract the reporting source identifier from the valid backfilled data, use the reporting source identifier to identify the source hierarchical node in the hierarchical node tree, obtain the node hierarchy and number of subordinate nodes of the source hierarchical node, and obtain the source hierarchy and number of subordinate nodes.
[0179] By associating valid backfill data, source hierarchy, and the number of source sub-nodes, hierarchical backfill nodes are obtained. By summarizing the hierarchical backfill nodes, a hierarchical backfill node set is obtained.
[0180] Sort the hierarchical backfill nodes in the hierarchical backfill node set in ascending order of source hierarchy to obtain the hierarchical backfill sequence;
[0181] The aggregate weight is calculated using a pre-constructed aggregate weight calculation formula, wherein the aggregate weight calculation formula is as follows:
[0182] ,
[0183] in, Indicates the first node in the hierarchical backfill node set. The aggregate weights corresponding to each level of backfill nodes. , These represent the preset first weighting coefficient and the preset second weighting coefficient, respectively. + =1, This represents the highest level in the hierarchical node tree. , These represent the nodes in the hierarchical backfill node set. The, the The source level corresponding to each level of backfill node. This indicates that the hierarchical backfill node set has a total of Each level of backfill node , They represent the first The, the The number of source subordinate nodes corresponding to each level of backfill node;
[0184] Summarize the aggregated weights to obtain multiple aggregated weights;
[0185] By using multiple aggregation weights to aggregate the hierarchical backfill sequence level by level, a summary data table is obtained.
[0186] It should be understood that the valid backfill dataset is a collection of backfill data that has passed verification and filtering. The reporting source identifier is a field value in the valid backfill data used to identify the data submitting unit, corresponding to the node identifier in the hierarchical node tree. The reporting timestamp is a field value in the valid backfill data that records the data submission time, used to trace the data submission point. The source hierarchical node is the hierarchical node in the hierarchical node tree that matches the reporting source identifier. The source hierarchy is the node hierarchy attribute value of the source hierarchy node, representing the hierarchical depth of the reporting unit in the organizational structure. The number of source subordinate nodes is the number of subordinate nodes attribute value of the source hierarchy node, representing the total number of subordinate organizational units directly or indirectly managed by the reporting unit. The hierarchical backfill node is a data structure formed by associating the valid backfill data, source hierarchy, and number of source subordinate nodes. The hierarchical backfill sequence is an ordered set of hierarchical backfill nodes sorted from low to high source hierarchy, facilitating subsequent step-by-step aggregation calculations. The first weighting coefficient is the contribution ratio of hierarchical factors to the aggregation weight, used to quantify the influence of the source hierarchy depth of the backfill node on its weight. The second weighting coefficient is the contribution ratio of the number of source subordinate nodes to the aggregation weight, used to quantify the influence of the size of the subordinate nodes of the backfill node on its weight. Furthermore, the aggregation weight is a weighting coefficient used during data aggregation to assign different importance to data from different source hierarchies; data with higher hierarchies or more subordinate nodes typically has a higher aggregation weight. The number of source subordinate nodes is the total number of subordinate nodes directly or indirectly managed by a node at a certain level in the hierarchical node tree.
[0187] In detail, during data aggregation, outlier handling and statistical correction are required for data at the same level. Therefore, the method of using multiple aggregation weights to aggregate the hierarchical backfill sequence level by level yields a summary data table, including:
[0188] In the hierarchical backfill sequence, identify backfill nodes from the same source level to obtain a set of sibling backfill nodes. Perform the following operation on each sibling backfill node group in the set:
[0189] In the same-level backfill node group, identify the set of numeric field values, perform outlier detection on the set of numeric field values, and obtain the outlier identifier set;
[0190] Outlier identifiers are used to remove outlier values from the set of numeric field values to obtain a set of valid values. The mean of the valid values in the set of valid values is then calculated to obtain the mean of the same level.
[0191] Calculate the standard deviation of the effective values in the effective value set to obtain the standard deviation of the same level, and construct a confidence interval based on the mean and standard deviation of the same level.
[0192] The values of numeric fields in the same-level backfill node group are corrected using confidence intervals to obtain the corrected backfill node group;
[0193] The corrected backfill node groups are summarized to obtain the corrected backfill sequence, and a summary data table is generated based on the corrected backfill sequence and multiple aggregation weights.
[0194] Understandably, the peer-level backfill node group is a set of groups formed by grouping hierarchical backfill nodes with the same source level in the hierarchical backfill sequence. The peer-level backfill node group is a single group within the peer-level backfill node group set, containing all hierarchical backfill nodes with the same source level. The numerical field value set is the set of values for all numerical fields in the peer-level backfill node group. Outlier detection is the process of identifying abnormal data in the numerical field value set that deviates from the normal range. Optionally, outlier detection can be achieved using box plots or the 3σ principle. Box plots determine outlier boundaries by calculating quartiles, while the 3σ principle marks data deviating from the mean by more than three times the standard deviation as outliers by calculating the mean and standard deviation. The outlier identifier set is the set of outlier data location identifiers obtained after outlier detection. The valid value set is the set of normal values remaining after removing outliers. The peer mean is the arithmetic mean of all valid values in the valid value set. The peer standard deviation is a measure of the dispersion of all valid values in the valid value set relative to the peer mean. The confidence interval is a range of values that can be allowed based on the mean and standard deviation of the same level. It typically uses a 95% confidence level and is calculated as follows: the lower limit of the confidence interval equals the mean of the same level minus 1.96 times the standard deviation of the same level, and the upper limit of the confidence interval equals the mean of the same level plus 1.96 times the standard deviation of the same level. The corrected backfill node group is a backfill node group obtained by boundary correction of numerical field values that exceed the range using the confidence interval. The corrected backfill sequence is an ordered data sequence formed by summarizing all corrected backfill node groups.
[0195] For example, suppose a market supervision administration receives 64 backfill data records from 32 grassroots regulatory offices' reporting terminals. Each backfill data record is validated. In one backfill data record submitted by a regulatory office, the number of inspected enterprises field has a negative value. The type validation passes, but the constraint validation fails; therefore, this backfill data fails validation. Another backfill data record has an enterprise type field value outside the valid value range defined by the data dictionary validation rules; this dynamic validation fails, and the data validation result for this backfill data also fails. After validation, a valid backfill dataset containing 60 valid backfill data records is selected. Further, the reporting source identifier of each valid backfill data record is extracted sequentially. The source level node is identified in the hierarchical node tree, and the source level and the number of its subordinate nodes are obtained. After associating the valid backfill data, source level, and the number of its subordinate nodes to generate hierarchical backfill nodes, they are sorted by source level to obtain a hierarchical backfill sequence. Assuming the maximum level of the hierarchical node tree is 3, the first weight coefficient is 0.6, and the second weight coefficient is 0.4, for a grassroots regulatory office hierarchical backfill node with a source level of 3 and 0 source subordinate nodes, its aggregate weight is calculated as follows: 0.6 multiplied by (3 minus 3 plus 1) divided by the sum of hierarchical weights, plus 0.4 multiplied by 0 divided by the sum of source subordinate nodes, resulting in a lower aggregate weight. In the hierarchical backfill sequence, the same-level backfill node group of the 3rd level is identified. The numerical field value set of the number of inspected enterprises is extracted as (156, 142, 168, 175, 892, 151, 163, 148), a total of 32 values. Using the 3σ principle for outlier detection, the calculated mean is approximately 180 and the standard deviation is approximately 185. Value 892 is identified as an outlier and removed, resulting in the valid value set. The mean of the effective numerical set within each category is calculated to be 158, and the standard deviation within each category is 12. A 95% confidence interval is constructed as (134, 182). After boundary correction for values exceeding the confidence interval, a corrected backfill node group is obtained. Finally, a summary data table is generated based on the corrected backfill sequence and aggregated weights, showing that the total number of food production enterprises inspected in the city is 4896.
[0196] It should be explained that after the data aggregation is completed, the data entry progress also needs to be monitored and statistical reports generated. Therefore, after generating the summary data table based on the corrected backfill sequence and multiple aggregation weights, the process also includes:
[0197] Task progress monitoring data is obtained based on the distributed task set and the backfilled dataset. The task progress monitoring data includes: the number of data filled, the number of data not filled, and the data filling completion rate.
[0198] The task progress monitoring data is updated in real time using a preset monitoring time interval to obtain the progress monitoring time sequence. A pre-built reporting risk prediction model is used to perform trend analysis on the progress monitoring time sequence to obtain the reporting trend prediction value and reporting risk level.
[0199] Compare the predicted trend value with the preset warning threshold. If the predicted trend value is less than the warning threshold, dynamically adjust the task deadline or reminder frequency of the corresponding task node in the task distribution set based on the risk level of the reported task. Generate a reminder message based on the task distribution set and send the reminder message to the reporting terminal that has not reported.
[0200] Data traceability records are generated based on the source identifier and timestamp of the valid backfilled data, and these records are then linked to the summary data table.
[0201] Statistical analysis reports are generated using summary data tables and hierarchical node trees, and then sent to the initiator of emergency reporting instructions.
[0202] Furthermore, the task progress monitoring data is statistical data reflecting the execution status of the data entry task. The number of tasks that have been entered is the number of tasks in the distribution task set that have received the data. The number of tasks that have not been entered is the number of tasks in the distribution task set that have not yet received the data. The data entry completion rate is the ratio of the number of entered tasks to the total number of distribution tasks, expressed as a percentage. The monitoring time interval is a manually set time difference used to control the update frequency of the task progress monitoring data. In the early stages of data entry, there may be situations where multiple parties have not yet entered data, so frequent monitoring is unnecessary and would waste resources. In the final stages of data entry, when data entry is close to completion, the monitoring frequency can be increased to obtain the remaining data in a timely manner, achieving a precise match between the monitoring frequency and the data entry progress. Therefore, the preset monitoring time interval includes:
[0203] The monitoring time interval is calculated based on the reported quantity and the unreported quantity, wherein the monitoring time interval is as follows:
[0204] ,
[0205] in, Indicates the monitoring time interval, This indicates the quantity that has been reported. This indicates the quantity that was not reported. This indicates the preset maximum time interval. This indicates the preset minimum time interval.
[0206] It is understood that the maximum time interval is the longest allowed monitoring period in emergency data reporting and monitoring. The minimum time interval is the shortest allowed monitoring period in emergency data reporting and monitoring. Optionally, the maximum and minimum time intervals can be set according to the emergency scenario.
[0207] It should be understood that the calculation principle of the monitoring time interval is as follows: using the proportion of unreported units to the total number of reporting terminals as the core adjustment factor, the monitoring time interval is controlled within a certain range. and In the meantime, when there are a large number of unreported entries, Approaching 1, the monitoring time interval approaches 1. To reduce the frequency of invalid monitoring in the initial stage, as the number of reported data increases, As the value gradually decreases, the monitoring time interval smoothly decreases and approaches a certain value. During the final stage of data entry, the monitoring frequency is increased to ensure the timeliness of task progress monitoring data updates. This method of dynamically adjusting the monitoring time interval can simultaneously ensure system operational stability and data collection timeliness, eliminating the need for manual adjustment of the monitoring cycle and improving the automation and intelligence level of emergency data reporting and monitoring.
[0208] Furthermore, the progress monitoring time series is a sequence of task progress monitoring data recorded according to monitoring time intervals, reflecting the trend of the completion rate over time. The reporting risk prediction model is a machine learning model built based on the completion rate and delay data of historical emergency reporting tasks, used to predict the reporting risk of the current task. Optionally, random forest or gradient boosting decision tree algorithms can be used to build the reporting risk prediction model. The model input features include: the average completion rate of similar tasks in history, the average delay time of similar tasks in history, the number of reports already submitted for the current task, and the remaining time for the current task. The trend analysis is a process of statistically analyzing the progress monitoring time series to predict future reporting progress. Optionally, linear regression or exponential smoothing can be used to implement trend analysis, predicting the completion rate before the deadline by fitting historical data points. The reporting trend prediction value is the predicted completion rate obtained from the trend analysis. The reporting risk level is the risk classification result output by the reporting risk prediction model, usually divided into three levels: low risk, medium risk, and high risk, used to guide subsequent task adjustment strategies. The warning threshold is a manually set judgment standard used to determine whether a reporting reminder is triggered, typically set to 80% to 95%. The reporting reminder message is an automatically generated notification containing information such as the task name, deadline, and reporting requirements. The data traceability record is traceability information generated based on the reporting source identifier and reporting timestamp in the valid backfilled data, used to record the source unit and submission time of each data entry, achieving data traceability. The statistical analysis report is a visual report generated based on the summary data table and hierarchical node tree, including a data summary table, hierarchical distribution chart, trend analysis chart, etc.
[0209] For example, suppose a technical engineer from a corporate group is responsible for an emergency data collection task for IT asset inventory. The task distribution set contains 28 task nodes, and the task progress monitoring data is updated every 10 minutes. 30 minutes after the task is issued, 9 data points have been submitted, 19 have not been submitted, and the completion rate is 32.1%. 60 minutes after the task is issued, 16 data points have been submitted, 12 have not been submitted, and the completion rate is 57.1%. A data collection risk prediction model built based on historical emergency reporting task data is used to analyze the trend of the progress monitoring timeline. This model comprehensively considers characteristics such as the average completion rate of 85% and the average data collection delay time of 15 minutes for similar historical IT asset inventory tasks. The predicted data collection trend at the deadline (48 hours after task issuance) is 76%, which is less than the warning threshold of 90%, and the output data collection risk level is medium risk. Based on the medium-risk level, the system dynamically adjusted the reminder frequency for unreported task nodes in the task distribution center from once every 4 hours to once every 2 hours, automatically generating reminder messages and sending them to the 12 unreported terminals. Before the deadline, all reporting terminals completed data submission. Data traceability records were generated based on the source identifier and timestamp of the valid submitted data, and these records were linked to the summary data table, ensuring traceability of the source of each data entry. Statistical analysis reports were generated based on the summary data table and hierarchical node tree. These reports displayed statistical information such as the total amount of IT assets in each region of the group, the asset distribution of each branch, and year-on-year changes. The statistical analysis reports were then sent to the management system of the group's information center, enabling operations and maintenance personnel to complete the data collection work in a timely, controllable, and accurate manner during operations and maintenance services. This invention, through the introduction of a dynamic mapping mechanism and a hierarchical data aggregation strategy, achieves intelligent parsing and standardized processing of multi-source heterogeneous Excel files. It improves the accuracy of data verification by dynamically generating verification rules through scanning cell formulas, realizes risk warning and dynamic task adjustment by constructing a reporting risk prediction model based on historical data, and achieves data traceability through reporting source identifiers and reporting timestamps. It effectively solves the problems of low efficiency, error-proneness and difficulty in traceability in traditional manual processing methods, and improves the accuracy and response speed of emergency data reporting and processing.
[0210] Understandably, when enterprises conduct important data statistics, this important data usually involves the core management of the enterprise. If the final statistical analysis report is leaked during the transmission to the initiating end, it will have a serious negative impact on the enterprise's management. Therefore, before sending the statistical analysis report to the initiating end of the emergency reporting instruction, the following steps are also included:
[0211] A valid backfill data is randomly extracted from the valid backfill dataset, and the filling timestamp is extracted from the valid backfill data. The filling timestamp is then converted into a filling time value using a preset conversion method.
[0212] The monitoring values are calculated based on the monitoring time interval;
[0213] The minimum screening value is calculated based on the reporting time, monitoring data, number of reported data, and number of unreported data. The formula for calculating the minimum screening value is as follows:
[0214] ,
[0215] in, This represents the minimum filtering value. This indicates the reported time value. This indicates the monitored value. This indicates the quantity that has been reported. This indicates the quantity that was not reported. Represents the smallest identifier;
[0216] If the minimum screening value is less than the preset screening threshold, then the number of filled entries is incremented by one to obtain the updated number of filled entries. The updated number of filled entries is then used as the number of filled entries, and the step of calculating the minimum screening value based on the filling time value, monitoring value, number of filled entries, and number of unfilled entries is returned until the minimum screening value is greater than or equal to the screening threshold.
[0217] The maximum filtering value is calculated based on the reporting time value, monitoring value, number of reported data, and number of unreported data, wherein the maximum filtering value is as follows:
[0218] ,
[0219] in, This represents the maximum filter value. This indicates the reported time value. This indicates the monitored value. This indicates the quantity that has been reported. This indicates the quantity that was not reported. Represents the largest identifier;
[0220] Construct a filtering interval based on the minimum and maximum filtering values, and identify multiple filtering values from the filtering interval, where all filtering values are positive integers;
[0221] Two filter values are extracted from the filter intervals, wherein the two filter values are not equal;
[0222] Use a preset prime number determination method to determine whether two filter values are prime numbers;
[0223] If both filter values are prime numbers, calculate the public key using the two filter values; otherwise, return to the step of extracting the two filter values from the filter interval until both extracted filter values are prime numbers. Calculate the key using the two filter values and record the key generation time. The key includes a public key and a private key, and the key has a preset validity period.
[0224] An initial statistical analysis report is generated using the summary data table and hierarchical node tree. The initial statistical analysis report is then encrypted using the public key to obtain the final statistical analysis report.
[0225] It should be explained that the reported time value is obtained by converting the reported timestamp using the conversion method. Optionally, the specific conversion process is as follows: For example, if the reported timestamp is 2025-11-01-14:30:00, the corresponding value is directly extracted, resulting in the reported time value: 20251101143000. Because the reported timestamp is unique, the reported time value corresponding to the reported timestamp is used as the minimum filtering value. The specific process of calculating the monitoring value based on the monitoring time interval is as follows: the time monitoring interval is converted into a time length in seconds, and the value in the time length is extracted as the monitoring value. Because it is frequency conversion monitoring, this value at the current moment can represent one of the specific situations in the current time period. Therefore, the monitoring value is used as a parameter to calculate the minimum filtering value. For example, assuming the monitoring time interval is 30 minutes, the monitoring time value is 1800 (30 multiplied by 60). The minimum screening value is a lower limit reference value obtained by exponential operation based on the reporting time value, monitoring value, number of reported data, and number of unreported data. It is used to dynamically determine the screening range of encryption parameters. The maximum screening value is an upper limit reference value obtained by multiplication and exponential operation based on the reporting time value, monitoring value, number of reported data, and number of unreported data. Together with the minimum screening value, it defines the candidate value range of encryption parameters. This candidate value range is the screening range. The screening range is a set of positive integers between the minimum and maximum screening values, serving as the range of prime numbers required for the encryption algorithm. The screening threshold is a pre-set numerical threshold used to determine whether the minimum screening value meets the standard (the minimum screening value is greater than the screening threshold). The reason is that when the minimum screening value is greater than or equal to the screening threshold, it indicates that the complexity of the dynamic parameter combination composed of the reporting time value, monitoring value, number of reported data, and number of unreported data is large enough, making the prime numbers selected from the screening range large enough, thus giving the generated public and private keys stronger resistance to attacks. The purpose of the increment operation is to ensure that the minimum screening value is greater than the screening threshold, so that the prime number used to calculate the key is sufficiently large. Specifically, the increment operation is used instead of reconstructing the dynamic parameter set because the goal of the calculation is to make the minimum screening value greater than the screening threshold. Using a reselected dynamic parameter set (reporting time value, monitoring value, number of reported data, and number of unreported data) might cause the recalculated minimum screening value to become smaller. Therefore, the increment operation ensures that the minimum screening value gradually increases above the screening threshold. The prime number determination method is a rule used to determine whether a screening value is a prime number. The core of the rule is to check whether the screening value can only be divided by 1 and itself, and has no other positive factors. Optionally, the prime number determination method includes, but is not limited to, Fermat primality tests, AKS primality tests, etc.
[0226] Understandably, the key is a core credential used for data encryption and decryption, generated based on two selection values (prime numbers), and comprises two related parts: a public key and a private key. The public key is the public portion of an asymmetric encryption key calculated using two prime numbers, used to encrypt the initial statistical analysis report. The initial statistical analysis report is the statistical report before encryption. The private key is a key generated by pairing the public key with the private key, used to decrypt the statistical analysis report. The prime numbers are positive integers used to generate the public and private keys. Optionally, the technique of using two prime numbers to calculate the public and private keys is existing technology and will not be elaborated upon here. The validity period is a preset time interval for key availability, calculated from the time the key is generated; the key becomes invalid after this interval. For example, if the key is generated at 10:30:01, then 10:30:01 is the generation time. For example, a validity period of 2 hours indicates that the key's available time interval is from 10:30:01 to 12:30:01. The purpose of setting an expiration period is to avoid the risk of leakage due to long-term use, reduce key management risks, improve key security, and automatically generate a temporary key associated with business parameters after the key expires, ensuring that decryption operations are not interrupted and suitable for handling emergency situations.
[0227] Furthermore, after the initiating end of the emergency reporting instruction receives the statistical analysis report, it needs to decrypt the report to obtain the decrypted version. Decryption requires the use of the private key corresponding to the public key. Therefore, after sending the statistical analysis report to the initiating end of the emergency reporting instruction, the process also includes:
[0228] Once the decryption time is determined, the absolute difference between the decryption time and the generation time is calculated to obtain the absolute time difference;
[0229] Compare the absolute time difference with the effective duration;
[0230] If the absolute time difference is less than or equal to the effective duration, the statistical analysis report is decrypted using the private key to obtain the decrypted statistical analysis report.
[0231] If the absolute time difference is greater than the effective duration, the decryption time is converted into a decryption time value using the aforementioned conversion method.
[0232] Two temporary prime numbers are calculated based on the decryption time value, monitoring value, number of reported data, number of unreported data, and the screening threshold.
[0233] A temporary key is calculated using two temporary prime numbers. The temporary key includes a temporary public key and a temporary private key. The time of temporary key generation is recorded, and the temporary key has a preset validity period.
[0234] The initial statistical analysis report is encrypted using the temporary public key to obtain a temporary statistical analysis report.
[0235] The temporary decryption deadline is calculated based on the temporary generation time and the temporary validity period.
[0236] Before the temporary decryption deadline, use the temporary private key to decrypt the statistical analysis report to obtain the decrypted statistical analysis report.
[0237] It should be understood that the decryption time is the specific point in time when the statistical analysis report decryption operation is initiated. The absolute time difference is the absolute value of the time interval between the decryption time and the generation time. The decryption time value is the value obtained after the decryption time has been processed using a conversion method. The temporary prime numbers are two prime numbers used to generate the temporary key, calculated based on parameters such as the decryption time value. The temporary key is a temporary encryption / decryption credential generated from the two temporary prime numbers, containing a temporary public key and a temporary private key. The temporary public key is the publicly available part of the temporary key pair, used to encrypt the initial statistical analysis report, while the temporary private key is the confidential part of the temporary key pair, used to decrypt the temporary statistical analysis report. The temporary generation time is the point in time when the temporary key is created. The temporary decryption deadline is the effective expiration time of the temporary key, calculated from the temporary generation time and the temporary validity period. The temporary validity period is based on the premise that the key has expired and is determined based on the business timeliness requirements of emergency reporting scenarios (for example, emergency data usually needs to be decrypted and analyzed within 3 hours). Therefore, the temporary validity period will be set to a reasonable time window to satisfy the decryption operation of the initiator, which is sufficient to cover the entire process from the generation of the temporary key to the execution of decryption by the initiator, and there will be no insufficient time.
[0238] It's important to clarify that the reason for choosing to construct a temporary key instead of returning a reconstructed key after the key expires is as follows: Returning a reconstructed key requires re-extracting the reporting timestamp from the valid backfilled data, calculating the monitoring time interval and monitoring values, iteratively generating the minimum screening value and screening range, and then regenerating a new key that is completely identical to the original key according to the original logic. Subsequently, the statistical analysis report is re-encrypted and sent back to the initiator. The entire process is a repetition of the original key generation chain, resulting in a lengthy process and the risk of data loss. In contrast, the generation of a temporary key does not require tracing back to the original key generation stage. Instead, it is completed independently within the current decryption process: when it is determined that the absolute time difference of the original key exceeds the valid duration, two temporary prime numbers are calculated directly based on the decryption time value converted at the current decryption time, the real-time monitoring values, the number of reported data, the number of unreported data, and the screening threshold. These are then used to generate a temporary key. Subsequently, only the temporary public key is used to encrypt the initial statistical analysis report (an unencrypted version identical to the original encrypted report) to obtain a temporary statistical analysis report. Decryption is then completed within the validity period of the temporary key, ensuring data version consistency and shortening decryption time. The temporary statistical analysis report is an initial statistical analysis report encrypted using a temporary public key.
[0239] For example, suppose the original key was generated at 2025-10-01-08:00:00, with a validity period of 2 hours (7200 seconds), and the conversion method is timestamp conversion (unit: seconds). If the decryption time is 2025-10-01-09:30:00, the calculated absolute time difference is 5400 seconds, which is less than 7200 seconds. Therefore, the decrypted statistical analysis report can be obtained directly using the private key. If the decryption time is 10:30:00 on October 1, 2025, and the absolute time difference is 9000 seconds, which is greater than 7200 seconds, the key becomes invalid. In this case, the decryption time is converted to the decryption time value 20251001103000, and a temporary key (temporary public key and temporary private key) is recalculated. If the temporary key is generated at 10:35:00 on October 1, 2025, and its validity period is 3 hours, a temporary statistical analysis report is obtained using the temporary public key. The temporary decryption deadline is calculated to be 13:35:00 on October 1, 2025. Before 13:35:00 on October 1, 2025, the temporary statistical analysis report is decrypted using the temporary private key to obtain the decrypted statistical analysis report. Optionally, the encryption and decryption process can be implemented using the RSA algorithm, which will not be elaborated here. This invention improves the security of encryption by dynamically generating encryption keys associated with monitoring time intervals, monitoring values, reported quantities, and unreported quantities. It avoids the risk of leakage caused by long-term use of fixed keys, enhances the confidentiality and anti-theft capabilities of statistical analysis reports during transmission, and balances security with adaptive flexibility.
[0240] To address the problems described in the background section, this invention receives an emergency reporting instruction and, based on this instruction, identifies an emergency processing system and an Excel file to be processed. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit. The Excel file to be processed contains multiple worksheets. The invention performs structural parsing on the Excel file to obtain a header field set and a data region set. The header field set includes multiple header fields, and the data region set includes multiple data regions, with each header field corresponding to a specific data region. Therefore, this invention employs a cell matrix combined with connected component analysis to automatically extract non-empty cells to form independent data. The system uses a block-based approach, combined with field naming rules to determine the table header, breaking the limitations of traditional fixed-format parsing. It adapts to Excel files from different sources, eliminating the need for manual pre-setting of header and data row positions, resulting in more accurate and efficient parsing. This provides a reliable structured data foundation for subsequent data processing. A field mapping rule library is constructed using dynamic mapping units and header field sets. Based on this rule library and data region sets, a dynamic data entry template set is generated. This dynamic data entry template set includes multiple dynamic data entry templates. Therefore, this invention achieves intelligent matching between the original header and standard fields based on a semantic similarity model, adapting to multi-source heterogeneous data. Combined with data region type clustering analysis, it automatically generates dynamic data entry that fits business characteristics. Templates eliminate the need for manual intervention in field alignment and template design, unify multi-source data field standards, improve data integration efficiency, and offer strong template adaptability. They acquire organizational-level data and reporting terminals, utilize task distribution units, organizational-level data, and dynamic reporting template sets to generate distribution task sets, which are then sent to the reporting terminals. This invention constructs a hierarchical node tree based on organizational-level data, combines it with the reporting range identifier of dynamic reporting templates, accurately matches target reporting units, automatically generates and distributes distribution task sets, and achieves intelligent association between templates and organizational levels. Task distribution is accurate and efficient, eliminating the need for manual distribution, adapts to multi-level organizational structures, and improves the task flow efficiency of emergency data reporting, enabling real-time data acquisition. The backfilled dataset from the reporting terminal is retrieved, and the dataset is validated using a field mapping rule base to obtain a validation result set. Based on the validation result set and the backfilled dataset, data aggregation is performed using a data aggregation unit to obtain a summary data table. This summary data table is then sent to the initiating end of the emergency reporting command, realizing the emergency reporting and processing of Excel data. It is evident that this invention, by introducing a dynamic mapping mechanism and a hierarchical data aggregation strategy, achieves intelligent parsing and standardized processing of multi-source heterogeneous Excel files, effectively solving the problems of low efficiency, error-proneness, and difficulty in traceability inherent in traditional manual processing methods, and improving the accuracy and response speed of emergency data reporting and processing. Therefore, this invention can improve the accuracy and efficiency of emergency reporting and processing of Excel data.
[0241] like Figure 2The diagram shown is a functional block diagram of an Excel data emergency reporting and processing system based on dynamic mapping, provided in an embodiment of the present invention.
[0242] The Excel data emergency reporting and processing system 100 based on dynamic mapping described in this invention can be installed in an electronic device. Depending on the functions implemented, the Excel data emergency reporting and processing system 100 based on dynamic mapping may include a structure parsing module 101, a mapping rule construction module 102, a data backfilling module 103, and a data verification module 104. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.
[0243] The structure parsing module 101 is used to receive emergency reporting instructions and, based on the emergency reporting instructions, identify the emergency processing system and the Excel file to be processed. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets.
[0244] The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one.
[0245] The mapping rule construction module 102 is used to construct a field mapping rule library using the dynamic mapping unit and the header field set, and to generate a dynamic filling template set based on the field mapping rule library and the data area set, wherein the dynamic filling template set includes multiple dynamic filling templates.
[0246] The data backfilling module 103 is used to acquire organizational-level data and a filling terminal, generate a distribution task set using the task distribution unit, organizational-level data and dynamic filling template set, and send the distribution task set to the filling terminal.
[0247] The data verification module 104 is used to acquire the backfilled dataset from the reporting terminal in real time, verify the backfilled dataset using the field mapping rule base, obtain a verification result set, and aggregate the data using the data aggregation unit based on the verification result set and the backfilled dataset to obtain a summary data table. The summary data table is then sent to the initiating end of the emergency reporting command to realize the emergency reporting and processing of Excel data.
[0248] In detail, the modules in the Excel data emergency reporting and processing system 100 based on dynamic mapping described in this embodiment of the invention adopt the same approach as described above. Figure 1The method used is the same as the emergency reporting and processing method for Excel data based on dynamic mapping described in the previous section, and it can produce the same technical effect, so it will not be repeated here.
[0249] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements an emergency reporting and processing method for Excel data based on dynamic mapping, according to an embodiment of the present invention.
[0250] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as an Excel data emergency reporting and processing method program based on dynamic mapping.
[0251] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of an Excel data emergency reporting and processing method program based on dynamic mapping, but also to temporarily store data that has been output or will be output.
[0252] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a dynamic mapping-based Excel data emergency reporting and processing method program), and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.
[0253] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.
[0254] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0255] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.
[0256] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.
[0257] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.
[0258] The emergency reporting and processing method program for Excel data based on dynamic mapping, stored in the memory 11 of the electronic device 1, is a combination of multiple instructions. When run in the processor 10, it can achieve the following:
[0259] Upon receiving an emergency reporting instruction, the system identifies the emergency processing system and the Excel file to be processed based on the instruction. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets.
[0260] The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one.
[0261] A field mapping rule base is constructed using dynamic mapping units and table header field sets. A dynamic filling template set is generated based on the field mapping rule base and data region set. The dynamic filling template set includes multiple dynamic filling templates.
[0262] Obtain organizational hierarchy data and data entry terminals, generate a distribution task set using the task distribution unit, organizational hierarchy data and dynamic data entry template set, and send the distribution task set to the data entry terminals.
[0263] The system acquires the backfilled dataset from the reporting terminal in real time, verifies the backfilled dataset using a field mapping rule base, obtains a verification result set, aggregates the data using a data aggregation unit based on the verification result set and the backfilled dataset, obtains a summary data table, and sends the summary data table to the initiator of the emergency reporting command to realize emergency reporting and processing of Excel data.
[0264] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.
[0265] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0266] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:
[0267] Upon receiving an emergency reporting instruction, the system identifies the emergency processing system and the Excel file to be processed based on the instruction. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets.
[0268] The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one.
[0269] A field mapping rule base is constructed using dynamic mapping units and table header field sets. A dynamic filling template set is generated based on the field mapping rule base and data region set. The dynamic filling template set includes multiple dynamic filling templates.
[0270] Obtain organizational hierarchy data and data entry terminals, generate a distribution task set using the task distribution unit, organizational hierarchy data and dynamic data entry template set, and send the distribution task set to the data entry terminals.
[0271] The system acquires the backfilled dataset from the reporting terminal in real time, verifies the backfilled dataset using a field mapping rule base, obtains a verification result set, aggregates the data using a data aggregation unit based on the verification result set and the backfilled dataset, obtains a summary data table, and sends the summary data table to the initiator of the emergency reporting command to realize emergency reporting and processing of Excel data.
[0272] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.
[0273] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0274] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0275] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0276] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. An emergency reporting and processing method for Excel data based on dynamic mapping, characterized in that, The method includes: Upon receiving an emergency reporting instruction, the system identifies the emergency processing system and the Excel file to be processed based on the instruction. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets. The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one. A field mapping rule base is constructed using dynamic mapping units and table header field sets. A dynamic filling template set is generated based on the field mapping rule base and data region set. The dynamic filling template set includes multiple dynamic filling templates. Obtain organizational hierarchy data and reporting terminals, generate a distribution task set using the task distribution unit, organizational hierarchy data and dynamic reporting template set, and send the distribution task set to the reporting terminals. The system acquires the backfilled dataset from the reporting terminal in real time, verifies the backfilled dataset using a field mapping rule base, obtains a verification result set, aggregates the data using a data aggregation unit based on the verification result set and the backfilled dataset, obtains a summary data table, and sends the summary data table to the initiating end of the emergency reporting command to realize emergency reporting and processing of Excel data. Based on the verification result set and the backfilled dataset, the data aggregation unit is used to perform data aggregation to obtain a summary data table, including: The valid backfill dataset is filtered out from the backfill dataset using the validation result set. The valid backfill dataset includes multiple valid backfill data, and each valid backfill data contains a data source identifier and a data timestamp. The validation results of the data corresponding to the valid backfill data are all passed. For each valid backfill data point in the valid backfill dataset, perform the following operation: Extract the reporting source identifier from the valid backfilled data, use the reporting source identifier to identify the source hierarchical node in the hierarchical node tree, obtain the node hierarchy and number of subordinate nodes of the source hierarchical node, and obtain the source hierarchy and number of subordinate nodes. By associating valid backfill data, source hierarchy, and the number of source sub-nodes, hierarchical backfill nodes are obtained. By summarizing the hierarchical backfill nodes, a hierarchical backfill node set is obtained. Sort the hierarchical backfill nodes in the hierarchical backfill node set in ascending order of source hierarchy to obtain the hierarchical backfill sequence; The aggregate weight is calculated using a pre-constructed aggregate weight calculation formula, wherein the aggregate weight calculation formula is as follows: ; in, Indicates the first node in the hierarchical backfill node set. The aggregate weights corresponding to each level of backfill nodes. , These represent the preset first weighting coefficient and the preset second weighting coefficient, respectively. + =1, This represents the highest level in the hierarchical node tree. , These represent the nodes in the hierarchical backfill node set. The, the The source level corresponding to each level of backfill node. This indicates that the hierarchical backfill node set has a total of Each level of backfill node , They represent the first The, the The number of source subordinate nodes corresponding to each level of backfill node; Summarize the aggregated weights to obtain multiple aggregated weights; By using multiple aggregation weights to aggregate the hierarchical backfill sequence level by level, a summary data table is obtained.
2. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 1, characterized in that, The step of parsing the structure of the Excel file to be processed to obtain the header field set and data area set includes: Perform the following operations on each worksheet in the Excel file to be processed: Obtain the cell matrix of the worksheet, identify the set of non-empty cells in the cell matrix, where the set of non-empty cells contains multiple non-empty cells; Identify the position coordinates of each non-empty cell in the set of non-empty cells to obtain multiple position coordinates, where each non-empty cell corresponds one-to-one with its position coordinate; Perform connectivity analysis on multiple location coordinates to obtain one or more connected regions; Perform the following operation on each of one or more connected regions: The sequence of first-row cells is determined based on the connected components. This sequence contains multiple first-row cells. The following operation is performed on each first-row cell in the sequence: Obtain the cell content and cell format characteristics of the first row cell, determine whether the cell content conforms to the preset field naming rules, and obtain the naming rule judgment result, wherein the naming rule judgment result is conforming or not conforming; The table header confidence is calculated based on the cell format features and the preset feature score mapping rule library. The table header confidence is compared with the preset confidence threshold. If the naming rule judgment result is consistent and the table header confidence is greater than or equal to the confidence threshold, the cell content is confirmed as the table header field. The table header fields are summarized to obtain the table header field set. Remove the first row of cells from the connected region to obtain the data range; The data regions are aggregated to obtain a data region set.
3. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 2, characterized in that, The construction of a field mapping rule base using dynamic mapping units and table header field sets includes: Obtain the standard field library, which includes multiple standard field nodes, and each standard field node includes: standard field name, field data type, and field constraints; The cell formulas in the scanned data area set are identified, and the formula function set is dynamically generated based on the formula function set. The verification rule node includes regular expression verification rules and data dictionary verification rules. Perform the following operation on each header field in the header field set: The semantic similarity between the header field and the standard field name corresponding to each standard field node in the standard field library is calculated using a pre-built semantic similarity model to obtain a similarity set. The maximum similarity is extracted from the similarity set to obtain the target similarity. Compare the target similarity with the preset similarity threshold. If the target similarity is greater than or equal to the similarity threshold, then associate the header field, the validation rule node, and the standard field node corresponding to the target similarity to obtain the mapping rule node. Otherwise, generate a custom field node based on the table header fields, associate the table header fields, validation rule nodes, and custom field nodes to obtain a mapping rule node; Summarize the mapping rule nodes to obtain the field mapping rule library.
4. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 3, characterized in that, The generation of a dynamic data entry template set based on the field mapping rule base and data region set includes: Extract data regions sequentially from the data region set, and perform the following operations on each extracted data region: Extract the data row set from the data area, and perform the following operation on each data row in the data row set: The data types of cells in the data row are inferred by using the mapping rule nodes in the field mapping rule base to obtain the inferred type set. The number of each inferred type in the inferred type set is counted to obtain the type count set. By summarizing the number of types, a number of regional types is obtained. Cluster analysis is then performed on the number of regional types to obtain one or more cluster type groups. Identify the clustering group with the largest number of clustering groups in one or more clustering groups to obtain the target group. Generate a dynamic filling template based on the target group and the field mapping rule base. The dynamic filling template includes a filling range identifier. The dynamic data entry templates are summarized to obtain a dynamic data entry template set.
5. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 4, characterized in that, The process of generating a distribution task set using task distribution units, organizational hierarchy data, and dynamic reporting template sets includes: Parse the organizational hierarchy data to obtain a hierarchical node tree, which includes multiple hierarchical nodes, and each hierarchical node includes: node identifier, node level, and parent node identifier; Perform the following operations on each dynamic data entry template in the dynamic data entry template set: Extract the filling range identifier from the dynamic filling template, and use the filling range identifier to perform matching and retrieval in the hierarchical node tree to obtain the target hierarchical node set; Perform the following operation on each target-level node in the target-level node set: By associating the dynamic data entry template with the target level node, the task distribution node is obtained. The task distribution node includes the task deadline and reminder frequency. The distribution task nodes are aggregated to obtain the distribution task set.
6. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 5, characterized in that, The validation of the backfilled dataset using the field mapping rule base yields a validation result set, including: Perform the following operation on each backfilled data in the backfilled dataset: Extract the backfill field value set from the backfill data, and perform the following operations on each backfill field value in the backfill field value set: Using the header field corresponding to the backfilled field value, the corresponding mapping rule node is retrieved in the field mapping rule base. Based on the field data type in the mapping rule node, the backfilled field value is type-validated to obtain the type validation result, which is either pass or fail. The backfill field values are constrained and validated based on the field constraints in the mapping rule node to obtain the constraint validation result, which is either pass or fail. The backfilled field values are dynamically validated based on the validation rule nodes in the mapping rule nodes to obtain dynamic validation results, which are either pass or fail. The field validation results are obtained by combining the association type validation results, constraint validation results, and dynamic validation results. The data validation results are summarized to obtain the data validation result set.
7. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 6, characterized in that, The method of using multiple aggregation weights to aggregate data level by level in the hierarchical backfill sequence to obtain a summary data table includes: In the hierarchical backfill sequence, identify backfill nodes from the same source level to obtain a set of sibling backfill nodes. Perform the following operation on each sibling backfill node group in the set: In the same-level backfill node group, identify the set of numeric field values, perform outlier detection on the set of numeric field values, and obtain the outlier identifier set; Outlier identifiers are used to remove outlier values from the set of numeric field values to obtain a set of valid values. The mean of the valid values in the set of valid values is then calculated to obtain the mean of the same level. Calculate the standard deviation of the effective values in the effective value set to obtain the standard deviation of the same level, and construct a confidence interval based on the mean and standard deviation of the same level. The values of numeric fields in the same-level backfill node group are corrected using confidence intervals to obtain the corrected backfill node group; The corrected backfill node groups are summarized to obtain the corrected backfill sequence, and a summary data table is generated based on the corrected backfill sequence and multiple aggregation weights.
8. The method for emergency reporting and processing of Excel data based on dynamic mapping as described in claim 7, characterized in that, After generating the summary data table based on the corrected backfill sequence and multiple aggregation weights, the process also includes: Task progress monitoring data is obtained based on the distributed task set and the backfilled dataset. The task progress monitoring data includes: the number of data filled, the number of data not filled, and the data filling completion rate. The task progress monitoring data is updated in real time using a preset monitoring time interval to obtain the progress monitoring time sequence. A pre-built reporting risk prediction model is used to perform trend analysis on the progress monitoring time sequence to obtain the reporting trend prediction value and reporting risk level. Compare the predicted trend value with the preset warning threshold. If the predicted trend value is less than the warning threshold, dynamically adjust the task deadline or reminder frequency of the corresponding task node in the task distribution set based on the risk level of the reported task. Generate a reminder message based on the task distribution set and send the reminder message to the reporting terminal that has not reported. Data traceability records are generated based on the source identifier and timestamp of the valid backfilled data, and these records are then linked to the summary data table. Statistical analysis reports are generated using summary data tables and hierarchical node trees, and then sent to the initiator of emergency reporting instructions.
9. An emergency reporting and processing system for Excel data based on dynamic mapping, characterized in that, The system includes: The structure parsing module is used to receive emergency reporting instructions and, based on the emergency reporting instructions, identify the emergency processing system and the Excel file to be processed. The emergency processing system includes a dynamic mapping unit, a task distribution unit, and a data aggregation unit, and the Excel file to be processed contains multiple worksheets. The Excel file to be processed is parsed to obtain a set of header fields and a set of data areas. The set of header fields includes multiple header fields, and the set of data areas includes multiple data areas. The header fields and data areas correspond one-to-one. The mapping rule construction module is used to construct a field mapping rule library using dynamic mapping units and table header field sets, and to generate a dynamic filling template set based on the field mapping rule library and data region set, wherein the dynamic filling template set includes multiple dynamic filling templates; The data backfilling module is used to acquire organizational-level data and filling terminals, generate a distribution task set using the task distribution unit, organizational-level data and dynamic filling template set, and send the distribution task set to the filling terminal. The data verification module is used to acquire the backfilled dataset from the reporting terminal in real time, verify the backfilled dataset using the field mapping rule base, obtain a verification result set, and aggregate the data using the data aggregation unit based on the verification result set and the backfilled dataset to obtain a summary data table. The summary data table is then sent to the initiating end of the emergency reporting command to realize the emergency reporting and processing of Excel data. Based on the verification result set and the backfilled dataset, the data aggregation unit is used to perform data aggregation to obtain a summary data table, including: The valid backfill dataset is filtered out from the backfill dataset using the validation result set. The valid backfill dataset includes multiple valid backfill data, and each valid backfill data contains a data source identifier and a data timestamp. The validation results of the data corresponding to the valid backfill data are all passed. For each valid backfill data point in the valid backfill dataset, perform the following operation: Extract the reporting source identifier from the valid backfilled data, use the reporting source identifier to identify the source hierarchical node in the hierarchical node tree, obtain the node hierarchy and number of subordinate nodes of the source hierarchical node, and obtain the source hierarchy and number of subordinate nodes. By associating valid backfill data, source hierarchy, and the number of source sub-nodes, hierarchical backfill nodes are obtained. By summarizing the hierarchical backfill nodes, a hierarchical backfill node set is obtained. Sort the hierarchical backfill nodes in the hierarchical backfill node set in ascending order of source hierarchy to obtain the hierarchical backfill sequence; The aggregate weight is calculated using a pre-constructed aggregate weight calculation formula, wherein the aggregate weight calculation formula is as follows: ; in, Indicates the first node in the hierarchical backfill node set. The aggregate weights corresponding to each level of backfill nodes. , These represent the preset first weighting coefficient and the preset second weighting coefficient, respectively. + =1, This represents the highest level in the hierarchical node tree. , These represent the nodes in the hierarchical backfill node set. The, the The source level corresponding to each level of backfill node. This indicates that the hierarchical backfill node set has a total of Each level of backfill node , They represent the first The, the The number of source subordinate nodes corresponding to each level of backfill node; Summarize the aggregated weights to obtain multiple aggregated weights; By using multiple aggregation weights to aggregate the hierarchical backfill sequence level by level, a summary data table is obtained.
Citation Information
Patent Citations
Realization method and equipment for reporting report
CN117611078A
Excel data intelligent import method and system based on dynamic mapping
CN121212099A