Real-time supervision data submission system of distributed architecture

Abstract

The invention discloses a real-time supervision data submission system of a distributed architecture, which relates to the technical field of data processing and comprises a data acquisition module, a category division module, a distribution framework module, a format conversion module and a packaging submission module, the system has the advantages that the composition and characteristics of supervised data can be deeply known through the category division module, a foundation is laid for subsequent classified management and analysis of the data according to different characteristics, and the orderliness and manageability of data storage can be improved by utilizing the distribution framework module; according to the method, data with specific characteristics can be quickly positioned and retrieved, the efficiency and accuracy of data processing are improved, the consistency and integrity of data of all auxiliary frames are maintained through data comparison and difference calculation of the verification unit, the change condition of the data is effectively monitored, the universality and compatibility of the data are improved through the format conversion module, and the data processing efficiency and accuracy are improved. And subsequent data processing and analysis work are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a real-time supervision data reporting system with a distributed architecture. Background Art

[0002] The data for supervision refers to the data set generated, collected, and used in various supervision activities for supervising, managing, and standardizing specific objects or behaviors. It is widely applied in multiple fields such as finance, healthcare, environmental protection, and government affairs, and is an important foundation and support for realizing effective supervision. The supervision department checks whether the supervised objects comply with relevant laws, regulations, policy provisions, and industry standards through the analysis of the supervision data.

[0003] When the common real-time supervision data reporting system is in use, in the prior art, data management is inefficient due to an unreasonable distribution framework, making it difficult to ensure the consistency and integrity of data. When the data changes, it cannot be discovered and processed in a timely manner. For this reason, we propose a real-time supervision data reporting system with a distributed architecture. Summary of the Invention

[0004] The purpose of the present invention is to provide a real-time supervision data reporting system with a distributed architecture.

[0005] To solve the problems raised in the above background art, the present invention provides the following technical solution: A real-time supervision data reporting system with a distributed architecture, including a data reporting system, where the data reporting system includes a data collection module, a type classification module, a distribution framework module, a format conversion module, and a packaging and reporting module;

[0006] The data collection module collects the data to be supervised and extracts the supervised data regularly to obtain the supervised change data;

[0007] The type classification module uses the supervised change data obtained by the data collection module for splitting to obtain sub-supervised change data and analyzes its hardware to obtain the supervised hardware, and then extracts the component features of the supervised hardware;

[0008] The distribution framework module establishes a distributed framework, where the distributed framework includes a main framework and auxiliary frameworks. Then, it analyzes the supervised change data of the component features of the same supervised hardware to obtain the feature-supervised data, and then records the different feature-supervised data into different auxiliary frameworks. At the same time, the different auxiliary frameworks communicate with each other to enable information interaction between adjacent auxiliary frameworks. The auxiliary framework includes a verification unit, and a verification value is established in the verification unit. All the sub-supervised change data is recorded into the main framework;

[0009] The verification unit includes a verification transmission file and a verification reception file. It extracts corresponding n sub-supervision change data from the auxiliary framework, where n represents the number of sub-supervision change data to be extracted, records it in the verification transmission file, and then different auxiliary frameworks will transmit the verification transmission file to adjacent auxiliary frameworks, record the information received from the adjacent auxiliary frameworks in the verification reception file, and analyze the supervision hardware of the information recorded in the verification reception file, extract the sub-supervision change data of the data corresponding to the supervision hardware from the main framework, and perform a difference calculation on the information recorded in the verification reception file to obtain a change index;

[0010] The encapsulation and submission module delivers the supervision format information obtained by the format conversion module to a higher level.

[0011] As a further solution of the present invention: It further includes a format conversion module, which is used to assign data numbers to different sub-supervision change data, obtain a number comparison table, and establish a supervision data format, where the supervision data format is data number + change index, convert the sub-supervision change data into the supervision data format, and obtain supervision format information.

[0012] As a further solution of the present invention: The sub-supervision change data in the type classification module includes data name, data extraction time, and corresponding change information. An extension unit is established, and the user has the right to edit the components of the sub-supervision change data. When the components recorded in the extension unit do not exist in the sub-supervision change data, the corresponding fields are set as blank columns;

[0013] Let the verification value be the normalized percentage value Y, the total number of sub-supervision change data in the auxiliary framework be Z, and let the extraction value corresponding to the verification value be T.Y, where T is the integer part and Y is the decimal part;

[0014] T.Y = Y 数值 ·Z 数量

[0015] Calculate the extraction value corresponding to the verification value according to the above formula.

[0016] As a further solution of the present invention: After the extraction value corresponding to the verification value in the distribution framework module is calculated, it will analyze the number of sub-supervision change data to be extracted according to the extraction value corresponding to the verification value. At this time, it is necessary to extract the decimal part of the extraction value corresponding to the verification value, and then process it by rounding.

[0017] As a further solution of the present invention: The distribution framework module further includes a transmission unit, and all the sub - supervision change data recorded in the corresponding auxiliary framework is recorded in the transmission unit. When the auxiliary framework conducts information interaction, it will synchronously transmit all the sub - supervision change data recorded in the transmission unit to the adjacent auxiliary framework, so that the sub - supervision change data in the adjacent auxiliary frameworks realizes two - way synchronization.

[0018] As a further solution of the present invention: When calculating the change index in the distribution framework module, assume that the information recorded in the verification received file is Y1, Y2, Y3, ……, Y X , assume that the sub - supervision change data for extracting data corresponding to the supervision hardware from the main framework is Z1, Z2, Z3, ……, Z X , and X is the total number of data entries in the verification received file;

[0019] When , the information recorded in the verification received file in the auxiliary framework is the same as the sub - supervision change data recorded in the main framework;

[0020] When , the information recorded in the verification received file in the auxiliary framework is different from the sub - supervision change data recorded in the main framework.

[0021] As a further solution of the present invention: When the information recorded in the verification received file in the auxiliary framework of the distribution framework module is different from the sub - supervision change data recorded in the main framework, synchronously analyze the number of different data between the information recorded in the verification received file in the auxiliary framework and the sub - supervision change data recorded in the main framework, and calculate the change index.

[0022] As a further solution of the present invention: The calculation formula of the change index in the distribution framework module is as follows:

[0023]

[0024] Where B 指数 is the change index B 指数 , X 数量 is the number of information recorded in the verification received file in the auxiliary framework, B 数量 is the number of different data, and the change index is calculated according to the above formula.

[0025] As a further solution of the present invention: The format conversion module further includes a format editing unit. The user has the editing permission of the format editing unit. The user edits and processes the supervision data format through the format editing unit, and at the same time establishes a change value table, then sorts the change index in descending order and records it in the change value table, and inserts the change value table into the metadata of the supervision format information in the form of a label.

[0026] Adopting the above technical solution, compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] 1. The category division module of the present invention helps to deeply understand the composition and characteristics of the supervised data, laying a foundation for subsequent classification management and analysis of data according to different characteristics. The distribution framework module can improve the organization and manageability of data storage, facilitate quick positioning and retrieval of data with specific characteristics, and enhance the efficiency and accuracy of data processing. Through data comparison and difference calculation of the verification unit, the consistency and integrity of each auxiliary framework data are maintained, effectively monitoring the data changes. The format conversion module improves the versatility and compatibility of data, which is beneficial to subsequent data processing and analysis work;

[0028] 2. The category division module of the present invention helps to adapt to different supervision scenarios and data characteristics, improves the compatibility and processing ability of the system for diverse data, ensures the integrity and accuracy of data records. The distribution framework module can improve the reliability of verification results, helps to promptly discover abnormal situations in data, helps to enrich the data resources of each auxiliary framework, promotes data fusion and collaborative work between different frameworks, and enhances the comprehensive analysis ability of the system for data;

[0029] 3. The distribution framework module of the present invention can accurately judge the consistency between the auxiliary framework and the main framework data, and precisely quantify the degree of data change, which helps to quickly locate and analyze the reasons for data anomalies, improves the efficiency and pertinence of supervision work, enhances the sensitivity and response ability of the system to data changes. The format conversion module enables users to customize data formats according to their own analysis habits and business requirements, improving the flexibility and readability of data display. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the system process in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The following further describes the specific embodiments of the present invention in conjunction with the drawings. It should be noted here that the description of these embodiments is for helping to understand the present invention, but does not limit the present invention.

[0032] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0033] Embodiment 1:

[0034] Therefore, in order to effectively solve the above problems, the present application proposes a real-time supervision data submission system with a distributed architecture, as shown in the accompanying drawings of the specification Figure 1As shown in the figure, the data reporting system includes a data collection module, a category division module, a distribution framework module, a format conversion module, and a packaging and reporting module;

[0035] The data collection module collects the data that needs to be supervised and extracts the supervised data regularly to obtain the supervised change data;

[0036] The category division module uses the supervised change data obtained by the data collection module to split, obtains the sub-supervised change data and analyzes its hardware to obtain the supervised hardware, and then extracts the component features of the supervised hardware;

[0037] The distribution framework module establishes a distributed framework, where the distributed framework includes a main framework and auxiliary frameworks. Then it analyzes the supervised change data of the component features of the same supervised hardware to obtain the data of feature supervision, and then records the data of different feature supervision into different auxiliary frameworks. At the same time, the different auxiliary frameworks communicate with each other to enable information interaction between adjacent auxiliary frameworks. The auxiliary framework includes a verification unit, and a verification value is established in the verification unit. All the sub-supervised change data is recorded into the main framework;

[0038] The verification unit includes a verification transmission file and a verification reception file. Extracts the corresponding n sub-supervised change data from the auxiliary framework, where n represents the number of sub-supervised change data to be extracted, records them into the verification transmission file, and then different auxiliary frameworks will transmit the verification transmission file to adjacent auxiliary frameworks, records the information received from the adjacent auxiliary frameworks into the verification reception file, analyzes the supervised hardware of the information recorded in the verification reception file, extracts the sub-supervised change data of the data corresponding to the supervised hardware from the main framework, and calculates the difference value of the information recorded in the verification reception file to obtain the change index;

[0039] The packaging and reporting module delivers the supervised format information obtained by the format conversion module to the superior for reporting;

[0040] One auxiliary framework stores the basic information of the enterprise, and the other auxiliary framework stores the market transaction data of the enterprise. The two auxiliary frameworks can interact through information, providing richer data support for a comprehensive analysis of the market behavior of the enterprise;

[0041] Specific work process: Obtain regulatory change data, split it, analyze the components of the sub-regulatory change data, then extract the component features of the regulatory hardware, establish a distributed framework, where the distributed framework includes a main framework and auxiliary frameworks, record the data with different feature regulations into different auxiliary frameworks, perform information interaction between adjacent auxiliary frameworks, record all the sub-regulatory change data into the main framework, calculate the difference of the information recorded in the verification received file to obtain a change index, assign data numbers to different sub-regulatory change data, obtain a number comparison table, establish the data format of the regulation, convert the sub-regulatory change data into the data format of the regulation, and report and process the regulatory format information;

[0042] Furthermore, the type classification module helps to deeply understand the composition and characteristics of the regulatory data, laying a foundation for subsequent classification management and analysis of data according to different features. Using the distribution framework module can improve the organization and manageability of data storage, facilitate quick positioning and retrieval of data with specific features, and enhance the efficiency and accuracy of data processing. Through the data comparison and difference calculation of the verification unit, the consistency and integrity of the data in each auxiliary framework are maintained, effectively monitoring the data change situation. The format conversion module improves the generality and compatibility of the data, which is beneficial to subsequent data processing and analysis work.

[0043] Embodiment 2:

[0044] Based on Embodiment 1, as shown in the accompanying drawings of the specification Figure 1 shown, it further includes a format conversion module for assigning data numbers to different sub-regulatory change data, obtaining a number comparison table, establishing the data format of the regulation, where the data format of the regulation is data number + change index, converting the sub-regulatory change data into the data format of the regulation, and obtaining regulatory format information;

[0045] The sub-regulatory change data in the type classification module includes data name, data extraction time, and the corresponding change information. An extension unit is established, and the user has the right to edit the components of the sub-regulatory change data. When the components recorded in the extension unit do not exist in the sub-regulatory change data, the corresponding fields are set as blank columns;

[0046] Let the verification value be the normalized percentage value Y, the total number of sub-regulatory change data in the auxiliary framework be Z, and let the extraction value corresponding to the verification value be T.Y, where T is the integer part and Y is the decimal part;

[0047] T.Y = Y 数值 ·Z 数量

[0048] Calculate the extraction value corresponding to the verification value according to the above formula;

[0049] The normalized percentage value Y is calculated by monitoring the ratio of the hardware characteristics to the total data volume;

[0050] In the category division module, an edit log recording subsystem is established. Whenever a user performs an edit operation, the system automatically records information such as the time of the edit, the user identity, the specific content of the edit, and the reason for the edit. This way, during subsequent data review or problem troubleshooting, the evolution process of the data can be clearly understood, improving the credibility and auditability of the data;

[0051] After the extraction value corresponding to the verification value is calculated in the distribution framework module, the number of sub-supervision change data to be extracted is analyzed based on the extraction value corresponding to the verification value. At this time, the decimal part of the extraction value corresponding to the verification value needs to be extracted and then processed by rounding;

[0052] The distribution framework module also includes a transmission unit, and all the sub-supervision change data recorded in the corresponding auxiliary framework is recorded in the transmission unit. When the auxiliary framework conducts information interaction, it will synchronously transmit all the sub-supervision change data recorded in the transmission unit to the adjacent auxiliary framework, enabling two-way synchronization of the sub-supervision change data in the adjacent auxiliary frameworks;

[0053] Specific workflow: The sub-supervision change data includes the data name, the data extraction time, and the corresponding change information. The user has the right to edit the components of the sub-supervision change data. When the components recorded in the extension unit do not exist in the sub-supervision change data, the corresponding fields are set to blank columns. Calculate the extraction value corresponding to the verification value, and analyze the number of sub-supervision change data to be extracted. At this time, the decimal part of the extraction value corresponding to the verification value needs to be extracted. When Y≥0.5, T is incremented by 1, then the decimal part is deleted, and T + 1 sub-supervision change data is extracted. When Y<0.5, the decimal part is directly deleted, and at this time, T sub-supervision change data is extracted. When the auxiliary framework conducts information interaction, it will synchronously transmit all the sub-supervision change data recorded in the transmission unit to the adjacent auxiliary framework;

[0054] Furthermore, the category division module helps to adapt to different supervision scenarios and data characteristics, improving the system's compatibility and processing ability for diverse data, ensuring the integrity and accuracy of data records. The distribution framework module can improve the reliability of verification results, help to detect abnormal situations in data in a timely manner, help to enrich the data resources of each auxiliary framework, promote data fusion and collaborative work between different frameworks, and enhance the system's comprehensive data analysis ability.

[0055] Embodiment III:

[0056] Based on Embodiment II, as shown in the accompanying drawings of the specification Figure 1As shown, when calculating the change index in the distribution framework module, let the information recorded in the verification received file be Y1, Y2, Y3, ……, Y X , and let the sub-supervision change data extracted from the main framework corresponding to the supervision hardware be Z1, Z2, Z3, ……, Z X , and X is the total number of data entries in the verification received file;

[0057] When , the information recorded in the verification received file in the auxiliary framework is the same as the sub-supervision change data recorded in the main framework;

[0058] When , the information recorded in the verification received file in the auxiliary framework is different from the sub-supervision change data recorded in the main framework;

[0059] Introduce the cosine similarity algorithm or the edit distance algorithm, etc. The cosine similarity algorithm can measure the cosine value of the angle between two vectors. By converting the information recorded in the verification received file and the sub-supervision change data extracted from the main framework into vector form, calculate the cosine similarity between them. The edit distance algorithm can calculate the minimum number of edit operations required to convert one string to another string, which is suitable for text type data comparison;

[0060] When the information recorded in the verification received file in the auxiliary framework of the distribution framework module is different from the sub-supervision change data recorded in the main framework, synchronously analyze the number of different data between the information recorded in the verification received file in the auxiliary framework and the sub-supervision change data recorded in the main framework, and calculate the change index;

[0061] The calculation formula of the change index in the distribution framework module is as follows:

[0062]

[0063] Where B 指数 is the change index B 指数 , X 数量 is the number of information recorded in the verification received file in the auxiliary framework, B 数量 is the number of different data, and calculate the change index according to the above formula;

[0064] The format conversion module also includes a format editing unit. The user has the editing permission of the format editing unit. The user edits and processes the supervision data format through the format editing unit, and at the same time establishes a change value table, then sorts the change index in descending order and records it in the change value table, and inserts the change value table into the metadata of the supervision format information in the form of a label;

[0065] Specific workflow: Determine whether the information recorded in the verification received file in the auxiliary framework is the same as the sub-supervision change data recorded in the main framework, analyze the quantity of different data between the information recorded in the verification received file in the auxiliary framework and the sub-supervision change data recorded in the main framework, calculate the change index. The user has the editing permission for the format editing unit. The user processes the editing of the supervision data format through the format editing unit, and at the same time establishes a change value table. Then, sort the change index in descending order and record it in the change value table. Insert the change value table into the metadata of the supervision format information in the form of a tag;

[0066] Furthermore, through the distribution framework module, it can accurately determine the consistency of the data between the auxiliary framework and the main framework, and precisely quantify the degree of data change, which helps to quickly locate and analyze the reasons for data anomalies, improves the efficiency and pertinence of the supervision work, enhances the sensitivity and response ability of the system to data changes. Through the format conversion module, the user can customize the data format according to their own analysis habits and business requirements, improving the flexibility and readability of data display.

[0067] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time supervision data reporting system with a distributed architecture, including a data reporting system, characterized in that: The data reporting system includes a data collection module, a category division module, a distribution framework module, a format conversion module, and a packaging and reporting module; The data collection module collects the data to be supervised and extracts the supervised data at regular intervals to obtain the supervised change data; The category division module uses the supervised change data obtained by the data collection module to perform splitting, obtains the sub-supervised change data and analyzes its hardware to obtain the supervised hardware, and then extracts the component features of the supervised hardware; The distribution framework module establishes a distributed framework, where the distributed framework includes a main framework and an auxiliary framework. Then it analyzes the supervised change data of the component features of the same supervised hardware to obtain the feature-supervised data, and then records the differently feature-supervised data into different auxiliary frameworks. At the same time, the different auxiliary frameworks communicate with each other to enable information interaction between adjacent auxiliary frameworks. The auxiliary framework includes a verification unit, and a verification value is established in the verification unit. All the sub-supervised change data is recorded into the main framework; The verification unit includes a verification transmission file and a verification reception file. Extracts corresponding n sub-supervised change data from the auxiliary framework, where n represents the number of sub-supervised change data to be extracted, and records them into the verification transmission file. Then different auxiliary frameworks will transmit the verification transmission file to the adjacent auxiliary frameworks, record the information received from the adjacent auxiliary frameworks into the verification reception file, and analyze the supervised hardware of the information recorded in the verification reception file. Extract the sub-supervised change data of the data corresponding to the supervised hardware from the main framework, and perform a difference calculation on the information recorded in the verification reception file to obtain the change index; The packaging and reporting module delivers the supervised format information obtained by the format conversion module to the superior for submission; 2. The real-time supervision data reporting system with a distributed architecture according to claim 1, characterized in that: It also includes a format conversion module, which is used to assign data numbers to different sub-supervised change data, obtain a number comparison table, and establish the data format of supervision, where the data format of supervision is the data number + the change index, and convert the sub-supervised change data into the supervised data format to obtain the supervised format information; 3. The real-time supervision data reporting system with a distributed architecture according to claim 2, characterized in that: The sub-supervised change data in the category division module includes the data name, the data extraction time, and the corresponding change information. An extension unit is established, and the user has the right to edit the components of the sub-supervised change data. When the component recorded in the extension unit does not exist in the sub-supervised change data, the corresponding field is set as a blank column; Let the verification value be the normalized percentage value Y, the total number of sub-supervised change data in the auxiliary framework be Z, and the extraction value corresponding to the verification value be T.Y, where T is the integer part and Y is the decimal part; T.Y = Y 数值 ·Z 数量 Calculate the extraction value corresponding to the verification value according to the above formula; 4. A real-time supervision data reporting system with a distributed architecture according to claim 3, characterized in that: After the extraction value corresponding to the verification value in the distribution framework module is calculated, it will analyze the number of sub-supervised change data to be extracted according to the extraction value corresponding to the verification value. At this time, it is necessary to extract the decimal part of the extraction value corresponding to the verification value, and then process it by rounding; 5. The real-time supervision data reporting system with a distributed architecture according to claim 4, characterized in that: The distribution framework module further includes a transmission unit, and all sub-supervision change data recorded in the corresponding auxiliary framework is recorded in the transmission unit. When the auxiliary framework conducts information interaction, it will synchronously transmit all sub-supervision change data recorded in the transmission unit to the adjacent auxiliary framework, enabling two-way synchronization of the sub-supervision change data in the adjacent auxiliary frameworks.

6. The real-time supervision data reporting system with a distributed architecture according to claim 5, characterized in that: When calculating the change index in the distribution framework module, assume that the information recorded in the verification received file is Y1, Y2, Y3, ……, Y X , and assume that the sub-supervision change data extracted from the main framework corresponding to the supervision hardware is Z1, Z2, Z3, ……, Z X , and X is the total number of data entries in the verification received file; When the information of the received file record verified in the auxiliary frame is the same as the sub-supervision change data recorded in the main frame; When the information of the received file record verified in the auxiliary framework is different from the sub-supervision change data recorded in the main framework.

7. The real-time supervision data reporting system with a distributed architecture according to claim 6, characterized in that: When the information recorded in the verified received file in the auxiliary framework of the distribution framework module is different from the sub-supervision change data recorded in the main framework, the number of different data between the information recorded in the verified received file in the auxiliary framework and the sub-supervision change data recorded in the main framework is synchronously analyzed to calculate the change index.

8. A real-time supervision data reporting system with a distributed architecture according to claim 7, characterized in that: The calculation formula of the change index in the distribution framework module is as follows: Among them, B 指数 is the change index B 指数 , X 数量 is the number of information records for verifying received files in the auxiliary framework, and B 数量 is the number of different data. The change index is calculated according to the above formula.

9. A real-time supervision data reporting system with a distributed architecture according to claim 1, characterized in that: The format conversion module further includes a format editing unit. The user has the editing permission of the format editing unit. The user edits and processes the supervision data format through the format editing unit, and at the same time establishes a change value table. Then, the change index is sorted in descending order and recorded in the change value table. The change value table is inserted into the metadata of the supervision format information in the form of a label.

Images