Government affair data cross-network synchronization method, government affair data cross-network synchronization system and related equipment

Abstract

The invention discloses a government affair data cross-network synchronization method, a government affair data cross-network synchronization system and related equipment, according to the scheme, a data dictionary engine is introduced, a government affair data cross-network transmission file in an innovative format is defined accordingly, and standardization and safety of data are ensured; on the basis, a data encryption and security verification mechanism is further integrated, and comprehensive security and secrecy guarantee is provided for government affair data cross-network synchronization. According to the government affair data cross-network synchronization scheme provided by the invention, the data can be ensured to cross different network boundaries (such as network boundaries between the government affair intranet and the government affair extranet which are mutually isolated), seamless, safe and rapid synchronization is realized, and the problem of real-time, accurate and efficient synchronization of service data among different network environments is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer science, and in particular to government data transmission and synchronization technology. Background Art

[0002] With the continuous deepening of information construction, the demand for data sharing and exchange between government departments is increasing. Different government networks (such as government intranet, government extranet, Internet, etc.) are isolated from each other due to security and management factors, but business collaboration requires cross-network data transmission.

[0003] As a confidential network, the government intranet needs to be physically isolated from the Internet and the government extranet to ensure the high security of the intranet information. Among them, the government extranet and the Internet are logically isolated through firewalls and network gates to ensure that the two networks are logically isolated from each other, but allow necessary data exchange under control.

[0004] Compared with conventional network systems, where the LAN and the extranet are isolated from each other, data synchronization between the LAN and the extranet can be achieved through proxy servers, VPN networks, etc. Logical isolation is required between the government extranet and the Internet, so proxy servers or VPNs cannot meet technical requirements and pose risks of illegal data access and data leakage.

[0005] In the prior art, cross-network synchronization of government data is mainly achieved through file synchronization. The existing file synchronization method requires that data from the government external network be sent to the internal network using a certain file transfer protocol.

[0006] Existing file synchronization solutions mainly achieve indiscriminate file transmission through transmission channels such as network gateways. Such synchronization solutions make it difficult to finely control the in and out of data flows, and cannot control the transmission process based on business logic. At the same time, such file synchronization solutions have problems with data security and data packet loss when achieving cross-network synchronization of government data.

[0007] The primary challenge facing cross-network transmission of government data is data security. Since government data often contains a wealth of personal information about citizens, these sensitive data are at serious risk of being leaked during transmission. Once the data is leaked, it will not only infringe on citizens' privacy rights, but may also trigger a crisis of social trust and cause great damage to the government's image and credibility. Therefore, it is crucial to ensure the security of cross-network transmission of government data.

[0008] Secondly, the problem of data packet loss is common during the cross-network transmission of government data. This is mainly due to the differences between different network environments, such as network bandwidth, stability, latency and other factors, which may cause data packets to be lost or damaged during transmission. Data packet loss not only affects the integrity and accuracy of the data, but may also cause interruptions or errors in business processing, and have a negative impact on the continuity and quality of government services. Summary of the invention

[0009] In view of the problems existing in the existing cross-network synchronization schemes for government data in terms of data synchronization security and reliability, the purpose of the present invention is to provide an adaptive, highly reliable, secure and confidential cross-network synchronization scheme for government data. This scheme defines cross-network transmission files of government data in a specified format, effectively standardizes the format of cross-network transmission files for government data, and ensures data standardization and security. At the same time, it introduces data encryption technology and security verification mechanism to provide all-round security and confidentiality protection for cross-network synchronization of government data.

[0010] In order to achieve the above-mentioned object, the present invention provides a method for synchronizing government data across networks, wherein the synchronization method transmits files across networks through the following steps for mutually isolated government intranets and government extranets:

[0011] External network data generation stage,

[0012] Define the file format for cross-network data synchronization through the data dictionary engine, and process the external network data to be synchronized based on the file format defined by the data dictionary engine;

[0013] Data ferry stage,

[0014] Using polling, according to the definition of the data dictionary engine, the data files that meet the conditions defined by the data dictionary engine are transferred to the intranet; at the same time, data transmission reconciliation is performed, so that each type of data file is transmitted only once;

[0015] Intranet data import phase,

[0016] The file format for cross-network data synchronization is defined through the data dictionary engine, and the incoming external network data is parsed and verified based on the file format defined by the data dictionary engine.

[0017] In some embodiments of the present invention, the external network data generation stage includes:

[0018] Data dictionary engine configuration steps,

[0019] Based on the Nacos configuration center, the data dictionary engine dynamically configures the transmission channel connection, data upload and backup directory, and the file prefix and suffix rules used for various data synchronization;

[0020] Statically configure the load and export rules and data tables in the data dictionary engine based on XML files;

[0021] Data export steps,

[0022] According to the rules configured in the data dictionary engine configuration step, read the corresponding data tables, fields, and attachments from the external network data to be synchronized, and write them into the specified encrypted data file or attachment folder through the encryption algorithm;

[0023] Bill generation steps:

[0024] Write statement information into the data file exported in the data export step.

[0025] In some embodiments of the present invention, the data ferrying stage includes:

[0026] Data upload: read the upload directory configuration in the data dictionary engine, scan the specified prefix and suffix file list in the specified directory, and transmit it to the specified server through the configured transmission channel to complete the data upload;

[0027] Data download, read the download directory related configuration in the data dictionary, scan the specified prefix and suffix file list in the specified directory, and use the configured transmission channel to download to the specified directory;

[0028] Reconciliation is configured to perform data transmission reconciliation for the data uploaded in the data upload step and the data downloaded in the data download step, so that each type of data file is transmitted once and only once.

[0029] In some embodiments of the present invention, the intranet data import stage includes:

[0030] Data dictionary engine configuration steps, based on the Nacos configuration center, dynamically configure the data dictionary engine's transmission channel connection, data upload and backup directory, and the file prefix and suffix rules used for various data synchronization;

[0031] Statically configure the load and export rules and data tables in the data dictionary engine based on XML files;

[0032] Data decryption step: for each type of data, obtain the corresponding encryption and decryption algorithm and key from the data dictionary engine configured in the data dictionary engine configuration step, decrypt the data and convert it into a Json object, and combine it with the corresponding storage type, field list and type to realize intranet data storage;

[0033] In the bill generation step, when the intranet side imports data, it reads the bill information in the data file and changes the file flag in all bills from .sync to .zip. After one polling, the data reconciliation is completed. Every time a file is imported on the intranet side, the file name and import time are written into the bill for the next reconciliation.

[0034] In order to achieve the above object, the present invention provides a cross-network synchronization system for government data, the synchronization system comprising:

[0035] An external network data generation unit, the external network data generation unit being configured to define a file format for cross-network data synchronization through a data dictionary engine, and to process the external network data to be synchronized based on the file format defined by the data dictionary engine;

[0036] A data ferry unit, which is configured to interact with the external network data generation unit, and adopts a polling method to ferry data files that meet specific conditions to the internal network according to the definition of the data dictionary engine; and at the same time, data transmission reconciliation is performed, so that each type of data file is transmitted only once;

[0037] The intranet data import unit is configured to interact with the data ferry unit, define the file format of cross-network data synchronization through the data dictionary engine, and parse and verify the incoming external network data based on the file format defined by the data dictionary engine.

[0038] In some embodiments of the present invention, the external network data generating unit includes:

[0039] The external network data dictionary engine module is configured to dynamically configure the transmission channel connection, data upload and backup directory, and file prefix and suffix rules used for various data synchronization in the data dictionary engine based on the Nacos configuration center; and statically configure the loading and exporting rules and data tables in the data dictionary engine based on XML files;

[0040] A data export module, wherein the data export module is configured to interact with the external network data dictionary engine module and can read the corresponding data tables, fields and attachments from the external network data to be synchronized according to the rules configured in the data dictionary engine configuration step, and write them into the specified encrypted data file or attachment folder through the encryption algorithm;

[0041] The external network bill generation module is configured to interact with the data export module and can write statement information to the data file exported by the data export module.

[0042] In some embodiments of the present invention, the data ferry unit includes:

[0043] A data upload module, wherein the data upload module is configured to read the upload directory configuration in the data dictionary engine, scan the specified prefix and suffix file list under the specified directory, and transmit the file to the specified server through the configured transmission channel to complete the data upload;

[0044] A data download module, wherein the data download module is configured to read the download directory related configuration in the data dictionary, scan the specified prefix and suffix file list under the specified directory, and download to the specified directory using the configured transmission channel;

[0045] A reconciliation module, wherein the reconciliation module is configured to reconcile data transmission for data uploaded by the data upload module and data downloaded by the data download module, so that each type of data file is transmitted only once;

[0046] In some embodiments of the present invention, the intranet data import unit includes:

[0047] Intranet data dictionary engine module, the intranet data dictionary engine module is configured to dynamically configure the transmission channel connection, data upload and backup directory, and file prefix and suffix rules used for various data synchronization in the data dictionary engine based on the Nacos configuration center; and statically configure the loading and exporting rules and data tables in the data dictionary engine based on XML files;

[0048] A data decryption module, which is configured to obtain the corresponding encryption and decryption algorithm and key from the data dictionary engine for each type of data, decrypt the data and convert it into a Json object, and realize intranet data storage in combination with the corresponding storage type, field list and type;

[0049] The intranet bill generation module is configured to read the bill information in the data file when the intranet side imports data, change the file flag in all bills from .sync to .zip, and complete the data reconciliation after one polling.

[0050] In order to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps of the above-mentioned cross-network synchronization method of government data are implemented.

[0051] In order to achieve the above-mentioned purpose, the present invention also provides a processor, which is used to run a program, and when the program is running, the steps of the above-mentioned cross-network synchronization method of government data are executed.

[0052] In order to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and runnable on the processor, and the program code is loaded and executed by the processor to implement the steps of the above-mentioned cross-network synchronization method of government data.

[0053] In order to achieve the above-mentioned objectives, the present invention also provides a computer program product which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned method for synchronizing government data across networks.

[0054] The cross-network synchronization solution for government data provided by the present invention can ensure that data can cross different network boundaries (such as the network boundary between mutually isolated government intranets and government extranets), achieve seamless, secure and fast synchronization, and solve the problem of real-time, accurate and efficient synchronization of business data between different network environments.

[0055] The cross-network synchronization solution for government data provided by the present invention can effectively solve the problems of real-time, accuracy and efficiency in cross-network transmission of government data in a variety of network environments.

[0056] The cross-network synchronization solution for government data provided by the present invention can be used in various business scenarios of intranet and extranet data synchronization in government systems to build an adaptive, highly available, secure and confidential data synchronization module; at the same time, the solution can adapt to changes in synchronized data and implement the scope of data synchronization in a configurable manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0058] Figure 1 This is a flow chart of cross-network synchronization of government data in the present invention;

[0059] Figure 2 A flow chart for configuring the data dictionary engine in the present invention;

[0060] Figure 3 This is a data export flow chart in the present invention;

[0061] Figure 4 This is a data reconciliation flow chart in the present invention;

[0062] Figure 5 This is a schematic diagram of the cross-network synchronization system for government data in the present invention. DETAILED DESCRIPTION

[0063] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below with reference to specific diagrams.

[0064] In response to the problems of data security and data loss faced by cross-network transmission of government data, the solution of the present invention innovatively introduces a data dictionary engine to define innovative formats of cross-network transmission files of government data to ensure data standardization and security; on this basis, it further integrates data encryption and security verification mechanisms to provide comprehensive security and confidentiality protection for cross-network synchronization of government data.

[0065] See also Figure 1 , which is a flowchart of the implementation of the cross-network synchronization scheme of government data formed by the present invention. The cross-network synchronization scheme of government data formed by the present invention mainly includes three parts: external network data generation stage, data ferry stage, and internal network data import stage.

[0066] First, in the stage of generating external network data, the file format of cross-network data synchronization is defined through the data dictionary engine, and the external network data to be synchronized is processed based on the file format defined by the data dictionary engine;

[0067] Then, after completing the external network data generation phase, the data ferry phase begins. In the data ferry phase, a polling method is used to ferry data files that meet the conditions defined by the data dictionary engine to the internal network according to the data dictionary engine definition; at the same time, data transmission reconciliation is performed so that each type of data file is transmitted only once.

[0068] As a further explanation, during the specific implementation of this stage, the data upload module can be used to realize the input of external network data and the output of internal network data, the data download module can be used to download the corresponding data from the external network or internal network, and the data reconciliation module can be used to compare the reconciliation statements in the incoming and outgoing data files.

[0069] Finally, after completing the data ferrying phase, the intranet data import phase begins. In the intranet data import phase, the file format for cross-network data synchronization is defined through the data dictionary engine, and the incoming external network data is parsed and verified based on the file format defined by the data dictionary engine.

[0070] The following describes in detail the specific implementation process and corresponding technical features of each stage in the cross-network synchronization solution for government data provided by the present invention.

[0071] The scheme of the present invention realizes adaptive data export and data security encryption processing by introducing a data dictionary engine during the external network data generation stage.

[0072] The specific implementation of this external network data generation stage mainly includes the external network data dictionary engine configuration steps, data export steps and bill generation steps.

[0073] In the external network data dictionary engine configuration step, the file format of cross-network data synchronization and the definition of data to be synchronized are defined by introducing a data dictionary engine.

[0074] The definition of data to be synchronized described here specifically refers to files whose file names meet preset conditions (such as file names ending with .sync, etc.), indicating data that has not yet been reconciled. If reconciliation has not been completed for a long time, it needs to be retransmitted.

[0075] Furthermore, in this step, the files for cross-network data synchronization defined by the data dictionary engine are organized using a compressed file format, and the resulting compressed package contains an index file, several encrypted data files, a statement, and an attachment directory, wherein the index file specifically is the type of data that needs to be exported this time.

[0076] The compressed file format formed here is a combination of index files, attached encrypted data files and attachment directories, so that data transmission supports multiple data types. Different data formats can be defined for each type of data, and data can be imported and exported in combination with data dictionary engine configuration and index files. At the same time, the bill file provides a basic basis for data reconciliation in the data ferry stage.

[0077] As an example, the files for cross-network data synchronization defined by the data dictionary engine in this step can be organized in zip format. Files in zip format have the characteristics of clear organizational structure, convenient transmission, and saving storage space, which can meet a variety of internal and external network data synchronization business scenarios.

[0078] The solution of the present invention further provides a corresponding configuration solution for the introduced data dictionary engine.

[0079] The configuration of the data dictionary engine in the solution of the present invention specifically includes a dynamic configuration based on Nacos and a static configuration based on XML.

[0080] Among them, Nacos is used to implement the configuration for the parts of the data dictionary engine that need to be dynamically adjusted, and synchronized to the data dictionary engine in real time. The dynamic configuration items here mainly include: data transmission channel connection parameters (FTP account or HTTP interface information), data download and backup directory addresses, and prefix and suffix rules used for synchronization of various data files. Based on such dynamic configuration, the data export module can export the corresponding data files in a predetermined format and transmit them to the intranet by the data ferry unit, and the intranet side can also perform data analysis.

[0081] XML static configuration limits the scope of data synchronization and is mainly used to configure loading and exporting rules and data tables. The configuration items of export rules include the data source type, fields and types, and attachments to be exported. The configuration items of data tables include: data table meta-information to be synchronized (including table type, table name, export statement), data file name, attachment field list information (including attachment field name, index name), encrypted field list information (including field name, field type, encryption algorithm and key), etc. Based on such static configuration, the data synchronization solution to which this solution belongs is highly adaptive, and the synchronized data scope and encryption method can be adjusted according to business needs.

[0082] This solution achieves high efficiency and flexibility of the engine by combining dynamic and static configuration modes.

[0083] When configuring the data dictionary engine based on the data dictionary engine configuration scheme given in the solution of the present invention, XML static configuration is first performed to form the main scheme of the data dictionary engine; subsequently, in the actual application process, the data dictionary engine related projects are dynamically configured based on Nacos according to the actual application situation.

[0084] As a further explanation, when the XML static configuration data dictionary engine is used, the export rules are first loaded according to the configuration. The configuration items include the data source type, fields and types, and attachments to be exported. After the configuration and loading of the export rules are completed, the data table is configured. The configuration of the data table includes: table type, export statement, table name, data file name, encrypted field list, and attachment field list. Among them, the encrypted fields need to be decrypted one by one during the export process. The configuration items include: field name, field type, encryption algorithm, secret key, etc.

[0085] See also Figure 2 , which shows an example of a process solution for data dictionary engine configuration given by the present invention.

[0086] Based on the diagram, this solution includes the following steps when configuring the data dictionary engine:

[0087] (1) The data dictionary engine configuration step starts;

[0088] (2) Configure the loading and exporting rules. The configuration items include the data source type, fields and types, and attachments to be exported.

[0089] (3) Configure the exported data table and add a data table to be configured;

[0090] (4) Configure basic information for the data table added in step (3): table name, table type, and export statement;

[0091] (5) Further configure the attachment field for the data table in which the basic information configuration is completed in step (4), specifically configuring the attachment field name (i.e., the original file name) and the attachment index name;

[0092] (6) Further configure the encryption field configuration for the data table that completes the attachment field configuration in step (5), specifically configuring the field name, field type, encryption algorithm, and secret key.

[0093] (7) Add the next data table to be configured and repeat steps (3) to (6) to complete the data table configuration.

[0094] After completing the above-mentioned external network data dictionary engine configuration steps, this external network data generation stage will enter the corresponding data export step. In this data export step, according to the configuration of the data dictionary engine according to the external network data dictionary engine configuration steps, the corresponding data tables, fields, and attachments are read from the source data (i.e., the external network data to be synchronized), and then the read data tables, fields, and attachments are written into the specified encrypted data file or attachment folder through the symmetric encryption algorithm configured in the data dictionary engine, thereby generating the corresponding data transmission format file. Here, the read data table and field information are encrypted using the encryption algorithm and secret key, and then written into the encrypted data file, and the corresponding attachment file is written into the attachment folder.

[0095] See also Figure 3 , which shows the specific implementation process scheme of the data export step given by the present invention.

[0096] Based on the diagram, when exporting data, this solution first loads the configured data dictionary engine and reads the data dictionary configuration information, such as exporting data table information;

[0097] Next, according to the read data dictionary engine configuration information, the corresponding data is read from the source data (i.e. the external network data to be synchronized) and converted into JSON format; here, the read data is converted into JSON format, thereby achieving the scalability of internal and external network data transmission and ensuring the diversity of data types;

[0098] Next, for the read data, determine whether there are encrypted fields. If there are encrypted fields, first use the corresponding secret keys configured in the data dictionary engine to decrypt the relevant fields one by one;

[0099] It should be noted here that different encryption and decryption algorithms and keys can be configured for each type of data in the data dictionary engine.

[0100] Next, for the read data, determine whether there is an attachment field. If so, download the attachment files one by one and write them into the attachment folder, and store the corresponding attachment file name in the JSON data.

[0101] Finally, after the preprocessing of the read data is completed, the JSON data is encrypted using a unified encryption key, written into the data file, and the data file name is written into the index file. The index file contains: data file name, data type, and number of data items.

[0102] After completing the above-mentioned data export step, the external network data generation stage will enter the corresponding bill generation step, which is used to write the statement information in a compressed file package (such as zip format).

[0103] Specifically, the reconciliation information generated here includes: the processed intranet data file name, the number of data items, and the file sequence number.

[0104] As a further explanation, the bill generation is achieved through the following steps in the solution of the present invention:

[0105] When the external network is importing data, it reads the bill information in the data file and changes the file flag in all bills from .sync to .zip. On this basis, after one polling, the data reconciliation is completed. Finally, all .sync files are re-uploaded to complete the data retransmission to the internal network.

[0106] The scheme of the present invention enters the data ferrying stage after the external network data generation stage. It is specifically configured to rely on the configuration of the data dictionary engine and adopt a polling method to ferry data files that meet specific conditions to the intranet server, and confirm which files have been confirmed to be received based on the statement information in the ferrying files, and retransmit the files that have not been confirmed to be received, so that each type of data file is transmitted once and only once.

[0107] Furthermore, the data ferrying stage is implemented specifically by the coordination of three steps: data upload step, data download step, and data reconciliation step.

[0108] The data upload step in this phase is used to transfer data from the external network to the internal network.

[0109] Furthermore, in the data uploading step, the upload directory related configuration in the data dictionary engine is read, the specified prefix and suffix file list under the specified directory is scanned, and the data is transmitted to the specified server through the configured transmission channel (FTP or HTTP), the data upload is completed, and the data is ferried to the intranet by the one-way network gate device.

[0110] The data downloading step in this phase is used to download corresponding data from the external network or the internal network.

[0111] Furthermore, the data downloading step reads the download directory related configuration in the data dictionary, scans the specified prefix and suffix file list under the specified directory, and uses the configured transmission channel (FTP or HTTP) to download to the specified directory.

[0112] The data reconciliation step in this phase is used to compare the reconciliation statements in the incoming and outgoing data files.

[0113] Furthermore, in the data reconciliation step, it is configured to perform data transmission reconciliation for the data uploaded in the data upload step and the data downloaded in the data download step, so that each type of data file is transmitted once and only once.

[0114] Specifically, when importing data on the external network side, the bill information in the data file is read, and the file flag in all bills is changed from .sync to .zip; on this basis, after a polling, the data reconciliation is completed; finally, the data is re-uploaded for all .sync files to complete the data retransmission to the internal network.

[0115] When importing data on the intranet side, the bill information in the data file is read and the file flags in all bills are changed from .sync to .zip. On this basis, after one polling, the data reconciliation is completed. Finally, the data is re-uploaded for all .sync files to complete the data retransmission to the external network. In addition, every time a file is imported on the intranet side, the file name and import time of the file are written into the bill for the next reconciliation.

[0116] As an example, here we take the scenario of transferring data from the external network to the internal network as an example to explain the data reconciliation process in this stage.

[0117] See also Figure 4 For scenarios where data files need to be transferred from the government extranet to the government intranet, during the data transfer phase, when reconciliation is not completed, the corresponding compressed file package name stored on the government extranet is test.zip.sync.

[0118] If the government intranet processing is completed and the reconciliation information is transmitted, the corresponding compressed file package name will be automatically changed to test.zip through the data reconciliation step during the data ferrying stage, indicating that the transmission has been completed.

[0119] If you do not receive the reconciliation information from the intranet within the deadline, you will need to retransmit all .sync files.

[0120] This ensures that when data is transmitted, relying on the configuration of the data dictionary engine, a polling method is used to ferry data files that meet specific conditions to the intranet server, and confirm which files have been confirmed to be received based on the statement information in the ferrying files, and retransmit the files that have not been confirmed to be received, so that each type of data file is transmitted once and only once, avoiding data loss caused by internal and external network data asynchrony; or multiple data transmissions caused by resource waste or business logic problems.

[0121] As a further explanation, during the data ferrying stage, the solution of the present invention specifically implements corresponding file data synchronization through a network gateway device to cross the boundary of the government network; to ensure data security, through the configuration of the data dictionary engine, data synchronization is limited to a specific range, and the data files during the synchronization process are all in encrypted format, avoiding data security issues.

[0122] The solution of the present invention enters the intranet data import stage after the data ferry stage, in which external network data is parsed and statements are generated. This stage specifically relies on the data dictionary engine, combined with Nacos and XML configuration information, and for each type of data, obtains the corresponding encryption and decryption algorithm and key from the data dictionary engine, decrypts the data and converts it into a Json object, combined with the corresponding storage type, field list and type, to realize intranet data storage.

[0123] Furthermore, the specific implementation of this intranet data import stage is mainly formed by the cooperation of three steps: intranet data dictionary engine configuration step, data decryption step, and bill generation step.

[0124] The intranet data dictionary engine configuration steps in this stage include dynamic configuration based on Nacos and static configuration based on XML.

[0125] Among them, Nacos is used to implement the configuration for the parts of the data dictionary engine that need to be dynamically adjusted, and synchronized to the data dictionary engine in real time. The dynamic configuration items here mainly include: data transmission channel connection parameters (FTP account or HTTP interface information), data download and backup directory addresses, and prefix and suffix rules used for synchronization of various data files. Based on such dynamic configuration, the data export module can export the corresponding data files in a predetermined format and transmit them to the intranet by the data ferry unit, and the intranet side can also perform data analysis.

[0126] XML static configuration limits the scope of data synchronization and is mainly used to configure loading and exporting rules and data tables. The configuration items of export rules include the data source type, fields and types, and attachments to be exported. The configuration items of data tables include: data table meta-information to be synchronized (including table type, table name, export statement), data file name, attachment field list information (including attachment field name, index name), encrypted field list information (including field name, field type, encryption algorithm and key), etc. Based on such static configuration, the data synchronization solution to which this solution belongs is highly adaptive, and the synchronized data scope and encryption method can be adjusted according to business needs.

[0127] This solution achieves high efficiency and flexibility of the engine by combining dynamic and static configuration modes.

[0128] The data decryption step in this phase obtains the corresponding encryption and decryption algorithm and key from the configured data dictionary engine for each type of data, decrypts the data and converts it into a Json object, and combines it with the corresponding storage type, field list and type to implement intranet data storage.

[0129] In the bill generation step of this phase, when the intranet side imports data, it reads the bill information in the data file and changes the file flag in all bills from .sync to .zip; and on this basis, after one polling, the data reconciliation is completed; finally, the data is re-uploaded for all .sync files to complete the data retransmission to the external network. Furthermore, every time a file is imported on the intranet side, the file name and import time of the file are written into the bill for the next reconciliation.

[0130] The resulting cross-network synchronization solution for government data can realize the export of data in a specific range of the external network through the configuration of the data dictionary engine, and encrypt the exported data. The files are transferred to the internal network through the data ferrying stage, and the internal network data import unit decrypts and imports the data according to the corresponding configuration. The data ferrying stage also reconciles the incoming and outgoing data files, and retransmits the files that have not been reconciled for a long time.

[0131] The cross-network synchronization scheme for government data provided by the present invention can be used to form a corresponding software program and a corresponding cross-network synchronization system for government data in specific applications. When the software program is running, it will execute the above-mentioned cross-network synchronization method for government data and store it in a corresponding storage medium for the processor to retrieve and execute.

[0132] See also Figure 5 The cross-network synchronization system 100 of government data thus formed mainly includes three functional units: an external network data generation unit 110, a data ferry unit 120, and an internal network data import unit 130.

[0133] The external network data generating unit 110 in the present system is configured to define the file format of cross-network data synchronization through a data dictionary engine, and process the external network data to be synchronized based on the file format defined by the data dictionary engine.

[0134] The data ferry unit 120 in this system is configured to interact with the external network data generation unit 110, and can adopt a polling method to ferry data files that meet specific conditions to the intranet server according to the definition of the data dictionary engine. It can also perform data transmission reconciliation to ensure that various data files are transmitted once and only once.

[0135] The intranet data import unit 130 in this system is configured to interact with the data ferry unit 120, and define the file format of cross-network data synchronization through the data dictionary engine, and parse and verify the incoming external network data based on the file format defined by the data dictionary engine.

[0136] As a further explanation, the external network data generation unit 110 in the present system can realize adaptive data export and data security encryption functions.

[0137] Specifically, the external network data generating unit 110 is specifically composed of an external network data dictionary engine module 111 , a data export module 112 , and an external network bill generating module 113 .

[0138] The external network data dictionary engine module 111 has a dynamic configuration submodule and a static configuration submodule.

[0139] The dynamic configuration submodule is set to dynamically configure the parts of the external network data dictionary engine that need to be dynamically adjusted based on the Nacos configuration center and synchronize them to the data dictionary engine in real time.

[0140] The dynamic configuration submodule is configured to dynamically configure the data transmission channel connection parameters (FTP account or HTTP interface information), the data download and backup directory address, and the prefix and suffix rules used for synchronization of various data files.

[0141] The static configuration submodule can generate XML files for static configuration, which is mainly used to configure the external network data dictionary engine to load export rules and data tables.

[0142] The configuration items of the static configuration submodule for export rules include the data source type, fields and types, and attachments to be exported; the configuration items for data tables include: data table meta-information to be synchronized (including table type, table name, export statement), data file name, attachment field list information (including attachment field name, index name), encrypted field list information (including field name, field type, encryption algorithm and key), etc.

[0143] The data export module 112 is configured to interact with the external network data dictionary engine module 111, and can read the corresponding data tables, fields and attachments from the external network data to be synchronized according to the rules configured in the data dictionary engine configuration steps, and write them into the specified encrypted data file or attachment folder through the encryption algorithm.

[0144] The external network bill generation module 113 is configured to interact with the data export module 112 and can write statement information to the data file exported by the data export module.

[0145] As a further explanation, the data ferry unit 120 in the present system is specifically composed of a data upload module 121 , a data download module 122 and a reconciliation module 123 .

[0146] The data upload module 121 is used to complete the transmission of external network data and internal network data.

[0147] Specifically, the data upload module 121 is configured to read the upload directory configuration in the data dictionary engine, scan the specified prefix and suffix file list in the specified directory, transmit it to the specified server through the configured transmission channel (FTP or HTTP), complete the data upload, and ferry it to the intranet network by the one-way network gate device;

[0148] The data download module 122 is used to download corresponding data from the external network or the internal network.

[0149] Specifically, the data download module 122 is configured to read the download directory related configuration in the data dictionary, scan the specified prefix and suffix file list under the specified directory, and download it to the specified directory using the configured transmission channel (FTP or HTTP);

[0150] The reconciliation module 123 is used to compare the reconciliation statements in the incoming and outgoing data files.

[0151] Specifically, the reconciliation module 123 is configured to perform data transmission reconciliation for data upload by the data upload module and data download by the data download module, so that each type of data file is transmitted once and only once.

[0152] As a further explanation, the reconciliation module 123 is configured to read the bill information in the data file when the external network is importing data, and change the file flag in all bills from .sync to .zip; on this basis, after one polling, the data reconciliation is completed; finally, the data upload operation is re-performed on all .sync files to complete the data retransmission to the internal network;

[0153] When importing data on the intranet side, the bill information in the data file is read and the file flags in all bills are changed from .sync to .zip. On this basis, after one polling, the data reconciliation is completed. Finally, the data is re-uploaded for all .sync files to complete the data retransmission to the external network. In addition, every time a file is imported on the intranet side, the file name and import time of the file are written into the bill for the next reconciliation.

[0154] As a further explanation, the intranet data import unit 130 in the present system implements the functions of extranet data analysis and statement generation.

[0155] Specifically, the intranet data import unit 130 is composed of an intranet data dictionary engine module 131 , a data decryption module 132 and an intranet bill generation module 133 .

[0156] The intranet data dictionary engine module 131 also has a dynamic configuration submodule and a static configuration submodule.

[0157] The dynamic configuration submodule is set to dynamically configure the parts of the intranet data dictionary engine that need to be dynamically adjusted based on the Nacos configuration center and synchronize them to the data dictionary engine in real time.

[0158] The dynamic configuration submodule is configured to dynamically configure the data transmission channel connection parameters (FTP account or HTTP interface information), the data download and backup directory address, and the prefix and suffix rules used for synchronization of various data files.

[0159] The static configuration submodule can generate XML files for static configuration, which is mainly used to configure the intranet data dictionary engine to load export rules and data tables.

[0160] The configuration items of the static configuration submodule for export rules include the data source type, fields and types, and attachments to be exported; the configuration items for data tables include: storage type, data table meta-information to be synchronized (including table type, table name, export statement), data file name, attachment field list information (including attachment field name, index name), encrypted field list information (including field name, field type, encryption algorithm and key), etc.

[0161] The data decryption module 132 is configured to obtain the corresponding encryption and decryption algorithm and key from the data dictionary engine generated by the intranet data dictionary engine module 131 for each type of data, decrypt the data and convert it into a Json object, and realize intranet data storage in combination with the corresponding storage type, field list and type;

[0162] The intranet bill generation module 133 is configured to read the bill information in the data file when the intranet side imports data, change the file flag in all bills from .sync to .zip, and on this basis, complete the data reconciliation after one polling; finally, re-upload the data for all .sync files to complete the data retransmission to the external network.

[0163] In order to further illustrate the operation process and corresponding technical characteristics of the cross-network synchronization solution for government data constituted by this solution, the following will use specific application examples to illustrate the implementation process of this cross-network synchronization solution for government data to realize the synchronization of data between the internal and external networks of the government system.

[0164] When this cross-network synchronization solution for government data is put into practice, the corresponding cross-network synchronization system or platform for government data is first built based on the aforementioned solution, and then cross-network transmission and real-time synchronization between government intranet data and government extranet data are carried out accordingly.

[0165] Before synchronizing intranet and extranet data, you first need to configure the data dictionary engine module of the intranet and extranet modules, including the data source type to be synchronized (relational database, full-text index library), data table list (table type, export statement, table name, data file name), encrypted field list (field name, field type, encryption algorithm, secret key), attachment list (field name, storage location), and storage type.

[0166] Then initialize the data dictionary engine according to the set format, and set the encryption and decryption keys used for internal and external network synchronization. The external network data export module will generate the corresponding data files according to the configuration, and the data ferry module will ferry the files to the internal network.

[0167] The intranet data import module imports relevant data according to the configuration of the data dictionary engine, and writes the list of parsed and imported data files into the reconciliation file. The data ferry module compares the list of files transmitted from the external network to the internal network and from the internal network to the external network, and promptly completes the data files that are lost or failed to synchronize. Here, at each reconciliation, the files that appear in the bill will be changed to the .zip suffix. After multiple rounds of data synchronization, files that have not been synchronized for a long time (that is, the file name still has the .sync suffix) need to be retransmitted.

[0168] In this way, by configuring the data dictionary engine on the external network side, configuring the data range to be synchronized to the internal network, the encryption algorithm and key used, and the data transmission channel type, the data export module exports the corresponding data through the data dictionary engine; the data ferry module realizes the cross-network transmission of data files through the network gateway device, and makes up for the data files that have not been synchronized for a long time; correspondingly, on the internal network side, the same type of data dictionary engine is configured to parse and import the received data files.

[0169] From the above examples, it can be seen that the cross-network synchronization solution for government data provided by the present invention can ensure that the data of the government intranet and the government external network can cross different network boundaries, achieve seamless, secure and fast synchronization, and solve the problem of real-time, accurate and efficient synchronization of business data between different network environments.

[0170] As further explanation, the present invention further provides the following implementation scheme.

[0171] An embodiment of the present invention also provides a processor, which is used to run a program, wherein the program executes the steps of the above-mentioned cross-network synchronization method for government data when running.

[0172] An embodiment of the present invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. The program code is loaded and executed by the processor to implement the steps of the above-mentioned cross-network synchronization method for government data.

[0173] The present invention also provides a computer program product which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned method for cross-network synchronization detection of government data.

[0174] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0175] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0176] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0178] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0180] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0181] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0182] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0183] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0184] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0185] The above-mentioned method of the present invention, or a specific system unit, or a part of the same, is a pure software architecture, which can be arranged on a physical medium, such as a hard disk, an optical disk, or any electronic device (such as a smart phone, a computer-readable storage medium) through program code. When the machine loads the program code and executes it (such as a smart phone loading and executing it), the machine becomes a device for implementing the present invention. The above-mentioned method and device of the present invention can also be transmitted in the form of program code through some transmission media, such as cables, optical fibers, or any transmission mode. When the program code is received, loaded and executed by a machine (such as a smart phone), the machine becomes a device for implementing the present invention.

[0186] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for synchronizing government data across networks, characterized in that: The synchronization method uses the following steps to transfer files across the isolated government intranet and government extranet: External network data generation stage, Define the file format for cross-network data synchronization through the data dictionary engine, and process the external network data to be synchronized based on the file format defined by the data dictionary engine; Data ferry stage, Using polling, according to the definition of the data dictionary engine, the data files that meet the conditions defined by the data dictionary engine are transferred to the intranet; at the same time, data transmission reconciliation is performed, so that each type of data file is transmitted only once; Intranet data import phase, The file format for cross-network data synchronization is defined through the data dictionary engine, and the incoming external network data is parsed and verified based on the file format defined by the data dictionary engine.

2. The cross-network synchronization method for government data according to claim 1, characterized in that: The external network data generation stage includes Data dictionary engine configuration steps, Based on the Nacos configuration center, the data dictionary engine dynamically configures the transmission channel connection, data upload and backup directory, and the file prefix and suffix rules used for various data synchronization; and statically configures the data dictionary engine's loading and export rules and data tables based on XML files; Data export steps, According to the rules configured in the data dictionary engine configuration step, read the corresponding data tables, fields, and attachments from the external network data to be synchronized, and write them into the specified encrypted data file or attachment folder through the encryption algorithm; Bill generation steps: Write statement information into the data file exported in the data export step.

3. The cross-network synchronization method for government data according to claim 1 is characterized in that: The data ferrying stage includes: Data upload: read the upload directory configuration in the data dictionary engine, scan the specified prefix and suffix file list in the specified directory, and transmit it to the specified server through the configured transmission channel to complete the data upload; Data download, read the download directory related configuration in the data dictionary, scan the specified prefix and suffix file list in the specified directory, and use the configured transmission channel to download to the specified directory; Reconciliation is configured to perform data transmission reconciliation for the data uploaded in the data upload step and the data downloaded in the data download step, so that each type of data file is transmitted once and only once.

4. The cross-network synchronization method for government data according to claim 1, characterized in that: The intranet data import phase includes: Data dictionary engine configuration steps, based on the Nacos configuration center, dynamically configure the data dictionary engine's transmission channel connection, data upload and backup directory, and file prefix and suffix rules used for various data synchronization; Statically configure the load and export rules and data tables in the data dictionary engine based on XML files; Data decryption step: for each type of data, obtain the corresponding encryption and decryption algorithm and key from the data dictionary engine configured in the data dictionary engine configuration step, decrypt the data and convert it into a Json object, and combine it with the corresponding storage type, field list and type to realize intranet data storage; In the bill generation step, when the intranet side imports data, it reads the bill information in the data file and changes the file flag in all bills from .sync to .zip. After one polling, the data reconciliation is completed. Every time a file is imported on the intranet side, the file name and import time are written into the bill for the next reconciliation.

5. A cross-network synchronization system for government data, characterized in that: The synchronization system comprises: An external network data generation unit, the external network data generation unit being configured to define a file format for cross-network data synchronization through a data dictionary engine, and to process the external network data to be synchronized based on the file format defined by the data dictionary engine; A data ferry unit, which is configured to interact with the external network data generation unit, and adopts a polling method to ferry data files that meet specific conditions to the internal network according to the definition of the data dictionary engine; and at the same time, data transmission reconciliation is performed, so that each type of data file is transmitted only once; The intranet data import unit is configured to interact with the data ferry unit, define the file format of cross-network data synchronization through the data dictionary engine, and parse and verify the incoming external network data based on the file format defined by the data dictionary engine.

6. The cross-network synchronization system for government data according to claim 5 is characterized in that: The external network data generating unit comprises: The external network data dictionary engine module is configured to dynamically configure the transmission channel connection, data upload and backup directory, and file prefix and suffix rules used for various data synchronization in the data dictionary engine based on the Nacos configuration center; and statically configure the loading and exporting rules and data tables in the data dictionary engine based on XML files; A data export module, wherein the data export module is configured to interact with the external network data dictionary engine module and can read the corresponding data tables, fields and attachments from the external network data to be synchronized according to the rules configured in the data dictionary engine configuration step, and write them into the specified encrypted data file or attachment folder through the encryption algorithm; The external network bill generation module is configured to interact with the data export module and can write statement information to the data file exported by the data export module.

7. The cross-network synchronization system for government data according to claim 5, characterized in that: The data ferry unit comprises: A data upload module, wherein the data upload module is configured to read the upload directory configuration in the data dictionary engine, scan the specified prefix and suffix file list under the specified directory, and transmit the file to the specified server through the configured transmission channel to complete the data upload; A data download module, wherein the data download module is configured to read the download directory related configuration in the data dictionary, scan the specified prefix and suffix file list under the specified directory, and download to the specified directory using the configured transmission channel; The reconciliation module is configured to perform data transmission reconciliation for data uploaded by the data upload module and data downloaded by the data download module, so that each type of data file is transmitted only once.

8. The cross-network synchronization system for government data according to claim 5, characterized in that: The intranet data import unit includes: Intranet data dictionary engine module, the intranet data dictionary engine module is configured to dynamically configure the transmission channel connection, data upload and backup directory, and file prefix and suffix rules used for various data synchronization in the data dictionary engine based on the Nacos configuration center; and statically configure the loading and exporting rules and data tables in the data dictionary engine based on XML files; A data decryption module, which is configured to obtain the corresponding encryption and decryption algorithm and key from the data dictionary engine for each type of data, decrypt the data and convert it into a Json object, and realize intranet data storage in combination with the corresponding storage type, field list and type; The intranet bill generation module is configured to read the bill information in the data file when the intranet side imports data, change the file flag in all bills from .sync to .zip, and complete the data reconciliation after one polling.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by the processor, the steps of the cross-network synchronization method of government data described in any one of claims 1-4 are implemented.

10. A computer program product, characterized in that When executed on a data processing device, it is suitable for executing the steps of the cross-network synchronization method of government data described in any one of claims 1-4.

Images