Cross-network data transmission method and cross-network data transmission system

Abstract

The present disclosure provides a cross-network data transmission method, comprising: when target ferry data satisfying a target ferry data rule in a first database in a first information site changes, a ferry data generation module generates a corresponding operation record, and writes the operation record into a target ferry log file corresponding to the target ferry data rule; when cross-network data transmission is required, the ferry data generation module generates a ferry data packet, and sends the ferry data packet to a data review and cross-network transmission module; the data review and cross-network transmission module performs a security audit on each target ferry log file in the ferry data packet, and writes the ferry data packet into a data storage medium after the security audit passes; a ferry data redo module reads the ferry data packet in the data storage medium, and redoes the operation on a second database in a second information site according to each operation record recorded in each target ferry log file in the ferry data packet.

Description

Technical Field

[0001] The present disclosure relates to the field of data communication technology, and in particular to a cross-network data transmission method and a cross-network data transmission system. Background Art

[0002] A company may establish multiple networks with different confidentiality control levels, such as the freely usable Internet (low control level), the company intranet that ordinary employees can use (medium control level), and the confidential network that core personnel can use (high control level). In order to ensure absolute information security, these networks are physically isolated. Although the physical networks are isolated, there is a need for data exchange in each network. Therefore, how to achieve cross-network data transmission between information sites in two physically isolated networks has become a technical problem that needs to be solved in this field. Summary of the invention

[0003] The present disclosure aims to solve at least one of the technical problems existing in the prior art, and proposes a cross-network data transmission method and a cross-network data transmission system.

[0004] In a first aspect, an embodiment of the present disclosure provides a cross-network data transmission method, which is used to implement cross-network data transmission from a first information site located in a first network to a second information site located in a second network, wherein the first network is physically isolated from the second network, and the cross-network data transmission method is based on a cross-network data transmission system, wherein the cross-network data transmission system includes: a ferry data generation module, a data review and cross-network transmission module, and a ferry data redo module, wherein the ferry data generation module is deployed in the first information site, and the ferry data redo module is deployed in the second information site;

[0005] The data ferrying method comprises:

[0006] When the target ferry data satisfying the pre-configured target ferry data rule in the first database of the first information site changes, the ferry data generation module deployed in the first information site generates an operation record corresponding to the change of the target ferry data, and writes the operation record into the target ferry log file corresponding to the target ferry data rule;

[0007] When cross-network data transmission is required, the ferry data generation module generates a ferry data packet and sends the ferry data packet to the data review and cross-network transmission module, wherein the ferry data packet includes: at least one target ferry log file;

[0008] The data review and cross-network transmission module performs a security audit on each of the target ferry log files in the ferry data packet, and writes the ferry data packet into the data storage medium after the security audit passes;

[0009] The ferry data redo module deployed at the second information site reads the ferry data packet in the data storage medium, and performs redo operations on the second database in the second information site according to each operation record recorded in each target ferry log file in the ferry data packet.

[0010] In some embodiments, the data transmission system further includes: a ferrying scheme design module;

[0011] Before the step of generating, by the ferry data generating module deployed at the first information site, an operation record corresponding to the change of the target ferry data, the method further includes:

[0012] In response to the target design instruction, the ferry scheme design module generates a corresponding target ferry scheme, wherein the target ferry scheme defines a target ferry route to be established between the first information site and the second information site, and the ferry data flow direction of the target ferry route is from the first information site to the second information site, and the rules satisfied by the ferry data allowed to be transmitted by the target ferry route are the target ferry data rules;

[0013] The ferrying scheme design module sends the target ferrying scheme to the first information site and the second information site;

[0014] The ferry data generation module deployed at the first information site creates a ferry log writing thread corresponding to the target ferry data rule.

[0015] In some embodiments, the step of writing the operation record into the target ferry log file corresponding to the target ferry data rule includes:

[0016] The ferry data generation module detects whether the file size of the target ferry log file to be written currently exceeds a preset threshold, and detects whether the writable time period corresponding to the target ferry log file to be written currently covers the current time;

[0017] When the file size of the target ferry log file to be written does not exceed the preset threshold and the writable period corresponding to the target ferry log file to be written covers the current time, the ferry data generation module writes the operation record into the target ferry log file to be written;

[0018] When the file size of the target ferry log file to be written currently exceeds a preset threshold, or when the writable period corresponding to the target ferry log file to be written currently does not cover the current time, the ferry data generation module creates a new target ferry log file and writes the operation record into the new target ferry log file.

[0019] In some embodiments, when the ferry data generation module creates a new target ferry log file, it also includes:

[0020] The ferry data generation module writes the ferry log file information of the newly created target ferry log file into the log file list corresponding to the target ferry data rule;

[0021] The ferry data packet also includes the log file list.

[0022] In some embodiments, after the ferry data redo module reads the ferry data packet in the data storage medium, and before the ferry data redo module performs a redo operation on the second database in the second information site according to each operation record recorded in the target ferry log file in the ferry data packet, the step further includes:

[0023] The ferry data redo module adds the ferry log file information of each target ferry log file that should be included in the ferry data packet to a preset log file entry record file according to the log file list;

[0024] The log file placement record file records different ferry log file information and corresponding status information;

[0025] The status information includes: already-swung-in status, partially-swung-in status, waiting-to-swung-in status or missing status; wherein, the already-swung-in status indicates that all operation records recorded in the ferry log file have been successfully redone in the second database, the partially-swung-in status indicates that at least part of the operation records recorded in the ferry log file have redone exceptions in the second database, the waiting-to-swung-in status indicates that the ferry log file is waiting to be redone in the second database, and the missing status indicates that the ferry log file is in a missing state.

[0026] In some embodiments, the ferry data redo module performs a step of redoing the second database in the second information site according to each operation record recorded in the target ferry log file in the ferry data packet, including:

[0027] For the target ferry log file currently to be processed, the ferry data redo module generates a database transaction instruction corresponding to each operation record recorded in the target ferry log file currently to be processed, and sends the database transaction instruction to the persistence layer framework in the second information site, so that the persistence layer framework performs corresponding redo operations on the second database according to the received database transaction instruction.

[0028] In some embodiments, it is characterized in that after the step of the ferry data redo module sending the database transaction instruction to the persistence layer framework in the second information site, it also includes:

[0029] The ferry data redo module records in a preset redo record file the execution result of the persistence layer framework performing the corresponding redo operation on the second database according to the received database transaction instruction.

[0030] In some embodiments, the operation record includes at least one of: data added by a adding operation, primary keys of data deleted by a deleting operation, and values ​​of data changed by a changing operation before and after the change.

[0031] In some embodiments, the step of the ferry data generation module writing the operation record into the target ferry log file corresponding to the target ferry data rule comprises:

[0032] The ferry data generation module encrypts the operation record to be written into the target ferry log file, and then writes the encrypted operation record into the target ferry log file;

[0033] The step of the data review and cross-network transmission module performing a security review on each of the target ferry log files in the ferry data packet includes:

[0034] The operation records in each of the target ferry log files are decrypted, and the decrypted operation records are displayed for external review.

[0035] In a second aspect, an embodiment of the present disclosure further provides a cross-network data transmission system, which is used to implement cross-network data transmission from a first information site located in a first network to a second information site located in a second network, wherein the first network is physically isolated from the second network, and the cross-network data transmission method is based on the cross-network data transmission system, wherein the cross-network data transmission system includes: a ferry data generation module, a data review and cross-network transmission module, and a ferry data redo module, wherein the ferry data generation module is deployed in the first information site, and the ferry data redo module is deployed in the second information site;

[0036] The ferry data generation module is configured to generate an operation record corresponding to the change of the target ferry data when the target ferry data satisfying the pre-configured target ferry data rule in the first database of the first information site changes, and write the operation record into the target ferry log file corresponding to the target ferry data rule, and is also configured to generate a ferry data packet when cross-network data transmission is required, and send the ferry data packet to the data review and cross-network transmission module, the ferry data packet including: at least one of the target ferry log files;

[0037] The data review and cross-network transmission module is configured to perform a security review on each of the target ferry log files in the ferry data packet, and write the ferry data packet into the data storage medium after the security review passes;

[0038] The ferry data redo module is configured to read the ferry data packet in the data storage medium, and perform a redo operation on the second database in the second information site according to each operation record recorded in each target ferry log file in the ferry data packet. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The present invention is a schematic diagram of an application scenario of the technical solution involved in the present invention.

[0040] Figure 2 A schematic diagram of a ferrying solution in the present disclosure.

[0041] Figure 3A A schematic diagram of ferrying log files in an embodiment of the present disclosure.

[0042] Figure 3B A schematic diagram of a ferry data generation module writing operation records into a ferry log file in an embodiment of the present disclosure.

[0043] Figure 4 It is a schematic diagram of the ferry data redo module performing data redo based on operation records in an embodiment of the present disclosure.

[0044] Figure 5 A flowchart of a cross-network data transmission method provided in an embodiment of the present disclosure.

[0045] Figure 6 A flowchart of another cross-network data transmission method provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the present disclosure is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0047] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described target changes, the relative positional relationship may also change accordingly.

[0048] In each of the accompanying drawings, identical elements are represented by similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not all drawn to scale. In addition, some well-known parts may not be shown in the drawings.

[0049] Many specific details of the present disclosure are described below, such as component structures, materials, dimensions, processing techniques, and technologies, in order to more clearly understand the present disclosure. However, as those skilled in the art will appreciate, the present disclosure may be implemented without following these specific details.

[0050] Figure 1 Schematic diagram of an application scenario of the technical solution involved in this disclosure. Figure 1 As shown, the technical solution of the present disclosure is based on a cross-network data transmission system, which can realize regular or irregular cross-network data transmission between two information sites located in two physically isolated networks. The cross-network data transmission system includes: a ferry data generation module, a data review and cross-network transmission module, and a ferry data redo module.

[0051] The ferry data generation module is deployed at the source information site, and the ferry data redo module is deployed at the target information site. Of course, in some scenarios, a certain information site needs to send cross-network data to other information sites and receive cross-network data sent by other information sites, so the information site is configured with both a ferry data generation module and a ferry data redo module. Therefore, in actual applications, which information sites in each network are only configured with a ferry data generation module, which information sites are only configured with a ferry data redo module, and which information sites are simultaneously configured with a ferry data generation module and a ferry data redo module can be configured accordingly according to actual needs, and this disclosure does not limit this.

[0052] In some embodiments, the cross-network data transmission system further includes: a ferrying scheme design module, which can define a ferrying scheme.

[0053] Some important functional modules and objects involved in this disclosure are described below:

[0054] 1) Ferry scheme design module

[0055] Considering the information sites of the entire unit deployed in different networks, we can use the ferry plan design module to define the ferry plan. The ferry plan is a series of rules, which include the information site identifiers involved in the plan (the information site must be developed based on the persistence layer framework provided by the present disclosure), the data transmission routes between the information sites (also called "ferry routes"), and the rules that must be met on each ferry route to allow data transmission (also called "ferry data rules", such as the type of data running the transmission and the conditions that each data should meet). The ferry route connects the upstream and downstream information sites, and the ferry data can only flow from the upstream information site (source information site) of the ferry route to the downstream information site (target information site). Data transmission is not allowed between sites without a defined ferry route.

[0056] Figure 2 FIG. 1 is a schematic diagram of a ferrying solution in the present disclosure. Figure 2 As shown, information site A, information site B and information site C are in different networks and the networks of any two are physically isolated. In the ferrying scheme, a ferrying route 1 is defined to allow information site A to transmit data across the network to information site B, a ferrying route 2 is defined to allow information site B to transmit data across the network to information site A, a ferrying route 3 is defined to allow information site A to transmit data across the network to information site C, a ferrying route 4 is defined to allow information site C to transmit data across the network to information site A, and a ferrying route 5 is defined to allow information site B to transmit data across the network to information site C; each ferrying route is configured with a corresponding standard ferrying data rule. Based on the ferrying scheme, it can be seen that the system does not allow information site C to transmit data across the network to information site B.

[0057] It should be noted that the "ferry route" in the attached figure is a virtual route, which does not mean that data can be directly transmitted between the information sites at both ends of the ferry route, but it means that cross-network transmission based on the technical solution disclosed in this disclosure is allowed between the information sites at both ends.

[0058] After the ferrying scheme is designed, it can be sent to information site A, information site B, and information site C. Information site A, information site B, and information site C are all deployed with ferrying data generation modules and ferrying data redo modules in advance. The ferrying data generation modules and ferrying data redo modules in each information site act autonomously by understanding the rules in the ferrying scheme. This rule-driven ferrying scheme can reduce the cost of implementing the entire ferrying mechanism. It can also improve flexibility. For example, at a certain stage, information site B may no longer be allowed to transmit data across the network to information site C. At this time, you only need to delete the ferry route between information site B and information site C and redeploy the ferrying scheme.

[0059] 2) Ferry data rules

[0060] Each ferry route is configured with a corresponding ferry data rule. The ferry data rule is like a filter, which only allows data that meets the corresponding rules (i.e. ferry data) to flow to the target information site; the ferry data rule can define the conditions under which the newly created / modified / deleted data can be ferryed to the downstream information site. From another perspective, the standard ferry data rule can define which data in the source information site is the ferry data that can be transmitted across the network on the corresponding ferry route.

[0061] The present disclosure does not limit the standard ferry data rules designed for the ferry routes, which can be pre-designed and adjusted according to actual needs.

[0062] 3) Ferry log files

[0063] In the present disclosure, the ferry log file is used as the smallest unit for cross-network data transmission and review. The content in the ferry log file is the operation record generated by the ferry data generation module of the source information site based on the changes in the ferry data. Mapping the changes in the ferry data to individual ferry log files can ensure the integrity of the data. For example, the changes in a transaction will not cross the ferry log file; at the same time, it can also facilitate the data ferry system to manage the ferry process, for example, it can record the ferry data generated by the information site and the imported data at the granularity of the ferry log file.

[0064] The ferry log file records the corresponding operation records (for example, adding ferry data, changing ferry data, deleting ferry data) when the ferry data that meets the standard ferry data rules in the source information site changes. The content of the ferry log file is a collection of operation records encrypted by encryption facilities. An operation record corresponds to a change in a database transaction. The operation record may include: the data added by the new operation, the primary key of the data deleted by the deletion operation, and the value of the data changed by the change operation before and after the change (including the value before and after the change). At least one of the values, in addition, the operation record may include the information of the executor of the operation, the execution time, etc. By characterizing the data changes in a database transaction in the form of operation records, it is possible to avoid the problem that only part of the data changes of a ferry data in the database are transmitted to the destination information site. In addition, for the changes in the ferry data, the values ​​before and after the data change are recorded in the operation record, and this information can be used for the automatic data conflict resolution process. For details, please refer to the following description.

[0065] In actual applications, the ferry data changes of an information site may flow to multiple sites, such as Figure 2 Among information site A and information site B, the information site will maintain the same number of ferry log file writing threads as the number of export channels.

[0066] In addition, due to the different data specifications of different export waterways, if multiple different ferry data are operated separately when a transaction is submitted, different operation records may be written into multiple ferry log files.

[0067] In the disclosed embodiment, the ferry route, the ferry data rule and the ferry log file set correspond to each other one by one, and each ferry log file set may include one or more ferry log files. In some scenarios, when the file size of the target ferry log file to be written has reached a preset threshold, or the current time is not within the writable period corresponding to the file of the target ferry log file to be written, a new target ferry log file can be created, and the operation record can be written into the new target ferry log file.

[0068] 4) Ferry data generation module

[0069] The ferry data generation module should be able to record the corresponding operation records when any ferry data changes. One of the key points is to be able to capture any changes made by the data persistence layer. In this disclosure, a persistence layer framework is designed to capture any data changes based on the framework.

[0070] The information site disclosed in the present invention is developed based on a persistence layer framework, which maps a row of data (Record) in a database table in the information site into an object (GeneralItem) in the framework. GeneralItem provides several key methods: calling the load method to load an object from the database (load operation); calling the setProperty method to change the value of a field (change operation); calling the delete method to delete the Record corresponding to the current object (delete operation); calling the save method to persist changes to the object in the database (save operation). The persistence layer framework tracks changes to any object and can generate SQL files based on metadata and submit them to the database for execution. At the same time, it also provides key extension points (for the ferry data generation module to connect to the persistence layer framework). Based on this extension point, the ferry data generation module learns whether the data that has changed in the database is ferry data (whether it is data that meets the ferry data rules) and what operations have occurred on the ferry data, so that the ferry data generation module can write operation records to the corresponding ferry log file.

[0071] The ferry data generation module will not only write the ferry log file, but also write the list of generated ferry log files (i.e., log file list). The log file list will also be transmitted to the destination information site for analysis and use by the ferry data redo module of the destination information site.

[0072] Figure 3A A schematic diagram of ferrying log files in an embodiment of the present disclosure is shown. Figure 3B A schematic diagram of the ferry data generation module writing operation records into the ferry log file in the embodiment of the present disclosure. As shown in FIG3 , it illustrates the case where the value of the NAME field in the object with ID 003 in the WorkReport table is changed from "X" to "XXX".

[0073] 5)Data review and cross-network transmission module

[0074] At present, most units use the form of burning CDs to copy data from isolated networks to CDs, and then read the data in the CDs to other networks. The cross-network data transmission mechanism based on ferry log files adopted by the present invention can support this usage. Most confidential units stipulate that any data must be manually reviewed before being burned to a ferry CD; for this reason, in the present disclosure, when cross-network data transmission is required, the corresponding ferry data generation module will package at least one ferry log file that needs to be transmitted across the network (for example, all ferry log files generated in the current cycle or a certain period of time), generate a ferry data packet, and then send the ferry data packet to the data review and cross-network transmission module in the form of FTP. The data review and cross-network transmission module decrypts the operation records in each target ferry log file, and displays the decrypted operation records for review by the data reviewer.

[0075] After the data auditor confirms that the ferry data packet has passed the security audit, the ferry data packet will be burned to a CD, and the destination information site can obtain the ferry data packet by reading the information on the CD.

[0076] 6) Ferry data redo module

[0077] The core work of the ferry data redo module is to automatically read the ferry log file list and the content of the ferry log file, and re-execute the operation records recorded in the ferry log file in the destination information site (ie data "redo").

[0078] The ferry log file list records all the ferry log files generated by the source site. If the redo module finds that the specified ferry log file is missing locally, it will stop the redo process. This design can prevent data inconsistency caused by the loss of ferry log files.

[0079] The ferry data redo module reads and decrypts the contents of the ferry log file. It opens a database transaction for each operation record in the ferry log file. For each database transaction, it uses the persistence layer framework mentioned above to generate corresponding SQL statements and apply them to the database system of the destination information site.

[0080] Data conflicts may occur during data redo, for example, if the modified data does not exist in the destination information site. There are two strategies for handling this situation: one is to ignore; the other is to create it if it does not exist (using the latest state of the data in the ferry log record).

[0081] Figure 4 FIG. 1 is a schematic diagram of a ferry data redo module performing data redo based on operation records in an embodiment of the present disclosure. Figure 4As shown, it indicates that when the redo operation of modifying the value of the NAME field in the object with ID 003 in the WorkReport table from "X" to "XXX" is performed in the destination information site, since the value of the NAME field in the object with ID 003 in the WorkReport table in the database of the destination information site is "XX", it is inconsistent with the value "X" before the data change in the operation record, that is, there is a data conflict. At this time, there are two coping strategies: one is to apply the changes of the source information site (the NAME field value is set to "XXX"); the other is to apply the changes of the destination information site (the NAME field value remains "XX"). The data conflict handling strategy can be defined in the "ferry data rule". In other words, the ferry data rule can not only define which data in the source information site can be used as ferry data, but also define which conflict handling strategy can be used to handle data conflicts when data conflicts occur during the redo operation at the destination information site, so as to enable the destination information site to automatically handle data conflicts, reduce the work that requires manual intervention, and improve the ferrying efficiency.

[0082] 7) Log files are placed in record files

[0083] The log file placement record file records the ferry log files that have been successfully ferried and those waiting to be ferried. It is used to ensure that the operation record in each ferry log is only "replayed" once, and can show the execution progress and history of the current ferry process to the system administrator. The log file placement record file is generated based on the ferry log file list, and the status can be changed in real time during the redo process. Each record may be in one of the following states:

[0084] The state of being redone is: it means that all the operation records recorded in the ferry log file have been successfully redone in the second database.

[0085] Partially ferry-in state: indicates that at least part of the operation records recorded in the ferry log file have redo exceptions in the second database.

[0086] Waiting for redo status: indicates that the ferry log file is waiting to be redone in the second database.

[0087] Missing state: Indicates that the ferry log file is in a missing state (the list records that the file exists), and the "redo" process cannot continue.

[0088] 8) Redo log file

[0089] The ferry data redo module also monitors the execution results of each operation record redo and records the execution results in the redo record file. If there is an exception in the redo execution process of an operation record recorded in the redo record file, the reason for the redo exception will be recorded in the redo record file.

[0090] The technical solution of the present disclosure will be described in detail below with reference to specific examples.

[0091] Figure 5 The following is a flow chart of a method for cross-network data transmission provided by an embodiment of the present disclosure. Figure 5 As shown, the cross-network data transmission method is used to realize cross-network data transmission from a first information site located in a first network to a second information site located in a second network. The first network is physically isolated from the second network. The cross-network data transmission method is based on a cross-network data transmission system. The cross-network data transmission system includes: a ferry data generation module, a data review and cross-network transmission module and a ferry data redo module. The ferry data generation module is deployed in the first information site, and the ferry data redo module is deployed in the second information site.

[0092] Data ferrying methods include:

[0093] Step S1, when the target ferry data satisfying the pre-configured target ferry data rules in the first database of the first information site changes, the ferry data generation module deployed in the first information site generates an operation record corresponding to the change in the target ferry data, and writes the operation record into the target ferry log file corresponding to the target ferry data rules.

[0094] Step S2: When cross-network data transmission is required, the ferry data generation module generates a ferry data packet and sends the ferry data packet to the data review and cross-network transmission module. The ferry data packet includes: at least one target ferry log file.

[0095] Step S3: The data review and cross-network transmission module performs a security audit on each target ferry log file in the ferry data packet, and writes the ferry data packet into the data storage medium after the security audit passes.

[0096] In some embodiments, the data storage carrier includes: a physical carrier with data storage function such as a USB flash drive and an optical disk.

[0097] Step S4: The ferry data redo module deployed at the second information site reads the ferry data packet in the data storage medium, and performs redo operations on the second database in the second information site according to each operation record recorded in each target ferry log file in the ferry data packet.

[0098] In the disclosed embodiment, cross-network data transmission from a first information site to a second information site can be achieved based on the ferry log file, and has extremely high data transmission efficiency and data security.

[0099] Figure 6 Flow chart of another cross-network data transmission method provided by the embodiment of the present disclosure. Figure 6As shown, in some embodiments, the data transmission system further includes: a ferry scheme design module. Before step S1, it also includes: steps S01 to S03.

[0100] Step S01, in response to the target design instruction, the ferry scheme design module generates a corresponding target ferry scheme, the target ferry scheme defines a target ferry route between the first information site and the second information site, and the ferry data flow direction of the target ferry route is from the first information site to the second information site, and the rules satisfied by the ferry data allowed to be transmitted by the target ferry route are target ferry data rules.

[0101] The target design instruction is an instruction manually input into the ferry scheme design module based on actual needs, and is used to create a corresponding target ferry scheme in the ferry scheme design module.

[0102] Step S02: The ferrying scheme design module sends the target ferrying scheme to the first information site and the second information site.

[0103] Step S03: The ferry data generation module deployed at the first information site creates a ferry log writing thread corresponding to the target ferry data rule.

[0104] In some embodiments, the step of writing the operation record into the target ferry log file corresponding to the target ferry data rule includes:

[0105] Step S101, the ferry data generation module detects whether the file size of the target ferry log file to be written currently exceeds a preset threshold, and detects whether the writable time period corresponding to the target ferry log file to be written currently covers the current time;

[0106] Step S102: When the file size of the target ferry log file to be written does not exceed the preset threshold and the writable period corresponding to the target ferry log file to be written covers the current time, the ferry data generation module writes the operation record into the target ferry log file to be written.

[0107] Step S103: When the file size of the target ferry log file to be written currently exceeds a preset threshold, or when the writable period corresponding to the target ferry log file to be written currently does not cover the current time, the ferry data generation module creates a new target ferry log file and writes the operation record into the new target ferry log file.

[0108] In some embodiments, while the ferry data generation module creates a new target ferry log file, it also includes: the ferry data generation module writes the ferry log file information of the newly created target ferry log file into the log file list corresponding to the target ferry data rule; the ferry data packet also includes a log file list.

[0109] In some embodiments, after the ferry data redo module reads the ferry data packet in the data storage medium, and before the ferry data redo module performs a step of redoing the operation on the second database in the second information site according to each operation record recorded in the target ferry log file in the ferry data packet, it also includes: the ferry data redo module adds the ferry log file information of each target ferry log file to be included in the ferry data packet to a pre-set log file placement record file according to the log file list; the log file placement record file records different ferry log file information and corresponding status information;

[0110] In some embodiments, the ferry data redo module performs the steps of redoing the second database in the second information site according to the operation records recorded in the target ferry log file in the ferry data packet, including: for the target ferry log file currently to be processed, the ferry data redo module generates a database transaction instruction corresponding to the operation record for each operation record recorded in the target ferry log file currently to be processed, and sends the database transaction instruction to the persistence layer framework in the second information site, so that the persistence layer framework performs the corresponding redo operation on the second database according to the received database transaction instruction.

[0111] In some embodiments, after the ferry data redo module sends the database transaction instruction to the persistence layer framework in the second information site, it also includes: the ferry data redo module records in a pre-set redo record file the execution results of the persistence layer framework performing the corresponding redo operation on the second database according to the received database transaction instruction.

[0112] In some embodiments, the step in which the ferry data generation module writes the operation record into the target ferry log file corresponding to the target ferry data rule includes: the ferry data generation module encrypts the operation record to be written into the target ferry log file, and then writes the encrypted operation record into the target ferry log file.

[0113] The steps of the data review and cross-network transmission module performing security review on each target ferry log file in the ferry data packet include: decrypting the operation records in each target ferry log file, and displaying the decrypted operation records for external review.

[0114] The technical solution disclosed in the present invention has the following advantages: (1) data can be transmitted between physically isolated networks; (2) it has a high data transmission efficiency; (3) it supports manual review of data before confidential data is exported from a confidential network to ensure that secrets are not leaked; (4) different data changes in one site may need to be transferred to networks with different confidentiality levels, and the transfer system can support the definition of which data that meets certain conditions should be transferred to which destination site; (5) it can avoid conflicts between data changes in one site and data changes from another site, or it can automatically handle data conflicts; (6) it can ensure that changes in the source information site are transmitted and only transmitted once to the destination information site.

[0115] Based on the same inventive concept, an embodiment of the present disclosure further provides a cross-network data transmission system, which can be used to implement the cross-network data transmission method provided in the previous embodiment.

[0116] In some embodiments, a cross-network data transmission system is used to implement cross-network data transmission from a first information site located in a first network to a second information site located in a second network. The first network is physically isolated from the second network. The cross-network data transmission method is based on the cross-network data transmission system. The cross-network data transmission system includes: a ferry data generation module, a data review and cross-network transmission module, and a ferry data redo module. The ferry data generation module is deployed in the first information site, and the ferry data redo module is deployed in the second information site.

[0117] The ferry data generation module is configured to generate an operation record corresponding to the change in the target ferry data when the target ferry data that meets the pre-configured target ferry data rules in the first database in the first information site changes, and write the operation record into the target ferry log file corresponding to the target ferry data rule, and is also configured to generate a ferry data packet when cross-network data transmission is required, and send the ferry data packet to the data review and cross-network transmission module, the ferry data packet includes: at least one target ferry log file.

[0118] The data review and cross-network transmission module is configured to perform a security review on each target ferry log file in the ferry data packet, and write the ferry data packet into the data storage medium after the security review is passed.

[0119] The ferry data redo module is configured to read the ferry data packet in the data storage medium, and perform redo operations on the second database in the second information site according to each operation record recorded in each target ferry log file in the ferry data packet.

[0120] For the detailed description of the above modules, please refer to the contents of the previous embodiments, which will not be repeated here.

[0121] According to an embodiment of the present disclosure, a computer-readable medium is further provided. The computer-readable medium stores a computer program, wherein when the program is executed by a processor, the steps in any of the cross-network data transmission methods in the above embodiments are implemented.

[0122] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, including a computer program carried on a machine-readable medium, the computer program containing a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the system of the present disclosure are executed.

[0123] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0124] Flowcharts and block diagrams in the accompanying drawings. The possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure are illustrated. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the aforementioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] The circuits or subcircuits involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The described circuits or subcircuits may also be arranged in a processor, for example, may be described as: a processor comprising: a receiving circuit and a processing circuit, the processing module comprising a writing subcircuit and a reading subcircuit. The names of these circuits or subcircuits do not, in some cases, constitute limitations on the circuits or subcircuits themselves, for example, a receiving circuit may also be described as "receiving a video signal".

[0126] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A cross-network data transmission method, characterized in that: Used to implement cross-network data transmission from a first information site located in a first network to a second information site located in a second network, the first network is physically isolated from the second network, the cross-network data transmission method is based on a cross-network data transmission system, the cross-network data transmission system includes: a ferry data generation module, a data review and cross-network transmission module and a ferry data redo module, the first information site is deployed with the ferry data generation module, and the second information site is deployed with the ferry data redo module; The cross-network data transmission method comprises: When the target ferry data satisfying the pre-configured target ferry data rule in the first database of the first information site changes, the ferry data generation module deployed in the first information site generates an operation record corresponding to the change of the target ferry data, and writes the operation record into the target ferry log file corresponding to the target ferry data rule; When cross-network data transmission is required, the ferry data generation module generates a ferry data packet and sends the ferry data packet to the data review and cross-network transmission module, wherein the ferry data packet includes: at least one of the target ferry log files; The data review and cross-network transmission module performs a security audit on each of the target ferry log files in the ferry data packet, and writes the ferry data packet into the data storage medium after the security audit passes; The ferry data redo module deployed at the second information site reads the ferry data packet in the data storage medium, and performs redo operations on the second database in the second information site according to each operation record recorded in each target ferry log file in the ferry data packet.

2. The cross-network data transmission method according to claim 1, characterized in that: The data transmission system also includes: a ferry scheme design module; Before the step of generating, by the ferry data generating module deployed at the first information site, an operation record corresponding to the change of the target ferry data, the method further includes: In response to the target design instruction, the ferry scheme design module generates a corresponding target ferry scheme, wherein the target ferry scheme defines a target ferry route to be established between the first information site and the second information site, and the ferry data flow direction of the target ferry route is from the first information site to the second information site, and the rules satisfied by the ferry data allowed to be transmitted by the target ferry route are the target ferry data rules; The ferrying scheme design module sends the target ferrying scheme to the first information site and the second information site; The ferry data generation module deployed at the first information site creates a ferry log writing thread corresponding to the target ferry data rule.

3. The cross-network data transmission method according to claim 1, characterized in that: The step of writing the operation record into the target ferry log file corresponding to the target ferry data rule comprises: The ferry data generation module detects whether the file size of the target ferry log file to be written currently exceeds a preset threshold, and detects whether the writable time period corresponding to the target ferry log file to be written currently covers the current time; When the file size of the target ferry log file to be written does not exceed the preset threshold and the writable period corresponding to the target ferry log file to be written covers the current time, the ferry data generation module writes the operation record into the target ferry log file to be written; When the file size of the target ferry log file to be written currently exceeds a preset threshold, or when the writable period corresponding to the target ferry log file to be written currently does not cover the current time, the ferry data generation module creates a new target ferry log file and writes the operation record into the new target ferry log file.

4. The cross-network data transmission method according to claim 1, characterized in that: When the ferry data generation module creates a new target ferry log file, it also includes: The ferry data generation module writes the ferry log file information of the newly created target ferry log file into the log file list corresponding to the target ferry data rule; The ferry data packet also includes the log file list.

5. The cross-network data transmission method according to claim 4, characterized in that: After the ferry data redo module reads the ferry data packet in the data storage medium, and before the ferry data redo module performs a redo operation on the second database in the second information site according to each operation record recorded in the target ferry log file in the ferry data packet, the method further includes: The ferry data redo module adds the ferry log file information of each target ferry log file that should be included in the ferry data packet to a preset log file entry record file according to the log file list; The log file placement record file records different ferry log file information and corresponding status information; The status information includes: already-swung-in status, partially-swung-in status, waiting-to-swung-in status or missing status; wherein, the already-swung-in status indicates that all operation records recorded in the ferry log file have been successfully redone in the second database, the partially-swung-in status indicates that at least part of the operation records recorded in the ferry log file have redone exceptions in the second database, the waiting-to-swung-in status indicates that the ferry log file is waiting to be redone in the second database, and the missing status indicates that the ferry log file is in a missing state.

6. The cross-network data transmission method according to claim 1, characterized in that: The step of the ferry data redo module performing a redo operation on the second database in the second information site according to each operation record recorded in the target ferry log file in the ferry data packet includes: For the target ferry log file currently to be processed, the ferry data redo module generates a database transaction instruction corresponding to each operation record recorded in the target ferry log file currently to be processed, and sends the database transaction instruction to the persistence layer framework in the second information site, so that the persistence layer framework performs corresponding redo operations on the second database according to the received database transaction instruction.

7. The method for cross-network data transmission according to claim 6, characterized in that: After the ferry data redo module sends the database transaction instruction to the persistence layer framework in the second information site, the method further includes: The ferry data redo module records in a preset redo record file the execution result of the persistence layer framework performing the corresponding redo operation on the second database according to the received database transaction instruction.

8. The cross-network data transmission method according to claim 1, characterized in that: The operation record includes at least one of: data added by a adding operation, primary keys of data deleted by a deleting operation, and values ​​of data changed by a changing operation before and after the change.

9. The cross-network data transmission method according to claim 1, characterized in that: The step of the ferry data generation module writing the operation record into the target ferry log file corresponding to the target ferry data rule includes: The ferry data generation module encrypts the operation record to be written into the target ferry log file, and then writes the encrypted operation record into the target ferry log file; The step of the data review and cross-network transmission module performing a security review on each of the target ferry log files in the ferry data packet includes: The operation records in each of the target ferry log files are decrypted, and the decrypted operation records are displayed for external review.

10. A cross-network data transmission system, characterized in that: Used to implement cross-network data transmission from a first information site located in a first network to a second information site located in a second network, the first network is physically isolated from the second network, the cross-network data transmission system comprises: a ferry data generation module, a data review and cross-network transmission module and a ferry data redo module, the first information site is deployed with the ferry data generation module, and the second information site is deployed with the ferry data redo module; The ferry data generation module is configured to generate an operation record corresponding to the change of the target ferry data when the target ferry data satisfying the pre-configured target ferry data rule in the first database of the first information site changes, and write the operation record into the target ferry log file corresponding to the target ferry data rule, and is also configured to generate a ferry data packet when cross-network data transmission is required, and send the ferry data packet to the data review and cross-network transmission module, the ferry data packet including: at least one of the target ferry log files; The data review and cross-network transmission module is configured to perform a security review on each of the target ferry log files in the ferry data packet, and write the ferry data packet into the data storage medium after the security review passes; The ferry data redo module is configured to read the ferry data packet in the data storage medium, and perform a redo operation on the second database in the second information site according to each operation record recorded in each target ferry log file in the ferry data packet.

Images