File processing method, device and equipment based on one-way shutter system and medium

Abstract

Embodiments of the present application provide a file processing method, device and equipment based on a one-way optical gate system and a medium. The method comprises: in the case that a host on an external network side receives service data, the host caches the service data in the form of a cache file into a first virtual file system (VFS), and adds a first event message corresponding to the service data and a read priority corresponding to the first event message into a priority message queue of the host on the external network side; the host on the external network side reads the first event message from the priority message queue in turn according to the read priority, reads service data corresponding to each first event message from the first VFS in the case that each first event message is read, and imports the service data into a host on an internal network side through the one-way optical gate. Embodiments of the present application can effectively improve the reliability of one-way optical gate transmission files.

Description

Technical Field

[0001] This invention relates to the field of cross-domain file transfer technology, and in particular to a file processing method, apparatus, device and medium based on a one-way optical shutter system. Background Technology

[0002] For cross-domain file transfer devices such as one-way optical shutters, they can be divided into "external network side" and "internal network side" according to the network security and confidentiality level. "External network side" belongs to the network side with a lower security and confidentiality level, while "internal network side" belongs to the network side with a higher security and confidentiality level.

[0003] Typically, a one-way shutter is used to import files from the "external network side" to the "internal network side" across domains.

[0004] However, during the file import process, there may be situations where the file on the "external network side" is not completely imported into the "internal network side". For example, the speed at which the front-end server transfers files to the "external network side" is greater than the speed at which the "external network side" imports files to the "internal network side". This may lead to file loss during cross-domain import and the reliability of unidirectional optical gate file transmission is low. Summary of the Invention

[0005] To address the problems in the prior art, embodiments of the present invention provide a file processing method, apparatus, device, and medium based on a unidirectional optical shutter system.

[0006] Specifically, the embodiments of the present invention provide the following technical solutions:

[0007] In a first aspect, embodiments of the present invention provide a file processing method based on a unidirectional optical shutter system, wherein the unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host, and the method includes:

[0008] When the external network host receives service data, it caches the service data in the form of a cache file in the first virtual file system (VFS), and adds the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0009] The external network host reads the first event message sequentially from the priority message queue according to the read priority order, and reads the service data corresponding to the first event message from the first VFS when reading each first event message, and imports the service data into the internal network host through the unidirectional optical gate.

[0010] Furthermore, the method also includes:

[0011] If the first condition is met, the external network host adds a second event message and the corresponding read priority of the second event message to the priority message queue.

[0012] When the external network host reads the second event message with the corresponding reading priority from the priority message queue, it reads the second service data corresponding to the second event message from the first VFS and imports the second service data into the internal network host through the unidirectional optical shutter.

[0013] The first condition includes at least one of the following:

[0014] The external network host has selected and recorded the corresponding event messages in the retransmission list as the second event message;

[0015] The external network host obtains the damage record of the internal network host, and the damage record is used to indicate that the internal network host did not accept the second service data;

[0016] The external network host obtains all records of the internal network host and determines, based on all records, that the internal network host has not accepted the second service data.

[0017] Furthermore, the business data cached in the first virtual file system (VFS) includes imported data and unimported data. The imported data is business data that has been imported from the external network host to the internal network host, and the unimported data is business data that has not yet been imported from the external network host to the internal network host.

[0018] The method further includes:

[0019] If the cache size occupied by the imported data exceeds a first threshold, the external network host performs a rollback operation on the imported data.

[0020] If the cache size occupied by the unimported data is greater than the second threshold, the external network host will suspend receiving service data from the front-end server until the cache size occupied by the unimported data is less than the third threshold, at which point it will start receiving service data from the front-end server. The third threshold is less than the second threshold.

[0021] Furthermore, the method also includes:

[0022] The external network host caches the file record information during the data transmission process of the unidirectional optical shutter in the first memory of the external network host;

[0023] When the second condition is met, the external network host performs an input operation on the file record information in the first memory;

[0024] The second condition includes at least one of the following:

[0025] The inbound operation was not performed within the timeout period;

[0026] The number of file record information cached in the first memory is greater than the maximum number of commits;

[0027] The space occupied by the file record information cached in the first memory is greater than the maximum length of a single commit statement.

[0028] Furthermore, upon receiving service data, the external network host caches the service data as a cache file in the first virtual file system (VFS), including:

[0029] Upon receiving the service data, the external network host caches the service data as a cache file in the cache directory of the first VFS.

[0030] The method further includes:

[0031] If the number of cache files corresponding to the business data in the first VFS is less than the fourth threshold, the external network host performs a rollback operation using the cache files in the cache directory as the operation unit.

[0032] If the number of cached files corresponding to the business data in the first VFS is greater than or equal to the fifth threshold, the external network host performs a rollback operation using the cache directory as the unit of operation.

[0033] Furthermore, the method also includes:

[0034] If the message body cache of the first event message in the priority message queue is damaged and the message body cache cannot be repaired, the external network host obtains the cache file path of the cache file corresponding to the service data in the first VFS;

[0035] The external network host determines the task to which the cache file belongs based on the cache file path;

[0036] Based on the task, the external network host reconstructs the message body cache of the first event message by parsing the file header information of the cache file;

[0037] The external network host adds the message body cache of the reconstructed first event message to the priority message queue.

[0038] Secondly, embodiments of the present invention provide a file processing method based on a unidirectional optical shutter system, wherein the unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host, and the method includes:

[0039] The internal network host receives service data from the external network host through the one-way optical shutter;

[0040] The intranet-side host caches the service data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and adds the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host.

[0041] The intranet-side host reads the third event messages sequentially from the task message queue according to the task number, and reads the corresponding business data from the second VFS for each third event message, and uploads the business data to the backend server.

[0042] Furthermore, after the intranet-side host receives service data from the extranet-side host through the unidirectional optical gate, the method further includes:

[0043] The internal network side host records the service data received by the internal network side host as a complete record.

[0044] Furthermore, the method also includes:

[0045] When the internal network host determines that the received service data is corrupted, it records the corrupted service data as a corruption record.

[0046] Furthermore, the method also includes:

[0047] If the cache size occupied by the service data cached in the second VFS is greater than the sixth threshold, the internal network host will suspend receiving service data from the external network host until the cache size occupied by the service data cached in the second VFS is less than the seventh threshold, at which point it will start receiving service data from the external network host. The seventh threshold is less than the sixth threshold.

[0048] Furthermore, the method also includes:

[0049] The intranet-side host caches the file record information during the data transmission process of the unidirectional optical shutter in the second memory of the intranet-side host.

[0050] Under the condition that the third condition is met, the intranet-side host performs an input operation on the file record information in the second memory;

[0051] The third condition includes at least one of the following:

[0052] The inbound operation was not performed within the timeout period;

[0053] The number of file record information cached in the second memory is greater than the maximum number of commits;

[0054] The space occupied by the file record information cached in the second memory is greater than the maximum length of a single commit statement.

[0055] Furthermore, the intranet-side host caches the service data as cache files in the second virtual file system (VFS) of the intranet-side host, including:

[0056] The intranet-side host caches the service data as cache files in the cache directory of the second VFS of the intranet-side host;

[0057] The method further includes:

[0058] If the number of cache files corresponding to the service data in the second VFS is less than the eighth threshold, the intranet-side host performs a rollback operation using the cache files in the cache directory as the operation unit.

[0059] If the number of cached files corresponding to the business data in the second VFS is greater than or equal to the ninth threshold, the intranet-side host performs a rollback operation using the cache directory as the unit of operation.

[0060] Furthermore, the method also includes:

[0061] If the message body cache of the third event message in the task message queue is damaged and the message body cache cannot be repaired, the intranet-side host obtains the cache file path of the cache file corresponding to the service data in the second VFS;

[0062] The internal network host determines the task to which the cache file belongs based on the cache file path;

[0063] Based on the task, the intranet-side host reconstructs the message body cache of the third event message by parsing the file header information of the cache file;

[0064] The intranet-side host adds the message body cache of the reconstructed third event message to the task message queue.

[0065] Thirdly, embodiments of the present invention also provide a file processing device based on a unidirectional optical shutter system. The unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host. The device includes:

[0066] The first caching module is used to cache the business data as a cache file in the first virtual file system (VFS) when the business data is received, and to add the first event message corresponding to the business data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0067] The import module is used to read the first event message sequentially from the priority message queue according to the reading priority order, and when reading each first event message, read the corresponding service data from the first VFS, and import the service data to the intranet-side host through the unidirectional optical gate.

[0068] Fourthly, embodiments of the present invention also provide a file processing device based on a unidirectional optical shutter system. The unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host. The device includes:

[0069] The receiving module is used to receive service data from the external network host through the unidirectional optical shutter;

[0070] The second caching module is used to cache the business data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and to add the third event message corresponding to the business data and the task number corresponding to the third event message to the task message queue of the intranet-side host.

[0071] The transmission module is used to read the third event messages sequentially from the task message queue according to the task number, and when reading each third event message, read the corresponding business data from the second VFS and upload the business data to the back-end server.

[0072] Fifthly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the file processing method based on a unidirectional light gate system as described in the first aspect, or implements the file processing method based on a unidirectional light gate system as described in the second aspect.

[0073] In a sixth aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the file processing method based on a unidirectional light gate system as described in the first aspect, or implements the file processing method based on a unidirectional light gate system as described in the second aspect.

[0074] In a seventh aspect, embodiments of the present invention also provide a computer program product having executable instructions stored thereon, which, when executed by a processor, cause the processor to implement the file processing method based on a unidirectional light gate system as described in the first aspect, or to implement the file processing method based on a unidirectional light gate system as described in the second aspect.

[0075] The file processing method, apparatus, device, and medium based on a unidirectional optical shutter system provided in this invention allow an external network host in the unidirectional optical shutter system to cache service data in a first Virtual File System (VFS) upon receiving the data. The external host then adds the corresponding first event message and its read priority to the priority message queue of the service data. The external host then reads the first event messages sequentially from the queue according to their read priority. For each first event message, the external host reads the corresponding service data from the first VFS and imports it to the internal network host via the unidirectional optical shutter. By caching the service data in the first VFS, this invention can effectively improve the reliability of file transmission via the unidirectional optical shutter, even if the speed at which the external host receives service data exceeds the speed at which it imports data to the internal network host, thus allowing technicians to retrieve lost service data from the first VFS. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a schematic diagram of the structure of the unidirectional optical shutter system in the file processing method based on the unidirectional optical shutter system provided in the embodiments of the present invention;

[0078] Figure 2 This is one of the flowcharts illustrating a file processing method based on a unidirectional light gate system provided in an embodiment of the present invention;

[0079] Figure 3 This is one of the flowcharts illustrating the batch storage of file record information in the file processing method based on a unidirectional light gate system provided in this embodiment of the invention.

[0080] Figure 4 This is the second flowchart illustrating the batch storage of file record information in the file processing method based on a unidirectional light gate system provided in this embodiment of the invention.

[0081] Figure 5 This is the second flowchart illustrating the file processing method based on a unidirectional light gate system provided in this embodiment of the invention.

[0082] Figure 6 This is a schematic diagram of the structure of the DAO data access interface provided in an embodiment of the present invention;

[0083] Figure 7 This is a schematic diagram of the file caching process in the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention;

[0084] Figure 8 This is a schematic diagram of the WebUI process for selecting resend in the file processing method based on a unidirectional light gate system provided in this embodiment of the invention;

[0085] Figure 9 This is a schematic diagram of the file processing method based on a one-way light gate system provided in the embodiments of the present invention, illustrating the process of recording, downloading, and uploading.

[0086] Figure 10 This is a schematic diagram illustrating the relationship between the FID generator and the multi-channel multitasking provided in this embodiment of the invention.

[0087] Figure 11 This is a schematic diagram of a dual-directory hierarchy provided in an embodiment of the present invention;

[0088] Figure 12 This is a schematic diagram of the process for performing a rollback operation on the record cache according to an embodiment of the present invention;

[0089] Figure 13 This is a schematic diagram of the process for retrieving file cache in the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention;

[0090] Figure 14 This is a schematic diagram of the process of querying record caching and downloading in the file processing method based on a unidirectional light gate system provided in the embodiments of the present invention;

[0091] Figure 15 This is a schematic diagram of the process of resending the selected record in the file cache list in the file processing method based on the unidirectional light gate system provided in the embodiment of the present invention;

[0092] Figure 16 This is a schematic diagram of the process of recording the cache list and downloading the file for retransmission on the external network host in the file processing method based on the unidirectional optical shutter system provided in the embodiment of the present invention;

[0093] Figure 17 This is one of the structural schematic diagrams of a file processing device based on a unidirectional optical shutter system provided in an embodiment of the present invention;

[0094] Figure 18 This is a second schematic diagram of the structure of the file processing device based on a one-way light gate system provided in this embodiment of the invention;

[0095] Figure 19 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0097] The following description, in conjunction with the accompanying drawings, describes the document processing method, apparatus, equipment, and medium based on a unidirectional optical shutter system of the present invention.

[0098] To facilitate a clearer understanding of the various embodiments of the present invention, some related technical knowledge will be introduced as follows.

[0099] Data Abstraction Unified Interface (DAO): The DAO layer mainly performs the work of the data persistence layer. All interface operations responsible for communicating with the database are encapsulated here. The design of the DAO layer begins with designing the DAO interface, which supports calling this interface in functional modules to process data business without having to worry about the specific implementation of this interface. The data source configuration of the DAO layer, as well as the parameters related to database connection, are uniformly configured and managed in the internal configuration file.

[0100] Figure 1 This is a schematic diagram of the structure of the unidirectional optical shutter system in the file processing method based on the unidirectional optical shutter system provided in the embodiments of the present invention, as shown below. Figure 1 As shown, the one-way light gate system 100 includes:

[0101] External network host 101, internal network host 102, and unidirectional optical shutter 103 for controlling the unidirectional transmission of data from the external network host 101 to the internal network host 102.

[0102] Figure 2 This is one of the flowcharts illustrating a file processing method based on a unidirectional light gate system provided in an embodiment of the present invention, such as... Figure 2 As shown, this file processing method based on a one-way optical gate system can be applied to external network hosts. The method includes the following steps:

[0103] Step 201: When the external network host receives service data, it caches the service data in the form of a cache file in the first virtual file system (VFS), and adds the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0104] Step 202: The external network host reads the first event message sequentially from the priority message queue according to the reading priority order, and reads the service data corresponding to the first event message from the first VFS when reading each first event message, and imports the service data to the internal network host through the unidirectional optical gate.

[0105] Specifically, in related technologies, the optical one-way data automatic import system, also known as the one-way optical shutter, is a security isolation product mainly used in government classified networks, military industry, and power industry; the one-way optical shutter can realize functions such as importing files, emails, and databases from low-security networks to high-security networks.

[0106] Typically, one-way optical shutters are used to import files from the "external network side" to the "internal network side" across domains. However, during the file import process, there may be situations where the files from the "external network side" are not completely imported into the "internal network side." For example, the speed at which the front-end server transmits files to the "external network side" may be greater than the speed at which the "external network side" imports files to the "internal network side." This may lead to file loss during the cross-domain file import process, resulting in low reliability of file transmission via one-way optical shutters.

[0107] In this embodiment of the invention, a deployment scenario for a unidirectional optical shutter system is first provided, such as... Figure 1 As shown, a one-way optical shutter 103 is installed between the external network host 101 and the internal network host 102. The one-way optical shutter 103 is used to control the external network host 101 to transmit data unidirectionally to the internal network host 102.

[0108] Optionally, such as Figure 1 As shown, the one-way light gate system 100 may also include a front-end server 104 and a back-end server 105; the deployed devices are described below.

[0109] 1) Front-end server 104: Provides business request access functionality to the front-end (business client) and interacts with the external network host 101 via a private Transmission Control Protocol (TCP) to the back-end; wherein, the business client is the role of the business party initiating business requests, which are requests based on TCP and may include, but are not limited to, Hypertext Transfer Protocol (HTTP) requests, File Transfer Protocol (FTP) access, MySQL (My Structured Query Language) queries, and Simple Mail Transfer Protocol (SMTP) email access;

[0110] 2) Backend server 105: It communicates with the internal network host 102 via a private TCP connection and performs business interactions with the network. The DMZ server is located in the isolation zone and provides business services. It may include, but is not limited to, a web server, FTP server, MySQL server, and SMTP server.

[0111] Based on the deployment scenario of the unidirectional optical shutter system 100 described above, this embodiment of the invention provides a file processing method based on the unidirectional optical shutter system.

[0112] Specifically, the external network host can receive service data from the front-end server. Upon receiving the service data, the external network host can first cache the service data in the first VFS and add the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host. The priority message queue includes at least one first event message and its corresponding read priority. For example, the read priority corresponding to the first event message added to the priority message queue first can be set to be higher, and the read priority corresponding to the first event message added to the priority message queue later can be set to be lower. That is, the read priority corresponding to the event message can be set according to the first-in-first-out principle.

[0113] The external network host reads the first event message from the queue in order of reading priority. When each first event message is read, it can be assumed that the corresponding service data of the first event message has been transmitted. Then, the service data corresponding to the first event message is read from the first VFS and imported into the internal network host through the one-way optical gate.

[0114] It should be noted that after the external network host imports service data to the internal network host through the one-way optical gate, the corresponding first event message and its corresponding read priority need to be deleted from the priority message queue to avoid duplicate transmission of service data.

[0115] It should also be noted that the external network host caches business data in the form of cache files in the first VFS. The external network host can also read business data in the form of cache files from the first VFS and import the business data in the form of cache files into the internal network host.

[0116] The file processing method based on a unidirectional optical shutter system provided in this invention allows the external network host in the unidirectional optical shutter system to cache service data in a first Virtual File System (VFS) upon receiving the data. The external host then adds the corresponding first event message and its read priority to the priority message queue of the external network host. The external network host then reads the first event messages sequentially from the queue according to their read priority. For each first event message, the external network host reads the corresponding service data from the first VFS and imports it into the internal network host via the unidirectional optical shutter. By caching the service data in the first VFS, this invention can effectively improve the reliability of file transmission via the unidirectional optical shutter, even if the speed at which the external network host receives service data exceeds the speed at which it imports data into the internal network host, thus allowing technicians to retrieve the lost service data from the first VFS in cases where service data loss occurs.

[0117] Optionally, if the first condition is met, the external network host may add a second event message and the corresponding reading priority of the second event message to the priority message queue;

[0118] When the external network host reads the second event message with the corresponding reading priority from the priority message queue, it reads the second service data corresponding to the second event message from the first VFS and imports the second service data into the internal network host through the unidirectional optical shutter.

[0119] The first condition includes at least one of the following:

[0120] 1) The event messages that have been selected and recorded in the retransmission list constructed by the external network host are used as the second event messages;

[0121] 2) The external network host obtains the damage record of the internal network host, and the damage record is used to indicate that the internal network host did not accept the second service data;

[0122] 3) The external network host obtains all records of the internal network host, and determines based on all records that the internal network host has not accepted the second service data.

[0123] Specifically, in addition to adding the first event message corresponding to the service data and its corresponding read priority to the priority message queue of the external network host when receiving service data, the external network host can also add a second event message and its corresponding read priority to the priority message queue when the first condition is met.

[0124] It can be divided into the following three situations:

[0125] Scenario 1: If the corresponding event message has been selected and recorded as the second event message in the retransmission list built on the external network host, the external network host adds the second event message and its corresponding reading priority to the priority message queue.

[0126] Specifically, users can select the service data that needs to be resent from the external network host to the internal network host through the WebUI interface of the external network host. When the user selects the data, the WebUI interface will display the service data cached in the first VFS of the external network host as the external network cache record, and the service data that has been imported from the external network host to the internal network host as the detailed record. Optionally, the detailed record can be divided according to the task number, and the detailed record can include a certain number of valid service data imported from the external network host to the internal network host.

[0127] Users can select business data that exists in both external network cache records and detailed records, enabling the external network host to resend the selected business data to the internal network host. The external network host can resend corrupted records through the external network cache records without having to retrieve the corresponding data from the server, which can reduce the amount of data queries on the server, thereby reducing the time cost of data transmission from the server to the external network host and improving file resending efficiency.

[0128] After the user selects the business data that needs to be resent from the external network host to the internal network host, the external network host can generate a WebUI resend list based on the selected business data. The WebUI resend list only displays the business data cached in the first VFS.

[0129] The WebUI resend process is as follows:

[0130] a) Add the event messages required for "File Import" by supplementing the selected records in the WebUI resend list with detailed records;

[0131] b) Add the newly constructed event message as the second event message and its corresponding read priority to the priority message queue;

[0132] c) When the external network host reads the second event message, it reads the second service data corresponding to the second event message in the first VFS and sends the second service data to the internal network host through a one-way optical gate.

[0133] Scenario 2: The external network host obtains and analyzes the corruption records of the internal network host. If the corruption record indicates that the internal network host has not received the second service data, a second event message and its corresponding read priority can be added to the priority message queue to resend the second service data.

[0134] It should be noted that the one-way import type of the one-way optical shutter system means that only the external network host can import business data to the internal network host through the one-way optical shutter. The internal network host cannot send data to the external network host through the one-way optical shutter. Therefore, users can manually download records on the internal network host and then manually upload them to the external network host, so that the external network host can obtain the manually uploaded records, such as corrupted records.

[0135] Scenario 3: If the external network host obtains all records of the internal network host and determines based on all records that the internal network host has not received the second service data, it can add a second event message and its corresponding read priority to the priority message queue to resend the second service data.

[0136] Specifically, users can manually download records on the intranet-side host and then manually upload them to the external-side host, so that the external-side host can obtain the manually uploaded records, such as all records; where all records include details of all business data received by the intranet-side host;

[0137] The external network host can compare all records and detailed records to filter out the second service data that the internal network host has not received. In this way, the second service data can be resent by adding a second event message and its corresponding read priority to the priority message queue.

[0138] It should be noted that due to the "one-way import" characteristic of file import in unidirectional optical shutters, the "external network side" is unaware of the file acceptance status of the "internal network side." This may result in situations where the internal network host does not accept the second service data, but the external network host cannot determine which service data was not accepted by the internal network host. In this embodiment of the invention, the external network host can add a second event message and its corresponding read priority to the priority message queue to retransmit the second service data, provided that the user selects the second service data for retransmission via the WebUI, and / or, by obtaining the internal network host's corrupted records or all records, confirming that the internal network host has not accepted the second service data. This retransmission improves the reliability of data transmission in the unidirectional optical shutter system.

[0139] Optionally, the service data cached in the first VFS includes imported data and unimported data. The imported data is the service data that has been imported from the external network host to the internal network host, and the unimported data is the service data that has not yet been imported from the external network host to the internal network host.

[0140] The above method also includes the following steps:

[0141] If the cache size occupied by the imported data exceeds a first threshold, the external network host performs a rollback operation on the imported data.

[0142] If the cache size occupied by the unimported data is greater than the second threshold, the external network host will suspend receiving service data from the front-end server until the cache size occupied by the unimported data is less than the third threshold, at which point it will start receiving service data from the front-end server. The third threshold is less than the second threshold.

[0143] Specifically, the business data in the first VFS cache includes imported data and unimported data. Imported data refers to business data that has been imported from the external network host to the internal network host through the one-way optical gateway, while unimported data refers to business data that has not yet been imported from the external network host to the internal network host through the one-way optical gateway.

[0144] For imported data, the cache size of the imported data can be controlled by setting parameters. When the cache size exceeds the set parameters, a rollback operation will be performed. For example, if the cache size occupied by the imported data on the external network host is greater than the first threshold, a rollback operation will be performed on the imported data.

[0145] For unimported data, the cache size is limited by system space. If a rollback operation is performed on the unimported data by setting parameters, it may result in the loss of cached data. In this embodiment of the invention, if the cache size occupied by unimported data is greater than a second threshold, the external network host suspends receiving service data from the front-end server until the cache size occupied by unimported data is less than a third threshold, at which point it resumes receiving service data from the front-end server. Optionally, the second and third thresholds can be the same or different.

[0146] For example, when the system space utilization rate is greater than 90%, the business modules on the external network host will no longer perform data drop-down operations. The files that have been stored need to be imported to the internal network host first. The business modules can then perform data drop-down operations when the system space utilization rate is less than 90%.

[0147] Optionally, the external network host caches the file record information during the data transmission process of the unidirectional optical shutter in the first memory of the external network host;

[0148] When the second condition is met, the external network host performs an input operation on the file record information in the first memory;

[0149] The second condition includes at least one of the following:

[0150] 1) The inbound operation was not performed within the timeout period;

[0151] 2) The number of file record information cached in the first memory is greater than the maximum number of commits;

[0152] 3) The space occupied by the file record information cached in the first memory is greater than the maximum length of a single commit statement.

[0153] Specifically, during the data transmission process of the one-way optical gate, the external network host can record the business data transmitted by the one-way optical gate in the form of a cached file, forming file record information, and cache it in the first memory of the external network host. When the second condition is met, the external network host performs a database entry operation on the file record information in the first memory. Specifically, the external network host transmits the file record information in the first memory to the database for storage through the DAO interface. The database includes, for example, Redis database and MySQL database.

[0154] In one embodiment, the file record information can be transmitted to a MySQL database for storage, facilitating subsequent retrieval.

[0155] This invention can solve the problem of low efficiency in single-process, single-instance record entry, improve the efficiency of file record information entry, and reduce the interaction frequency between the business modules of the external network host and the MySQL database.

[0156] Figure 3 This is one of the flowcharts illustrating the batch storage of file record information in the file processing method based on a unidirectional light gate system provided in this embodiment of the invention. Figure 3 As shown, a business module may have business tasks 1 to N. These tasks are first divided into N working processes by the main process. Each working process may also include N task resource blocks (RBs). Each task RB corresponds to a buffer. The following method for batch importing file record information into the database is then executed.

[0157] Batch data entry of file records can be completed in two steps: record insertion and record submission. Batch processing technology allows control over the submission frequency, improving entry efficiency. Record submission control strategies are mainly divided into: submission time control, submission record quantity control, and submission record space control. Detailed explanations of these strategies are shown below, specifically categorized into three cases:

[0158] Scenario 1: Submission time control is accomplished by listening for the inbound event. If the inbound event does not occur within the timeout period (e.g., 300ms), that is, if the inbound operation is not performed within the timeout period, the cached record is submitted and the inbound operation is completed.

[0159] Case 2: The submission record control is completed by comparing the number of submitted but not yet entered into the database with the maximum submission number. When the number of submitted records is greater than the maximum submission number (e.g., 8192), the cached records are submitted and the database entry operation is completed.

[0160] Case 3: The space limit for committed records is determined by comparing the space occupied by committed but not yet entered into the database with the maximum length of a single commit statement. If the total length of the committed records is greater than the maximum length of a single commit statement, the cached records are committed and the database entry operation is completed. The maximum length of a single commit statement is, for example, 1MB, depending on the maximum length limit of a single MySQL statement.

[0161] In one embodiment, Figure 4 This is the second schematic diagram of the process for batch storage of file record information in the file processing method based on a unidirectional light gate system provided in this embodiment of the invention, as shown below. Figure 4 As shown;

[0162] Step 401: Determine whether the data entry operation was performed within the timeout period. If yes, proceed to step 402; otherwise, proceed to step 404.

[0163] Step 402: Determine whether the number of file record information cached in the first memory is less than or equal to the maximum number of submissions. If yes, proceed to step 403; otherwise, proceed to step 404.

[0164] Step 403: Determine whether the space occupied by the file record information cached in the first memory is less than or equal to the maximum length of a single commit statement. If yes, return to step 401; otherwise, proceed to step 404.

[0165] Step 404: Perform an input operation on the file record information in the first memory.

[0166] Optionally, the implementation method for the external network host to cache the service data as a cache file in the first virtual file system (VFS) upon receiving the service data may include:

[0167] Upon receiving the service data, the external network host caches the service data as a cache file in the cache directory of the first VFS.

[0168] The above method also includes the following steps:

[0169] If the number of cache files corresponding to the business data in the first VFS is less than the fourth threshold, the external network host performs a rollback operation using the cache files in the cache directory as the operation unit.

[0170] If the number of cached files corresponding to the business data in the first VFS is greater than or equal to the fifth threshold, the external network host performs a rollback operation using the cache directory as the unit of operation.

[0171] Specifically, when an external network host receives service data, it can cache the service data as a cache file in the cache directory of the first VFS;

[0172] To prevent the unlimited increase of cached data on the external network host, which could affect the performance and functionality of business modules, this invention proposes a rollback strategy for file processing based on a unidirectional optical shutter system to control the number of cached records and keep the number of cached records within the set parameter range.

[0173] To quickly roll back cached files, the rollback strategy can be dynamically adjusted between rollback strategy one and rollback strategy two, where:

[0174] Rollback Strategy 1: Precise rollback, which involves scanning the cache directory and deleting cache files in the cache directory one by one;

[0175] Rollback Strategy 2: Coarse rollback, directly deleting from the cache directory to achieve a fast rollback;

[0176] For example, the cache directory is first scanned to obtain the cached files and their total number. When the total number of cached files is less than the set rollback policy switching value (e.g., 5000), rollback policy one can be used to roll back the cached files; when the total number of cached files is not less than the set rollback policy switching value, rollback policy two can be used to roll back the cached files.

[0177] Optionally, rollback strategy one: precise rollback based on cached files. Cached files refer to the "tarball" files where business modules are stored. A single "tarball" file may consist of multiple small files or fragments of a large file. Precise rollback involves sequentially retrieving detailed information about the backend cached files from the scan list, calculating the actual number and size of each cached file, and processing the cached files according to their size. This includes deleting detected cached files until the total size of the cached files is less than the set cache file size.

[0178] Rollback Strategy Two: A coarse rollback is performed based on the cache directories where the cached files are located. The cache directories are divided according to the number and size of the cached files. When the total size of the cached files is not less than the cache file size setting, cache directories are deleted sequentially until the total size of the remaining cached files is less than the cache file size setting. The file path format is " / base directory / quota quantity directory / quota size directory / cache files". The parent directory of the cached files consists of three parts: the base directory, the quota quantity directory, and the quota size directory. The base directory is determined by the task number, the quota quantity directory specifies the number of cached files in this directory (default number: 0x4E20), and the quota size directory specifies the total size of the cached files in this directory (default size: 100MB).

[0179] Optionally, if the message body cache of the first event message in the priority message queue is damaged and the message body cache cannot be repaired, the external network host obtains the cache file path of the cache file corresponding to the service data in the first VFS;

[0180] The external network host determines the task to which the cache file belongs based on the cache file path;

[0181] Based on the task, the external network host reconstructs the message body cache of the first event message by parsing the file header information of the cache file;

[0182] The external network host adds the message body cache of the reconstructed first event message to the priority message queue.

[0183] Specifically, if the message body cache of the first event message in the priority message queue is damaged and cannot be repaired, the external host can obtain the cache file path of the cache file corresponding to the business data in the first VFS. Based on the cache file path, it can determine the task to which the cache file belongs. Then, based on the task, it can reconstruct the message body cache of the first event message by parsing the file header information of the cache file. The reconstructed message body cache of the first event message is added to the priority message queue. Then, the external host can read the business data corresponding to the newly constructed first event message and import the business data to the internal host.

[0184] Figure 5 This is a second flowchart illustrating the file processing method based on a unidirectional light gate system provided in this embodiment of the invention, as shown below. Figure 5 As shown, this file processing method based on a one-way optical gate system can be applied to hosts on the intranet side. The method includes the following steps:

[0185] Step 501: The intranet-side host receives service data from the extranet-side host through the unidirectional optical shutter;

[0186] Step 502: The intranet-side host caches the service data in the form of a cache file in the second VFS of the intranet-side host, and adds the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host;

[0187] Step 503: The intranet-side host reads the third event messages sequentially from the task message queue according to the task number, and reads the corresponding business data from the second VFS for each third event message, and uploads the business data to the backend server.

[0188] Specifically, the internal network host receives service data from the external network host through a one-way optical gate, caches the service data in the form of a cache file in the second VFS of the internal network host, and adds the third event message corresponding to the service data and its corresponding task number to the task message queue of the internal network host. The task message queue includes at least one event message and its corresponding task number. The internal network host then reads the third event message from the task message queue in order of task number, and reads the service data corresponding to the third event message from the second VFS when reading each third event message, and uploads the service data to the back-end server.

[0189] Optionally, the task number in the task message queue can be read using a first-in, first-out (FIFO) approach.

[0190] It should be noted that after the internal network host imports business data to the backend server, the corresponding third event message and its corresponding task number need to be deleted from the task message queue to avoid duplicate transmission of business data.

[0191] It should also be noted that the internal network host caches business data in the form of cache files in the second VFS. The internal network host can also read business data in the form of cache files from the second VFS and upload the business data in the form of cache files to the backend server.

[0192] The file processing method based on a unidirectional optical shutter system provided in this invention allows the internal network host in the unidirectional optical shutter system to cache service data received from the external network host in a second Virtual File System (VFS). The method also adds the corresponding third event message and its task number to the task message queue of the internal network host. The internal network host then reads the third event messages sequentially from the queue according to their task numbers. For each third event message, the internal network host reads the corresponding service data from the second VFS and uploads it to the backend server. By caching the service data in the second VFS, this invention can effectively improve the reliability of file transmission via the unidirectional optical shutter system, even if the internal network host's data reception speed exceeds its data upload speed to the backend server, thus allowing technicians to retrieve lost service data from the second VFS.

[0193] Optionally, after the internal network host receives service data from the external network host through the unidirectional optical gate, the internal network host records the service data received by the internal network host as a complete record.

[0194] Specifically, after receiving service data from the external network host through the one-way optical shutter, the internal network host can record the received service data as a complete record. Subsequently, users can manually download the complete record through the internal network host and upload it to the external network host. The external network host can then compare the complete record with its detailed record to determine which service data was not received by the internal network device and retransmit it, thus improving the reliability of data transmission in the one-way optical shutter system. The detailed record can include a certain number of valid service data imported from the external network host to the internal network host.

[0195] Optionally, if the internal network host determines that the received service data is corrupted, it records the corrupted service data as a corruption record.

[0196] Specifically, after receiving service data from the external network host through the one-way optical shutter, the internal network host can record the damaged service data as a corruption record if it is determined that the received service data is corrupted. Subsequently, users can manually download the corruption record through the internal network host and upload it to the external network host, so that the external network host can retransmit the corrupted service data based on the corruption record, thereby improving the reliability of data transmission in the one-way optical shutter system.

[0197] Optionally, if the cache size occupied by the service data cached in the second VFS is greater than the sixth threshold, the internal network host suspends receiving service data from the external network host until the cache size occupied by the service data cached in the second VFS is less than the seventh threshold, and then starts receiving service data from the external network host, where the seventh threshold is less than the sixth threshold.

[0198] Specifically, if the cache size occupied by the service data cached in the second VFS is greater than the sixth threshold, the internal network host will suspend receiving service data from the external network host until the cache size occupied by the service data cached in the second VFS is less than the seventh threshold, and then will resume receiving service data from the external network host; wherein the sixth threshold and the seventh threshold can be the same or different.

[0199] The second VFS on the internal network host is limited by system space. For example, when the system space utilization rate in the second VFS is greater than 90%, the business modules do not perform persistence processing on the business data imported from the external network host that has been accepted, which can avoid file corruption or file loss problems.

[0200] Optionally, the intranet-side host caches the file record information during the data transmission process of the unidirectional optical shutter in the second memory of the intranet-side host;

[0201] Under the condition that the third condition is met, the intranet-side host performs an input operation on the file record information in the second memory;

[0202] The third condition includes at least one of the following:

[0203] 1) The inbound operation was not performed within the timeout period;

[0204] 2) The number of file record information cached in the second memory is greater than the maximum number of commits;

[0205] 3) The space occupied by the file record information cached in the second memory is greater than the maximum length of a single commit statement.

[0206] Specifically, during the data transmission process of the one-way optical gateway, the intranet-side host can record the business data transmitted by the one-way optical gateway in the form of a cached file, forming file record information, and cache it in the second memory of the intranet-side host. When the third condition is met, the intranet-side host performs a database entry operation on the file record information in the second memory. Specifically, the intranet-side host transmits the file record information in the second memory to the database for storage through the DAO interface. The database may include Redis database and MySQL database.

[0207] Specifically, it can be divided into the following three situations:

[0208] Scenario 1: If the data entry operation is not performed within the timeout period (e.g., 300ms), the internal network host will submit the cached records and complete the data entry operation.

[0209] Scenario 2: When the number of file record information cached in the second memory is greater than the maximum number of submissions (e.g., 8192), the internal network host submits the cached records and completes the database entry operation.

[0210] Scenario 3: When the space occupied by the file record information cached in the second memory is greater than the maximum length of a single commit statement (e.g., 1MB, depending on the maximum length limit of a single MySQL statement), the internal network host commits the cached record and completes the database entry operation.

[0211] Optionally, the implementation method of the intranet-side host caching the service data as cache files in the second virtual file system (VFS) of the intranet-side host may include:

[0212] The intranet-side host caches the service data as cache files in the cache directory of the second VFS of the intranet-side host;

[0213] The above method also includes the following steps:

[0214] If the number of cache files corresponding to the service data in the second VFS is less than the eighth threshold, the intranet-side host performs a rollback operation using the cache files in the cache directory as the operation unit.

[0215] If the number of cached files corresponding to the business data in the second VFS is greater than or equal to the ninth threshold, the intranet-side host performs a rollback operation using the cache directory as the unit of operation.

[0216] Specifically, the internal network host can cache service data from the external network host in the form of cache files in the cache directory of the second VFS of the internal network host;

[0217] To prevent the unlimited increase of cached data on the internal network host, which could affect the performance and functionality of business modules, this invention proposes a rollback strategy for file processing based on a unidirectional optical shutter system to control the number of cached records and keep the number of cached records within the set parameter range.

[0218] If the number of cached files corresponding to business data in the second VFS on the internal network host is less than the eighth threshold, a precise rollback will be performed. Specifically, the rollback operation will be performed using the cached files in the cache directory as the unit of operation.

[0219] If the number of cached files corresponding to business data in the second VFS on the internal network host is greater than or equal to the ninth threshold, a coarse rollback will be performed, specifically with the cache directory as the unit of operation, and the rollback operation will be performed.

[0220] The eighth and ninth thresholds can be the same or different.

[0221] Optionally, if the message body cache of the third event message in the task message queue is damaged and the message body cache cannot be repaired, the intranet-side host obtains the cache file path of the cache file corresponding to the service data in the second VFS;

[0222] The internal network host determines the task to which the cache file belongs based on the cache file path;

[0223] Based on the task, the intranet-side host reconstructs the message body cache of the third event message by parsing the file header information of the cache file;

[0224] The intranet-side host adds the message body cache of the reconstructed third event message to the task message queue.

[0225] Specifically, if the message body cache of the third event message in the task message queue is damaged and cannot be repaired, the intranet host can obtain the cache file path of the cache file corresponding to the business data in the second VFS. Based on the cache file path, it can determine the task to which the cache file belongs. Then, based on the task, it can reconstruct the message body cache of the third event message by parsing the file header information of the cache file. The reconstructed message body cache of the third event message is added to the task message queue. Then, the intranet host can read the business data corresponding to the newly constructed third event message and upload the business data to the backend server.

[0226] The following example illustrates the file processing method based on a unidirectional optical shutter system provided in this embodiment of the invention.

[0227] This invention is applicable to file retransmission operations during cross-domain file import, primarily addressing file corruption caused by network jitter, human error, and other issues. It aims to improve the efficiency of data recovery during cross-domain file import, primarily through two aspects: first, enhancing the detection rate of lost data through "recording import / export" technology; and second, improving the recovery rate of lost data through "fast retransmission" technology. This invention mainly targets recovery processing for message loss, file corruption, and internal database anomalies during file import. It retains recently transmitted files on the external network host's hard drive, performs cache management, and provides disaster recovery, thus resolving situations where the source file cannot be found.

[0228] I. This invention also includes a DAO data access interface designed to uniformly manage operations related to Redis and MySQL, providing services to the upper layer in the form of modules. Based on the data retransmission function, the module services provided by the data access interface layer are mainly divided into message queue, data statistics, record statistics, and record caching. Figure 6 This is a schematic diagram of the structure of the DAO data access interface provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the modules and their functions included in the DAO data access interface are as follows:

[0229] 1) Message queue: It mainly serves as a cache pool for the business modules and file tunnels of the external network host. Due to the different requirements of the internal and external network hosts for the cache pool, the message queues of the external network host are divided by priority, while the message queues of the internal network host are divided by task number.

[0230] 2) Data statistics: Record the number and size of files received and sent by the internal and external network hosts during data transmission. Users can compare the number of files received and sent on the same side to infer the current data caching status and data transmission rate. By comparing the number of files received by the internal network host with the number of files sent by the external network host, the reliability of data transmission during cross-domain file import can be inferred.

[0231] 3) Record statistics: It mainly caches the file records of the one-way optical shutter during the cross-domain file import process, on the one hand ensuring the visibility of data transmission, and on the other hand providing the function of fast file resending;

[0232] 4) Record caching: mainly divided into intranet cache and extranet cache. Extranet cache can be further subdivided into unimported cache and imported cache. Business data unimported in the cache provides data resources for file transfer, while imported cache provides data resources for retransmission. Intranet cache provides upload data resources for business modules on the intranet side host. Data cached on the intranet side host is deleted after being uploaded to the backend server by the business module. Data already sent in the extranet side host is transferred to the imported cache after being transferred by the file transfer layer. The imported cache on the extranet side host can be rolled back through configuration information to control the total amount of cache resources on the extranet side host.

[0233] II. File caching process

[0234] Cache files can improve data recovery speed through "fast retransmission" technology. "Fast retransmission" technology is guaranteed through data caching. Data caching is mainly divided into "internal network caching" and "external network caching." "Internal network caching" primarily addresses data loss caused by the data import speed from external network hosts to internal network hosts exceeding the transmission speed from internal network hosts to backend servers. "External network caching" primarily addresses data loss caused by the transmission speed from frontend servers to external network hosts exceeding the import speed from external network hosts to internal network hosts, and also provides rapid retransmission after data loss during the data import process from external network hosts to internal network hosts.

[0235] Figure 7 This is a schematic diagram of the file caching process in the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the file caching process is as follows:

[0236] 1. The business module stores the business data from the front-end server to the external network host and puts the event messages corresponding to the business data into a priority message queue; at the same time, the business data is stored in the first VFS file system in the form of cache files; statistical information and detailed records are recorded in the DAO. The statistical information includes, for example, the size of each of the 1000 files, and the detailed records include, for example, the specific details of each file, timestamps, etc.

[0237] It should be noted that statistical information can be stored in the Rsdis database via the DAO interface, which is more convenient and efficient; detailed records can be stored in the MySQL database via the DAO interface for easy retrieval later.

[0238] 2. The file tunnel of the external network host reads event messages from the priority message queue and reads the corresponding business data from the event message to the data tunnel of the one-way optical gate to complete the data import from the external network host to the internal network host.

[0239] 3. The internal network host receives service data from the data tunnel, completes the finalization process in the file tunnel, and puts the event message corresponding to the service data into the task message queue; if file corruption occurs, the corruption information is recorded in the DAO.

[0240] 4. The business modules of the internal network host read the event messages in the task message queue, and finally upload the business data corresponding to the event messages to the backend server in the form of cached files, thus completing the entire file import process.

[0241] III. Document Resending

[0242] Method 1: Select "Resend" in the WebUI;

[0243] Figure 8 This is a schematic diagram of the WebUI process for selecting resend in the file processing method based on a unidirectional light gate system provided in this embodiment of the invention, as shown below. Figure 8 As shown.

[0244] The WebUI's resend option is primarily implemented through "External Network Cache" and "Detailed Records". Resending business data that exists in both "Detailed Records" and "External Network Cache" reduces the time cost of server data queries and data transfer from the server to the external network host, thus improving file resend efficiency.

[0245] The WebUI list of pending entries can be accessed through "Detailed Records." "Detailed Records" are categorized by task number and include a certain number of valid file import records from external network hosts to internal network hosts. The WebUI retransmission list only displays records that exist in the cache of the first VFS.

[0246] The WebUI resend process is as follows:

[0247] 1. Add the selected records in the WebUI resend list to the detailed records to form the event messages required for "file import";

[0248] 2. Add the newly constructed event message as the second event message to the priority message queue;

[0249] 3. When the external network host reads the second event message, it reads the second service data corresponding to the second event message in the first VFS and sends the second service data to the internal network host through the one-way optical gate.

[0250] It should be noted that the business modules may include file business modules, database business modules, and email business modules.

[0251] Method 2: Record downloads and uploads;

[0252] The file retransmission record download and upload primarily addresses the rapid location and recovery of lost files during cross-domain file import. Record download and upload refers to downloading all records and / or corrupted records from the internal network host and manually uploading the relevant records to the external network host. The external network host filters the uploaded records and retransmits any unreceived business data from the internal network host. Record download from the internal network host is divided into "corrupted record download" and "all record download." The external network host can directly retransmit uploaded "corrupted records" without comparison.

[0253] Figure 9 This is a schematic diagram of the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention, illustrating the process of recording, downloading, and uploading. Figure 9 As shown, the upload and download process is as follows:

[0254] 1. The internal network host downloads records (damaged records and / or all records);

[0255] 2. Users manually upload downloaded records to the external network host;

[0256] 3. The external network host performs retransmission operations based on the uploaded records.

[0257] IV. Cache Management Technology

[0258] Caching management technologies include file caching (VFS), record caching, and rollback processes. File caching facilitates rapid file retransmission; record caching facilitates visualization of transmitted files and rapid location of lost files; and rollback processes protect device space and ensure the normal operation of the one-way optical shutter system.

[0259] (I) File Identifier Descriptor (FID) Generator

[0260] FID (File Identifier) ​​is used to transfer files and is used for file indexing and retransmission, as well as for identifying file corruption. The relevant specifications for FID are shown below:

[0261] 1) FID is divided by task, and there is no correlation between FIDs of tasks; they are all monotonically increasing.

[0262] 2) FID is maintained in shared memory and supports persistent storage on disk;

[0263] 3) FID is strictly auto-incrementing, and the interface design supports parallel calls from multiple threads and processes;

[0264] For example, Figure 10 This is a schematic diagram illustrating the relationship between the FID generator and the multi-channel multitasking provided in this embodiment of the invention, as shown below. Figure 10As shown, an FID generator can be set to correspond to task 1, and this FID generator can correspond to N channels of task 1; similarly, the correspondence between the FID generator and tasks 2 to N can be set respectively.

[0265] (II) File Caching Scheme

[0266] File caching is mainly divided into external network-side caching and internal network-side caching. Internal network-side caching refers to business data imported from the external network-side host to the internal network-side host that has not yet been transmitted to the back-end server. External network-side caching refers to business data downloaded from the front-end server by the external network-side host that has not yet been imported to the internal network-side host, as well as retained data that has been imported to the internal network-side host. It is possible to configure the deletion of cached data not uploaded by the internal network-side host and cached data to be imported from the external network-side host after data upload / import. External network-side caching may not be deleted within a certain range, but will be rolled back after exceeding the specified limit.

[0267] The business data cached in the first VFS of the external network host can be divided into imported data and non-imported data. The size of the imported data can be set, while the size of the non-imported data cannot be set. Specifically:

[0268] 1) The size of the imported data is controlled by the setting parameters. When the cache size exceeds the setting parameters, a rollback operation will be performed. Figure 11 This is a schematic diagram of a dual-directory hierarchy provided in an embodiment of the present invention, as shown below. Figure 11 As shown, this embodiment of the invention designs a dual-directory hierarchical structure to implement the rollback operation of imported data. First, based on the FID dimension, a first-level directory is divided to maintain a fixed cache size. Then, based on the size dimension, for example, a total size of 100MB is divided into a second-level directory. For example, the second-level directory SIZE:100MB_02 includes files of 90MB and 1MB.

[0269] It should be noted that, to ensure the accuracy of the "coarse rollback," the directory cache size may fluctuate and may be greater than, less than, or equal to the fragment value. Figure 11 The median fragment size is 100MB. For Figure 11In the SIZE setting (e.g., 100MB_03), for example, if the current actual size of the first directory is 70MB, and a 100MB file to be cached arrives, since the total size (100 + 70 > 100), the directory needs to be switched, and the 100MB file to be cached is stored in the second directory, whose actual size is 100MB. To ensure the accuracy of the "coarse rollback," the cacheable file size in the current directory can be adjusted. For example, the cache size in the second directory can be adjusted to (100 + 70)% * 100 = 70MB. Next, when the cache size of the file to be cached is 20MB, since 70 + 20 < 100, the 20MB file to be cached can also be stored in the second directory.

[0270] 2) The size of the unimported data is limited by the system space. For example, when the system space utilization rate is greater than 90%, the business modules of the external network host will not perform data pull-down operations and will wait for the already landed business data to be imported to the internal network host.

[0271] The business data cached in the second VFS of the intranet host is limited by system space. For example, when the system space utilization rate is greater than 90%, the business modules of the intranet host do not perform persistence processing on the imported business data, but wait for the persistence business data to be uploaded to the back-end server.

[0272] (III) Record caching scheme

[0273] The record cache refers to the import records of files imported during cross-domain file import in a one-way optical gateway system. These records are primarily used to quickly locate corrupted files and query imported file records by comparing records on internal and external network hosts. The number of records in the cache is configurable and can be managed from the backend, for example, by performing rollback operations to prevent the cache from growing indefinitely.

[0274] Record caching can be partitioned and stored in tables based on task numbers, and then managed through partitioning using FID. By default, the record cache table consists of 32 partitions, each capable of storing 100,000 records. Records are managed through partitioning using the FID field. Within a single task, FID is a locally incrementing field, meaning that the FID increments during the time interval between two cache clearing operations. FID needs to be reset during cache clearing to ensure that the number of cached records matches the configured cacheable value. Failure to reset FID during cache clearing may result in unexpected record cache behavior, such as the following:

[0275] 1) Enable record caching and configure the threshold for the number of records cached for rollback operations to be 100,000, which means that at most one partition will be retained;

[0276] 2) The external network host stores 60,000 units of business data to be imported;

[0277] 3) After the external network host has sent all the business data to be imported, the number of records cached in the external network host is 60,000;

[0278] 4) Clear the record cache of the external network host;

[0279] 5) After clearing the record cache, the current record cache count is 0, and the current partition count is 0;

[0280] 6) Place another 50,000 units of business data to be imported on the external network host;

[0281] 7) After the external network host has sent all the business data to be imported, the number of records cached in the external network host is 10,000, and the current partition is 1;

[0282] During the cross-domain file import process of the aforementioned one-way optical shutter system, the expected number of records in the cache was 50,000, but the actual number was 10,000. This is because the record cache is managed by partitioning using FIDs. However, since the record FIDs were not reset during the clearing task in step 4), after all the business data to be imported in the second batch was imported, the current FID was 110,000, and the current partition was 1. Performing a rollback operation on the record cache based on the configured threshold of 100,000 records for rollback operations would delete all partition records except the current partition, thus mistakenly deleting all records with FIDs between 60001 and 100000.

[0283] Optionally, a larger value can be configured for the threshold number of records cached for performing rollback operations. For example, the threshold number of records cached can be configured to be greater than the actual required threshold. Instead of configuring 100,000, it can be configured to 200,000 or 300,000. In this way, even if records are accidentally deleted, the number of accidentally deleted records will cause a smaller error compared to the larger number of records cached.

[0284] (iv) Rollback Strategy Design

[0285] Rollback operations can be divided into record cache rollback and file cache rollback. The purpose of performing rollback operations is mainly to prevent the cached data from increasing indefinitely, which would affect the performance and functionality of business modules.

[0286] Performing a rollback operation on the record cache is to control the number of records in the cache, ensuring it remains within the set parameter range. The record cache can be stored in a MySQL table. To address the slow rollback issue, the record cache can be partitioned into tables, by task and by record fileid (FID), improving rollback speed. The backend MySQL table can be partitioned based on fileid, with a default number of partitions of 32. Each partition can store 100,000 records, so the configurable number of records in the cache is 0 to 3.2 million (0 indicates no record caching, and other configuration parameters are rounded down). During the rollback operation, rolling back a single record is inefficient and time-consuming, while rolling back by partitions effectively improves efficiency. The scenario analysis is as follows:

[0287] Scenario 1: The threshold for the number of records cached is configured to 0, the record caching function is not enabled, and no data is written to the database from the record cached records.

[0288] Scenario 2: The threshold for the number of records cached is configured to be 90,000. If the number of records cached in a single partition is set to 100,000, the backend can adjust the configured number. If the number is less than 100,000, it is considered an abnormal configuration and will be handled according to 200,000, which means that two partitions will be retained.

[0289] Figure 12 This is a flowchart illustrating the process of performing a rollback operation on the record cache according to an embodiment of the present invention, as shown below. Figure 12 As shown, the rollback of the record cache is as follows: the record is inserted into the database by taking the remainder of the fileid attribute to obtain the MySQL storage location (step 1). When the number of records in the cache is greater than the record cache number threshold (step 2), that is, the three partitions p3, p4 and p5, the record cache can be rolled back through the partitions (step 3). All other partitions except the one to be retained are cleared, that is, the rollback operation is performed on the p3 partition, and the p4 and p5 partitions are retained.

[0290] Scenario 3: The threshold for the number of records to be cached is configured as 110,000. Following the rounding up strategy, the backend adjusts the configured number to 100,000, which means that one partition is retained.

[0291] Performing a rollback operation on the file cache is to control the data imported into the external network host, ensuring that the cache size remains within the set parameter range. Business data is stored in the first VFS as cache files, and the rollback strategy can be dynamically adjusted to quickly roll back the cache files.

[0292] Rollback Strategy 1: Precise rollback, which involves scanning the cache directory of the first VFS and deleting the cache files in the cache directory one by one;

[0293] Rollback Strategy 2: Coarse rollback, directly deleting the cache directory of the first VFS to achieve a fast rollback.

[0294] For example, the cache directory is first scanned to obtain the cached files and their total number. When the total number of cached files is less than the set rollback policy switching value (e.g., 5000), rollback policy one can be used to roll back the cached files; when the total number of cached files is not less than the set rollback policy switching value, rollback policy two can be used to roll back the cached files.

[0295] V. WebUI Integration Solution

[0296] The WebUI integration provides a visual interface for the file cache list and record cache list. The file cache list allows users to query detailed information for all cached files (business data in cached file format) on the external network host, and select individual or multiple cached files for resending. The record cache list allows users to query statistics for imported files on the internal network host, and download all records or upload corrupted records to the external network host for resending.

[0297] (i) Retrieve file cache and resend

[0298] The WebUI file cache list is a visual representation of the file cache retrieved from external network hosts, making it easy for users to query detailed information about the file cache.

[0299] Figure 13 This is a schematic diagram of the process for retrieving the file cache in the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention, as shown below. Figure 13 As shown, the file retrieval cache process is as follows:

[0300] 1) The user logs into the external network host and clicks the "File Resend" menu;

[0301] 2) Query all task tables in the MySQL database through the WebUI. Users can switch between different task tables for querying. The first task table is selected by default. If there is no data, it will display "No data available".

[0302] 3) The data source is dynamically switched according to the user's selected task table. The query method uses pagination query, which can optimize the query speed, reduce bandwidth usage, reduce memory pressure, and sort according to FID in reverse order, always displaying the latest cached file at the top.

[0303] 4) Optimize the query results. First, iterate through the query results and check if the file exists in each cached data information. If the file exists, the page can select it and resend it. If the file does not exist, the selection box is grayed out to prevent the user from selecting it for resending.

[0304] 5) Return the processed results to the page for display.

[0305] The WebUI records the cache list to query intranet-side file cache statistics, enabling users to quickly locate lost files, download records, and log in to the one-way shutter's external network side to resend the files.

[0306] Figure 14 This is a schematic diagram illustrating the process of querying record cache and downloading in the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention, as shown below. Figure 14 As shown, the query record caching and download process is as follows:

[0307] 1) The user logs into the intranet host and clicks the file resend menu;

[0308] 2) Query the Redis database via the WebUI and display the file cache list;

[0309] 3) Users can quickly locate lost files on the page and select different records to download;

[0310] 4) Download all records, and then dynamically connect to the MySQL database to export the relevant SQL files based on the task table of the records selected by the user;

[0311] 5) Download the corrupted records, then query the corrupted record data table, and write the retrieved data to a temporary file;

[0312] 6) Encrypt the file to prevent information leakage, export the encrypted file to the user, and the page will indicate that the download was successful.

[0313] (III) Batch import records and select data to resend

[0314] WebUI file resending is divided into two types: resending files selected from the file cache list on the external network host, and downloading files from the record cache list on the internal network host and uploading them to the external network host for resending.

[0315] Figure 15 This is a schematic diagram of the process of resending a selected record in the file cache list in the file processing method based on a unidirectional light gate system provided in an embodiment of the present invention, as shown below. Figure 15 As shown, the resending process for selected records in the file cache list is as follows:

[0316] 1) Batch record selection resend: When a user selects multiple records to resend, the system first iterates through the FID set, queries the complete data of each FID, and queries the detailed configuration information of the current task table. Then, it performs data deduplication based on whether the absolute file paths in the data are the same. Finally, it iterates through the deduplicated data set again.

[0317] 2) Select a single record to resend: When a user selects a single record to resend, the WebUI queries the complete data for each FID and retrieves the detailed configuration information for the current task table.

[0318] 3) By checking the number of data shards, determine if the file corresponding to each record is sharded. If the total number of file shards is greater than 1, query the database again, retrieving the complete file dataset based on the absolute file path and one hour before and after the current record. After traversing the dataset again, assemble it into the data structure required for file retransmission and transmit it to the Redis database via an asynchronous lpush operation. Simultaneously, check if the consumer queue is full. If the queue is full, activate the loading mechanism to prevent data loss. If the total number of shards is equal to 1, directly perform an asynchronous lpush operation; where lpush refers to the enqueue operation, that is, adding the record to be retransmitted to the priority message queue of the Redis database.

[0319] 4) Because it is an asynchronous lpush operation, the WebUI does not need to wait for all data to be resent before returning the message "Operation successful, file is being resent" to the page.

[0320] Figure 16 This is a schematic diagram illustrating the process of recording the cache list for downloaded files and uploading and retransmitting them on the external network host in the file processing method based on a unidirectional optical gate system provided in this embodiment of the invention. Figure 16 As shown, the process for uploading and retransmitting downloaded files from the cache list to the external host is as follows:

[0321] 1) Record Upload: Users log in to the external network host, click the record upload button on the WebUI interface, and select to upload files downloaded from the record cache list of the internal network host;

[0322] 2) After receiving the file through the WebUI, the file is decrypted. If decryption fails, the page is notified of the upload failure and the specific reason. If decryption is successful, the upload record is determined based on the file type to determine whether it is a complete record or a corrupted record.

[0323] 3) If the file is an SQL file, it means the uploaded file contains all records. You need to first import the SQL file into a temporary MySQL database table. The SQL file content includes database reset statements. Use the UNION ALL statement to join the temporary table and the task table, compare the data, quickly compare for differences, and identify the datasets with discrepancies.

[0324] 4) If it is a temporary file type, move the temporary file to a fixed temporary directory and read all the datasets in that file;

[0325] 5) The subsequent process is the same as the batch resend operation of the selected files in the file cache list.

[0326] VI. Disaster Recovery and Rescue Tool Design

[0327] The above embodiments illustrate how the unidirectional optical shutter system performs file caching and retransmission during normal transmission. This invention provides a data disaster recovery and handling scheme for severe abnormal scenarios. It assumes that disk aging or abnormal power outages may cause data table corruption and database service startup failures within the unidirectional optical shutter system.

[0328] (I) Database Rescue

[0329] The database self-check and repair buttons on the WebUI page employ two methods internally: one method prioritizes using MySQL's check table and repair table SQL statements to attempt recovery, and the other method uses multiple data detection and recovery tools provided by MySQL, such as myisamchk and isamchk.

[0330] (II) File Cache Recovery

[0331] If the cached information in the data table cannot be recovered through database recovery, then the second-level recovery method will be used.

[0332] File cache recovery is primarily used for restoring file cache data in situations where there is no message body cache. When the message body cache is damaged and the relevant message body data cannot be repaired, restoring the file cache data is equivalent to rebuilding the file cache message body. Depending on the functional requirements of the internal and external networks, file cache recovery can be divided into rebuilding message bodies awaiting transmission in priority message queues and rebuilding message bodies awaiting transmission in task message queues. The specific reconstruction process is as follows:

[0333] 1) Priority message queue message body reconstruction: Scan the cached data of the pending transmission record of the external network host, determine the task to which the file belongs by the cached file path, reconstruct the message body to be transmitted by parsing the file header information, and insert the reconstructed message body into the high priority message queue so that the external network host can import the corresponding business data to the internal network host through the file tunnel.

[0334] 2) Reconstruction of message body to be transmitted in task message queue: Scan the record cache data of the internal network host, determine the task to which the file belongs by the file path, reconstruct the message body to be transmitted by parsing the file header information, and insert the reconstructed message body into the task message queue corresponding to the task, so that the business modules of the internal network host can upload the corresponding business data to the back-end server.

[0335] The technical innovation of this invention lies in the design of a file caching and file retransmission technical solution based on a unidirectional light gate system. This solution covers everything from the design of the front-end WebUI to the design of the back-end data caching structure, caching management, and rollback strategy, and also includes the handling of normal scenarios and the rescue of abnormal scenarios.

[0336] 1. The embodiments of the present invention are used in the scenario of a one-way light gate system, and support a unified solution for file caching with upper-layer business modules (such as file business, database business, email business).

[0337] 2. The embodiments of the present invention can also be used for a unified file caching solution for a unidirectional optical gate system including front and back servers.

[0338] In the embodiments of the present invention, the following beneficial effects exist:

[0339] 1) By employing file caching and record caching technologies, the reliability and visibility of cross-domain file imports in the one-way shutter system are ensured at the application layer. A rollback mechanism prevents unlimited increases in memory and disk usage, guaranteeing business stability. Batch processing technology using record caching improves record loading performance and reduces the impact of record loading on cross-domain file imports.

[0340] 2) By designing the record statistics service, batch data record import processing is realized. Compared with the solution of processing each task independently, the CPU utilization rate is reduced by 10%, the record import performance is improved by 60%, and the number of operations per second (ops) reaches 110,000, which fully meets the requirements of the unidirectional light gate whole machine business processing.

[0341] 3) By using FID generator technology to replace the original Redis persistence solution, FID is required for identification during file transfer. First, the external host needs to initialize the FID of the file to be transferred. mmap's performance specifications are two orders of magnitude (one hundred times) higher than rRedis. Based on the experience of multi-task and multi-concurrency, concurrency of 2, 4, and 8 is more meaningful. The mmap method is basically stable at ops=1551w, which fully meets the business processing of the entire optical shutter.

[0342] 4) The WebUI integrated design ensures the product's ease of use and maintainability.

[0343] 5) The design of disaster recovery tools ensures that even in extreme and abnormal scenarios, the one-way optical shutter system can recover user data, reducing the risk of data loss.

[0344] The following describes the file processing device based on a one-way optical shutter system provided by the present invention. The file processing device based on a one-way optical shutter system described below can be referred to in correspondence with the file processing method based on a one-way optical shutter system described above.

[0345] The unidirectional optical shutter system includes an external network-side host, an internal network-side host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network-side host to the internal network-side host. Figure 17 This is one of the structural schematic diagrams of a file processing device based on a unidirectional optical shutter system provided in an embodiment of the present invention, such as... Figure 17 As shown, the file processing device 1700 based on a one-way optical shutter system can be applied to an external network host. The device includes:

[0346] The first caching module 1701 is used to cache the service data in the form of a cache file in the first VFS when the service data is received, and to add the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0347] Import module 1702 is used to read the first event message sequentially from the priority message queue according to the reading priority order, and when reading each first event message, read the service data corresponding to the first event message from the first VFS, and import the service data to the intranet-side host through the unidirectional optical gate.

[0348] The file processing device based on a unidirectional optical shutter system provided in this invention includes a first caching module that, upon receiving service data, first caches the service data in a first Virtual File System (VFS) and adds the corresponding first event message and its read priority to the priority message queue of the external network host. The import module then reads the first event messages sequentially from the queue according to their read priority. For each first event message, the corresponding service data is read from the first VFS and imported into the internal network host via the unidirectional optical shutter. By caching the service data in the first VFS, this invention can effectively improve the reliability of file transmission via the unidirectional optical shutter, even if the speed at which the external network host receives service data exceeds the speed at which it imports data into the internal network host, thus allowing technicians to retrieve lost service data from the first VFS.

[0349] Optionally, the document processing device 1700 based on the one-way light gate system further includes:

[0350] The first processing unit is used for:

[0351] If the first condition is met, a second event message and its corresponding read priority are added to the priority message queue.

[0352] If the second event message with the highest read priority is read from the priority message queue, the second service data corresponding to the second event message is read from the first VFS, and the second service data is imported into the intranet-side host through the unidirectional optical gate;

[0353] The first condition includes at least one of the following:

[0354] 1) The event messages that have been selected and recorded in the retransmission list constructed by the external network host are used as the second event messages;

[0355] 2) The external network host obtains the damage record of the internal network host, and the damage record is used to indicate that the internal network host did not accept the second service data;

[0356] 3) The external network host obtains all records of the internal network host, and determines based on all records that the internal network host has not accepted the second service data.

[0357] Optionally, the service data cached in the first VFS includes imported data and unimported data. The imported data is the service data that has been imported from the external network host to the internal network host, and the unimported data is the service data that has not yet been imported from the external network host to the internal network host.

[0358] The first processing unit is also used for:

[0359] If the cache size occupied by the imported data exceeds a first threshold, a rollback operation is performed on the imported data.

[0360] If the cache size occupied by the unimported data is greater than the second threshold, the reception of business data from the front-end server is suspended until the cache size occupied by the unimported data is less than the third threshold, at which point the reception of business data from the front-end server resumes. The third threshold is less than the second threshold.

[0361] Optionally, the first processing unit is further configured to:

[0362] The file record information during the data transmission process of the unidirectional optical shutter is cached in the first memory of the external network host;

[0363] If the second condition is met, perform an input operation on the file record information in the first memory.

[0364] The second condition includes at least one of the following:

[0365] 1) The inbound operation was not performed within the timeout period;

[0366] 2) The number of file record information cached in the first memory is greater than the maximum number of commits;

[0367] 3) The space occupied by the file record information cached in the first memory is greater than the maximum length of a single commit statement.

[0368] Optionally, the first caching module 1701 is specifically used to: upon receiving the service data, cache the service data in the form of a cache file in the cache directory of the first VFS;

[0369] The first processing unit is also used for:

[0370] If the number of cache files corresponding to the business data in the first VFS is less than the fourth threshold, a rollback operation is performed using the cache files in the cache directory as the operation unit.

[0371] If the number of cached files corresponding to the business data in the first VFS is greater than or equal to the fifth threshold, a rollback operation is performed using the cache directory as the unit of operation.

[0372] Optionally, the first processing unit is further configured to:

[0373] If the message body cache of the first event message in the priority message queue is damaged and the message body cache cannot be repaired, obtain the cache file path of the cache file corresponding to the business data in the first VFS;

[0374] Based on the cache file path, determine the task to which the cache file belongs;

[0375] Based on the task, the message body cache of the first event message is reconstructed by parsing the file header information of the cache file;

[0376] The message body cache of the reconstructed first event message is added to the priority message queue.

[0377] Figure 18 This is a second schematic diagram of the structure of a file processing device based on a unidirectional optical shutter system provided in an embodiment of the present invention, as shown below. Figure 18 As shown, the file processing device 1800 based on a one-way optical shutter system can be applied to an intranet-side host. The device includes:

[0378] The receiving module 1801 is used to receive service data from the external network host through the unidirectional optical shutter;

[0379] The second caching module 1802 is used to cache the service data in the form of a cache file in the second VFS of the intranet-side host, and to add the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host; wherein, the task message queue includes at least one event message and its corresponding task number;

[0380] The transmission module 1803 is used to read the third event messages sequentially from the task message queue according to the task number, and when reading each third event message, read the corresponding business data from the second VFS and upload the business data to the back-end server.

[0381] The file processing device based on a unidirectional optical shutter system provided in this embodiment of the invention includes a receiving module that receives service data from an external network host. Upon receiving the service data from the external network host, the second caching module first caches the service data in a second Virtual File System (VFS) and adds the corresponding third event message and its task number to the task message queue of the internal network host. The transmission module then reads the third event messages sequentially from the queue according to their task numbers. For each third event message, the corresponding service data is read from the second VFS and uploaded to the backend server. By caching the service data in the second VFS, this embodiment of the invention can effectively improve the reliability of file transmission via the unidirectional optical shutter, even if the speed at which the internal network host receives service data exceeds the speed at which it uploads data to the backend server, thus allowing technicians to retrieve lost service data from the second VFS.

[0382] Optionally, the document processing device 1800 based on the one-way light gate system further includes:

[0383] The second processing module is used to record the service data received by the intranet-side host as a complete record.

[0384] Optionally, the second processing module is also used to record the corrupted service data as a corruption record if it is determined that the received service data is corrupted.

[0385] Optionally, the second processing module is further configured to: suspend receiving service data from the external network host when the cache size occupied by the service data cached in the second VFS is greater than the sixth threshold, until the cache size occupied by the service data cached in the second VFS is less than the seventh threshold, and then start receiving service data from the external network host, wherein the seventh threshold is less than the sixth threshold.

[0386] Optionally, the second processing module is also used for:

[0387] The file record information during the data transmission process of the unidirectional optical shutter is cached in the second memory of the intranet-side host;

[0388] If the third condition is met, the file record information in the second memory is inserted into the database.

[0389] The third condition includes at least one of the following:

[0390] 1) The inbound operation was not performed within the timeout period;

[0391] 2) The number of file record information cached in the second memory is greater than the maximum number of commits;

[0392] 3) The space occupied by the file record information cached in the second memory is greater than the maximum length of a single commit statement.

[0393] Optionally, the second caching module 1802 is specifically used for: the intranet-side host caching the service data in the form of cache files to the cache directory of the second VFS of the intranet-side host;

[0394] The second processing module is also used for:

[0395] If the number of cache files corresponding to the business data in the second VFS is less than the eighth threshold, a rollback operation is performed using the cache files in the cache directory as the operation unit.

[0396] If the number of cached files corresponding to the business data in the second VFS is greater than or equal to the ninth threshold, a rollback operation is performed using the cache directory as the unit of operation.

[0397] Optionally, the second processing module is also used for:

[0398] If the message body cache of the third event message in the task message queue is damaged and the message body cache cannot be repaired, obtain the cache file path of the cache file corresponding to the business data in the second VFS;

[0399] Based on the cache file path, determine the task to which the cache file belongs;

[0400] Based on the task, the message body cache of the third event message is reconstructed by parsing the file header information of the cache file;

[0401] The message body cache of the reconstructed third event message is added to the task message queue.

[0402] Figure 19 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of the present invention, such as... Figure 19 As shown, the electronic device 1900 may include: a processor 1910, a communication interface 1920, a memory 1930, and a communication bus 1940, wherein the processor 1910, the communication interface 1920, and the memory 1930 communicate with each other through the communication bus 1940. The processor 1910 can call logical instructions in the memory 1930 to execute the following file processing method based on a unidirectional optical shutter system. The unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host. The method includes:

[0403] When the external network host receives service data, it caches the service data in the form of a cache file in the first virtual file system (VFS), and adds the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0404] The external network host reads the first event message sequentially from the priority message queue according to the reading priority order, and reads the service data corresponding to the first event message from the first VFS when reading each first event message, and imports the service data into the internal network host through the unidirectional optical gate;

[0405] or,

[0406] Processor 1910 can call logic instructions in memory 1930 to execute the following file processing method based on a unidirectional optical shutter system, wherein the unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host, and the method includes:

[0407] The internal network host receives service data from the external network host through the one-way optical shutter;

[0408] The intranet-side host caches the service data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and adds the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host.

[0409] The intranet-side host reads the third event messages sequentially from the task message queue according to the task number, and reads the corresponding business data from the second VFS for each third event message, and uploads the business data to the backend server.

[0410] Furthermore, the logical instructions in the aforementioned memory 1930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0411] On the other hand, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the following file processing method based on a unidirectional optical shutter system. The unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host. The method includes:

[0412] When the external network host receives service data, it caches the service data in the form of a cache file in the first virtual file system (VFS), and adds the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0413] The external network host reads the first event message sequentially from the priority message queue according to the reading priority order, and reads the service data corresponding to the first event message from the first VFS when reading each first event message, and imports the service data into the internal network host through the unidirectional optical gate;

[0414] or,

[0415] A file processing method based on a unidirectional optical shutter system is implemented, wherein the unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host. The method includes:

[0416] The internal network host receives service data from the external network host through the one-way optical shutter;

[0417] The intranet-side host caches the service data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and adds the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host.

[0418] The intranet-side host reads the third event messages sequentially from the task message queue according to the task number, and reads the corresponding business data from the second VFS for each third event message, and uploads the business data to the backend server.

[0419] In another aspect, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, implement the following file processing method based on a unidirectional optical shutter system, the unidirectional optical shutter system including an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host, the method including:

[0420] When the external network host receives service data, it caches the service data in the form of a cache file in the first virtual file system (VFS), and adds the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host.

[0421] The external network host reads the first event message sequentially from the priority message queue according to the reading priority order, and reads the service data corresponding to the first event message from the first VFS when reading each first event message, and imports the service data into the internal network host through the unidirectional optical gate;

[0422] or,

[0423] A file processing method based on a unidirectional optical shutter system is implemented, wherein the unidirectional optical shutter system includes an external network host, an internal network host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network host to the internal network host. The method includes:

[0424] The internal network host receives service data from the external network host through the one-way optical shutter;

[0425] The intranet-side host caches the service data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and adds the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host.

[0426] The intranet-side host reads the third event messages sequentially from the task message queue according to the task number, and reads the corresponding business data from the second VFS for each third event message, and uploads the business data to the backend server.

[0427] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0428] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0429] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A file processing method based on a unidirectional optical shutter system, characterized in that, The unidirectional optical shutter system includes an external network-side host, an internal network-side host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network-side host to the internal network-side host. The method includes: Upon receiving service data, the external network host caches the service data as a cache file in the first virtual file system (VFS), and adds the first event message corresponding to the service data and the read priority corresponding to the first event message to the priority message queue of the external network host. The external network host reads the first event message sequentially from the priority message queue according to the read priority order, and reads the service data corresponding to the first event message from the first VFS when reading each first event message, and imports the service data to the internal network host through the unidirectional optical gate. If the message body cache of the first event message in the priority message queue is damaged and the message body cache cannot be repaired, the external network host obtains the cache file path of the cache file corresponding to the service data in the first VFS. The reasons for the damage to the message body cache include disk aging or abnormal power failure of the device. The external network host determines the task to which the cache file belongs based on the cache file path; Based on the task, the external network host reconstructs the message body cache of the first event message by parsing the file header information of the cache file; The external network host adds the message body cache of the reconstructed first event message to the priority message queue.

2. The method according to claim 1, characterized in that, The method further includes: If the first condition is met, the external network host adds a second event message and the corresponding read priority of the second event message to the priority message queue. When the external network host reads the second event message with the corresponding reading priority from the priority message queue, it reads the second service data corresponding to the second event message from the first VFS and imports the second service data into the internal network host through the unidirectional optical shutter. The first condition includes at least one of the following: The external network host has selected and recorded the corresponding event messages in the retransmission list as the second event message; The external network host obtains the damage record of the internal network host, and the damage record is used to indicate that the internal network host did not accept the second service data; The external network host obtains all records of the internal network host and determines, based on all records, that the internal network host has not accepted the second service data.

3. The method according to claim 1, characterized in that, The business data cached in the first virtual file system (VFS) includes imported data and unimported data. The imported data is business data that has been imported from the external network host to the internal network host, and the unimported data is business data that has not yet been imported from the external network host to the internal network host. The method further includes: If the cache size occupied by the imported data exceeds a first threshold, the external network host performs a rollback operation on the imported data. If the cache size occupied by the unimported data is greater than the second threshold, the external network host will suspend receiving service data from the front-end server until the cache size occupied by the unimported data is less than the third threshold, at which point it will start receiving service data from the front-end server. The third threshold is less than the second threshold.

4. The method according to claim 1, characterized in that, The method further includes: The external network host caches the file record information during the data transmission process of the unidirectional optical shutter in the first memory of the external network host; When the second condition is met, the external network host performs an input operation on the file record information in the first memory; The second condition includes at least one of the following: The inbound operation was not performed within the timeout period; The number of file record information cached in the first memory is greater than the maximum number of commits; The space occupied by the file record information cached in the first memory is greater than the maximum length of a single commit statement.

5. The method according to claim 1, characterized in that, Upon receiving service data, the external network host caches the service data as a cache file in the first virtual file system (VFS), including: Upon receiving the service data, the external network host caches the service data as a cache file in the cache directory of the first VFS. The method further includes: If the number of cache files corresponding to the business data in the first VFS is less than the fourth threshold, the external network host performs a rollback operation using the cache files in the cache directory as the operation unit. If the number of cached files corresponding to the business data in the first VFS is greater than or equal to the fifth threshold, the external network host performs a rollback operation using the cache directory as the unit of operation.

6. A file processing method based on a one-way light gate system, characterized in that, The unidirectional optical shutter system includes an external network-side host, an internal network-side host, and a unidirectional optical shutter for controlling the unidirectional transmission of data from the external network-side host to the internal network-side host. The method includes: The internal network host receives service data from the external network host through the unidirectional optical gate. The service data is cached in the form of a cache file in the first virtual file system (VFS) when the external network host receives the service data. The first event message corresponding to the service data and the read priority corresponding to the first event message are added to the priority message queue of the external network host. The first event message is read from the priority message queue in the order of read priority. When reading each first event message, the service data corresponding to the first event message is read from the first VFS. The intranet-side host caches the service data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and adds the third event message corresponding to the service data and the task number corresponding to the third event message to the task message queue of the intranet-side host. The intranet-side host reads the third event messages sequentially from the task message queue according to the task number, and reads the corresponding business data from the second VFS for each third event message, and uploads the business data to the backend server. If the message body cache of the third event message in the task message queue is damaged and the message body cache cannot be repaired, the intranet-side host obtains the cache file path of the cache file corresponding to the service data in the second VFS; The internal network host determines the task to which the cache file belongs based on the cache file path; Based on the task, the intranet-side host reconstructs the message body cache of the third event message by parsing the file header information of the cache file; The intranet-side host adds the message body cache of the reconstructed third event message to the task message queue. The message body cache may be damaged due to disk aging or abnormal power failure of the device.

7. The method according to claim 6, characterized in that, After the internal network host receives service data from the external network host through the unidirectional optical gate, the method further includes: The internal network side host records the service data received by the internal network side host as a complete record.

8. The method according to claim 6 or 7, characterized in that, The method further includes: When the internal network host determines that the received service data is corrupted, it records the corrupted service data as a corruption record.

9. The method according to claim 6, characterized in that, The method further includes: If the cache size occupied by the service data cached in the second VFS is greater than the sixth threshold, the internal network host will suspend receiving service data from the external network host until the cache size occupied by the service data cached in the second VFS is less than the seventh threshold, at which point it will start receiving service data from the external network host. The seventh threshold is less than the sixth threshold.

10. The method according to claim 6, characterized in that, The method further includes: The intranet-side host caches the file record information during the data transmission process of the unidirectional optical shutter in the second memory of the intranet-side host. Under the condition that the third condition is met, the intranet-side host performs an input operation on the file record information in the second memory; The third condition includes at least one of the following: The inbound operation was not performed within the timeout period; The number of file record information cached in the second memory is greater than the maximum number of commits; The space occupied by the file record information cached in the second memory is greater than the maximum length of a single commit statement.

11. The method according to claim 6, characterized in that, The intranet-side host caches the service data as cache files in the second virtual file system (VFS) of the intranet-side host, including: The intranet-side host caches the service data as cache files in the cache directory of the second VFS of the intranet-side host; The method further includes: If the number of cache files corresponding to the service data in the second VFS is less than the eighth threshold, the intranet-side host performs a rollback operation using the cache files in the cache directory as the operation unit. If the number of cached files corresponding to the business data in the second VFS is greater than or equal to the ninth threshold, the intranet-side host performs a rollback operation using the cache directory as the unit of operation.

12. A document processing device based on a unidirectional optical shutter system, characterized in that, The unidirectional optical shutter system includes an external network-side host, an internal network-side host, and a unidirectional optical shutter for controlling the unidirectional data transmission from the external network-side host to the internal network-side host. The device includes: The first caching module is used to cache the business data as a cache file in the first virtual file system (VFS) when the business data is received, and to add the first event message corresponding to the business data and the read priority corresponding to the first event message to the priority message queue of the external network host. The import module is used to read the first event message sequentially from the priority message queue according to the reading priority order, and when reading each first event message, read the service data corresponding to the first event message from the first VFS, and import the service data to the intranet-side host through the unidirectional optical gate; If the message body cache of the first event message in the priority message queue is damaged and the message body cache cannot be repaired, the external network host obtains the cache file path of the cache file corresponding to the service data in the first VFS. The reasons for the damage to the message body cache include disk aging or abnormal power failure of the device. The external network host determines the task to which the cache file belongs based on the cache file path; Based on the task, the external network host reconstructs the message body cache of the first event message by parsing the file header information of the cache file; The external network host adds the message body cache of the reconstructed first event message to the priority message queue.

13. A document processing device based on a unidirectional optical shutter system, characterized in that, The unidirectional optical shutter system includes an external network-side host, an internal network-side host, and a unidirectional optical shutter for controlling the unidirectional data transmission from the external network-side host to the internal network-side host. The device includes: The receiving module is used to receive service data from the external network host through the unidirectional optical shutter; The second caching module is used to cache the business data in the form of a cache file in the second virtual file system (VFS) of the intranet-side host, and to add the third event message corresponding to the business data and the task number corresponding to the third event message to the task message queue of the intranet-side host. The transmission module is used to read the third event messages sequentially from the task message queue according to the task number, and when reading each third event message, read the corresponding business data from the second VFS and upload the business data to the backend server. If the message body cache of the third event message in the task message queue is damaged and the message body cache cannot be repaired, the intranet-side host obtains the cache file path of the cache file corresponding to the service data in the second VFS; The internal network host determines the task to which the cache file belongs based on the cache file path; Based on the task, the intranet-side host reconstructs the message body cache of the third event message by parsing the file header information of the cache file; The intranet-side host adds the message body cache of the reconstructed third event message to the task message queue. The message body cache may be damaged due to disk aging or abnormal power failure of the device.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the file processing method based on a unidirectional optical shutter system as described in any one of claims 1 to 5, or implements the file processing method based on a unidirectional optical shutter system as described in any one of claims 6 to 11.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the file processing method based on the unidirectional light gate system as described in any one of claims 1 to 5, or implements the file processing method based on the unidirectional light gate system as described in any one of claims 6 to 11.

16. A computer program product having executable instructions stored thereon, characterized in that, When executed by the processor, the instruction causes the processor to implement the file processing method based on the unidirectional light gate system as described in any one of claims 1 to 5, or to implement the file processing method based on the unidirectional light gate system as described in any one of claims 6 to 11.

Images