Multi-terminal cooperative office method and system based on unidirectional isolation transmission equipment

Abstract

The invention relates to the technical field of data transmission, and provides a multi-terminal cooperative office method and system based on unidirectional isolation transmission equipment. The method comprises the steps that a desktop transmission tool / web end / app end uploads file data to an external network, if the size of the file data is smaller than a first threshold value, a specific number of file data in a period of time is written into a folder, and each file database index is configured and written into file meta-information; if the size of the file data is greater than a second threshold value, performing slice transmission on the file data, configuring a private protocol of file service segment information, and writing file meta-information; the extranet performs security check on the file data based on a preset service strategy, and sends the file data to the intranet after the audit is passed; and the intranet distributes the received file data to a database and a redis cache, and the desktop transmission tool / web end / app end initiates a request to the intranet, so that the intranet sends the file data corresponding to the request to the desktop transmission tool / web end / app end.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and in particular to a multi-terminal collaborative office method and system based on a unidirectional isolated transmission device. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In highly confidential network environments, there is a significant conflict between the need for multi-terminal collaborative work and the security mechanisms of physical isolation. Existing solutions include: establishing encrypted tunnels via VPNs to access intranet resources, remotely controlling office hosts to transmit commands, building unified intranet links using Software-Defined Wide Area Networks (SD-WAN), and manually transferring data between physically isolated mobile storage devices. However, these solutions all have obvious drawbacks: VPN solutions suffer from coarse-grained control and are prone to interruptions due to gateway failures; remote control solutions require extremely high network stability and struggle to achieve file collaboration between multiple devices; SD-WAN solutions cannot directly address employees' remote access needs; and manual data transfer solutions are inefficient, rely on manual operation for security, and lack audit and traceability capabilities. The core contradiction of these technical solutions lies in the difficulty of balancing security and efficiency. Overemphasizing security sacrifices convenience and real-time performance, while prioritizing efficiency may introduce security risks.

[0004] Furthermore, the transmission process involves data transfer. Small files, due to their small size and large number, result in a high proportion of file metadata, requiring frequent disk seek operations during access, significantly reducing disk I / O efficiency and creating a significant disk performance bottleneck. To alleviate this problem, existing solutions often involve upgrading hardware configurations, such as increasing the number of high-performance disks or upgrading the storage controller, which undoubtedly increases hardware procurement and maintenance costs considerably. At the same time, traditional storage technologies lack the ability to automatically identify the characteristics of small files, and their poor packing strategies lead to ineffective packing, failing to fundamentally solve the performance and cost problems caused by small file storage. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a multi-terminal collaborative office method and system based on a unidirectional isolation transmission device. Through a collaborative design of "threshold-triggered transmission strategy - unidirectional isolation guarantee - distributed cache optimization - full protocol compatibility", this invention achieves a balance between security and efficiency in multi-terminal collaborative office work under a highly confidential network. At the same time, it breaks through the performance bottleneck of small file storage, reduces hardware costs, and has good scalability and audit traceability capabilities.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a multi-terminal collaborative office method based on a unidirectional isolated transmission device.

[0007] A multi-terminal collaborative office method based on unidirectional isolated transmission equipment includes: Desktop transfer tools / web clients / app clients upload file data to the external network. If the file data size is less than a first threshold, a specific number of files within a certain period are written to a folder, and file metadata is written to the database index of each file. If the file data size is greater than a second threshold, the file data is transmitted in segments, and a private protocol for configuring business segment information is written to the file metadata. After the external network performs security checks and audits on the file data based on pre-set business policies and it passes the audit, it is sent to the internal network. The intranet distributes the received file data to the database and Redis cache. Desktop transfer tools / web clients / app clients send requests to the intranet so that the intranet can send the corresponding file data to the desktop transfer tools / web clients / app clients.

[0008] Furthermore, the desktop transfer tool uses a proprietary TCP protocol for communication and supports SSL and GMSSSL; it supports a pure pass-through strategy, forwarding data received from the desktop transfer tool through application layer protocols and writing it into a one-way channel for transmission to the intranet. The receiving side persists one or more TCP long connections to distribute the received data; cross-network file transfer is supported between desktop transfer tools.

[0009] Furthermore, the desktop transfer tool / web client / app uploads the file data to the external network. After the external network performs security checks and audits on the file data, it is sent to the internal network. The methods include: The desktop transfer tool uploads file data to the server. The server sequentially writes the data to disk and returns an ACK to the desktop transfer tool. After the data is written to disk, the server notifies the security check module of the file information via Kafka. The security check module checks the data sequentially according to the configured check strategy. If an error occurs, the server returns the information to the file receiving module via Kafka and then to the desktop transfer tool. If the approval is initiated, and the approver approves it, the file data is notified to the file sending module. The file sending module communicates with the one-way channel program. If the module does not provide an ACK response, it will resend the file every half hour until an ACK response is provided. The module communicates directly with the physical device isolation card to send the file data to the intranet. After receiving the file data, the file receiving module on the intranet responds with an ACK and notifies the file sending module of the file information via Kafka.

[0010] Furthermore, the file sending module is used to consume Kafka messages in batches and parse the recipient field in the file information. This field is a JSON body containing person, group, and department, which represent the recipient set, group set, and department set, respectively. The corresponding personnel in the personnel table are obtained through these three fields to obtain a deduplicated set of personnel to be received. Write to Redis cache using a queue data structure, with the key being client_wait_files_receiver ID. At the same time, batch store file records in the database, distinguishing between desktop transfer tools, web applications, and mobile applications. Once the desktop transfer tool is online, it initiates a heartbeat request every 5 seconds to obtain the number of files to be distributed by the user from the server. If the number is greater than 0, it initiates a file receiving request to the server. The server retrieves the oldest file from the queue and sends it to the desktop transfer tool. The desktop transfer tool returns an ACK until the file is successfully written to disk. Each file needs to be added to the Redis cache at the start of the transfer, using a sorted set data structure. The key is client_ack_files_receiver_id, and the value is the file information.

[0011] Furthermore, the web / app client uses the HTTPS protocol and is configured with a RESTful interface. The web / app client calls the server's file upload interface to upload file data to the server. The file data is written to Kafka to notify the security check module. If the security check fails, the error reason is directly returned on the page. The web / app client can then select the application page to initiate an application for the file and submit it for approval.

[0012] Furthermore, the web / app interface is used to query the file data list and retrieve the list of file data to be received.

[0013] Furthermore, when file data is uploaded to the external network, the core business data of the file data is extracted, parsed and standardized, and stored in a database table with a flag bit designed; the file metadata includes file name, size, format, access time, storage path and parsing status.

[0014] Furthermore, the intranet is equipped with a backup caching mechanism, in which file data in the cache will not be deleted within a set time period until a preset expiration condition is reached before being cleared.

[0015] Furthermore, to enable the intranet to send the requested file data to the desktop transfer tool / web client / app client, the method includes: querying the database using the file ID carried in the request, locating the corresponding file storage path, and returning the file data to the desktop transfer tool / web client / app client via streaming, supporting multiple downloads of the same file.

[0016] A second aspect of the present invention provides a multi-terminal collaborative office system based on a unidirectional isolated transmission device.

[0017] A multi-terminal collaborative office system based on a unidirectional isolated transmission device includes: The external network module is used by desktop transfer tools / web clients / app clients to upload file data to the external network. If the file data size is less than a first threshold, a specific number of files within a certain period of time are written into a folder, and the database index of each file is configured and written into the file metadata information. If the file data size is greater than a second threshold, the file data is transmitted in segments, and the private protocol of the configuration business segment information is written into the file metadata information. After the external network performs security checks and audits on the file data based on the pre-set business policies and the data passes the audit, it is sent to the internal network. The intranet module is used to distribute received file data to the database and Redis cache. Desktop transfer tools / web clients / app clients send requests to the intranet so that the intranet can send the corresponding file data to the desktop transfer tool / web client / app client.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention significantly reduces the proportion of metadata in small files and the risk of large file transmission by writing folder merging indexes at the first threshold (small files), transmitting slices at the second threshold (large files), and encapsulating with a private protocol. This reduces disk I / O pressure and improves overall transmission efficiency.

[0019] On the external network, the Kafka message queue is used to achieve asynchronous decoupling of file writing to disk, security checks, and approval processes. Combined with a heartbeat mechanism and ACK retransmission, data integrity is ensured, while avoiding transmission interruptions caused by single points of failure.

[0020] This invention achieves absolute isolation between the external network and the internal network through a physical isolation card. Combined with a pure transparent transmission strategy and application layer protocol forwarding, it eliminates the risk of data leakage and supports cross-network file transfer to meet the needs of multi-terminal collaboration.

[0021] This invention triggers a differentiated processing strategy based on file size thresholds, automatically identifies the characteristics of small files and merges their indexes, reducing disk seek times; large file slice transmission adapts to network bandwidth, avoiding the compatibility issues of traditional packaging strategies.

[0022] The intranet uses Redis caching (queue + Sorted Set data structure) to implement file distribution and ACK management. Combined with database streaming, it supports multiple downloads of the same file, reduces storage redundancy, and improves access efficiency. Attached Figure Description

[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0024] Figure 1 This is a flowchart illustrating a multi-terminal collaborative office method based on a unidirectional isolated transmission device, as shown in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating cross-network communication of desktop transmission tools according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating web page data transmission according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating multi-terminal collaborative office work according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the packaging of a large number of small files according to an embodiment of the present invention; Figure 6 This is a structural diagram of a multi-terminal collaborative office system based on a unidirectional isolated transmission device, as shown in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0027] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0028] To facilitate understanding of the technical solutions of this invention, some technical terms involved in this invention will be introduced below.

[0029] This desktop file transfer tool is a cross-platform desktop application that supports Windows and Linux operating systems. It consists of a sender and a receiver. After logging in, users simply drag and drop files onto the sender's icon to automatically initiate the file transfer process. The receiver only needs to log in, and the file will automatically download to their local machine.

[0030] As introduced in the background technology, there is a significant conflict between the demand for multi-terminal collaborative office and the security guarantee mechanism of physical isolation. Existing solutions include: (1) First, establish an encrypted "tunnel" to connect to the intranet through VPN (Virtual Private Network) and obtain an intranet IP address before accessing internal OA system, file sharing server and other resources. This method is equivalent to logically accessing the company's local area network. However, this method has a coarse control granularity, usually based on the entire network or network segment. There may be delays in the access experience, and if the VPN gateway fails, all remote access will be interrupted. (2) Install the controlled terminal software on the office computer (host) and use the control terminal software on any device via the Internet to remotely connect to and operate the computer. All calculations are completed on the office computer, and the local device is only responsible for displaying the screen and transmitting keyboard and mouse commands. This method has high requirements for network bandwidth and stability. All data is on the company's intranet and is not easy to leak. However, the experience may be affected by network latency, and convenient file collaboration between multiple devices is usually not possible. (3) Through software-defined methods, multiple network links (such as MPLS leased lines, the Internet, 4G / 5G) are intelligently managed, and the optimal path is automatically selected to transmit data, ensuring the smoothness of critical applications (such as video conferencing, ERP systems). Although this method solves the network performance problem across regions, it usually does not directly solve the remote access problem of individual employees, but rather provides access for branch offices. (4) For traditional high-security network environments, the internal and external networks are absolutely physically disconnected. Data exchange is carried out manually, such as using CDs, USB flash drives, and other mobile storage media to copy data from external network computers and then manually import it into internal network computers, and vice versa. This is the most primitive and theoretically the safest method. However, this method is extremely inefficient: the process is cumbersome, time-consuming, and labor-intensive, and cannot support real-time or near-real-time collaboration; security depends on people, and it is very easy for viruses to enter the internal network due to operational errors, or for data leakage to occur due to the cross-use of USB flash drives; there is no audit traceability, and the copying behavior is difficult to monitor and audit throughout the process, resulting in management blind spots.

[0031] To address at least one of the aforementioned technical problems, this invention provides a multi-terminal collaborative office method and system based on a unidirectional isolated transmission device. The following describes the solution of this invention in detail through several embodiments.

[0032] Figure 1This is a flowchart illustrating a multi-terminal collaborative office method based on a unidirectional isolated transmission device, as shown in an embodiment of the present invention; see reference. Figure 1 The method includes: Desktop transfer tools / web clients / app clients upload file data to the external network. If the file data size is less than a first threshold, a specific number of files within a certain period are written to a folder, and file metadata is written to the database index of each file. If the file data size is greater than a second threshold, the file data is transmitted in segments, and a private protocol for configuring business segment information is written to the file metadata. After the external network performs security checks and audits on the file data based on pre-set business policies and it passes the audit, it is sent to the internal network. The intranet distributes the received file data to the database and Redis cache. Desktop transfer tools / web clients / app clients send requests to the intranet so that the intranet can send the corresponding file data to the desktop transfer tools / web clients / app clients.

[0033] This invention utilizes modern unidirectional isolation transmission equipment (also known as an "optical unidirectional import system" or "data diode") technology to achieve efficient and controllable multi-terminal collaborative work while ensuring the absolute security of the core network. It supports web pages, desktop transfer tools, and mobile apps, and users can log in to any terminal to synchronize files to any terminal on the other side of the network. It supports self-sending and receiving, one-to-many data transmission, group or departmental data synchronization. For massive amounts of data at the business system level, it supports synchronizing large amounts of data using silent desktop transfer tools. The desktop transfer tool supports mainstream CPU architectures and operating systems, such as ARM architecture Phytium and Kunpeng CPUs, x86 architecture Hygon, Zhaoxin, and Intel CPUs, MIPSLE Loongson CPUs, and Windows. The web interface supports multiple browsers, such as 360, Firefox, and Google Chrome. The app supports Huawei, Xiaomi, and Vivo systems.

[0034] 1. Desktop transfer tool The communication uses a proprietary TCP protocol, supporting SSL and GMSSSL, enhancing communication security, meeting compliance requirements, and improving protocol trustworthiness. The SSL uses TCNative technology, providing efficient SSL / TLS encryption and decryption capabilities, significantly reducing CPU overhead for encryption and decryption in high-concurrency scenarios, with performance approaching that of plaintext transmission. GMSSSL uses the TLSpcpv1.1 protocol and employs Chinese national cryptographic algorithms, avoiding the potential "backdoor" risks associated with relying on foreign encryption algorithms. Especially in proprietary protocols involving critical national information infrastructure, GMSSL is a crucial choice for ensuring network security and independent controllability.

[0035] Data persistence is based on NIO using FileChannel technology, which has significant advantages in handling large files and high-concurrency scenarios. It also differentiates between files of different sizes and processes them differently. For extremely large files, a custom threshold can be set; for example, files exceeding 2GB are identified as extremely large and a fragmented transmission strategy is adopted. When the files arrive at the other network domain, they are automatically assembled into a complete file according to their sequence numbers. For extremely small files, a custom size can also be set, such as 1KB files. Simultaneously persisting a large number of small files to disk places extremely high demands on disk performance (IOPS). On one hand, solid-state drives are used; on the other hand, the software uses tar compression with customizable parameters, such as a maximum of 1000 small files per second, which are sequentially written into a large file block. Preloading the file blocks improves performance and compresses the file size. Most importantly, the file's metadata is also compressed within the compression, allowing for file distribution to the other network domain without decompression.

[0036] For high-performance scenarios, a pure pass-through strategy is also supported. Data received from desktop transfer tools is written into a unidirectional channel and transmitted to the intranet via application-layer protocol forwarding. The receiving end maintains one or more persistent TCP connections to distribute the received data. Multiple TCP connections operate on a file-by-file basis, using a round-robin modulo operation to ensure data arrives at its destination securely and accurately.

[0037] Desktop file transfer tools support cross-network file sharing. For example, user A on the external network can send a file to themselves on the internal network, or to user B on the internal network. It also supports importing organizational charts or creating new groups to send files to departments or multiple users on the internal network.

[0038] like Figure 2As shown, users install the corresponding desktop transfer tool program on both internal and external network terminals. Administrators create user accounts on the web interface, and users log in to establish a TCP long connection with the server. All protocols between the desktop transfer tool and the server use TCP GMSSSL encryption to ensure data security. When a user drags a file to the desktop transfer tool, the file is transferred to the server in chunks. The server sequentially writes the files to disk and returns an ACK to the desktop transfer tool. After the files are written to disk, the file information is notified to the security check module via Kafka. The security check module checks the files sequentially according to the configured check policy. If an error occurs, it returns the information to the file receiving module via Kafka, and then to the desktop transfer tool to display the file to the user. Users can choose to initiate an approval process. If the approver approves the file, the file is notified to the file sending module. The file sending module communicates with the unidirectional channel program. If this module does not respond with an ACK, it will resend the file every half hour until an ACK is received, ensuring the reliability of file transmission. This module communicates directly with the physical device isolation card to send files to the internal network side. The file receiving module on the internal network side also needs to respond with an ACK after receiving the file, and then notify the file sending module of the file information via Kafka.

[0039] The file sending module process is as follows: First, it consumes Kafka messages in batches, parsing the recipient field in the file information. This field is a JSON body containing `person`, `group`, and `depart`, representing the recipient set, group set, and department set, respectively. Using these three fields, it retrieves the corresponding personnel from the personnel table, obtaining a deduplicated set of recipients. To save disk space, although there are multiple recipients, only one copy of the file is kept on disk. Next, it writes to a Redis cache, using a queue data structure to ensure FIFO of elements. The key uses `client_wait_files_recipient_id`. Simultaneously, file records are batch-added to the database, distinguishing between desktop transfer tools, web, and app types. When the desktop transfer tool comes online, it sends a heartbeat request every 5 seconds to obtain the number of files to be distributed for that user from the server. If the number is greater than 0, it sends a file receiving request to the server. The server retrieves the oldest file from the queue and sends it to the desktop transfer tool. The desktop transfer tool returns an ACK until the file is successfully written to disk. When files are available, the desktop transfer tool continuously sends file receiving requests until the server returns "no files," at which point the short TCP connection is closed.

[0040] For each file, at the start of transmission, it needs to be cached in Redis using a sorted set data structure. The key is `client_ack_files_receiver_id`, the value is the file information, and the score is the current timestamp plus the number of attempts. The initial request has one attempt. If the desktop transfer tool returns an ACK, this cache is deleted. If no return is received, it is added back to the `client_wait_files_receiver_id` queue after 5 minutes so the desktop transfer tool can receive it again. If the desktop transfer tool encounters an error, such as a disk full error, and has not yet returned an ACK, the next attempt count is 2, and the time interval doubles to 10 minutes, and so on, up to 20 minutes, 40 minutes, and 1 hour, with a maximum retry interval of 1 hour. This scheme enables automatic file retransmission, improving system reliability.

[0041] 2. Web pages and mobile apps Using the HTTPS protocol, terminals must install the corresponding certificate to access the system. Only users with valid certificates and valid signatures can use the service, ensuring security. The system includes rich RESTful APIs: sending and receiving emails, upload lists, download lists, applications, approvals, and group functions. Multiple files can be uploaded in batches and cached on the server. When a submission is received, a subsequent scheduling process is triggered. If approval is required, a designated approver must agree or reject the submission in the approval system before the data is transmitted to the internal network via a one-way channel. Internal network users can view their files to be downloaded and selectively download the files they need. The mobile app, similar to the web page, uses HTTPS and a certificate-based secure access strategy, running on native systems like Android using Java / Kotlin. It supports batch selection, resumeable uploads, background uploads, and version updates.

[0042] Both the web page and the mobile app support one-to-one and one-to-many cross-domain file transfers within this application. Unlike the push method used by desktop file transfer tools, it adopts a pull mode, retrieving the necessary files from the server.

[0043] like Figure 3As shown, all interactions use RESTful HTTP interfaces. Users enter their username and password to log in via the login interface. Selecting a contact, group, or department, and clicking or dragging to upload a file, calls the server-side file upload interface to upload the file from the desktop transfer tool to the server. Clicking the send button calls the server-side batch submission interface, writing these files to Kafka to notify the security check module. If the security check fails, the error message is returned directly. Users can initiate a file application on the application page. The approver logs in, calls the approval query interface to retrieve the current list of pending approvals, selects an approval form, fills in the results and comments, and completes the approval. If the approval fails, the process ends directly; if approved, the file sending module is notified. The isolation card transfer works the same as the desktop transfer tool, transferring files to the intranet.

[0044] On the intranet side, users log in via a web page or app; the login interfaces differ, but the database is the same. The homepage's file list query interface retrieves a list of files to be received, categorized into three states: downloaded, not downloaded, and expired. When a user selects a record, the download interface is invoked, and the server returns the file to the desktop transfer tool, marking the record as downloaded. Multiple records can also be selected for batch downloading, which is done by default into a single compressed file. All files can be downloaded repeatedly on the intranet side, which has a unified file expiration cleanup policy. Generally, two policies exist: disk space and time. Upon reaching any policy setting value, expired files are automatically cleaned up, their status marked as expired, and they can no longer be downloaded.

[0045] The ability of desktop transfer tools, web applications, and apps to achieve cross-domain data sharing and interoperability is key to enabling multi-terminal collaborative work. Users can send the same file anytime, anywhere via any terminal, according to their needs. After passing through a unified security policy check and approval system, the file securely and reliably enters the other party's domain. Users, groups, and departments on the other end can access the file via any terminal, greatly improving work efficiency and user experience.

[0046] The following is a detailed solution for multi-terminal collaborative work, divided into two parts: external network and internal network. Figure 4 As shown.

[0047] A. External network: Regardless of the access method used, all files accessed through these three different methods will be processed by a unified parsing engine to extract core business data (including field information, format structure, content details, etc.). This parsed data will be standardized and stored in the same database table, while specially designed flags (such as access channel codes, file type identifiers, etc.) will be used to distinguish file data from different sources, ensuring data traceability.

[0048] Once the file data is successfully written to disk (i.e., the database storage is completed), the system will automatically collect the file's metadata (including filename, size, format, access time, storage path, parsing status, etc.), encapsulate it into a message according to the preset message format, and write it to the Kafka message queue as a trigger signal for subsequent processes.

[0049] Subsequently, a unified scheduling service monitors and distributes file metadata in the message queue. Based on pre-configured business policies (such as rules on file type, sensitivity level, and source channel), the scheduling service determines whether the file needs to enter the security inspection system (for virus scanning, content compliance testing, etc.) and the approval system (for manual review or automatic approval by relevant personnel).

[0050] After all the above pre-processing steps are completed and confirmed, the file will finally be transmitted to the intranet system through a strictly controlled one-way channel (ensuring that data can only be transmitted from the outside to the inside to prevent information leakage), completing the entire process of file access and processing.

[0051] B. Intranet The intranet-side file receiving service serves as the unified entry point for files entering the intranet, distributing and storing all incoming files to corresponding directories according to rules. The system predefines independent root folders for different business types, ensuring business isolation of files from the storage source.

[0052] After a file is successfully written to the target path, the system automatically generates a message containing key information such as file ID, storage path, and business type through the Kafka message-driven mechanism and sends it to the specified topic. This triggers the file distribution module to consume information from the message queue, complete the insertion operation of file metadata into the business database, and realize the association between file entity and metadata.

[0053] To support the automatic download function of desktop transfer tools, a continuous heartbeat interaction mechanism is designed between the server and the desktop transfer tools: the server writes the file information to be distributed to each desktop transfer tool in real time to the Redis cache, and for the same file, a composite key of "recipient's unique identifier + file ID" is used for storage to ensure the uniqueness and relevance of the cached data. Each desktop transfer tool (corresponding to a specific user) periodically sends a heartbeat request. The server retrieves the number of files currently to be distributed for that user by searching the user's independent cache space. When a desktop transfer tool detects a file to be downloaded, it will proactively establish a short TCP connection with the server. The server then retrieves the earliest file from the user's distribution queue according to the file creation time and sends it to the desktop transfer tool through the connection. After the desktop transfer tool successfully writes the file to its local disk, it returns an acknowledgment (ACK) signal to the server. Upon receiving the ACK signal, the server immediately marks the file as "distributed" and removes it from the distribution queue.

[0054] In addition to the real-time distribution queue, the system also establishes an extra file backup caching mechanism. File information in this cache will remain undeleted for a set time period (e.g., 7 or 30 days) until preset expiration conditions are met (e.g., exceeding the retention period or manual cleanup). This mechanism ensures that if the user needs to retrieve the file again after the desktop transfer tool has automatically downloaded it, they can manually trigger the specified file download function to retrieve it again, improving the flexibility of file retrieval.

[0055] For file download requests on both the web and app platforms, the system employs a direct database indexing mechanism: when file metadata is entered into the database, its complete path in the storage system is recorded. When a user initiates a download request through a web page or app, the system can quickly query the database using the file ID carried in the request, locate the corresponding file storage path, and then return the file content to the user via streaming. This supports multiple downloads of the same file, meeting the file acquisition needs of multiple platforms and scenarios.

[0056] The entire file receiving, storage, distribution, and download system, through layered design (storage layer, message layer, cache layer, database layer) and multi-mechanism collaboration (hash hashing, message-driven, heartbeat interaction, cache management), not only ensures the efficiency of large-scale file storage, but also meets the diverse needs of different terminals (desktop transfer tools, Web, App), while taking into account the system's scalability and data reliability.

[0057] Users have three ways to exchange files across domains: desktop transfer tools, web pages, and mobile apps, choosing the most convenient method according to their needs. Regardless of the method, protocol encryption is used to ensure data transmission security; for example, TCP exchanges use GMSSSL, and HTTP exchanges use HTTPS with two-way certificate authentication. The first step on the external network side is file storage, handled uniformly by the data access service. Administrators can configure file security policies: whether security checks are required, the strategies for security checks (such as filename keywords, file content keywords, viruses, file format blacklists / whitelists, password labels, etc.), and whether approval is required. Multiple customized policies can be set and bound to users for flexible security checks. All check or approval results are asynchronously returned to the terminal via a message queue.

[0058] After security checks are passed, files communicate via a one-way channel through the sending module. The internal network file receiving module receives files from the one-way channel, completing the file transfer from the outside to the inside. This process is done on a file-by-file basis, using a one-file-one-ack mechanism. If no ack is received between any two nodes, the file will be automatically retransmitted. After the file receiving module finishes writing the file to disk, it notifies the data distribution module of the file information. The module then handles the data differently depending on the business type: for desktop transfer tools, it parses the recipient's ID, queries the database to find all recipients, writes the same file to multiple Redis queues, and records the file in the database; for web and app businesses, it directly batches the data into the database. Once a desktop transfer tool user is online, they can obtain the number of files they need to receive via a heartbeat protocol and actively initiate a file receiving request. The server then transmits the files to the desktop transfer tool sequentially from the queue according to time order. Once all files have been received, the TCP short connection is closed. Web and app users can call the interface to query all files and select one or more files to download.

[0059] The process of transferring files between these three methods is the same; only the form and interaction method at both ends differ. Throughout the process, the file remains a single copy on the disk. Files transferred via web can be received by desktop transfer tools and apps, and similarly, files transferred via desktop transfer tools can be received by web and apps, and vice versa. Furthermore, it's not limited to self-sending and self-receiving; files can be sent to others, groups, or departments. This allows users to achieve convenient and efficient office work in cross-domain scenarios, while simultaneously ensuring legal compliance and secure and reliable data transmission.

[0060] This embodiment supports a large number of small file packaging technology, automatically identifies the characteristics of small files and packages them under a specified strategy, solving the disk performance bottleneck problem and reducing hardware costs.

[0061] The main reason for the poor performance of small file disk transfer mode is the frequent time consumption of creating, opening, and closing files, as well as random disk writes. The core of optimizing small files is to merge a large number of small files into one large file, reducing the number of times files are opened, closed, and created. Based on the characteristics of the business, this can be done using EventLoops, with each EventLoop managing a large file, writing all the small files received by that EventLoop from different desktop transfer tools into that large file.

[0062] like Figure 5 As shown, the desktop transfer tool has three small files on its hard drive that need to be sent. First, it logs in and opens a management channel, which is responsible for business functions such as policy distribution and heartbeat. Then, it establishes a TCP channel with the server based on the number of files. This connection is a short connection, which can send different files continuously within 5 seconds. If there are no files to send, it will close the short connection.

[0063] The filegroup contains a 512-byte tar header, file metadata (1536 bytes reserved), and the actual file data. Server-side configuration includes: small file criteria (files smaller than a set threshold are considered small), maximum number of files per filegroup, and maximum write time for each filegroup. The server's default thread count is twice the CPU's. At service startup, filegroups are bound to threads, and 2N filegroup blocks are pre-written to disk. The size of each block is equal to 2N × (file header + file metadata + configured small file criteria).

[0064] When establishing a file connection with a desktop file transfer tool, multiple file channels are bound to an event loop, and each event loop corresponds one-to-one with a file group. This means each file group can be considered to use the same thread for sequential disk writing. There are no concerns about thread safety or out-of-order file writes, and performance is guaranteed due to the reduced thread race conditions. Even with a low-performance hard drive like a regular mechanical hard drive, it can handle high-speed writing of a large number of small files.

[0065] For filegroup strategies, such as 1KB, 1000, 2 seconds, it means that files less than or equal to 1KB will be written to the smaller filegroup. If 1000 files are written in less than 2 seconds, they will be counted as 1000 files, and the 1001st file will be written to the next filegroup. If only 500 files are written in 2 seconds, this filegroup will be counted as 500 files, and the 501st file will be written to the next filegroup.

[0066] Each filegroup has its own attributes: the current number of files, and the filegroup initialization time. When writing files in the event loop, each small filegroup checks if the conditions for generating the next filegroup have been met. If so, the cache is destroyed and a new filegroup is generated, updating the cache with the new filegroup. An asynchronous thread is then invoked to notify subsequent modules via Kafka to continue transmitting the original filegroup. Since it's impossible to know when the last file has finished sending, a new triggering process is needed. This can be achieved by creating a new scheduled task that internally iterates through all event loops, retrieves the bound small filegroups from the current thread, and notifies subsequent modules using the same logic as the normal process.

[0067] To improve performance, the receiving side does not need to parse the filegroup into individual small files and write them to disk. The filegroup contains an index for each file, and this information is also written into the file metadata on the sending side. When the file receiving module receives the file, it parses out the information of each small file and writes it to the Redis cache. When the desktop transfer tool tries to retrieve the file, it obtains the file index through the metadata, starts reading from the starting index in the tar package, and continues reading until it reaches the length of the file, thus directly retrieving the small file and sending it to the desktop transfer tool.

[0068] For low-end servers, the CPU may not be able to handle high-speed business, especially high-frequency database operations. This invention uses asynchronous batch processing for all database insertion operations and sets a CPU threshold. For example, if the CPU usage exceeds 60%, it is considered to be under significant performance pressure, and only file transfer operations are performed without database logging to avoid increasing CPU load. Batch insertion only occurs when the real-time percentage falls below this value.

[0069] This embodiment supports chunked transmission technology for ultra-large files, automatically identifies the characteristics of large files (such as files exceeding 2GB), transmits large files in chunks, and automatically reassembles them on the receiving side, greatly improving the performance of large file transmission, accurately identifying lost chunks, and improving the efficiency of file retransmission. The process is as follows: The large file service segment in the private protocol is defined as follows: large file transfer identifier, large file directory level plus file name, file unique identifier, total file size, current large file data packet number, and total number of large file data packets.

[0070] The desktop file transfer tool splits large files exceeding a specified size into multiple smaller files and subtasks. Simultaneously, it adds records to the database and adds them to the queue of large files to be sent. After each smaller file is sent, the corresponding record is deleted until the count reaches 0, indicating the large file sending is complete.

[0071] The system receives data from a desktop file transfer tool. The first packet uses the file's unique identifier as the key, combines file information into key-value pairs, and initially creates the file, ending with ".tmp" to indicate a temporary disk write status. Packet numbers are generally incremental, but after multiple transmission processes, sequence number continuity cannot be guaranteed. Subsequent packets directly retrieve key-value pairs; if the received packet number matches the expected packet number, it is written to disk; otherwise, it is added to the cache. Upon receiving the last packet, the system calculates whether large files were lost. If no packet loss occurred, the file was successfully written to disk, and the ".tmp" suffix was removed. If packet loss occurred, the packet loss details were written to the database. The system automatically writes the loss details to a .txt file and reports it to the server via API.

[0072] In addition, this embodiment also supports file loss retransmission. If a large file is partially lost, it is not necessary to retransmit the entire file; only the lost parts need to be retransmitted. As mentioned above, if packet loss occurs, the server will receive a TXT record. The administrator downloads this record via the webpage and uses compliant storage tools to export it to the external network, then uploads the TXT file via the external network page.

[0073] The desktop file transfer tool listens for heartbeat parameters to determine if there are any files to be retransmitted. If so, the .txt file is received via a large file retransmission protocol. The file is parsed, and each lost packet record is added to a send task. Consecutive fragments can be sent in the same task. The sending thread calculates the absolute path of the large file based on the records, obtains the breakpoint position, and retransmits the number of lost file fragments.

[0074] The receiving desktop transfer tool calculates the offset position based on the current packet number and writes the file data packets to disk in sequence. The process is the same as the normal logic for writing large files to disk. If packet loss occurs again, the above process is repeated until the entire file is successfully written to disk.

[0075] This embodiment supports a high-performance transparent transmission mode. For file-intensive business needs, transparent transmission mode can be enabled without security checks or approvals, improving transmission performance. The process is extremely simplified. When a data stream is received from the desktop transfer tool, multiple TCP channels can be established based on the number of files. Simultaneously, the server and the one-way transmission program establish multiple long-lived TCP connections upon startup, forming a many-to-many relationship. A round-robin algorithm is used, with file IDs as the unit, ensuring that the same file is forwarded on the same TCP channel, and the next file enters a different channel. The one-way transmission program interacts directly with the one-way transmission card, sending data to the one-way receiving program on the internal network. To ensure receiving-side performance, the desktop transfer tool can selectively establish multiple long-lived TCP connections with the server, again using a round-robin algorithm based on file IDs to distribute different files into different channels. The receiving desktop transfer tool's disk can become a performance bottleneck; when performance is insufficient, it needs to be replaced with an SSD. The server controls the cache queue, selectively discarding data when too much accumulates, thus ensuring normal service operation.

[0076] This embodiment supports flow control, dynamically adjusting the maximum flow threshold based on disk usage. Limits can be applied to individual TCP connections or the entire server. A scheduled task scans disk usage every 30 seconds. If the current disk usage exceeds 60%, it indicates that current disk writes are too fast, subsequent modules are processing slowly, and there is some backlog. The speed is automatically reduced to half the default value. Once the usage decreases, the speed limit is reset in the scheduled task, automatically restoring the original speed. If the business pressure is too high, reaching 80%, no requests are accepted. After processing is complete, the speed automatically returns to half the default value. If disk usage decreases again, it returns to full speed. This dynamic approach more rationally handles business needs, ensuring efficient processing without causing service crashes due to excessively high speeds leading to disk overload.

[0077] The file storage rules utilize a file ID-based hashing mechanism to generate evenly distributed storage paths. Specifically, the system employs a file ID-based hashing mechanism: first, a hash value is calculated using the file's unique identifier (FileID); then, the last four digits of this hash value are used for a modulo-16 operation, resulting in a two-level nested folder index (each level containing 256 folders, forming a 256×256 two-level structure). This design evenly distributes a large number of files across 65,536 folders, effectively avoiding file system performance bottlenecks (such as slow retrieval and operation delays) caused by an excessive number of files in a single folder, while ensuring the orderliness and traceability of file storage.

[0078] The above combination Figure 1 The multi-terminal collaborative office method based on a unidirectional isolated transmission device provided in the embodiments of the present invention has been described in detail. Next, the multi-terminal collaborative office system based on a unidirectional isolated transmission device provided in the embodiments of the present invention will be described in conjunction with the accompanying drawings.

[0079] Figure 6 This is a schematic diagram of the structure of a multi-terminal collaborative office system based on a unidirectional isolated transmission device, as shown in an embodiment of the present invention. Figure 6 The system described in this invention includes: The external network module is used by desktop transfer tools / web clients / app clients to upload file data to the external network. If the file data size is less than a first threshold, a specific number of files within a certain period of time are written into a folder, and the database index of each file is configured and written into the file metadata information. If the file data size is greater than a second threshold, the file data is transmitted in segments, and the private protocol of the configuration business segment information is written into the file metadata information. After the external network performs security checks and audits on the file data based on the pre-set business policies and the data passes the audit, it is sent to the internal network. The intranet module is used to distribute received file data to the database and Redis cache. Desktop transfer tools / web clients / app clients send requests to the intranet so that the intranet can send the corresponding file data to the desktop transfer tool / web client / app client.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-terminal collaborative office method based on a unidirectional isolated transmission device, characterized in that, include: Desktop transfer tools / web clients / app clients upload file data to the external network. If the file data size is less than a first threshold, a specific number of files within a certain period are written to a folder, and file metadata is written to the database index of each file. If the file data size is greater than a second threshold, the file data is transmitted in segments, and a private protocol for configuring business segment information is written to the file metadata. After the external network performs security checks and audits on the file data based on pre-set business policies and it passes the audit, it is sent to the internal network. The intranet distributes the received file data to the database and Redis cache. Desktop transfer tools / web clients / app clients send requests to the intranet so that the intranet can send the corresponding file data to the desktop transfer tools / web clients / app clients.

2. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, The desktop transmission tool uses a proprietary TCP protocol for communication and supports SSL and GMSSSL. It supports a pure transparent transmission strategy, which forwards data received from the desktop transmission tool through an application layer protocol and writes it into a one-way channel for transmission to the intranet. The other side persists one or more long TCP connections to distribute the received data. Desktop file transfer tools support cross-network file transfer.

3. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, Desktop transfer tools / web-based / app-based applications upload file data to the external network. The external network performs security checks and audits on the file data before sending it to the internal network. Methods include: The desktop transfer tool uploads file data to the server. The server sequentially writes the data to disk and returns an ACK to the desktop transfer tool. After the data is written to disk, the server notifies the security check module of the file information via Kafka. The security check module checks the data sequentially according to the configured check strategy. If an error occurs, the server returns the information to the file receiving module via Kafka and then to the desktop transfer tool. If the approval is initiated, and the approver approves it, the file data is notified to the file sending module. The file sending module communicates with the one-way channel program. If the module does not provide an ACK response, it will resend the file every half hour until an ACK response is provided. The module communicates directly with the physical device isolation card to send the file data to the intranet. After receiving the file data, the file receiving module on the intranet responds with an ACK and notifies the file sending module of the file information via Kafka.

4. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 3, characterized in that, The file sending module is used to consume Kafka messages in batches and parse the recipient field in the file information. This field is a JSON body containing person, group, and department, which represent the recipient set, group set, and department set, respectively. The corresponding personnel in the personnel table are obtained through these three fields to obtain a deduplicated set of personnel to be received. Write to Redis cache using a queue data structure, with the key being client_wait_files_receiver ID. At the same time, batch store file records in the database, distinguishing between desktop transfer tools, web applications, and mobile applications. Once the desktop transfer tool is online, it initiates a heartbeat request every 5 seconds to obtain the number of files to be distributed by the user from the server. If the number is greater than 0, it initiates a file receiving request to the server. The server retrieves the oldest file from the queue and sends it to the desktop transfer tool. The desktop transfer tool returns an ACK until the file is successfully written to disk. Each file needs to be added to the Redis cache at the start of the transfer, using a sorted set data structure. The key is client_ack_files_receiver_id, and the value is the file information.

5. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, The web / app client uses the HTTPS protocol and is configured with a RESTful interface. The web / app client calls the server's file upload interface to upload file data to the server. The file data is written to Kafka to notify the security check module. If the security check fails, the error reason is returned directly to the page. On the web or app side, select the application page to initiate the application for this file and submit it for approval.

6. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, The web / app interface is used to query the file data list and retrieve the list of files to be received.

7. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, When file data is uploaded to the external network, the core business data of the file data is extracted, parsed and standardized, and stored in a database table with a flag bit. The file metadata includes file name, size, format, access time, storage path and parsing status.

8. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, The intranet is equipped with a backup caching mechanism. The file data in the cache will not be deleted within a set time period until the preset expiration conditions are met before it is cleared.

9. The multi-terminal collaborative office method based on a unidirectional isolated transmission device according to claim 1, characterized in that, To enable the intranet to send the requested file data to the desktop transfer tool / web client / app client, the method includes: querying the database using the file ID carried in the request, locating the corresponding file storage path, and returning the file data to the desktop transfer tool / web client / app client via streaming, supporting multiple downloads of the same file.

10. A multi-terminal collaborative office system based on a unidirectional isolated transmission device, characterized in that, include: The external network module is used by desktop transfer tools / web clients / app clients to upload file data to the external network. If the file data size is less than a first threshold, a specific number of files within a certain period of time are written into a folder, and the database index of each file is configured and written into the file metadata information. If the file data size is greater than a second threshold, the file data is transmitted in segments, and the private protocol of the configuration business segment information is written into the file metadata information. After the external network performs security checks and audits on the file data based on the pre-set business policies and the data passes the audit, it is sent to the internal network. The intranet module is used to distribute received file data to the database and Redis cache. Desktop transfer tools / web clients / app clients send requests to the intranet so that the intranet can send the corresponding file data to the desktop transfer tool / web client / app client.

Images