Isolation environment-oriented large file and heterogeneous data one-way processing system

Abstract

The invention provides an isolation environment-oriented large file and heterogeneous data one-way processing system, which relates to the technical field of data processing, and comprises an input adaptation module and a security isolation module which are connected in series, the input adaptation module is configured with a dynamic fragmentation unit, primitives, characters and topologies are associated through multi-modal matching, and the security isolation module is configured with a dynamic fragmentation unit; the security isolation module adopts optical shutter physical isolation and comprises a one-way protocol stack and a dual-check unit, and the one-way protocol stack only receives fragmented data output by the input adaptation module; through a multi-module linkage dynamic adjustment mechanism, the limitation of the prior art in an isolation environment is solved in a targeted manner: dynamic fragmentation and dual verification linkage balance transmission efficiency and security, and the transmission success rate is significantly improved compared with a fixed fragmentation mode; the parameter cooperation of preprocessing and heterogeneous data processing avoids feature distortion, and the data association precision is improved; dynamic scheduling based on load and storage state optimizes resource allocation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a one-way processing system for large files and heterogeneous data in isolated environments. Background Technology

[0002] To ensure data security, physical isolation or strong logical isolation environment architecture is usually adopted. In such environments, data must strictly follow the principle of unidirectional flow (such as transmission from low security domain to high security domain). In addition, the number of large files (such as high-definition surveillance videos and massive logs) and heterogeneous data (text, images, structured tables, etc.) that need to be processed is increasing, which puts forward higher requirements for data transmission efficiency, processing accuracy and security. Existing technologies for data processing in isolated environments mainly fall into the following categories: Fixed-fragmentation transmission technology: Files are split into segments of preset size and then transmitted through a one-way channel. Its core purpose is to simplify the complexity of large file transfers, but the segment size is statically set and does not consider factors such as bandwidth fluctuations and channel stability differences in isolated environments. Independent security verification and data preprocessing technology: Security isolation devices perform signature verification independently, while the data preprocessing module independently performs noise reduction and filtering operations. Both focus on security protection and data standardization respectively, but no linkage mechanism is established. Static priority scheduling for heterogeneous data processing: This technique allocates computing resources with fixed priorities for large files and multimodal data, pre-setting the processing order according to file type. Its design goal is to simplify scheduling logic, but it does not consider the dynamic changes in limited resources within an isolated environment. Fixed-rule unidirectional transmission and storage management technology: Transmission rules and storage quotas are pre-configured, such as a fixed upper limit for single transmission volume and allocation of fixed storage partitions according to type. Its core is to ensure the stability of transmission and storage, but it does not consider the impact of dynamic changes in storage capacity on transmission in an isolated environment; Single-modal or fixed-weight multimodal integration techniques: These techniques extract independent features from text, images, and other data and store them directly, or use fixed-weight models for association. While designed to integrate basic data, they fail to optimize the association logic for the characteristics of isolated environments, such as data loss and quality fluctuations. Under the special constraints of isolated environments, the aforementioned existing technologies have certain limitations in adaptability: fixed fragmentation transmission struggles to balance the contradiction between "excessively large fragments leading to high verification failure rates" and "excessively small fragments leading to low transmission efficiency"; independent security verification and preprocessing may result in feature distortion due to parameter mismatch; static priority scheduling is prone to causing delays in core task processing when resources are scarce; transmission and storage management based on fixed rules may lead to data loss due to storage overflow; and multimodal integration with single or fixed weights suffers from decreased correlation accuracy when data is missing. These limitations make it difficult for existing technologies to fully meet the needs of efficient, secure, and accurate processing of large files and heterogeneous data in isolated environments. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a one-way processing system for large files and heterogeneous data in isolated environments.

[0004] A one-way processing system for large files and heterogeneous data in isolated environments includes an input adaptation module and a security isolation module, which are connected in series. The input adaptation module is configured with a dynamic fragmentation unit, which calculates the fragment size S using the following formula:

[0005] Where k is the file type coefficient, D is the original file size in MB, B is the unidirectional channel bandwidth in the isolation environment in MB / s, and α is the redundancy check coefficient. The unit is MB; The security isolation module uses optical gate physical isolation and includes a one-way protocol stack and a dual verification unit. The one-way protocol stack only receives fragmented data output by the input adaptation module. The dual verification unit performs feature code matching and behavior entropy value detection on the fragmented data. Fragments that fail the dual verification unit verification are regenerated by the input adaptation module and transmitted to the security isolation module. It also includes a preprocessing module, which is connected to the heterogeneous data processing module. The preprocessing module performs reassembly and standardization on the fragmented data output by the security isolation module. The standardization includes improved median filtering for images, filtering of non-ASCII characters in text, and improved Z-Score outlier removal for structured data. The data processed by the preprocessing module is then transmitted to the heterogeneous data processing module. The heterogeneous data processing module includes a large file processing unit, a multimodal feature extraction unit, and a priority scheduling unit. The priority scheduling unit is shared by the large file processing unit and the multimodal feature extraction unit. The priority scheduling unit calculates the file priority N and allocates computing resources to the large file processing unit and the multimodal feature extraction unit according to the value of N. The one-way transmission control module and the storage module are connected to form a write closed loop. The one-way transmission control module includes a transmission rule base and a blockchain log unit. The transmission rule base restricts data to be transmitted only from the heterogeneous data processing module to the storage module. The blockchain log unit records the data transmission trajectory. The storage module adopts a read-only partition architecture, only receives write requests from the one-way transmission control module, stores data in partitions according to data type, and enables a type-adaptive compression algorithm. The multimodal integration module is integrated into the storage module. Based on the standardization results of the preprocessing module and the feature extraction results of the heterogeneous data processing module, the multimodal integration module maps the features of text, images, and structured data to a unified space by improving the autoencoder, and establishes a cross-modal index. The output module is connected to the storage module via a read-only interface. The output module performs format, syntax, and semantic checks on the integrated data before outputting it.

[0006] Preferably, the dynamic fragmentation unit includes a fragmentation calculation subunit and a retransmission probability prediction subunit. The fragmentation calculation subunit and the retransmission probability prediction subunit are linked, and the retransmission probability prediction subunit calculates the fragment loss probability P using a formula:

[0007] in This is a retransmission correction factor, dynamically adjusted based on historical retransmission behavior, reflecting the degree of influence of the network environment on retransmissions. This refers to the fragmentation transmission time parameter; When P > 5%, the retransmission probability prediction subunit triggers the fragment calculation subunit to reduce the fragment size by a ratio of P × 0.2. At the same time, the input adaptation module sends a "high-risk fragment" flag to the security isolation module. When the dual verification unit of the security isolation module performs behavior entropy value detection on the fragment containing the flag, it adjusts the detection threshold H0 to 3.0. When the calculated behavioral entropy value exceeds the adjusted detection threshold, the dual verification unit determines that the segment has an abnormal risk, thereby triggering subsequent security policies. The dynamic fragmentation unit generates a C32 checksum and file fingerprint hash for the deep compression ratio of each fragment. After the dual verification unit of the security isolation module verifies the C32 checksum and file fingerprint hash of the deep compression ratio, it sends a "verification passed" signal back to the input adaptation module. The signal is transmitted in one direction only and there is no data return.

[0008] Preferably, the security isolation module includes a behavior entropy detection unit and an offline feature library update subunit. The behavior entropy detection unit is linked with the offline feature library update subunit, and the behavior entropy detection unit calculates the file behavior entropy H using the following formula:

[0009] Where n is the total number of events involved in the behavioral entropy calculation, and the number of different types of feature items in the file behavior. Let i be the probability of the i-th feature term occurring. When H > H0 (H0 = 3.5) is detected three times consecutively and no matching feature library is found, the offline feature library update subunit extracts a 512-byte feature fragment to supplement the local feature library. At the same time, the hash value of the feature fragment is sent to the preprocessing module. The preprocessing module automatically performs depth standardization on subsequent files containing the same hash value. The depth standardization includes multi-round image denoising and text granular refinement.

[0010] Preferably, the preprocessing module includes an improved median filtering unit, and the heterogeneous data processing module includes an improved SIFT feature extraction unit. The improved median filtering unit is linked with the improved SIFT feature extraction unit, and the improved median filtering unit calculates the window size using the following formula. :

[0011] Where ρ is the image noise density; the image filtered by the improved median filtering unit is transmitted to the improved SIFT feature extraction unit, and the scale space σ range of the improved SIFT feature extraction unit is based on... Dynamic adjustment (σ= / 5), the feature vector dimension is simplified by the following formula, where the feature vector is the SIFT feature vector extracted by the improved SIFT feature extraction unit:

[0012] in The simplified SIFT feature vector dimension. This is the baseline scale parameter.

[0013] Preferably, the transmission rule base of the one-way transmission control module is linked to the quota dynamic adjustment mechanism of the storage module. When the utilization rate of a certain partition of the storage module is ≥80%, the storage module sends a "quota insufficient" signal to the one-way transmission control module. The transmission rule base prioritizes the transmission of this type of data using the following formula. Reduced by 30%

[0014] in, Prioritize data transmission. To indicate the level of urgency, This represents the current used capacity of the storage module. The total capacity of the storage module, These are the characteristic values ​​corresponding to the data type.

[0015] Preferably, the improved autoencoder of the multimodal integration module is linked with the normalized unit of the preprocessing module, and the loss function L of the improved autoencoder is calculated by the following formula:

[0016] in , These are the feature vectors of the preprocessed text and image, respectively. The weighting coefficients are the terms representing the absolute difference between the text feature vector and the image feature vector. The weighting coefficients are the complement terms for the similarity between text and image features.

[0017] Preferably, it also includes a monitoring module, which forms a feedback closed loop with the unidirectional transmission control module and the heterogeneous data processing module. The monitoring module calculates the load index G using the formula: G = (CPU utilization / 80%) × 0.4 + (Memory usage / 90%) × 0.3 + (Data backlog / Maximum queue length) × 0.3; When G > 1.2, the monitoring module reduces the transmission rate to the unidirectional transmission control module to the set value, and at the same time sends a resource reallocation signal to the heterogeneous data processing module. The heterogeneous data processing module increases the proportion of computing resources allocated to the large file processing unit to the set value.

[0018] Preferably, the output module includes a semantic verification unit, which is linked to the association index of the multimodal integration module. The semantic verification unit is based on the power domain ontology library and calculates the confidence level C using the formula:

[0019] Where C is the confidence level output by the semantic verification unit. It is a matching degree function used to calculate the features to be verified. Domain rules The degree of matching; when C < 0.95, the output module calls the association index of the multimodal integration module to extract high-confidence data with the same topic and association strength ≥ the judgment threshold for auxiliary verification. If the auxiliary verification passes, it is judged as qualified.

[0020] Preferably, the file fingerprint hash is generated by the dynamic sharding unit based on the shard data, the shard sequence number, and the file's unique identifier using SHA-256. After the dual verification unit of the security isolation module verifies the hash, the blockchain log unit records the hash value, which serves as the index key value for partitioned storage in the storage module.

[0021] Preferably, the type-adaptive compression algorithm of the storage module is linked with the standardization result of the preprocessing module: data determined by the preprocessing module to be "text-intensive" is subjected to LZMA deep compression, specifically with a compression ratio of ≥3:1; data determined to be "image-intensive" is subjected to WebP quantization compression, specifically with a compression parameter QP=28.

[0022] This invention provides a one-way processing system for large files and heterogeneous data in isolated environments, which has the following advantages: Through a dynamic adjustment mechanism involving multiple modules, the limitations of existing technologies in isolated environments are specifically addressed: dynamic fragmentation and dual verification work together to balance transmission efficiency and security, significantly improving transmission success rate compared to fixed fragmentation mode; parameter coordination between preprocessing and heterogeneous data processing avoids feature distortion and improves data association accuracy; dynamic scheduling based on load and storage status optimizes resource allocation, solving resource waste or overflow problems under static management; and adaptive weight adjustment through multimodal integration enhances the robustness of association in data missing scenarios, fully adapting to the special constraints of bandwidth fluctuations, limited resources, and uneven data quality in isolated environments, achieving efficient, secure, and accurate unidirectional processing of large files and heterogeneous data. Attached Figure Description

[0023] Figure 1 This is a system flowchart of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, this invention proposes a one-way processing system for large files and heterogeneous data in isolated environments, including an input adaptation module and a security isolation module, wherein the input adaptation module and the security isolation module are connected in series. The input adaptation module is configured with a dynamic fragmentation unit, which calculates the fragment size S using the following formula:

[0026] Where k is the file type coefficient, D is the original file size in MB, B is the unidirectional channel bandwidth in the isolation environment in MB / s, and α is the redundancy check coefficient. The unit is MB; It should be noted that the value of the file type coefficient k is usually derived from the technical specifications documents within the enterprise or industry. The value of k is specified for different file types: for text files, due to their regular data structure and low transmission and processing overhead, k is 1.0; for image files (such as .jpg and .png), which need to consider the integrity of image blocks and have slightly higher processing overhead, k is 1 or 5; for video files (such as .mp4 and .avi), which have inter-frame dependencies and whose segments need to be aligned with the encoding blocks, the processing logic is more complex, so k is 2.

[0027] The redundancy check coefficient is usually derived from specific algorithm design documents and needs to be determined by combining redundancy algorithm theory and fault simulation verification. To address the problem of fragment loss or damage in one-way transmission, Reed-Solomon erasure coding is used. If the requirement is "the original data can still be fully recovered when no more than 25% of fragments are lost", the theoretical formula of the erasure coding is used for calculation, and multiple sets of experiments are conducted to verify the "simulated fragment loss of 25%" (all 100 sets of experiments can fully recover the original data through redundant fragments). Finally, the value is determined to be 1.25 (that is, the total fragment data volume is 1.25 times the original data volume, including the original data and redundancy check data). The security isolation module uses optical gate physical isolation and includes a one-way protocol stack and a dual verification unit. The one-way protocol stack only receives fragmented data output by the input adaptation module. The dual verification unit performs feature code matching and behavior entropy value detection on the fragmented data. Fragments that fail the dual verification unit verification are regenerated by the input adaptation module and transmitted to the security isolation module. It also includes a preprocessing module, which is connected to the heterogeneous data processing module. The preprocessing module performs reassembly and standardization on the fragmented data output by the security isolation module. The standardization includes improved median filtering for images, filtering of non-ASCII characters in text, and improved Z-Score outlier removal for structured data. The data processed by the preprocessing module is then transmitted to the heterogeneous data processing module. The heterogeneous data processing module includes a large file processing unit, a multimodal feature extraction unit, and a priority scheduling unit. The priority scheduling unit is shared by the large file processing unit and the multimodal feature extraction unit. The priority scheduling unit calculates the file priority N and allocates computing resources to the large file processing unit and the multimodal feature extraction unit according to the value of N. It should be noted that: N = I_urgent × 0.5 + (1 - S / S_max) × 0.3 + T_type × 0.2, where I_urgent is the urgency coefficient (0-1), S is the file size, S_max is the maximum file capacity that the system can process, and T_type is the data type weight (0-1, with critical types taking 1). The one-way transmission control module and the storage module are connected to form a write closed loop. The one-way transmission control module includes a transmission rule base and a blockchain log unit. The transmission rule base restricts data to be transmitted only from the heterogeneous data processing module to the storage module. The blockchain log unit records the data transmission trajectory. The storage module adopts a read-only partition architecture, only receives write requests from the one-way transmission control module, stores data in partitions according to data type, and enables a type-adaptive compression algorithm. The multimodal integration module is integrated into the storage module. Based on the standardization results of the preprocessing module and the feature extraction results of the heterogeneous data processing module, the multimodal integration module maps the features of text, images, and structured data to a unified space by improving the autoencoder, and establishes a cross-modal index. The output module is connected to the storage module via a read-only interface. The output module performs format, syntax, and semantic checks on the integrated data before outputting it.

[0028] It should be noted that by achieving integrated processing such as dynamic fragmentation, dual verification, standardization and feature extraction through multi-module collaboration, the limitations of existing technologies such as isolated modules, fixed fragmentation and single verification are overcome. The design aims to solve the problems of imbalance between transmission efficiency and security and poor data processing correlation in the one-way processing of large files and heterogeneous data in isolated environments, and adapt to the special constraints of isolated environments.

[0029] As an optional embodiment, the dynamic fragmentation unit includes a fragmentation calculation subunit and a retransmission probability prediction subunit. The fragmentation calculation subunit and the retransmission probability prediction subunit are linked, and the retransmission probability prediction subunit calculates the fragment loss probability P using a formula:

[0030] in This is a retransmission correction factor, dynamically adjusted based on historical retransmission behavior, reflecting the degree of influence of the network environment on retransmissions. This refers to the fragmentation transmission time parameter; When P > 5%, the retransmission probability prediction subunit triggers the fragment calculation subunit to reduce the fragment size by a ratio of P × 0.2. At the same time, the input adaptation module sends a "high-risk fragment" flag to the security isolation module. When the dual verification unit of the security isolation module performs behavior entropy value detection on the fragment containing the flag, it adjusts the detection threshold H0 to 3.0. When the calculated behavioral entropy value exceeds the adjusted detection threshold, the dual verification unit determines that the segment has an abnormal risk, thereby triggering subsequent security policies. The dynamic fragmentation unit generates a C32 checksum and file fingerprint hash for the deep compression ratio of each fragment. After the dual verification unit of the security isolation module verifies the C32 checksum and file fingerprint hash of the deep compression ratio, it sends a "verification passed" signal back to the input adaptation module. The signal is transmitted in one direction only and there is no data return.

[0031] It should be noted that linking fragment size adjustment with retransmission probability prediction and dynamic adjustment of verification threshold overcomes the shortcomings of existing technologies where fixed fragment size and fixed verification threshold cannot adapt to network fluctuations. The design aims to reduce the risk of fragment loss and enhance the strictness of verification, thereby improving the reliability of data transmission when the isolated network is unstable.

[0032] As an optional embodiment, the security isolation module includes a behavior entropy detection unit and an offline feature library update subunit. The behavior entropy detection unit is linked with the offline feature library update subunit, and the behavior entropy detection unit calculates the file behavior entropy H using the following formula:

[0033] Where n is the total number of events involved in the behavioral entropy calculation, and the number of different types of feature items in the file behavior. Let i be the probability of the i-th feature term occurring. When H > H0 (H0 = 3.5) is detected three times consecutively and no matching feature library is found, the offline feature library update subunit extracts a 512-byte feature fragment to supplement the local feature library. At the same time, the hash value of the feature fragment is sent to the preprocessing module. The preprocessing module automatically performs depth standardization on subsequent files containing the same hash value. The depth standardization includes multi-round image denoising and text granular refinement.

[0034] It should be noted that the deep cleaning and preprocessing of threat detection that achieves secure isolation solves the problems of isolated feature database updates and lagging handling of unknown threats in existing technologies. The design aims to improve the efficiency of handling unknown threats through "one-time detection, full-domain protection" when the threat database cannot be updated in real time in the isolated environment.

[0035] As an optional embodiment, the preprocessing module includes an improved median filtering unit, and the heterogeneous data processing module includes an improved SIFT feature extraction unit. The improved median filtering unit is linked with the improved SIFT feature extraction unit, and the improved median filtering unit calculates the window size using the following formula. :

[0036] Where ρ is the image noise density; the image filtered by the improved median filtering unit is transmitted to the improved SIFT feature extraction unit, and the scale space σ range of the improved SIFT feature extraction unit is based on... Dynamic adjustment (σ= / 5), the feature vector dimension is simplified by the following formula, where the feature vector is the SIFT feature vector extracted by the improved SIFT feature extraction unit:

[0037] in The simplified SIFT feature vector dimension. This is the baseline scale parameter.

[0038] As an optional embodiment, the transmission rule base of the one-way transmission control module is linked with the quota dynamic adjustment mechanism of the storage module. When the utilization rate of a certain partition of the storage module is ≥80%, the storage module sends a "quota insufficient" signal to the one-way transmission control module. The transmission rule base prioritizes the transmission of this type of data using the following formula. Reduced by 30%

[0039] in, Prioritize data transmission. To indicate the level of urgency, This represents the current used capacity of the storage module. The total capacity of the storage module, These are the characteristic values ​​corresponding to the data type.

[0040] It should be noted that dynamically linking transmission control with storage quotas overcomes the problem that static quota management in existing technologies can easily lead to storage overflow or resource waste. The design aims to avoid data loss and achieve "priority storage of urgent data" in isolated environments with limited resources, thereby improving storage resource utilization.

[0041] As an optional embodiment, the improved autoencoder of the multimodal integration module is linked with the normalized unit of the preprocessing module, and the loss function L of the improved autoencoder is calculated by the following formula:

[0042] in , These are the feature vectors of the preprocessed text and image, respectively. The weighting coefficients are the terms representing the absolute difference between the text feature vector and the image feature vector. These are the weighting coefficients of the complement term for the similarity between text and image features. =0.3, =0.7; When the preprocessing module detects a data missing rate >10%, the multimodal integration module will... The weight has been adjusted to 0.9.

[0043] It should be noted that dynamically adjusting the weights of the autoencoder loss function based on the data missing rate solves the problem of poor association performance in existing fixed-weight models when data is incomplete. The design aims to enhance the role of semantic similarity in cross-modal association and improve matching robustness in scenarios where data is easily missing in isolated environments.

[0044] As an optional embodiment, a monitoring module is also included. This monitoring module, together with the unidirectional transmission control module and the heterogeneous data processing module, forms a feedback closed loop. The monitoring module calculates the load index G using the formula: G = (CPU utilization / 80%) × 0.4 + (Memory usage / 90%) × 0.3 + (Data backlog / Maximum queue length) × 0.3; When G > 1.2, the monitoring module reduces the transmission rate to the unidirectional transmission control module to the set value, and at the same time sends a resource reallocation signal to the heterogeneous data processing module. The heterogeneous data processing module increases the proportion of computing resources allocated to the large file processing unit to the set value.

[0045] The set value represents the range within which staff can implement the settings. It should be noted that linking load monitoring with transmission rate and computing resource allocation breaks through the limitation of existing technologies where single monitoring alarms cannot dynamically adjust system load; the design aims to prevent system overload and crash in an isolated environment, prioritize core processing tasks, and improve system stability.

[0046] As an optional embodiment, the output module includes a semantic verification unit, which is linked to the association index of the multimodal integration module. The semantic verification unit is based on the power domain ontology library and calculates the confidence level C using the following formula:

[0047] Where C is the confidence level output by the semantic verification unit. It is a matching degree function used to calculate the features to be verified. Domain rules The degree of matching; When C < 0.95, the output module calls the association index of the multimodal integration module to extract high-confidence data with the same topic and association strength ≥ the association strength judgment threshold for auxiliary verification. If the auxiliary verification passes, it is judged as qualified.

[0048] It should be noted that by combining multimodal association indexes for auxiliary verification, the problem of output blocking caused by excessively strict single data verification in existing technologies is solved; the design aims to improve output efficiency while ensuring the compliance of data in the isolated environment.

[0049] As an optional embodiment, the file fingerprint hash is generated by the dynamic sharding unit based on the shard data, the shard sequence number, and the file's unique identifier using SHA-256. After the dual verification unit of the security isolation module verifies the hash, the blockchain log unit records the hash value, which serves as the index key value for partitioned storage in the storage module.

[0050] It should be noted that by using file fingerprint hashes throughout the entire transmission, verification, and storage chain as index keys to achieve end-to-end association, the limitations of existing technologies where hashes are only used for verification at a single stage are overcome. The design aims to ensure the consistency of data transmission, verification, and storage in an isolated environment and improve the ability to detect data tampering.

[0051] As an optional embodiment, the storage module's type-adaptive compression algorithm is linked with the standardization results of the preprocessing module: data determined by the preprocessing module to be "text-intensive" is subjected to LZMA deep compression, specifically with a compression ratio of ≥3:1; data determined to be "image-intensive" is subjected to WebP quantization compression, specifically with a compression parameter QP=28.

[0052] It should be noted that the compression algorithm is dynamically selected based on the actual data type after preprocessing, which overcomes the defect of fixed compression algorithms in existing technologies that are not suitable for heterogeneous data. The design aims to optimize compression efficiency and speed and save computing resources in isolated environments with limited computing resources.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A one-way processing system for large files and heterogeneous data in isolated environments, characterized in that, It includes an input adaptation module and a security isolation module, which are connected in series. The input adaptation module is configured with a dynamic fragmentation unit, which calculates the fragment size S using the following formula: Where k is the file type coefficient, D is the original file size in MB, and α is the redundancy check coefficient. The unit is MB; The security isolation module uses optical gate physical isolation and includes a one-way protocol stack and a dual verification unit. The one-way protocol stack only receives fragmented data output by the input adaptation module. The dual verification unit performs feature code matching and behavior entropy value detection on the fragmented data. Fragments that fail the dual verification unit verification are regenerated by the input adaptation module and transmitted to the security isolation module. It also includes a preprocessing module, which is connected to the heterogeneous data processing module. The preprocessing module performs reassembly and standardization on the fragmented data output by the security isolation module. The standardization includes improved median filtering for images, filtering of non-ASCII characters in text, and improved Z-Score outlier removal for structured data. The data processed by the preprocessing module is then transmitted to the heterogeneous data processing module. The heterogeneous data processing module includes a large file processing unit, a multimodal feature extraction unit, and a priority scheduling unit. The priority scheduling unit is shared by the large file processing unit and the multimodal feature extraction unit. The priority scheduling unit calculates the file priority N and allocates computing resources to the large file processing unit and the multimodal feature extraction unit according to the value of N. The one-way transmission control module and the storage module are connected to form a write closed loop. The one-way transmission control module includes a transmission rule base and a blockchain log unit. The transmission rule base restricts data to be transmitted only from the heterogeneous data processing module to the storage module. The blockchain log unit records the data transmission trajectory. The storage module adopts a read-only partition architecture, only receives write requests from the one-way transmission control module, stores data in partitions according to data type, and enables a type-adaptive compression algorithm. The multimodal integration module is integrated into the storage module. Based on the standardization results of the preprocessing module and the feature extraction results of the heterogeneous data processing module, the multimodal integration module maps the features of text, images, and structured data to a unified space by improving the autoencoder, and establishes a cross-modal index. The output module is connected to the storage module via a read-only interface. The output module performs format, syntax, and semantic checks on the integrated data before outputting it.

2. The system for one-way processing of large files and heterogeneous data in isolated environments according to claim 1, characterized in that: The dynamic fragmentation unit includes a fragmentation calculation subunit and a retransmission probability prediction subunit. The fragmentation calculation subunit and the retransmission probability prediction subunit work together. The retransmission probability prediction subunit calculates the fragment loss probability P using a formula: in This is a retransmission correction factor, dynamically adjusted based on historical retransmission behavior, reflecting the degree of influence of the network environment on retransmissions. Here, B represents the fragmented transmission time parameter, and B represents the unidirectional channel bandwidth in the isolated environment. When P > 5%, the retransmission probability prediction subunit triggers the fragment calculation subunit to reduce the fragment size by a ratio of P × 0.

2. At the same time, the input adaptation module sends a "high-risk fragment" flag to the security isolation module. When the dual verification unit of the security isolation module performs behavior entropy value detection on the fragment containing the flag, it adjusts the detection threshold H0 to 3.

0. When the calculated behavioral entropy value exceeds the adjusted detection threshold, the dual verification unit determines that the segment has an abnormal risk, thereby triggering subsequent security strategies. The dynamic fragmentation unit generates a C32 checksum and file fingerprint hash for the deep compression ratio of each fragment. After the dual verification unit of the security isolation module verifies the C32 checksum and file fingerprint hash of the deep compression ratio, it sends a "verification passed" signal back to the input adaptation module. The signal is transmitted in one direction only and there is no data return.

3. The system for one-way processing of large files and heterogeneous data in isolated environments according to claim 2, characterized in that: The security isolation module includes a behavior entropy detection unit and an offline feature library update subunit. The behavior entropy detection unit works in conjunction with the offline feature library update subunit. The behavior entropy detection unit calculates the file behavior entropy H using the following formula: Where n is the total number of events involved in the behavioral entropy calculation, and the number of different types of feature items in the file behavior. Let i be the probability of the i-th feature term occurring. When H > H0 (H0 = 3.5) is detected three times consecutively and no matching feature library is found, the offline feature library update subunit extracts a 512-byte feature fragment to supplement the local feature library. At the same time, the hash value of the feature fragment is sent to the preprocessing module. The preprocessing module automatically performs depth standardization on subsequent files containing the same hash value. The depth standardization includes multi-round image denoising and text granular refinement.

4. The system for one-way processing of large files and heterogeneous data in isolated environments according to claim 3, characterized in that: The preprocessing module includes an improved median filtering unit, and the heterogeneous data processing module includes an improved SIFT feature extraction unit. The improved median filtering unit and the improved SIFT feature extraction unit are linked. The improved median filtering unit calculates the window size using the following formula. : Where ρ is the image noise density; The image filtered by the improved median filtering unit is transmitted to the improved SIFT feature extraction unit. The scale space σ range of the improved SIFT feature extraction unit is based on... Dynamic adjustment, σ= / 5, the feature vector dimension is simplified by the following formula, and the feature vector is the SIFT feature vector extracted by the improved SIFT feature extraction unit: in The simplified SIFT feature vector dimension. This is the baseline scale parameter.

5. A one-way processing system for large files and heterogeneous data in isolated environments according to claim 1, characterized in that: The transmission rule base of the unidirectional transmission control module is linked with the quota dynamic adjustment mechanism of the storage module. When the utilization rate of a certain partition of the storage module is ≥80%, the storage module sends a "quota insufficient" signal to the unidirectional transmission control module. The transmission rule base reduces the transmission priority Q of this type of data by 30% using the following formula: Where Q represents the data transmission priority. To indicate the urgency level, M represents the current used capacity of the storage module. The total capacity of the storage module, These are the characteristic values ​​corresponding to the data type.

6. The system for one-way processing of large files and heterogeneous data in isolated environments according to claim 1, characterized in that: The improved autoencoder of the multimodal integration module is linked with the normalized unit of the preprocessing module, and the loss function L of the improved autoencoder is calculated by the following formula: in , The preprocessed image feature vector. These are the weighting coefficients of the absolute difference term between the text feature vector and the image feature vector. The weighting coefficients are the complement terms for the similarity between text and image features.

7. The system for one-way processing of large files and heterogeneous data in isolated environments according to claim 1, characterized in that: It also includes a monitoring module, which forms a feedback loop with the unidirectional transmission control module and the heterogeneous data processing module. The monitoring module calculates the load index G using the following formula: G = (CPU utilization / 80%) × 0.4 + (Memory usage / 90%) × 0.3 + (Data backlog / Maximum queue length) × 0.3; When G > 1.2, the monitoring module reduces the transmission rate to the unidirectional transmission control module to the set value, and at the same time sends a resource reallocation signal to the heterogeneous data processing module. The heterogeneous data processing module increases the proportion of computing resources allocated to the large file processing unit to the set value.

8. The system for one-way processing of large files and heterogeneous data in isolated environments according to claim 1, characterized in that: The output module includes a semantic verification unit, which is linked to the association index of the multimodal integration module. The semantic verification unit is based on the power domain ontology library and calculates the confidence level C using the following formula: Where C is the confidence level output by the semantic verification unit. It is a matching degree function used to calculate the features to be verified. Domain rules The degree of matching; When C < 0.95, the output module calls the association index of the multimodal integration module to extract high-confidence data with the same topic and association strength ≥ the association strength judgment threshold for auxiliary verification. If the auxiliary verification passes, it is judged as qualified.

9. A one-way processing system for large files and heterogeneous data in isolated environments according to claim 2, characterized in that: The file fingerprint hash is generated by the dynamic sharding unit based on the shard data, the shard sequence number, and the file's unique identifier using SHA-256. After the dual verification unit of the security isolation module verifies the hash, the blockchain log unit records the hash value, which serves as the index key value for partitioned storage in the storage module.

10. A one-way processing system for large files and heterogeneous data in isolated environments according to claim 1, characterized in that: The storage module's type-adaptive compression algorithm is linked with the standardization results of the preprocessing module: data determined by the preprocessing module to be text-intensive is subjected to LZMA deep compression, specifically with a compression ratio of ≥3:1; data determined to be "image-intensive" is subjected to WebP quantization compression, specifically with a compression parameter QP=28.

Images