A data transmission method and system based on optimizing weak network environment

Abstract

The present invention discloses a data transmission method and system based on optimizing a weak network environment, and relates to the technical field of network data transmission, including on-site deployment of edge computing nodes based on a weak network environment, and using distributed network technology to coordinate the operation of edge computing nodes; for edge computing nodes under coordinated operation, encrypting data successfully transmitted during the coordinated operation process; implementing a hierarchical storage strategy for encrypted transmitted data, establishing a multi-dimensional retrieval mechanism for the hierarchical storage strategy, and completing data transmission work under a weak network environment. The present invention significantly improves the reliability and efficiency of data transmission, realizes coordinated transmission and processing of data, and improves the ability to quickly find and obtain data. In the case of network fluctuations or interruptions, the present invention can effectively manage the transmission of unfinished data, avoid data loss and repeated transmission, and thus improve the overall performance of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of network data transmission, and in particular to a method and system for optimizing data transmission in a weak network environment. Background Art

[0002] The current society is in a stage of rapid development of informatization and digitalization. Various data-driven applications and services are increasing, which puts higher requirements on the efficient, secure and stable transmission of data. However, in many application scenarios, such as edge computing and IoT devices, they are often restricted by weak network conditions such as unstable network environment and limited bandwidth, resulting in low data transmission efficiency, increased latency and data loss. In this case, how to ensure reliable transmission and storage of data in a weak network environment has become one of the key areas of current technical research.

[0003] The existing technology has the following major defects in data transmission and storage in weak network environments. First, there is a lack of effective packet loss rate prediction and processing mechanism, which makes it impossible to adjust the data transmission strategy in time when the network condition deteriorates; second, the existing encrypted data storage and retrieval methods are low in complexity and cannot effectively distinguish data of different priorities, resulting in low storage efficiency and difficulty in searching. In response to these problems, the present invention proposes a data transmission method based on optimizing weak network environments, which effectively solves the defects in the above-mentioned existing technologies through technical means such as collaborative edge computing nodes, encrypted transmission data, hierarchical storage, and multi-dimensional retrieval. Summary of the invention

[0004] In view of the problems existing in the existing data transmission method and system based on optimizing a weak network environment, the present invention is proposed.

[0005] Therefore, in order to solve the problems of low data transmission efficiency, data loss and difficult storage and retrieval in weak network environments in the prior art, the present invention adopts edge computing node collaborative operation, encrypted transmission, layered storage and multi-dimensional retrieval technical means to solve them.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for optimizing data transmission in a weak network environment, which includes deploying edge computing nodes on-site in a weak network environment and using distributed network technology to coordinate the operation of the edge computing nodes;

[0008] For the edge computing nodes in the collaborative operation, data transmission processing tasks are performed to encrypt the data successfully transmitted during the collaborative operation;

[0009] A tiered storage strategy is implemented for the encrypted transmitted data, and a multi-dimensional retrieval mechanism for the tiered storage strategy is established using distributed indexing technology to complete data transmission in a weak network environment.

[0010] As a preferred solution of the method for optimizing data transmission in a weak network environment described in the present invention, the collaborative operation of edge computing nodes includes recording the transmission status of the transmitted data using blockchain technology, displaying the packet loss parameters using a network adaptive data transmission protocol, and analyzing the packet loss parameters using a machine learning algorithm to predict the changing trend of the network status;

[0011] The packet loss parameter is analyzed based on the underlying logic of the machine learning algorithm, and the long short-term memory network LSTM of the time series corresponding to the length of the packet loss parameter is selected, the LSTM model is constructed, the structural hyperparameters of the LSTM model are set, and the packet loss rate of the packet loss parameter is calculated based on the structural hyperparameters of the LSTM model. The packet loss rate includes calculating the mean and standard deviation of the packet loss parameter, and the calculated result ratio is used as the packet loss rate. ;

[0012] The specific calculation formula of the mean is:

[0013] ,

[0014] in, represents the mean value of the packet loss parameter, represents the total number of samples, Indicates The packet loss parameter of samples, Indicates a sample;

[0015] The specific calculation formula of the standard deviation is:

[0016] ,

[0017] in, Indicates the standard deviation of the packet loss parameter;

[0018] The specific calculation formula of the packet loss rate is:

[0019] ,

[0020] in, Indicates the packet loss rate.

[0021] As a preferred solution of the method for optimizing data transmission in a weak network environment described in the present invention, wherein: the prediction of the change trend of the network status includes fitting the packet loss rate through a normal distribution process, selecting a confidence interval, calculating the threshold of the packet loss rate within the selected confidence interval, and predicting the change trend of the network status according to the packet loss rate threshold obtained by calculation, and the specific calculation formula of the packet loss rate threshold is:

[0022] ,

[0023] in, Indicates the packet loss rate threshold. represents the confidence interval, Indicates The confidence intervals correspond to value, represents the weight of the confidence interval, represents the number of confidence intervals, Represents the weight factor of the error in the underlying logic of the machine learning algorithm, represents the time series smoothing factor, Indicates the length of the time series data, Indicates The packet loss rate predicted by the machine learning algorithm at each time point, Indicates The average packet loss rate at each time point;

[0024] When the packet loss rate is less than or equal to the packet loss rate threshold, it is determined that the network is in a stable state, and at this time, a batch transmission mode is used to transmit data;

[0025] When the packet loss rate is greater than the packet loss rate threshold, it is determined that the network is in a fluctuating state and the network is interrupted. At this time, data compression breakpoint technology is used for transmission, and the breakpoint data after transmission is recorded on the blockchain of the edge computing node;

[0026] When the network is interrupted, the system automatically checks the transmission status records on the blockchain, verifies the unfinished transmission tasks through smart contracts, and generates a list of unfinished data blocks;

[0027] When the network is restored, the system continues to transmit data from the last breakpoint according to the unfinished data block list, and uses the hash value recorded during the transmission status to check the retransmitted data block list with the unfinished data block list;

[0028] After the data block list is successfully transmitted, the transmission status record on the blockchain is updated and the data block list is marked as having completed transmission.

[0029] As a preferred solution of the method for optimizing data transmission in a weak network environment described in the present invention, wherein: the execution of the data transmission processing task includes dynamically adjusting the data transmission strategy according to the hash value, and the hash value includes a hash value generated in the unfinished data block list by calculating the SHA-256 hash algorithm;

[0030] The transmission status record includes recording the data block list identifier, the data block list hash value and the data block list transmission status;

[0031] When the identifier of the retransmitted data block list is the same as the identifier of the unfinished data block list, the data block list hash value comparison process is started;

[0032] When the retransmitted data block list identifier is different from the unfinished data block list identifier, the mismatch status of this transmission is recorded on the blockchain, and the mismatch status includes the mismatched data block list identifier and the corresponding transmission timestamp and node information. The machine learning algorithm is used to analyze the cause of the mismatch, and the data transmission strategy is dynamically adjusted for a second time according to the hash value. The secondary transmission request identifier is recorded and tracked until the match is successful and the data block list hash value comparison process is entered;

[0033] The specific steps of entering the data block list hash value comparison process are as follows:

[0034] If the hash value of the retransmitted data block list is consistent with the hash value of the unfinished data block list, the data block transmission success status is recorded, and the transaction of the successful transmission is recorded on the blockchain. At this time, the three transmissions are no longer performed;

[0035] If the hash value of the retransmitted data block list is inconsistent with the hash value of the unfinished data block list, the data block transmission failure status is recorded, and the transaction of the failed transmission is recorded on the blockchain. At this time, the data transmission strategy is dynamically adjusted three times according to the hash value, and the three transmission request hash values ​​are recorded and tracked until the hash value of the retransmitted data block list is consistent with the hash value of the unfinished data block list.

[0036] As a preferred solution of the method for optimizing data transmission in a weak network environment described in the present invention, wherein: the encrypting of the data successfully transmitted during the collaborative operation includes using an asymmetric encryption algorithm to distribute data encryption keys to the data transmission after network recovery and the process of executing the data transmission processing task, and encrypting the data transmission after network recovery and the process of executing the data transmission processing task based on the distribution result;

[0037] When the network is restored, the transmitted data is layered according to the distribution of the data encryption key, and when the data is received, the edge computing node decrypts the data and verifies the integrity;

[0038] If the verification fails, the data is discarded and the transmission is requested again;

[0039] If the verification is successful, the role-based access control policy is used to limit the access rights of different user roles to the data. When the role permissions change, the data access rights are adjusted;

[0040] When the adjusted data access permissions exceed the normal range, the alarm mechanism is triggered;

[0041] The stratification of the transmitted data includes determining the priority of the transmitted data according to the sensitivity of the data. When the system senses sensitive business information, the data enters a high-priority processing queue, which is called superior data.

[0042] When the system senses general business information, the data enters the medium processing queue, which is called medium data. After the superior data transmission is completed, the medium data begins to be transmitted;

[0043] When the system senses ordinary business information, the data enters the poor processing queue, which is called poor data. After the excellent data and medium data are transmitted, the poor data begins to be transmitted;

[0044] When the network is restored, the system performs detection and performs hierarchical processing on the transmitted data for a second run.

[0045] As a preferred solution of the method for optimizing data transmission in a weak network environment described in the present invention, wherein: the implementation of a hierarchical storage strategy for the encrypted transmitted data includes hierarchical storage according to the high priority processing queue, the medium processing queue and the poor processing queue based on the data access frequency;

[0046] When data enters the high priority processing queue, the system marks the data as a type of transmission data, and the system determines that the access frequency of the type of transmission data is high access frequency data;

[0047] When data enters the medium processing queue, the system marks the data as Class II transmission data, and the system determines that the access frequency of the Class II transmission data is medium access frequency data;

[0048] When data enters the differential processing queue, the system marks the data at this time as the third type of transmission data. At this time, the system determines that the access frequency of the three types of transmission data is low access frequency data.

[0049] As a preferred solution of the method for optimizing data transmission in a weak network environment described in the present invention, the multi-dimensional retrieval mechanism includes creating a global index table using distributed indexing technology, and storing different types of transmission data in evenly distributed solid-state hard disk media respectively;

[0050] When the system detects that the transmitted data has changed, it triggers the data migration mechanism to update the changed data in storage in real time and complete the precise instructions for data transmission.

[0051] In a second aspect, an embodiment of the present invention provides a data transmission system based on optimizing a weak network environment, which includes: a collaborative operation module, which deploys edge computing nodes on-site in a weak network environment and uses distributed network technology to collaboratively operate the edge computing nodes;

[0052] An encryption module, which performs data transmission processing tasks for the edge computing nodes in the collaborative operation and encrypts the data successfully transmitted during the collaborative operation;

[0053] The transmission module implements a hierarchical storage strategy for the encrypted transmitted data, uses distributed indexing technology to establish a multi-dimensional retrieval mechanism for the hierarchical storage strategy, and completes data transmission in a weak network environment.

[0054] In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the processor executes the computer program, any step of the above-mentioned method based on optimizing data transmission in a weak network environment is implemented.

[0055] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the above-mentioned method for optimizing data transmission in a weak network environment is implemented.

[0056] The beneficial effects of the present invention are as follows: the present invention significantly improves the reliability and efficiency of data transmission by optimizing the data transmission method in a weak network environment; realizes the collaborative transmission and processing of data by deploying edge computing nodes and adopting distributed network technology; and the encrypted transmission and layered storage strategies ensure the security and access efficiency of data; at the same time, the multi-dimensional retrieval mechanism is utilized to improve the ability to quickly search and obtain data; in the event of network fluctuations or interruptions, the present invention can effectively manage the transmission of unfinished data to avoid data loss and repeated transmission, thereby improving the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0058] Figure 1 A flowchart of a method and system for optimizing data transmission in a weak network environment is provided according to an embodiment of the present invention.

[0059] Figure 2 A schematic diagram of data transmission based on a method and system for optimizing data transmission in a weak network environment is provided as an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0062] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0063] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0064] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0065] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0066] Example 1

[0067] Reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a method for optimizing data transmission in a weak network environment, comprising:

[0068] S1: Deploy edge computing nodes on-site in a weak network environment and use distributed network technology to coordinate the operation of edge computing nodes.

[0069] Among them, the coordinated operation of edge computing nodes includes using blockchain technology to record the transmission status of transmitted data, using the network adaptive data transmission protocol to display packet loss parameters, using machine learning algorithms to analyze packet loss parameters, and predicting the changing trend of network conditions;

[0070] The analysis of packet loss parameters includes selecting the long short-term memory network LSTM of the corresponding time series according to the length of the packet loss parameters based on the underlying logic of the machine learning algorithm, building an LSTM model, setting the structural hyperparameters of the LSTM model, and calculating the packet loss rate of the packet loss parameters based on the structural hyperparameters of the LSTM model. The packet loss rate includes calculating the mean and standard deviation of the packet loss parameters, and taking the calculated result ratio as the packet loss rate. ;

[0071] The specific calculation formula of the mean is:

[0072] ,

[0073] in, represents the mean value of the packet loss parameter, represents the total number of samples, Indicates The packet loss parameter of samples, Indicates a sample;

[0074] The specific calculation formula for standard deviation is:

[0075] ,

[0076] in, Indicates the standard deviation of the packet loss parameter;

[0077] The specific calculation formula for packet loss rate is:

[0078] ,

[0079] in, Indicates the packet loss rate.

[0080] Furthermore, for the collaborative operation of edge computing nodes, the system uses blockchain technology to record the status of each data transmission, and uses the network adaptive data transmission protocol to display the packet loss parameters. The packet loss parameters are analyzed by machine learning algorithms, and the system uses the long-short memory network model to predict the changing trend of network conditions. Specific data include: in a certain network environment, the system records the mean values ​​of packet loss parameters as 0.04, 0.06, and 0.08, and the standard deviations are 0.01, 0.02, and 0.03, respectively. Based on these statistical data, the system calculates the packet loss rate and dynamically adjusts the data transmission strategy according to the network conditions. If the mean value is 0.06 and the standard deviation is 0.02, the packet loss rate is 0.15. The system will optimize the transmission strategy to reduce data loss and ensure stability and transmission efficiency in a weak network environment.

[0081] S1.1: Predicting the changing trend of the network status includes fitting the packet loss rate through a normal distribution process, selecting a confidence interval, calculating a threshold of the packet loss rate within the selected confidence interval, and predicting the changing trend of the network status according to the calculated packet loss rate threshold. The specific calculation formula of the packet loss rate threshold is:

[0082] ,

[0083] in, Indicates the packet loss rate threshold. represents the confidence interval, Indicates The confidence intervals correspond to value, represents the weight of the confidence interval, represents the number of confidence intervals, Represents the weight factor of the error in the underlying logic of the machine learning algorithm, represents the time series smoothing factor, Indicates the length of the time series data, Indicates The packet loss rate predicted by the machine learning algorithm at each time point, Indicates The average packet loss rate at each time point;

[0084] When the packet loss rate is less than or equal to the packet loss rate threshold, the network is determined to be in a stable state, and batch transmission is used to transmit data;

[0085] When the packet loss rate is greater than the packet loss rate threshold, the network is determined to be in a fluctuating state and the network is interrupted. At this time, data compression breakpoint technology is used for transmission, and the breakpoint data after transmission is recorded on the blockchain of the edge computing node;

[0086] When the network is interrupted, the system automatically checks the transmission status records on the blockchain, verifies the unfinished transmission tasks through smart contracts, and generates a list of unfinished data blocks;

[0087] When the network is restored, the system continues to transmit data from the last breakpoint based on the list of unfinished data blocks, and uses the hash value recorded during the transmission status to check the retransmitted data block list against the unfinished data block list;

[0088] After the data block list is successfully transmitted, the transmission status record on the blockchain is updated and the data block list is marked as having been transmitted.

[0089] Furthermore, in order to predict the changing trend of network conditions, the system first fits the data distribution of the packet loss rate through the normal distribution, and selects the confidence interval to determine the threshold of the packet loss rate. For example, in a weak network environment, the mean of the packet loss parameter is 0.03, the standard deviation is 0.01, and the system sets a 99% confidence interval, and the threshold of the packet loss rate is 0.05. When the actual measured packet loss rate is less than or equal to 0.05, the system determines that the network is in a stable state. At this time, batch transmission is adopted at a rate of 200 data packets per minute. When the packet loss rate exceeds 0.05, the network is considered to be in a fluctuating state or interrupted. At this time, the system starts the data compression breakpoint technology. The transmission breakpoint data is recorded in the blockchain. For example, in an interruption event, the system recorded 150 unfinished data blocks. When the network is restored, the system automatically checks the transmission status record on the blockchain, verifies the unfinished transmission tasks through smart contracts, and generates an unfinished data block list. The system uses the hash value in the transmission status to check the retransmitted data block list to ensure that it is consistent with the unfinished data block list. After the transmission is successful, the system updates the transmission status record on the blockchain and marks the data block list as completed. Through this process, the continuity and accuracy of data transmission in a complex weak network environment are ensured. The packet loss rate and data transmission data are shown in Table 1:

[0090] Table 1 Packet loss rate and data transmission data table

[0091] ,

[0092] Table 1 summarizes how to set thresholds and corresponding transmission strategies based on packet loss rate data in the process of predicting the trend of network status changes. When the network is stable, batch transmission is performed at a rate of 200 data packets per minute. When the packet loss rate exceeds the threshold, the system will start the data compression block technology and record the unfinished data blocks. After the network is restored, the system verifies the integrity of the transmission task through blockchain and smart contracts to ensure the continuity and accuracy of data transmission. For example, in an interruption event, 150 unfinished data blocks were recorded and proofread through the data block list and hash value to ensure that the retransmitted data is consistent with the unfinished data.

[0093] S2: For edge computing nodes in collaborative operation, perform data transmission processing tasks and encrypt data that has been successfully transmitted during the collaborative operation.

[0094] The performing of the data transmission processing task includes dynamically adjusting the data transmission strategy according to the hash value, and the hash value includes a hash value generated in the unfinished data block list by using the SHA-256 hash algorithm;

[0095] The transmission status record includes recording the data block list identifier, the data block list hash value and the data block list transmission status;

[0096] When the identifier of the retransmitted data block list is the same as the identifier of the unfinished data block list, the data block list hash value comparison process is started;

[0097] When the retransmitted data block list identifier is different from the unfinished data block list identifier, the mismatch status of this transmission is recorded on the blockchain. The mismatch status includes the mismatched data block list identifier and the corresponding transmission timestamp and node information. The machine learning algorithm is used to analyze the cause of the mismatch, and the data transmission strategy is dynamically adjusted according to the hash value. The secondary transmission request identifier is recorded and tracked until the match is successful and the data block list hash value comparison process is entered;

[0098] The specific steps to enter the data block list hash value comparison process are as follows:

[0099] If the hash value of the retransmitted data block list is consistent with the hash value of the unfinished data block list, the data block transmission success status is recorded, and the transaction of successful transmission is recorded on the blockchain. At this time, three transmissions are no longer performed;

[0100] If the hash value of the retransmitted data block list is inconsistent with the hash value of the unfinished data block list, the data block transmission failure status is recorded, and the failed transmission transaction is recorded on the blockchain. At this time, the data transmission strategy is dynamically adjusted three times according to the hash value, and the three transmission request hash values ​​are recorded and tracked until the hash value of the retransmitted data block list is consistent with the hash value of the unfinished data block list.

[0101] Furthermore, the data transmission processing task includes dynamically adjusting the data transmission strategy according to the hash value. When the network transmission encounters an interruption or problem, the system first uses the SHA-256 hash algorithm to generate a hash value for the unfinished data block list. For example, the hash value of the unfinished data block list is abc123. The transmission status record includes the data block list identifier, the data block list hash value and the data block list transmission status. When the retransmitted data block list identifier is the same as the unfinished data block list identifier, the system compares the hash values ​​of the two. If they are consistent, such as the hash value of the retransmitted data block list is also abc123, the transmission is recorded as successful, and the successful transaction is recorded on the blockchain, and no three transmissions are performed. If they are inconsistent, such as the hash value of the retransmitted data block list is xyz789, the system records the transmission failure status and the failed transaction on the blockchain. At this time, the system uses a machine learning algorithm to analyze the cause of the mismatch, and dynamically adjusts the transmission strategy for the second time according to the hash value, records the secondary transmission request identifier and tracks it. If the hash value of the second retransmission does not match the unfinished hash value, the system will record the three transmission request hash values ​​and continue to track until the match is successful or the three transmissions fail. This mechanism ensures the accuracy and security of data transmission. For example, in a certain transmission, after the system adjusted the strategy three times, the hash value of the final successful transmission was abc123 and was marked as completed on the blockchain.

[0102] S2.1: Encrypting the data after successful transmission during the collaborative operation includes using an asymmetric encryption algorithm to distribute data encryption keys to the data transmission after network recovery and the process of executing the data transmission processing task, and encrypting the data transmission after network recovery and the process of executing the data transmission processing task based on the distribution result;

[0103] When the network is restored, the transmitted data is layered according to the distribution of data encryption keys. When the data is received, the edge computing node decrypts the data and verifies the integrity.

[0104] If the verification fails, the data is discarded and the transmission is requested again;

[0105] If the verification is successful, the role-based access control policy is used to limit the access rights of different user roles to the data. When the role permissions change, the data access rights are adjusted;

[0106] When the adjusted data access permissions exceed the normal range, the alarm mechanism is triggered;

[0107] The stratification of transmitted data includes determining the priority of transmitted data according to the sensitivity of the data. When the system senses sensitive business information, the data enters the high-priority processing queue, which is called superior data.

[0108] When the system senses general business information, the data enters the medium processing queue, which is called medium data. After the superior data transmission is completed, the medium data begins to be transmitted;

[0109] When the system senses ordinary business information, the data enters the poor processing queue, which is called poor data. After the excellent data and medium data are transmitted, the poor data begins to be transmitted;

[0110] When the network is restored, the system performs detection and performs hierarchical processing on the transmitted data for a second run.

[0111] Furthermore, in collaborative operation, after the network is restored, the system uses an asymmetric encryption algorithm to distribute encryption keys for the transmitted data. After the data is encrypted, the system processes the received data in layers, giving priority to sensitive data, such as data containing personal identity information, which accounts for 35% of the total transmission volume and is called premium data; then processes medium data, such as internal communication records, which accounts for 45% of the transmission volume; and finally processes ordinary data, such as system logs, which accounts for 20%. When receiving data, the edge computing node first uses the private key to decrypt and verify the integrity of the data. For example, the decrypted hash value should be consistent with the recorded hash value def456. If there is any inconsistency, the data is discarded and the transmission is requested again. After successful decryption, the system applies the role access control policy, such as only the management can access premium data. If the user's role permissions change, such as the permissions are upgraded from the medium level to the high level, the system will automatically adjust its access permissions. If the permission adjustment exceeds the prescribed range, such as ordinary users are granted high-level permissions, the system will trigger an alarm mechanism. In addition, the system will process the data in layers again after the network is restored to ensure that sensitive data is transmitted first, ensuring the security of the data and the effectiveness of the transmission.

[0112] S3: Implement a tiered storage strategy for encrypted data, use distributed indexing technology to establish a multi-dimensional retrieval mechanism for the tiered storage strategy, and complete data transmission in a weak network environment.

[0113] Among them, implementing a tiered storage strategy for encrypted and transmitted data includes tiered storage based on data access frequency in a high priority processing queue, a medium processing queue, and a poor processing queue;

[0114] When data enters the high priority processing queue, the system marks the data as Class I transmission data. At this time, the system determines that the access frequency of Class I transmission data is high access frequency data.

[0115] When data enters the medium processing queue, the system marks the data as Class II transmission data. At this time, the system determines that the access frequency of Class II transmission data is medium access frequency data.

[0116] When data enters the differential processing queue, the system marks the data at this time as Class III transmission data. At this time, the system determines that the access frequency of Class III transmission data is low access frequency data.

[0117] S3.1: The multi-dimensional retrieval mechanism includes using distributed indexing technology to create a global index table and store different types of transmission data in evenly distributed solid-state drive media;

[0118] When the system detects that the transmitted data has changed, it triggers the data migration mechanism to update the changed data in storage in real time and complete the precise instructions for data transmission.

[0119] Preferably, the system performs tiered storage based on data priority. High-priority data, such as 500 requests per second, is stored in SSD A, medium-priority data, such as 2000 data packets per day, is stored in SSD B, and low-priority data, such as monthly logs, is stored in SSD C. Distributed indexing technology is used to create a global index table, and the data in storage is updated in real time to ensure accurate data transmission in a weak network environment.

[0120] In a preferred embodiment, a data transmission system based on optimizing a weak network environment includes a collaborative operation module, which deploys edge computing nodes on-site in a weak network environment and uses distributed network technology to collaboratively operate the edge computing nodes;

[0121] An encryption module, which performs data transmission processing tasks for edge computing nodes in collaborative operation and encrypts data that has been successfully transmitted during the collaborative operation;

[0122] The transmission module implements a tiered storage strategy for encrypted transmitted data, uses distributed indexing technology to establish a multi-dimensional retrieval mechanism for the tiered storage strategy, and completes data transmission in a weak network environment.

[0123] The above-mentioned unit modules may be embedded in or independent of the processor in the computer device in the form of hardware, or may be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0124] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0125] In summary, the present invention significantly improves the reliability and efficiency of data transmission by optimizing the data transmission method in a weak network environment. By deploying edge computing nodes and adopting distributed network technology, the collaborative transmission and processing of data are realized. The encrypted transmission and layered storage strategies ensure the security and access efficiency of the data. At the same time, the multi-dimensional retrieval mechanism is utilized to improve the ability to quickly search and obtain data. In the event of network fluctuations or interruptions, the present invention can effectively manage the transmission of unfinished data to avoid data loss and repeated transmission, thereby improving the overall performance of the system.

[0126] Example 2

[0127] Reference Figure 1 and Figure 2 , which is the second embodiment of the present invention, and this embodiment provides a method for data transmission based on optimizing a weak network environment. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through simulation experiments.

[0128] During the experiment of the technical solution, a test environment was first configured to simulate different network conditions. At the beginning of the experiment, the mean of the packet loss rate was set to 0.04, the standard deviation was set to 0.015, and a 95% confidence interval was set. Based on these parameters, the packet loss rate threshold was calculated to be 0.06. In the experiment, when the actual measured packet loss rate was less than or equal to 0.06, the system performed batch transmission at a rate of 150 data packets per minute. If the packet loss rate exceeded 0.06, the system started the data compression breakpoint technology and recorded the unfinished data blocks. For example, in a network fluctuation event, the system recorded 120 unfinished data blocks. When the network recovered, the system automatically checked the transmission status records on the blockchain, verified the unfinished transmission tasks through smart contracts, and generated a list of unfinished data blocks. In this process, the system uses the recorded hash value for proofreading to ensure that the retransmitted data blocks are consistent with the unfinished data blocks. The experimental results show that the retransmitted data blocks are consistent with the unfinished data block list, and the transmission status is successfully updated on the blockchain, ensuring the accuracy and continuity of data transmission. The experimental process data is shown in Table 2:

[0129] Table 2 Experimental process data table

[0130] ,

[0131] Table 2 summarizes the experimental process data under different network conditions. In the experimental setting, the mean of the packet loss parameter is 0.04, the standard deviation is 0.02, and the packet loss rate threshold calculated under a 95% confidence interval is 0.06. In the actual experiment, when the packet loss rate is 0.05, the network is stable and transmits at a rate of 300 data packets per minute; when the packet loss rate reaches 0.07, the network state is judged to be fluctuating or interrupted, the data compression breakpoint technology is started, and 200 unfinished data blocks are recorded. After the network is restored, the system verifies the list of unfinished data blocks through blockchain and smart contracts to ensure that the retransmitted data blocks are consistent with the original data blocks, and updates the blockchain records after the transmission is successful. This process verifies the system's ability to maintain the reliability and continuity of data transmission under different network conditions. A comparison of the present invention and the prior art is shown in Table 3:

[0132] Table 3 Comparison table between the present invention and the prior art

[0133] ,

[0134] Compared with the existing technology, the present invention has significant advantages. It adopts a dynamic adjustment strategy in data transmission, can adapt to network fluctuations and support batch transmission and data compression breakpoint technology, improves transmission efficiency and reliability, and dynamically calculates the packet loss rate threshold through normal distribution fitting, realizing flexible packet loss processing. In addition, the use of blockchain to record breakpoint data and combined with smart contracts for data recovery ensures data integrity. The system also performs priority processing according to data sensitivity, improves the security and efficiency of transmission, and real-time monitoring and adjustment strategies improve the adaptability of the system.

[0135] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for data transmission based on optimizing a weak network environment, characterized in that: include, Deploy edge computing nodes on-site in a weak network environment and use distributed network technology to coordinate the operation of edge computing nodes; For the edge computing nodes in the collaborative operation, data transmission processing tasks are performed to encrypt the data successfully transmitted during the collaborative operation; Implement a tiered storage strategy for encrypted transmitted data, use distributed indexing technology to establish a multi-dimensional retrieval mechanism for the tiered storage strategy, and complete data transmission in a weak network environment; The collaborative operation of the edge computing nodes includes recording the transmission status of the transmitted data using blockchain technology, displaying the packet loss parameters using the network adaptive data transmission protocol, analyzing the packet loss parameters using a machine learning algorithm, and predicting the changing trend of the network status; The prediction of the change trend of the network status includes fitting the packet loss rate through a normal distribution process, selecting a confidence interval, calculating a threshold of the packet loss rate within the selected confidence interval, and predicting the change trend of the network status according to the calculated packet loss rate threshold. The specific calculation formula of the packet loss rate threshold is: in, Indicates the packet loss rate threshold. represents the confidence interval, Indicates The confidence intervals correspond to value, represents the weight of the confidence interval, represents the number of confidence intervals, Represents the weight factor of the error in the underlying logic of the machine learning algorithm, represents the time series smoothing factor, Indicates the length of the time series data, Indicates The packet loss rate predicted by the machine learning algorithm at each time point, Indicates The average packet loss rate at each time point; When the packet loss rate is less than or equal to the packet loss rate threshold, it is determined that the network is in a stable state, and at this time, a batch transmission mode is used to transmit data; When the packet loss rate is greater than the packet loss rate threshold, it is determined that the network is in a fluctuating state and the network is interrupted. At this time, data compression breakpoint technology is used for transmission, and the breakpoint data after transmission is recorded on the blockchain of the edge computing node; When the network is interrupted, the system automatically checks the transmission status records on the blockchain, verifies the unfinished transmission tasks through smart contracts, and generates a list of unfinished data blocks; When the network is restored, the system continues to transmit data from the last breakpoint according to the unfinished data block list, and uses the hash value recorded during the transmission status to check the retransmitted data block list with the unfinished data block list; After the data block list is successfully transmitted, the transmission status record on the blockchain is updated and the data block list is marked as having completed transmission.

2. The method for optimizing data transmission in a weak network environment according to claim 1, characterized in that: The packet loss parameter is analyzed based on the underlying logic of the machine learning algorithm, and the long short-term memory network LSTM of the time series corresponding to the length of the packet loss parameter is selected, the LSTM model is constructed, the structural hyperparameters of the LSTM model are set, and the packet loss rate of the packet loss parameter is calculated based on the structural hyperparameters of the LSTM model. The packet loss rate includes calculating the mean and standard deviation of the packet loss parameter, and the calculated result ratio is used as the packet loss rate. ; The specific calculation formula of the mean is: in, represents the mean value of the packet loss parameter, represents the total number of samples, Indicates The packet loss parameter of samples, Indicates a sample; The specific calculation formula of the standard deviation is: in, Indicates the standard deviation of the packet loss parameter; The specific calculation formula of the packet loss rate is: in, Indicates the packet loss rate.

3. The method for optimizing data transmission in a weak network environment according to claim 2, characterized in that: The performing of the data transmission processing task includes dynamically adjusting the data transmission strategy according to the hash value, wherein the hash value includes a hash value generated in the unfinished data block list by calculating the hash value using the SHA-256 hash algorithm; The transmission status record includes recording the data block list identifier, the data block list hash value and the data block list transmission status; When the identifier of the retransmitted data block list is the same as the identifier of the unfinished data block list, the data block list hash value comparison process is started; When the retransmitted data block list identifier is different from the unfinished data block list identifier, the mismatch status of this transmission is recorded on the blockchain, and the mismatch status includes the mismatched data block list identifier and the corresponding transmission timestamp and node information. The machine learning algorithm is used to analyze the cause of the mismatch, and the data transmission strategy is dynamically adjusted for a second time according to the hash value. The secondary transmission request identifier is recorded and tracked until the match is successful and the data block list hash value comparison process is entered; The specific steps of the comparison process of the hash value of the data block list are as follows: If the hash value of the retransmitted data block list is consistent with the hash value of the unfinished data block list, the data block transmission success status is recorded, and the transaction of the successful transmission is recorded on the blockchain. At this time, the three transmissions are no longer performed; If the hash value of the retransmitted data block list is inconsistent with the hash value of the unfinished data block list, the data block transmission failure status is recorded, and the transaction of the failed transmission is recorded on the blockchain. At this time, the data transmission strategy is dynamically adjusted three times according to the hash value, and the three transmission request hash values ​​are recorded and tracked until the hash value of the retransmitted data block list is consistent with the hash value of the unfinished data block list.

4. The method for optimizing data transmission in a weak network environment according to claim 3, characterized in that: The encrypting of the data after successful transmission during the collaborative operation includes distributing data encryption keys to the data transmission after network recovery and the process of executing the data transmission processing task using an asymmetric encryption algorithm, and encrypting the data transmission after network recovery and the process of executing the data transmission processing task based on the distribution result; When the network is restored, the transmitted data is layered according to the distribution of the data encryption key, and when the data is received, the edge computing node decrypts the data and verifies the integrity; If the verification fails, the data is discarded and the transmission is requested again; If the verification is successful, the role-based access control policy is used to limit the access rights of different user roles to the data. When the role permissions change, the data access rights are adjusted; When the adjusted data access permissions exceed the normal range, the alarm mechanism is triggered; The stratification of the transmitted data includes determining the priority of the transmitted data according to the sensitivity of the data. When the system senses sensitive business information, the data enters a high-priority processing queue, which is called superior data. When the system senses general business information, the data enters the medium processing queue, which is called medium data. After the superior data transmission is completed, the medium data begins to be transmitted; When the system senses ordinary business information, the data enters the poor processing queue, which is called poor data. After the excellent data and medium data are transmitted, the poor data begins to be transmitted; When the network is restored, the system performs detection and performs hierarchical processing on the transmitted data for a second run.

5. The method for optimizing data transmission in a weak network environment according to claim 4, characterized in that: The implementing of a hierarchical storage strategy for the encrypted transmitted data includes performing hierarchical storage according to the high priority processing queue, the medium processing queue and the poor processing queue based on the data access frequency; When data enters the high priority processing queue, the system marks the data as a type of transmission data, and the system determines that the access frequency of the type of transmission data is high access frequency data; When data enters the medium processing queue, the system marks the data as Class II transmission data, and the system determines that the access frequency of the Class II transmission data is medium access frequency data; When data enters the differential processing queue, the system marks the data at this time as the third type of transmission data. At this time, the system determines that the access frequency of the three types of transmission data is low access frequency data.

6. The method for optimizing data transmission in a weak network environment according to claim 5, characterized in that: The multi-dimensional retrieval mechanism includes using distributed indexing technology to create a global index table, and storing different types of transmission data in evenly distributed solid-state hard disk media; When the system detects that the transmitted data has changed, it triggers the data migration mechanism to update the changed data in storage in real time and complete the precise instructions for data transmission.

7. A data transmission system based on optimizing a weak network environment, based on the data transmission method based on optimizing a weak network environment according to any one of claims 1 to 6, characterized in that: include, The collaborative operation module deploys edge computing nodes on-site in a weak network environment and uses distributed network technology to coordinate the operation of edge computing nodes; An encryption module, which performs data transmission processing tasks for the edge computing nodes in the collaborative operation and encrypts the data successfully transmitted during the collaborative operation; The transmission module implements a hierarchical storage strategy for the encrypted transmitted data, uses distributed indexing technology to establish a multi-dimensional retrieval mechanism for the hierarchical storage strategy, and completes data transmission in a weak network environment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for optimizing data transmission in a weak network environment according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for optimizing data transmission in a weak network environment according to any one of claims 1 to 7 are implemented.

Images