Data link transmission method for memory

Abstract

The invention discloses a data link transmission method for a memory, relates to the technical field of data transmission, and solves the technical problems of resource mismatching, transmission delay increase and throughput reduction caused by mixed transmission of large-capacity data and small-capacity data. Small-capacity data are matched with fast and slow channels according to requirements, the problem of mismatching of data and channel resources is solved, the channels are screened and sorted based on dynamic indexes such as channel real-time loads, transmission speed mean values and periodic bandwidth utilization rates, the data are distributed in combination with a load balancing principle, the channels are prevented from being overloaded or idle, the overall channel utilization rate is improved, and the data transmission efficiency is improved. Secondly, through link optimization of combined packaging, repeated data replacement and priority scheduling, the control overhead and repeated transmission quantity of small data are reduced; meanwhile, a cache mechanism is introduced to multiplex high-frequency data, invalid transmission is further reduced, and delay is reduced.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, specifically to a data link transmission method for memory. Background Technology

[0002] With the development of information technology, data transmission scenarios in memory are becoming increasingly complex, with significant differences in the capacity of data to be transmitted, and the transmission requirements for data of different capacities are fundamentally different.

[0003] According to patent application CN114826656A, a trusted data link transmission method and system are disclosed, comprising: a message sender encrypting and signing the original data to be sent using a hybrid encryption method of symmetric encryption and asymmetric encryption, and sending the encrypted and signed ciphertext block and the symmetric encryption key processed by asymmetric encryption to the message receiver via a data link; the message receiver decrypts the ciphertext block based on the symmetric encryption key and verifies the signature, and if the signature verification is successful, the data reception is successful.

[0004] However, some existing technologies do not develop targeted strategies based on differences in data capacity when transmitting data through links. Large-capacity data and small-capacity data are transmitted together, resulting in large-capacity data occupying the resources of small-capacity data channels and small-capacity data occupying the high-speed channels of large-capacity data, causing resource mismatch. At the same time, they do not combine dynamic indicators such as channel load and transmission speed to select suitable channels, which may allocate data to channels with excessive load or mismatched speed, resulting in increased transmission latency and decreased throughput. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a data link transmission method for memory, which solves the problem of resource mismatch caused by the mixed transmission of large-capacity and small-capacity data, resulting in increased transmission delay and decreased throughput.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a data link transmission method for memory, which specifically includes the following steps: Based on the data capacity of the data to be transferred to the memory, large-capacity data and small-capacity data are classified, and transmission analysis is performed separately for each. For the analysis of large-capacity data, the transmission channel index is calculated, and the difference between the index and the preset value is calculated. At the same time, the pre-selected transmission channels are obtained. Large-capacity data is fragmented based on the average transmission value of the pre-selected transmission channel, and then matched with the corresponding queue length and idle bandwidth to generate matching transmission information. For the analysis of small-capacity data, the data is classified into primary and secondary capacity data according to its data capacity, and the channels to be analyzed are selected according to the total load of the transmission channels. At the same time, the average transmission speed is used as the standard for secondary classification to obtain fast and slow transmission channels. The first-level and second-level capacity data are combined to obtain corresponding combination packets based on the average transmission speed. Duplicate data is replaced, and the packets are sorted and transmitted according to their corresponding transmission priorities to generate matching transmission information.

[0007] As a further aspect of the present invention, the specific method for obtaining large-capacity data and small-capacity data through classification is as follows: The system acquires the data to be transferred from the storage device and its corresponding data capacity. It then compares the data capacity with a classification threshold, the specific value of which is set by the operator. Data with a capacity greater than or equal to the classification threshold is classified as large-capacity data, while data with a capacity less than or equal to the classification threshold is classified as small-capacity data.

[0008] As a further aspect of the present invention, the specific method for obtaining the pre-selected transmission channel through screening is as follows: All transmission channels are numbered i=1, 2, ..., j, where j is the total number of channels. The periodic bandwidth utilization and periodic transmission delay of each channel are collected and standardized to obtain the bandwidth utilization index and transmission delay index. The comprehensive index is calculated by the formula Transmission Channel Index = Bandwidth Utilization Index × Weight 1 + Transmission Delay Index × Weight 2. The comprehensive index is then compared with the preset index value, and channels with differences within the preset range are selected as pre-selected transmission channels, numbered a=1, 2, ..., b, where b is the number of pre-selected channels.

[0009] As a further aspect of the present invention, the specific method for collecting and standardizing the periodic bandwidth utilization and periodic transmission delay of each channel is as follows: The standardized formula for periodic broadband utilization rate is: Periodic Broadband Utilization Rate Index = The original value here represents the average periodic bandwidth utilization. The standardized formula for periodic transmission delay is: Transmission delay index = .

[0010] As a further aspect of the present invention, the specific method for generating the matching transmission information is as follows: The categorized large-capacity data is merged into ultra-large frame data packets and transmitted in parallel through multiple channels. First, the average transmission value of all pre-selected transmission channels a is calculated, and the ultra-large frame data packets are fragmented based on this value. Then, the number of tasks to be transmitted in each pre-selected channel is obtained and sorted from smallest to largest. The number of fragments to be allocated to each channel is calculated in combination with the available bandwidth, and the transmission information is generated after matching.

[0011] As a further aspect of the present invention, the specific method for obtaining the channel to be analyzed based on the total load of the transmission channel is as follows: Obtain the small-capacity data and corresponding data capacity obtained from the classification. Then, perform secondary classification based on the data capacity to obtain primary-capacity data and secondary-capacity data. At the same time, obtain the total load corresponding to the remaining transmission channels n, where n=1, 2, ..., m, where m represents the number of remaining transmission channels and the remaining transmission channels represent the transmission channels after removing the pre-selected transmission channels. Compare the total load with the load preset value, filter the transmission channels whose total load is less than the load preset value, record them as channels to be analyzed, and perform secondary classification to obtain fast and slow transmission channels.

[0012] As a further aspect of the present invention, the specific method for obtaining the fast and slow transmission channels is as follows: The transmission speed of the channel to be analyzed is acquired and calculated to obtain the average transmission speed. The channels are then sorted from largest to smallest based on the average transmission speed. The channels to be analyzed are then classified according to the average transmission speed. The average transmission speed is compared with a preset average. Channels with a speed greater than the preset average are classified as fast transmission channels, and those with a speed less than the preset average are classified as slow transmission channels. At the same time, primary capacity data is matched with slow transmission channels for transmission, and secondary capacity data is matched with fast transmission channels for transmission, generating transmission matching information.

[0013] As a further aspect of the present invention, the specific method for generating and transmitting matching information is as follows: For the first-level capacity data of the slow transmission channel, it is combined into a first-level combined packet according to the average transmission speed of the channel. Similarly, a second-level combined packet is generated for the second-level capacity data of the fast transmission channel. After processing duplicate data using the system replacement template for the two types of combined packets, the transmission priority is determined according to the real-time nature of the data, and the combined packets are transmitted in descending order of their transmission priority to generate matching transmission information.

[0014] This invention provides a data link transmission method for memory. Compared with the prior art, it has the following advantages: This invention employs a multi-level classification strategy to match large-capacity data with multi-channel parallel transmission and small-capacity data with fast and slow channels according to demand, thus solving the problem of data and channel resource mismatch. Simultaneously, it filters and sorts channels based on dynamic indicators such as real-time channel load, average transmission speed, and periodic bandwidth utilization, and allocates data according to load balancing principles to avoid channel overload or idleness, thereby improving overall channel utilization. Furthermore, it reduces control overhead and redundant transmission volume for small data through link optimization such as combination packaging, duplicate data replacement, and priority scheduling. Additionally, it introduces a caching mechanism to reuse high-frequency data, further reducing invalid transmissions and lowering latency. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the steps and methods of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 This application provides a data link transmission method for memory, which specifically includes the following steps: Step 1: Obtain the data to be transferred from the storage device and its corresponding data capacity. Simultaneously, compare the data capacity with the classification threshold. The specific value of the classification threshold is set by the operator. Classify the data with a data capacity greater than or equal to the classification threshold as large-capacity data, and classify the data with a data capacity less than the classification threshold as small-capacity data. Then, transmit and analyze the large-capacity data and small-capacity data obtained from the classification separately.

[0018] For example, in daily office work, when employees need to transfer files to the company server, the operator sets the classification threshold to 100MB. At this time, a video training material with a size of 1.2GB will be classified as large data because its size exceeds 100MB, while a 20MB Word document with a size of less than 100MB will be regarded as small data.

[0019] Step 2: Obtain the large-capacity data obtained from the classification, and merge the continuous large-capacity data into a super-large frame data packet. At the same time, perform multi-channel parallel transmission of the super-large frame data packet. For example, a cloud data center needs to transmit 5 consecutive large-capacity backup files (20GB each, 100GB in total) to an off-site disaster recovery center. Merge the 5 files into 1 100GB super-large frame data packet to reduce frame header overhead (the single frame header is reduced from 5×20 bytes in 5 transmissions to 20 bytes in 1 transmission). Obtain all transmission channels and label them as i, where i=1, 2, ..., j, and j represents the number of transmission channels. There are 8 available transmission channels (j=8), labeled i=1 to i=8 (all are 100Gbps fiber optic links).

[0020] Next, the periodic bandwidth utilization and periodic transmission delay corresponding to transmission channel i are obtained, and both are standardized. The standardized formula for periodic bandwidth utilization is: Periodic bandwidth utilization index = The original value here represents the average periodic bandwidth utilization. The closer the value is to 1, the more fully the bandwidth is utilized. The standardized formula for periodic transmission delay is: Transmission Delay Index = , the closer the specific value is to 0, the better the latency; if the latency exceeds the threshold, it is directly recorded as 1, indicating non-compliance. The corresponding broadband utilization rate index and transmission latency index are obtained. At the same time, the two are substituted into the formula: transmission channel index = broadband utilization rate index × weight one + transmission latency index × weight two, and the specific values of weight one and weight two are set by the operator; For example, the statistical period is set to 10 seconds, the average cycle broadband utilization rate of each channel is 30% - 80% (maximum 80%, minimum 30%), and the maximum allowable latency is 50 ms. Taking channel i = 3 as an example, the average utilization rate is 50%, then the broadband utilization rate index = (50 - 30) / (80 - 30) = 0.4; the average latency is 30 ms, then the transmission latency index = 30 / 50 = 0.6.

[0021] Calculate the difference between the transmission channel index and the index preset value. The specific value of the index preset value is set by the operator. Screen all transmission channels whose differences are within the preset interval, denoted as preselected transmission channels. The specific value of the preset area here is set by the operator and labeled as a, and a = 1, 2,..., b, where b represents the number of preselected transmission channels. At the same time, calculate the transmission mean value corresponding to all preselected transmission channels a, and perform fragmentation processing on the super-large frame data packets based on the transmission mean value to obtain fragmented data; For example, the index preset value is set to 0.5, and the preset interval is [-0.05, 0.05]. Channels with transmission channel indices between 0.45 - 0.55 are screened out. Finally, b = 4 (preselected channels a = 1 to a = 4). The transmission mean value of the 4 preselected channels is 8 GB / s. Therefore, the 100 GB super-large frame is split into 13 fragments (the first 12 are 8 GB, and the last one is 4 GB). The queue lengths of each channel are: a = 1 (2 tasks), a = 2 (1 task), a = 3 (3 tasks), a = 4 (0 tasks). Sorted by queue length as a = 4 < a = 2 < a = 1 < a = 3. The idle bandwidths are: a = 4 (80 Gbps), a = 2 (60 Gbps), a = 1 (40 Gbps), a = 3 (20 Gbps). The allocated fragment numbers are 4, 3, 3, 3 (total 13), generate matching information and start the transmission.

[0022] Then obtain the queue lengths corresponding to the preselected transmission channels a, where the queue length here represents the number of tasks to be completed for transmission, and sort them from small to large. At the same time, obtain the idle bandwidth corresponding to the preselected transmission channels a. Then calculate the number of fragmented data based on the idle bandwidth as the standard, and match the corresponding number of fragments with the transmission channels a to generate matching transmission information.

[0023] Step 3: Obtain the classified small-capacity data and its corresponding data capacity, set the secondary classification threshold (set by the operator according to business needs), and further divide the small-capacity data into primary capacity data and secondary capacity data. The single packet capacity of primary capacity data is smaller than that of secondary capacity data (for example, primary capacity data is less than 1KB, and secondary capacity data is 1KB to 10KB). Call all remaining transmission channels (labeled n, n=1, 2, ..., m, where m is the total number of remaining transmission channels), and collect the total load of each remaining transmission channel (including comprehensive indicators such as the current number of tasks to be transmitted and bandwidth utilization). Set a load preset value (e.g., total load ≤ 30%, set by the operator), and filter out the remaining transmission channels with a total load less than the preset value as "channels to be analyzed".

[0024] Calculate the average transmission speed of all channels to be analyzed (the average transmission rate over a fixed period, such as the average over a 100ms period), and set a preset average value (e.g., 50Mbps, set by the operator). Classify according to the following rules: Channels with an average transmission speed greater than the preset average are marked as fast transmission channels; Channels with an average transmission speed less than the preset average are marked as slow transmission channels.

[0025] Transmission channels are allocated based on the secondary classification results: Level 1 capacity data (smaller capacity, lower speed requirements) is matched with slow transmission channels to avoid occupying high-speed resources; Secondary capacity data (relatively large capacity, requiring a slightly faster response) is matched with a fast transmission channel to ensure transmission efficiency.

[0026] Step 4: Perform transmission analysis on the primary capacity data obtained from the slow transmission channel. Using the average transmission speed of the slow transmission channel as the standard, obtain the primary capacity data corresponding to the total capacity and combine them to obtain the primary combined data packet. Similarly, process the secondary capacity data corresponding to the fast transmission channel to obtain the secondary combined data packet. Specifically, the total data capacity corresponding to the combined data packet obtained here does not exceed the average transmission speed of the corresponding fast or slow transmission channel. For slow transmission channels, the average transmission speed (e.g., in KB / s) is used as a benchmark, combined with a preset maximum combination duration (e.g., 100ms, set by the operator), to calculate the maximum allowable capacity of a single data packet (average transmission speed × maximum combination duration). Data is selected from the primary capacity data and combined to ensure that the total capacity of the combined data does not exceed this maximum value, forming a primary combined data packet. Similarly, for high-speed transmission channels, secondary capacity data is combined to form secondary combined data packets based on the average transmission speed and the shortest maximum combination duration (e.g., 50ms, since the secondary data capacity is slightly larger, it is necessary to balance latency and efficiency). For example, for a slow transmission channel (average transmission speed 200KB / s, maximum combination duration 100ms): maximum allowable capacity = 200KB / s × 0.1s = 20KB. Combine 30 temperature and humidity data points (30 × 200B = 6KB) and 5 door and window status data points (5 × 100B = 500B) into one primary combination packet (total capacity 6.5KB ≤ 20KB). High-speed transmission channel (average transmission speed 1MB / s, maximum combination duration 50ms): maximum allowed capacity = 1MB / s × 0.05s = 50KB. Combine 10 "turn on lights" commands (10 × 1.5KB = 15KB) into one secondary combination packet (15KB ≤ 50KB).

[0027] Simultaneously, duplicate data in the obtained first-level and second-level combination packets are replaced using pre-stored replacement templates to generate corresponding first-level and second-level replacement combination packets. The transmission priority of these combination packets is analyzed, specifically based on data real-time performance, and transmitted sequentially from highest to lowest priority to generate matching transmission information. Corresponding buffers are set at both the sending and receiving ends, classifying data into high-frequency and low-frequency categories based on usage frequency. High-frequency data is then buffered, and the system continuously monitors the data to be transmitted, checking for its presence in the buffers. If present, the data is read directly from the buffer; otherwise, normal analysis and transmission processing are performed.

[0028] The generated first-level and second-level combined data packets are compressed using the system's pre-stored "duplicate data replacement template": Replace frequently repeated fixed data segments stored in the template (such as the sensor status identifier "normal=0x01", the instruction prefix "CMD_", etc.) with short identifiers (such as 1-byte codes) to replace long segments (such as 4-byte instruction prefixes). For example, the repeated occurrence of "temperature sensor ID=001" (5 bytes) in the first-level combined data packet can be replaced with "0x0A" (1 byte) using a template, reducing the amount of duplicate data transmitted. The processed data are called the first-level replacement combined packet and the second-level replacement combined packet, respectively.

[0029] The bundles are prioritized based on the "real-time requirements" of the data (from high to low priority): High priority: Data with high real-time requirements (such as primary alarm data for abnormal opening of doors and windows in smart homes, and secondary fault signal data for industrial equipment). Medium priority: Periodic but non-urgent data (such as temperature and humidity level 1 data every 30 seconds); Low priority: Non-real-time data (such as secondary statistics from device logs). Data is transmitted in descending order of priority to ensure that critical data occupies channel resources first and avoids being blocked by non-urgent data.

[0030] Deploy small data buffer pools at both the sending and receiving ends to optimize the transmission efficiency of duplicate data. Based on the frequency of data usage (e.g., the number of transmissions per minute), data is divided into high-frequency data (e.g., fixed configuration parameters of the device, recurring status codes) and low-frequency data (e.g., temporary commands triggered only once). High-frequency data is stored in a cache pool, and a LRU (Least Recently Used) strategy is used to evict data that has not been used for a long time. Before transmission, the cache pool is checked. If the data to be transmitted is already in the cache (and has not expired), it is read directly from the cache and reused without retransmission. If the cache is not hit, the normal combination, replacement, and priority transmission process is executed.

[0031] The data in the above formulas are all calculated using numerical values, without substituting the units of the parameters. In addition, the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0032] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A data link transmission method for a memory, characterized in that, The method specifically includes the following steps: Based on the data capacity of the data to be transferred to the memory, large-capacity data and small-capacity data are classified, and transmission analysis is performed separately for each. For the analysis of large-capacity data, the transmission channel index is calculated, and the difference between the index and the preset value is calculated. At the same time, the pre-selected transmission channels are obtained. Large-capacity data is fragmented based on the average transmission value of the pre-selected transmission channel, and then matched with the corresponding queue length and idle bandwidth to generate matching transmission information. For the analysis of small-capacity data, the data is classified into primary and secondary capacity data according to its data capacity, and the channels to be analyzed are selected according to the total load of the transmission channels. At the same time, the average transmission speed is used as the standard for secondary classification to obtain fast and slow transmission channels. The first-level and second-level capacity data are combined to obtain corresponding combination packets based on the average transmission speed. Duplicate data is replaced, and the packets are sorted and transmitted according to their corresponding transmission priorities to generate matching transmission information.

2. The data link transmission method for a memory according to claim 1, characterized in that, The specific method for obtaining large-volume and small-volume data through classification is as follows: The system acquires the data to be transferred from the storage device and its corresponding data capacity. It then compares the data capacity with a classification threshold, the specific value of which is set by the operator. Data with a capacity greater than or equal to the classification threshold is classified as large-capacity data, while data with a capacity less than or equal to the classification threshold is classified as small-capacity data.

3. The data link transmission method for a memory according to claim 1, characterized in that, The specific method for obtaining the pre-selected transmission channels through the screening process is as follows: All transmission channels are numbered i=1, 2, ..., j, where j is the total number of channels. The periodic bandwidth utilization and periodic transmission delay of each channel are collected and standardized to obtain the bandwidth utilization index and transmission delay index. The comprehensive index is calculated by the formula Transmission Channel Index = Bandwidth Utilization Index × Weight 1 + Transmission Delay Index × Weight 2. The comprehensive index is then compared with the preset index value, and channels with differences within the preset range are selected as pre-selected transmission channels, numbered a=1, 2, ..., b, where b is the number of pre-selected channels.

4. The data link transmission method for a memory according to claim 3, characterized in that, The specific method for collecting and standardizing the periodic bandwidth utilization and periodic transmission delay of each channel is as follows: The standardized formula for periodic broadband utilization rate is: Periodic Broadband Utilization Rate Index = The original value here represents the average periodic bandwidth utilization. The standardized formula for periodic transmission delay is: Transmission delay index = .

5. A data link transmission method for a memory according to claim 1, characterized in that, The specific method for generating matching transmission information is as follows: The categorized large-capacity data is merged into ultra-large frame data packets and transmitted in parallel through multiple channels. First, the average transmission value of all pre-selected transmission channels a is calculated, and the ultra-large frame data packets are fragmented based on this value. Then, the number of tasks to be transmitted in each pre-selected channel is obtained and sorted from smallest to largest. The number of fragments to be allocated to each channel is calculated in combination with the available bandwidth, and the transmission information is generated after matching.

6. A data link transmission method for a memory according to claim 1, characterized in that, The specific method for selecting the channel to be analyzed based on the total load of the transmission channel is as follows: Obtain the small-capacity data and corresponding data capacity obtained from the classification. Then, perform secondary classification based on the data capacity to obtain primary-capacity data and secondary-capacity data. At the same time, obtain the total load corresponding to the remaining transmission channels n, where n=1, 2, ..., m, where m represents the number of remaining transmission channels and the remaining transmission channels represent the transmission channels after removing the pre-selected transmission channels. Compare the total load with the load preset value, filter the transmission channels whose total load is less than the load preset value, record them as channels to be analyzed, and perform secondary classification to obtain fast and slow transmission channels.

7. A data link transmission method for a memory according to claim 6, characterized in that, The specific method for obtaining the fast and slow transmission channels is as follows: The transmission speed of the channel to be analyzed is acquired and calculated to obtain the average transmission speed. The channels are then sorted from largest to smallest based on the average transmission speed. The channels to be analyzed are then classified according to the average transmission speed. The average transmission speed is compared with a preset average. Channels with a speed greater than the preset average are classified as fast transmission channels, and those with a speed less than the preset average are classified as slow transmission channels. At the same time, primary capacity data is matched with slow transmission channels for transmission, and secondary capacity data is matched with fast transmission channels for transmission, generating transmission matching information.

8. A data link transmission method for a memory according to claim 7, characterized in that, The specific method for generating and transmitting matching information is as follows: For the first-level capacity data of the slow transmission channel, it is combined into a first-level combined packet according to the average transmission speed of the channel. Similarly, a second-level combined packet is generated for the second-level capacity data of the fast transmission channel. After processing duplicate data using the system replacement template for the two types of combined packets, the transmission priority is determined according to the real-time nature of the data, and the combined packets are transmitted in descending order of their transmission priority to generate matching transmission information.

Images