Big data transmission method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a big data transmission method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target data packet which carries a unique identifier, and the unique identifier is obtained through coding according to the timestamp of the target data packet, the address information of transmitting end equipment and the serial number of the data packet; and sending the target data packet to the receiving end equipment, so that the receiving end equipment judges the unique identifier of the target data packet to obtain a response data packet, receives the response data packet returned by the receiving end equipment, and performs data transmission according to the response data packet. In the data transmission process of the sending end equipment and the receiving end equipment, the unique identifier of the data packet is judged, and then whether retransmission is needed is judged, the transmission process is optimized, data verification and an intelligent retransmission mechanism are enhanced, so that the problems of repeated transmission and wrong transmission in the existing data transmission process are effectively solved, and the transmission efficiency is improved. And the accuracy and efficiency of data transmission are obviously improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and more specifically, to a method, device, electronic device, and storage medium for transmitting large amounts of data. Background Art

[0002] In current data transmission scenarios, whether based on wired or wireless networks, duplicate and erroneous transmissions are common. From a network architecture perspective, in complex network topologies, packets can be forwarded multiple times due to routing errors or network congestion, leading to duplicate transmissions. For example, in large enterprise networks, where multiple switches and routers work together, if routing tables are not updated promptly or conflicts occur, packets can easily be forwarded in loops, wasting network bandwidth and reducing overall transmission efficiency.

[0003] Transmission errors also frequently occur at the data link layer. Wireless networks are susceptible to environmental interference, such as signal fading and noise, which can cause bit flips during data transmission, leading to transmission errors. For example, while 5G communications offer high theoretical transmission speeds, signals in urban environments with densely populated buildings are easily obstructed and affected by reflections, resulting in increased packet error rates. Failure of the receiver to accurately identify and correct these errors can lead to data loss or application errors. Traditional data transmission protocols, such as TCP (Transmission Control Protocol), have retransmission mechanisms to handle errors, but these mechanisms are inefficient when faced with numerous repeated transmissions and complex error conditions, failing to meet today's stringent requirements for high-speed and accurate data transmission. Improving the accuracy and efficiency of data transmission is an urgent issue. Summary of the Invention

[0004] Some embodiments of the present application aim to provide a method, device, electronic device and storage medium for large data transmission. Through the technical solutions of the embodiments of the present application, a target data packet is obtained, wherein the target data packet carries a unique identifier, and the unique identifier is encoded according to the timestamp of the target data packet, the address information of the sending device and the data packet sequence number; the target data packet is sent to the receiving device so that the receiving device judges the unique identifier of the target data packet and obtains a response data packet, and the response data packet is used to judge whether to retransmit; the response data packet returned by the receiving device is received, and data is transmitted according to the response data packet. The embodiments of the present application uniquely identify the data packet, and during the data transmission process between the sending device and the receiving device, the unique identifier of the data packet is judged, and then whether retransmission is required, thereby optimizing the transmission process, strengthening the data verification and intelligent retransmission mechanism, effectively solving the problems of repeated transmission and erroneous transmission in the existing data transmission process, and significantly improving the accuracy and efficiency of data transmission.

[0005] In a first aspect, some embodiments of the present application provide a method for transmitting large amounts of data, applied to a transmitting device, comprising: Acquire a target data packet, wherein the target data packet carries a unique identifier, and the unique identifier is encoded according to a timestamp of the target data packet, address information of a sending end device, and a data packet sequence number; Sending the target data packet to a receiving device, so that the receiving device determines the unique identifier of the target data packet and obtains a response data packet, wherein the response data packet is used to determine whether to retransmit; Receive a response data packet returned by the receiving device, and perform data transmission according to the response data packet.

[0006] Some embodiments of the present application uniquely identify the data packet, and during the data transmission process between the sending device and the receiving device, determine the unique identifier of the data packet, and then determine whether retransmission is required, thereby optimizing the transmission process, strengthening data verification and intelligent retransmission mechanism, effectively solving the problems of repeated transmission and erroneous transmission in the existing data transmission process, and significantly improving the accuracy and efficiency of data transmission.

[0007] Optionally, the unique identifier is obtained in the following manner: Performing an XOR operation on the timestamp of the target data packet and the address information of the sending end device to obtain an XOR result; The XOR result and the data packet sequence number are concatenated to obtain a unique identifier for the target data packet.

[0008] Some embodiments of this application use a unique identifier for data packets. This unique identifier, along with a duplicate detection mechanism at the receiving end, can accurately identify and discard duplicate data packets. Compared to traditional transmission methods, this reduces the duplicate transmission rate by over 80%, significantly saving network bandwidth resources and improving transmission efficiency. In large file transfer scenarios, this can significantly shorten transmission time.

[0009] Optionally, before obtaining the target data packet, the method further includes: Send a test packet to the receiving device; Receive a response packet returned by the receiving device, and determine a target sending path based on the response packet; the target sending path is used to send the target data packet.

[0010] Some embodiments of the present application use dynamic network maps and optimal path selection to ensure that data is always transmitted on the best path, reducing transmission delay by an average of 30%, meeting application scenarios with high real-time requirements.

[0011] Optionally, determining a target sending path according to the response packet includes: Obtaining network status information in the response data packet, the network status information being at least one or more of bandwidth information, delay information, and packet loss rate; The network status information is calculated using the Dijkstra algorithm to determine the target sending path.

[0012] In some embodiments of the present application, before transmitting data, the sending device sends a probe packet to the network to collect network status information of each node, including bandwidth, delay, packet loss rate, etc. Based on this real-time information, the Dijkstra algorithm is used to calculate an optimal transmission path to improve data transmission efficiency. Optionally, the receiving device determines the unique identifier of the target data packet to obtain a response data packet, where the response data packet is used to determine whether to retransmit, including: Matching the unique identifier of the target data packet with a pre-stored data packet identifier list to obtain a matching result; According to the matching result, it is determined whether to retransmit the target data packet.

[0013] In some embodiments of the present application, after receiving a data packet, the receiving device first checks whether it is a duplicate based on the packet identifier. By maintaining a list of received data packet identifiers, the newly received data packet identifier is quickly compared. If a duplicate is found, the data packet is directly discarded and a confirmation message is sent to the sending end.

[0014] Optionally, the method further includes: The receiving device performs LDPC code verification on the target data packet received for the first time; If the verification is passed, the target data packet is stored and identification information of correct reception is returned to the sending end device; If the verification fails, the erroneous target data packet is recorded and error identification information is returned to the sending end device.

[0015] To improve data anti-interference capabilities, some embodiments of this application employ low-density parity-check (LDPC) codes to encode data packet content. LDPC codes offer robust error correction capabilities, effectively detecting and correcting bit errors during transmission while adding a small amount of redundant information.

[0016] Optionally, receiving a response data packet returned by the receiving device and performing data transmission according to the response data packet includes: Judging the response data packet; If the response data packet is confirmation discard information, stop retransmitting the target data packet; If the response data packet is correct reception information, then continue to transmit other data packets.

[0017] In some embodiments of the present application, after the sending device receives the feedback message from the receiving device, if it is a confirmation discard message, it stops retransmitting the data packet; if it is a correct reception confirmation message, it continues to transmit subsequent data packets; if it is an error feedback message, it performs intelligent retransmission based on the error type and network status.

[0018] In a second aspect, some embodiments of the present application provide a large data transmission apparatus, applied to a receiving device, comprising: An acquisition module is used to acquire a target data packet, wherein the target data packet carries a unique identifier, and the unique identifier is obtained by encoding the timestamp of the target data packet, the address information of the sending end device, and the data packet sequence number; a sending module, configured to send the target data packet to a receiving device, so that the receiving device determines the unique identifier of the target data packet and obtains a response data packet, wherein the response data packet is used to determine whether to retransmit; The receiving module is used to receive the response data packet returned by the receiving device and perform data transmission according to the response data packet.

[0019] Some embodiments of the present application uniquely identify the data packet, and during the data transmission process between the sending device and the receiving device, determine the unique identifier of the data packet, and then determine whether retransmission is required, thereby optimizing the transmission process, strengthening data verification and intelligent retransmission mechanism, effectively solving the problems of repeated transmission and erroneous transmission in the existing data transmission process, and significantly improving the accuracy and efficiency of data transmission.

[0020] Optionally, the unique identifier is obtained in the following manner: Performing an XOR operation on the timestamp of the target data packet and the address information of the sending end device to obtain an XOR result; The XOR result and the data packet sequence number are concatenated to obtain a unique identifier for the target data packet.

[0021] Some embodiments of this application use a unique identifier for data packets. This unique identifier, along with a duplicate detection mechanism at the receiving end, can accurately identify and discard duplicate data packets. Compared to traditional transmission methods, this reduces the duplicate transmission rate by over 80%, significantly saving network bandwidth resources and improving transmission efficiency. In large file transfer scenarios, this can significantly shorten transmission time.

[0022] Optionally, the acquisition module is used to: Send a test packet to the receiving device; Receive a response packet returned by the receiving device, and determine a target sending path based on the response packet; the target sending path is used to send the target data packet.

[0023] Some embodiments of the present application use dynamic network maps and optimal path selection to ensure that data is always transmitted on the best path, reducing transmission delay by an average of 30%, meeting application scenarios with high real-time requirements.

[0024] Optionally, the acquisition module is used to: Obtaining network status information in the response data packet, the network status information being at least one or more of bandwidth information, delay information, and packet loss rate; The network status information is calculated using the Dijkstra algorithm to determine the target sending path.

[0025] In some embodiments of the present application, before transmitting data, the sending device sends a probe packet to the network to collect network status information of each node, including bandwidth, delay, packet loss rate, etc. Based on this real-time information, the Dijkstra algorithm is used to calculate an optimal transmission path to improve data transmission efficiency. Optionally, the sending module is configured to: Matching the unique identifier of the target data packet with a pre-stored data packet identifier list to obtain a matching result; According to the matching result, it is determined whether to retransmit the target data packet.

[0026] In some embodiments of the present application, after receiving a data packet, the receiving device first checks whether it is a duplicate based on the packet identifier. By maintaining a list of received data packet identifiers, the newly received data packet identifier is quickly compared. If a duplicate is found, the data packet is directly discarded and a confirmation message is sent to the sending end.

[0027] Optionally, the receiving module is configured to: The receiving device performs LDPC code verification on the target data packet received for the first time; If the verification is passed, the target data packet is stored and identification information of correct reception is returned to the sending end device; If the verification fails, the erroneous target data packet is recorded and error identification information is returned to the sending end device.

[0028] To improve data anti-interference capabilities, some embodiments of this application employ low-density parity-check (LDPC) codes to encode data packet content. LDPC codes offer robust error correction capabilities, effectively detecting and correcting bit errors during transmission while adding a small amount of redundant information.

[0029] Optionally, the receiving module is configured to: Judging the response data packet; If the response data packet is confirmation discard information, stop retransmitting the target data packet; If the response data packet is correct reception information, then continue to transmit other data packets.

[0030] In some embodiments of the present application, after the sending device receives the feedback message from the receiving device, if it is a confirmation discard message, it stops retransmitting the data packet; if it is a correct reception confirmation message, it continues to transmit subsequent data packets; if it is an error feedback message, it performs intelligent retransmission based on the error type and network status.

[0031] In a third aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the big data transmission method as described in any embodiment of the first aspect can be implemented.

[0032] In a fourth aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the big data transmission method as described in any embodiment of the first aspect.

[0033] In a fifth aspect, some embodiments of the present application provide a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it can implement the big data transmission method as described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the following is a brief introduction to the drawings required for use in some embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A flowchart of a large data transmission method provided in an embodiment of the present application; Figure 2 A flowchart of another big data transmission method provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a large data transmission device provided in an embodiment of the present application; Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0037] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0038] In current data transmission scenarios, whether based on wired or wireless networks, duplicate and erroneous transmissions are common. From a network architecture perspective, in complex network topologies, packets can be forwarded multiple times due to routing errors or network congestion, leading to duplicate transmissions. For example, in large enterprise networks, where multiple switches and routers work together, if routing tables are not updated promptly or conflicts occur, packets can easily be forwarded in loops, wasting network bandwidth and reducing overall transmission efficiency.

[0039] Transmission errors also frequently occur at the data link layer. Wireless networks are susceptible to environmental interference. Factors such as signal fading and noise can cause bit flips during data transmission, leading to transmission errors. For example, while 5G communications offer high theoretical transmission speeds, signals are easily obstructed and affected by reflections from tall buildings in urban environments, resulting in increased packet error rates. If the receiving end cannot accurately identify and correct these errors, data loss or application errors may result. Traditional data transmission protocols, such as TCP (Transmission Control Protocol), have certain retransmission mechanisms to handle errors. However, this mechanism is inefficient when faced with large numbers of repeated transmissions and complex error situations, and cannot meet today's strict requirements for high-speed and accurate data transmission. How to improve the accuracy and efficiency of data transmission is an urgent problem to be solved. In view of this, some embodiments of the present application provide a large data transmission method, which includes obtaining a target data packet, wherein the target data packet carries a unique identifier, which is encoded based on the timestamp of the target data packet, the address information of the sending device, and the data packet sequence number; sending the target data packet to a receiving device so that the receiving device determines the unique identifier of the target data packet and obtains a response data packet, which is used to determine whether to retransmit; receiving the response data packet returned by the receiving device, and performing data transmission based on the response data packet. The embodiments of the present application uniquely identify the data packet, and during the data transmission process between the sending device and the receiving device, determine the unique identifier of the data packet, and then determine whether retransmission is required. This optimizes the transmission process, strengthens data verification and intelligent retransmission mechanisms, effectively solves the problems of repeated transmission and erroneous transmission in the existing data transmission process, and significantly improves the accuracy and efficiency of data transmission.

[0040] like Figure 1As shown, an embodiment of the present application provides a large data transmission method, which is applied to a sending end device, and the method includes: S101. Acquire a target data packet, wherein the target data packet carries a unique identifier, which is encoded based on a timestamp of the target data packet, address information of a sending end device, and a data packet sequence number; Specifically, the embodiment of the present application is applied to a data transmission system, in which a sending end device and a receiving end device are sent, the sending end device sends data to the receiving end device, and the original data packet is split according to a preset rule to obtain split data packets, each of which has a unique identifier, wherein the preset rule can be split according to byte size or according to the number of data packets; When sending a data packet, the sending device obtains the sending time of the data packet, that is, the timestamp of the target data packet, and obtains the address information of the sending device. Each split data packet has its own data packet sequence number. The sending device encodes the timestamp of the target data packet, the address information of the sending device and the data packet sequence number to obtain a unique identifier for each target data packet.

[0041] S102, sending a target data packet to a receiving device, so that the receiving device determines the unique identifier of the target data packet and obtains a response data packet, where the response data packet is used to determine whether to retransmit; The sending device sends a target packet to the receiving device. After receiving the target packet, the receiving device determines the unique identifier of the target packet, obtains a determination result, and returns the determination result to the sending device as a response packet. The determination result includes at least correct reception information and loss confirmation information. If the information is correct, the subsequent data packets are transmitted. If the information is lost, the target packet is retransmitted.

[0042] S103: Receive a response data packet returned by the receiving device, and perform data transmission according to the response data packet.

[0043] Specifically, the sending end device receives the response data packet returned by the receiving end device, parses the response data packet, obtains the judgment result in the response data packet, and performs data transmission according to the judgment result.

[0044] Some embodiments of the present application uniquely identify the data packet, and during the data transmission process between the sending device and the receiving device, determine the unique identifier of the data packet, and then determine whether retransmission is required, thereby optimizing the transmission process, strengthening data verification and intelligent retransmission mechanism, effectively solving the problems of repeated transmission and erroneous transmission in the existing data transmission process, and significantly improving the accuracy and efficiency of data transmission.

[0045] Another embodiment of the present application further supplements the big data transmission method provided in the above embodiment.

[0046] Optionally, the unique identifier is obtained as follows: Perform an XOR operation on the timestamp of the target data packet and the address information of the sending device to obtain an XOR result; Concatenate the XOR result and the data packet sequence number to obtain the unique identifier of the target data packet.

[0047] Specifically, if Figure 2 As shown, the sending device uniquely encodes each data packet using a hybrid encoding scheme that combines a timestamp, the sending device's address (i.e., the sending IP address), and the packet sequence number. For example, the timestamp, accurate to the millisecond level, is XORed with the binary form of the sending IP address and then concatenated with the incrementing packet sequence number to generate a unique packet identifier. This uniquely identifies each data packet throughout its transmission, facilitating subsequent identification and tracking. Furthermore, to improve data robustness against interference, the data packet content is encoded using a low-density parity-check (LDPC) code. LDPC codes offer robust error correction capabilities, effectively detecting and correcting bit errors during transmission while adding a small amount of redundant information.

[0048] Some embodiments of this application use a unique identifier for data packets. This unique identifier, along with a duplicate detection mechanism at the receiving end, can accurately identify and discard duplicate data packets. Compared to traditional transmission methods, this reduces the duplicate transmission rate by over 80%, significantly saving network bandwidth resources and improving transmission efficiency. In large file transfer scenarios, this can significantly shorten transmission time.

[0049] Optionally, before obtaining the target data packet, the method further includes: Send a test packet to the receiving device; Receive the response packet returned by the receiving device and determine the target sending path based on the response packet; the target sending path is used to send the target data packet.

[0050] Some embodiments of the present application use dynamic network maps and optimal path selection to ensure that data is always transmitted on the best path, reducing transmission delay by an average of 30%, meeting application scenarios with high real-time requirements.

[0051] Optionally, determining a target sending path according to the response packet includes: Obtaining network status information in the response data packet, the network status information including at least one or more of bandwidth information, delay information, and packet loss rate; The Dijkstra algorithm is used to calculate the network status information and determine the target sending path.

[0052] In some embodiments of the present application, before transmitting data, the sending device sends a probe packet to the network to collect network status information of each node, including bandwidth, delay, packet loss rate, etc. Based on this real-time information, the Dijkstra algorithm is used to calculate an optimal transmission path to improve data transmission efficiency. Optionally, the receiving device determines the unique identifier of the target data packet and obtains a response data packet, where the response data packet is used to determine whether to retransmit, including: Matching the unique identifier of the target data packet with a pre-stored data packet identifier list to obtain a matching result; According to the matching result, it is determined whether to retransmit the target data packet.

[0053] In some embodiments of the present application, after receiving a data packet, the receiving device first checks whether it is a duplicate based on the packet identifier. By maintaining a list of received data packet identifiers, the newly received data packet identifier is quickly compared. If a duplicate is found, the data packet is directly discarded and a confirmation message is sent to the sending end.

[0054] Optionally, the method further includes: The receiving device performs LDPC code verification on the target data packet received for the first time; If the verification is passed, the target data packet is stored and the identification information of correct reception is returned to the sending device; If the verification fails, the erroneous target data packet is recorded and error identification information is returned to the sending device.

[0055] Specifically, after the receiving device receives the target data packet, it first checks whether it is a duplicate reception based on the identifier of the target data packet. A data packet identifier list is stored on the receiving device, and the newly received data packet identifier and the data packet identifier list are quickly compared. If a duplicate is found, that is, the data packet identifier list contains the unique identifier of the target data packet, it is directly discarded and a confirmation discard message is sent to the sending device.

[0056] For the first received data packet, an LDPC code check is performed. If the check passes, the data packet is stored in the database and a confirmation message containing the correct received data packet is returned to the sending device. If the check finds an error, the error packet identifier is recorded and a feedback message containing the error packet identifier and error type (such as bit flip location information) is sent to the sending device.

[0057] To improve data anti-interference capabilities, some embodiments of this application employ low-density parity-check (LDPC) codes to encode data packet content. LDPC codes offer robust error correction capabilities, effectively detecting and correcting bit errors during transmission while adding a small amount of redundant information.

[0058] Optionally, receiving a response data packet returned by the receiving device and performing data transmission according to the response data packet includes: Judge the response data packet; If the response data packet is a confirmation discard message, then retransmission of the target data packet is stopped; If the response data packet is correct reception information, the target data packet will continue to be retransmitted.

[0059] Specifically, after the transmitting device receives the feedback message, i.e., the response data packet, from the receiving device, if the response data packet is a confirmation discard message, it stops retransmitting the target data packet; if the response data packet is a correct reception confirmation message, it continues to transmit subsequent data packets; If the error is a feedback message, including a response packet that confirms discard, intelligent retransmission is performed based on the error type and network status. If the error is a small number of bits and the current network bandwidth is sufficient, the sender uses forward error correction (FEC) technology to add more redundant error correction information to the retransmitted packet to correct the error in one go. If the error is more serious or the network status is poor, the transmission path is reselected and retransmission is performed.

[0060] In some embodiments of the present application, after the sending device receives the feedback message from the receiving device, if it is a confirmation discard message, it stops retransmitting the data packet; if it is a correct reception confirmation message, it continues to transmit subsequent data packets; if it is an error feedback message, it performs intelligent retransmission based on the error type and network status.

[0061] It should be noted that each implementable method in this embodiment can be implemented separately, or can be implemented in combination in any combination without conflict, and this application does not limit it.

[0062] Another embodiment of the present application provides a big data transmission device for executing the big data transmission method provided in the above embodiment.

[0063] like Figure 3 , which is a schematic diagram of the structure of a large data transmission device provided in an embodiment of the present application, and is applied to a receiving device. The large data transmission device includes an acquisition module 301, a sending module 302, and a receiving module 303, wherein: The acquisition module 301 is used to acquire a target data packet, wherein the target data packet carries a unique identifier, which is obtained by encoding the timestamp of the target data packet, the address information of the sending end device, and the data packet sequence number; The sending module 302 is used to send a target data packet to a receiving device, so that the receiving device can determine the unique identifier of the target data packet and obtain a response data packet, which is used to determine whether to retransmit; The receiving module 303 is configured to receive a response data packet returned by the receiving device and perform data transmission according to the response data packet.

[0064] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method and will not be elaborated here.

[0065] Some embodiments of the present application uniquely identify the data packet, and during the data transmission process between the sending device and the receiving device, determine the unique identifier of the data packet, and then determine whether retransmission is required, thereby optimizing the transmission process, strengthening data verification and intelligent retransmission mechanism, effectively solving the problems of repeated transmission and erroneous transmission in the existing data transmission process, and significantly improving the accuracy and efficiency of data transmission.

[0066] Another embodiment of the present application provides further supplementary explanation of the big data transmission device provided in the above embodiment.

[0067] Optionally, the unique identifier is obtained as follows: Perform an XOR operation on the timestamp of the target data packet and the address information of the sending device to obtain an XOR result; Concatenate the XOR result and the data packet sequence number to obtain the unique identifier of the target data packet.

[0068] Some embodiments of this application use a unique identifier for data packets. This unique identifier, along with a duplicate detection mechanism at the receiving end, can accurately identify and discard duplicate data packets. Compared to traditional transmission methods, this reduces the duplicate transmission rate by over 80%, significantly saving network bandwidth resources and improving transmission efficiency. In large file transfer scenarios, this can significantly shorten transmission time.

[0069] Optionally, obtain a module for: Send a test packet to the receiving device; Receive the response packet returned by the receiving device and determine the target sending path based on the response packet; the target sending path is used to send the target data packet.

[0070] Some embodiments of the present application use dynamic network maps and optimal path selection to ensure that data is always transmitted on the best path, reducing transmission delay by an average of 30%, meeting application scenarios with high real-time requirements.

[0071] Optionally, obtain a module for: Obtaining network status information in the response data packet, the network status information including at least one or more of bandwidth information, delay information, and packet loss rate; The Dijkstra algorithm is used to calculate the network status information and determine the target sending path.

[0072] In some embodiments of the present application, before transmitting data, the sending device sends a probe packet to the network to collect network status information of each node, including bandwidth, delay, packet loss rate, etc. Based on this real-time information, the Dijkstra algorithm is used to calculate an optimal transmission path to improve data transmission efficiency. Optionally, the sending module is configured to: Matching the unique identifier of the target data packet with a pre-stored data packet identifier list to obtain a matching result; According to the matching result, it is determined whether to retransmit the target data packet.

[0073] In some embodiments of the present application, after receiving a data packet, the receiving device first checks whether it is a duplicate based on the packet identifier. By maintaining a list of received data packet identifiers, the newly received data packet identifier is quickly compared. If a duplicate is found, the data packet is directly discarded and a confirmation message is sent to the sending end.

[0074] Optionally, the receiving module is configured to: The receiving device performs LDPC code verification on the target data packet received for the first time; If the verification is passed, the target data packet is stored and the identification information of correct reception is returned to the sending device; If the verification fails, the erroneous target data packet is recorded and error identification information is returned to the sending device.

[0075] To improve data anti-interference capabilities, some embodiments of this application employ low-density parity-check (LDPC) codes to encode data packet content. LDPC codes offer robust error correction capabilities, effectively detecting and correcting bit errors during transmission while adding a small amount of redundant information.

[0076] Optionally, the receiving module is configured to: Judge the response data packet; If the response data packet is a confirmation discard message, then retransmission of the target data packet is stopped; If the response data packet is correct reception information, the target data packet will continue to be retransmitted.

[0077] In some embodiments of the present application, after the sending device receives the feedback message from the receiving device, if it is a confirmation discard message, it stops retransmitting the data packet; if it is a correct reception confirmation message, it continues to transmit subsequent data packets; if it is an error feedback message, it performs intelligent retransmission based on the error type and network status.

[0078] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method and will not be elaborated here.

[0079] It should be noted that each implementable method in this embodiment can be implemented separately, or can be implemented in combination in any combination without conflict, and this application does not limit it.

[0080] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the operation of the method corresponding to any embodiment of the big data transmission method provided in the above embodiments can be implemented.

[0081] An embodiment of the present application further provides a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any embodiment of the big data transmission method provided in the above embodiments.

[0082] like Figure 4 As shown, some embodiments of the present application provide an electronic device 400, which includes: a memory 410, a processor 420, and a computer program stored in the memory 410 and executable on the processor 420, wherein the processor 420 can implement a method of any embodiment of the above-mentioned large data transmission method when reading the program from the memory 410 through the bus 430 and executing the program.

[0083] Processor 420 can process digital signals and can include various computing architectures, such as a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements a combination of multiple instruction sets. In some examples, processor 420 can be a microprocessor.

[0084] The memory 410 can be used to store instructions executed by the processor 420 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all functions of one or more modules described in the embodiments of this application. The processor 420 of the embodiment of the present disclosure can be used to execute the instructions in the memory 410 to implement the method shown above. The memory 410 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memory known to those skilled in the art.

[0085] The above are merely examples of the present application and are not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application. It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0086] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0087] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for transmitting large amounts of data, characterized in that: Applied to a transmitting device, the method includes: Acquire a target data packet, wherein the target data packet carries a unique identifier, and the unique identifier is encoded according to a timestamp of the target data packet, address information of a sending end device, and a data packet sequence number; Sending the target data packet to a receiving device, so that the receiving device determines the unique identifier of the target data packet and obtains a response data packet, wherein the response data packet is used to determine whether to retransmit; Receive a response data packet returned by the receiving device, and perform data transmission according to the response data packet.

2. The big data transmission method according to claim 1, characterized in that: The unique identifier is obtained as follows: Performing an XOR operation on the timestamp of the target data packet and the address information of the sending end device to obtain an XOR result; The XOR result and the data packet sequence number are concatenated to obtain a unique identifier for the target data packet.

3. The big data transmission method according to claim 1, characterized in that: Before obtaining the target data packet, the method further includes: Send a test packet to the receiving device; Receive a response packet returned by the receiving device, and determine a target sending path based on the response packet; the target sending path is used to send the target data packet.

4. The big data transmission method according to claim 3, characterized in that: The step of determining a target sending path according to the response packet includes: Obtaining network status information in the response data packet, the network status information being at least one or more of bandwidth information, delay information, and packet loss rate; The network status information is calculated using the Dijkstra algorithm to determine the target sending path.

5. The big data transmission method according to claim 1, characterized in that: The receiving end device determines the unique identifier of the target data packet to obtain a response data packet, where the response data packet is used to determine whether to retransmit, including: Matching the unique identifier of the target data packet with a pre-stored data packet identifier list to obtain a matching result; According to the matching result, it is determined whether to retransmit the target data packet.

6. The big data transmission method according to claim 1, characterized in that: The method further comprises: The receiving device performs LDPC code verification on the target data packet received for the first time; If the verification is passed, the target data packet is stored and identification information of correct reception is returned to the sending end device; If the verification fails, the erroneous target data packet is recorded and error identification information is returned to the sending end device.

7. The big data transmission method according to claim 1, characterized in that: The receiving of a response data packet returned by the receiving end device and performing data transmission according to the response data packet includes: Judging the response data packet; If the response data packet is confirmation discard information, stop retransmitting the target data packet; If the response data packet is correct reception information, then continue to transmit other data packets.

8. A large data transmission device, characterized in that: Applied to a receiving device, the apparatus comprises: An acquisition module is used to acquire a target data packet, wherein the target data packet carries a unique identifier, and the unique identifier is obtained by encoding the timestamp of the target data packet, the address information of the sending end device, and the data packet sequence number; a sending module, configured to send the target data packet to a receiving device, so that the receiving device determines the unique identifier of the target data packet and obtains a response data packet, wherein the response data packet is used to determine whether to retransmit; The receiving module is used to receive the response data packet returned by the receiving device and perform data transmission according to the response data packet.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor can implement the big data transmission method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the program is executed by a processor, the method for transmitting big data according to any one of claims 1 to 7 can be implemented.

Images