A data transmission method and a data transmission system for remote takeover of unmanned mining trucks

Abstract

The present invention discloses a data transmission method and a data transmission system for remote takeover of unmanned mining trucks, belonging to the technical field of data transmission. When the data transmission method is executed by the sending end, it includes: obtaining the data to be transmitted and its data type; in response to the data type of the data to be transmitted being video data, encoding and encapsulating the video data in sequence to obtain the original data packet and FEC redundant packet; sending the original data packet and FEC redundant packet to the receiving end; based on the retransmission request sent by the receiving end, sending the original data packet specified by the retransmission request to the receiving end; wherein, the encoding code rate and the transmission code rate sent to the receiving end are obtained by adjusting the code rate by the GCC congestion control algorithm according to the prediction result output by the pre-trained network prediction model, the actual delay sent by the receiving end, and the bandwidth detection result. The present invention can ensure the real-time performance of data transmission and improve the security and controllability of remote takeover.

Description

Technical Field

[0001] The invention relates to a data transmission method and a data transmission system for remote takeover of an unmanned mining truck, belonging to the technical field of data transmission. Background Art

[0002] With the widespread application of 5G private network technology in the field of unmanned open-pit mines, the characteristics of high bandwidth, low latency and high reliability provide guarantees for remote takeover. The remote takeover of unmanned vehicles is used when there is a problem with the vehicle. The driver takes over in time to eliminate the vehicle abnormality and avoid harm to personnel and other vehicles. 5G private networks have the characteristics of relatively fixed network nodes and network links, and the network can adapt to direct connection or NAT traversal scenarios.

[0003] 5G private networks provide security strategies that can offer ideas for simplifying transmission methods and further reducing latency. At the same time, in open-pit mine unmanned driving scenarios under 5G private networks, there are also problems such as multiple vehicles competing for the same base station, frequency band, accessing cells with poor signals, blocked signals, poor air interface quality, and excessive load on base stations and network equipment. Given the weak network conditions of 5G private networks, how to ensure real-time data transmission while taking into account video quality and control experience is an issue that needs to be addressed urgently.

[0004] The patent with publication number CN115052209A discloses a 5G remote takeover and control system, which realizes ultra-low latency data transmission through an end-to-end multi-path automatic optimization algorithm. The system provided by the patent does not optimize data transmission using network technologies unique to 5G scenarios, such as end-to-end network slicing, MEC, security policies, etc. The multi-path optimization algorithm proposed in the patent requires network detection for switching. First, the switching cost is very high. Second, if all channels have weak network problems, the algorithm still faces problematic paths and fails to optimize transmission performance from the transmission mechanism, such as congestion control and resistance to weak networks. In addition, data transmission needs to balance cost, delay and quality. Low delay does not meet the needs of all scenarios. Low delay will increase the cost. Data types are not classified. For example, video data transmission and other data have different characteristics, and the transmission methods and optimization strategies are also different.

[0005] The patent with publication number CN114355866A discloses an unmanned remote takeover system, which uses MEC data forwarding to avoid network resource occupation and uses the MPTCP protocol to improve transmission efficiency and throughput. The MPTCP protocol is implemented in the Linux kernel. The cost of deploying, upgrading and optimizing the protocol in the device is high, the throughput model is simple, and it cannot be applied to common scenarios. For example, in wireless environments and networks with long paths, a path with a high packet loss rate will be selected, increasing the retransmission rate and causing a decrease in network quality. At the same time, the lack of multi-path congestion control mechanism optimization will amplify the impact of TCP congestion control under multi-path. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a data transmission method and a data transmission system for remote takeover of unmanned mining trucks, which can ensure the real-time performance of data transmission and improve the safety and controllability of remote takeover. To achieve the above purpose, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a data transmission method, which is performed by a sending end and includes:

[0008] Get the data to be transmitted and its data type;

[0009] In response to the data type of the data to be transmitted being video data, encoding processing and encapsulation processing are sequentially performed on the video data to obtain an original data packet and an FEC redundant packet;

[0010] Send the original data packet and FEC redundant packet to the receiving end;

[0011] Based on the retransmission request sent by the receiving end, the original data packet specified by the retransmission request is sent to the receiving end;

[0012] The encoding bit rate and the sending bit rate to the receiving end are obtained by adjusting the bit rate using the GCC congestion control algorithm based on the prediction results output by the pre-trained network prediction model, the actual delay sent by the receiving end, and the bandwidth detection results.

[0013] In combination with the first aspect, optionally, the input of the pre-trained network prediction model is the actual delay sent by the receiving end in the current time period, and the output result is the predicted delay in the next time period.

[0014] In combination with the first aspect, optionally, the GCC congestion control algorithm includes an overload detector and a rate controller, and the rate controller includes a finite state machine and a rate outputter;

[0015] The encoding bit rate and the transmission bit rate sent to the receiving end are obtained by adjusting the bit rate according to the prediction result output by the pre-trained network prediction model and the bandwidth detection result by the GCC congestion control algorithm, including:

[0016] The overload detector determines the network status according to the predicted delay of the next time period output by the pre-trained network prediction model, and outputs the network status of the next time period; the network status is an overload state, an underload state or a normal state;

[0017] In response to the network state of the next time period being different from the network state of the current time period, triggering the finite state machine;

[0018] The finite state machine adjusts the bandwidth according to the network status in the next time period and outputs the estimated bandwidth, including:

[0019] Get the maximum bandwidth and current bandwidth of the bandwidth detection result, and calculate the receiving bit rate based on the current bandwidth;

[0020] Calculate the first bandwidth threshold = maximum bandwidth + 3 * maximum bandwidth standard deviation, and calculate the second bandwidth threshold = maximum bandwidth - 3 * maximum bandwidth standard deviation;

[0021] When the state of the finite state machine in the next time period is an ascending state, the bandwidth is increased, including:

[0022] If the received bit rate is greater than the first bandwidth threshold or the received bit rate is less than the second bandwidth threshold, the predicted bandwidth is increased by multiplication, and the estimated bandwidth is the current bandwidth increasing by 8% per second;

[0023] If the second bandwidth threshold ≤ the receiving bit rate ≤ the first bandwidth threshold, the predicted bandwidth is increased by additive increase, and the estimated bandwidth is the current bandwidth plus half the size of the original data packet every RTT time;

[0024] When the state of the finite state machine in the next time period is a decreasing state, the bandwidth is reduced, and the estimated bandwidth is 0.85*receiving bit rate;

[0025] When the state of the finite state machine in the next time period is normal, the estimated bandwidth is the current bandwidth;

[0026] The code rate outputter outputs the corresponding code rate according to the estimated bandwidth output by the finite state machine.

[0027] In combination with the first aspect, optionally, the pre-trained network prediction model is obtained by the following steps:

[0028] Initialize the original sequence for:

[0029] ,

[0030] in, The first individual data;

[0031] The original sequence is smoothed and the smoothed sequence is obtained after wavelet decomposition and reconstruction. for:

[0032] ,

[0033] in, is the first individual data;

[0034] The accumulated sequence obtained by accumulating the smoothed sequence once for:

[0035] ,

[0036] in, is the first The data are expressed by the following formula:

[0037] ,

[0038] in, is the data in the smoothed series;

[0039] Initialize neighbor value generation sequence for:

[0040] ,

[0041] in, Generate the first The data are expressed by the following formula:

[0042] ,

[0043] in, For taking Indicates the first The coefficient of the data;

[0044] According to the definition of cumulative sequence, we get:

[0045] ;

[0046] After bringing in It is expressed as:

[0047] ;

[0048] Before use Forecast No. Item, then is known, after adjustment, we get the grey differential equation:

[0049] ,

[0050] in, is the gray action, is the development coefficient, For time;

[0051] Pick ,but:

[0052] ;

[0053] The matrix representation is:

[0054] ,

[0055] in:

[0056] ;

[0057] Use the least squares method to get the parameters and Estimated value:

[0058] ,

[0059] in, For the parameter and The matrix of estimated values ​​of ;

[0060] Pick ,but:

[0061] ;

[0062] According to Leibniz formula and the geometric meaning of definite integral, we get:

[0063] ;

[0064] Then the whitening equation is:

[0065] ,

[0066] Solving the differential equations yields the predictive cumulative model:

[0067] ,

[0068] in, is the base of natural logarithms;

[0069] Use the cumulative reduction formula to restore the predicted value:

[0070] ;

[0071] Compensate the residuals and establish the GM(1,1) model to obtain the predicted values. The residual sequence is:

[0072] ,

[0073] in:

[0074] ;

[0075] right Taking the absolute value, we get the sequence:

[0076] ,

[0077] in:

[0078] ,

[0079] right Establish a grey model and get the residual prediction value:

[0080] ,

[0081] in, is the development coefficient of the residual value, is the grey action of the residual value;

[0082] For the predicted value After correction, we get:

[0083] ;

[0084] Use Markov chain to correct the prediction results and improve the prediction accuracy. According to the deviation, it is divided into states, establish the state transfer matrix :

[0085] ,

[0086] in, Indicates from the state To status The one-step transition probability is expressed as follows:

[0087] ,

[0088] in, Indicates status Transfer to state The number of times Indicates status Total number of;

[0089] The upper limit of the residual value is , the lower limit of the residual value is , the prediction model is corrected as follows:

[0090] ,

[0091] in, Status The initial state vector.

[0092] In combination with the first aspect, optionally, when the data type of the data to be transmitted is non-video data, the method includes:

[0093] Encapsulate the acquired non-video data to obtain the original data packet and FEC redundant packet;

[0094] Use a sliding window to send the original data packet and FEC redundant packet to the receiving end;

[0095] Based on the retransmission request sent by the receiving end, the original data packet and the FEC redundant packet specified by the retransmission request are sent to the receiving end using a sliding window;

[0096] Among them, the sending window and sending rate of the sliding window are obtained by the BBR congestion control algorithm through congestion window control and sending rate adjustment.

[0097] In a second aspect, the present invention provides a data transmission method, which is performed by a receiving end and includes:

[0098] Perform packet loss detection on the acquired original data packets and FEC redundant packets to obtain the packet loss location;

[0099] Send the actual delay to the sender;

[0100] Prioritize the packet loss positions, and use the packet loss positions with a priority higher than a preset first priority threshold as the packet loss positions of the original data packets to be retransmitted in the retransmission request;

[0101] Send a retransmission request to the sender.

[0102] In combination with the second aspect, optionally, when the data type corresponding to the acquired original data packet and the FEC redundant packet is video data, decoding and displaying are performed, including:

[0103] In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet;

[0104] Decapsulating and assembling the restored data packets and the original data packets to obtain video frames;

[0105] Prioritize the video frames, delete the video frames whose priorities are lower than a preset second priority threshold, and sort and assemble the remaining video frames to obtain a picture group;

[0106] The picture groups are sorted based on the time sequence to obtain a picture sequence. When the sorting reaches a buffer threshold, a decodable picture sequence is output, and the decodable picture sequence is decoded and then displayed.

[0107] In combination with the second aspect, optionally, when the data types corresponding to the acquired original data packet and the FEC redundant packet are non-video data, splicing is performed, including:

[0108] In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet;

[0109] Decapsulating the restored data packet and the original data packet to obtain decapsulated non-video data;

[0110] The decapsulated non-video data is spliced ​​according to preset categories to obtain non-video data that can be used.

[0111] In a third aspect, the present invention provides a data transmission system for remote takeover of unmanned mining trucks, including a vehicle and a remote cab;

[0112] The vehicle comprises a sending computing unit and a CPE, a camera and a sensor component which are communicatively connected to the sending computing unit, wherein the sending computing unit is configured to execute the method according to the first aspect, wherein the CPE is used to send an original data packet and an FEC redundant packet, receive a retransmission request and an actual delay sent by a receiving end, and perform bandwidth detection, wherein the camera is used to collect video data, and the sensor component is used to collect non-video data;

[0113] The remote cab includes a receiving computing unit and an MEC and a remote takeover platform to which the receiving computing unit is communicatively connected. The receiving computing unit is configured to execute the method described in the second aspect. The MEC is used to send retransmission requests and actual delays and receive original data packets and FEC redundant packets. The remote takeover platform is used to remotely take over the unmanned mining truck based on video data and non-video data.

[0114] In combination with the third aspect, optionally, a 5G base station is also included, and the CPE and the MEC are communicatively connected through the 5G base station.

[0115] Compared with the prior art, the data transmission method and the data transmission system for remote takeover of unmanned mining trucks provided by the embodiments of the present invention have the following beneficial effects:

[0116] In response to the data type of the data to be transmitted being video data, the present invention sequentially encodes and encapsulates the video data to obtain an original data packet and an FEC redundant packet; the original data packet and the FEC redundant packet are sent to a receiving end; based on a retransmission request sent by the receiving end, the original data packet specified by the retransmission request is sent to the receiving end; the encoding bit rate and the transmission bit rate sent to the receiving end are obtained by adjusting the bit rate by a GCC congestion control algorithm according to a prediction result output by a pre-trained network prediction model, an actual delay sent by the receiving end, and a bandwidth detection result; the present invention provides a video data transmission protocol, adds a prediction result output by a network prediction model as a criterion of a GCC congestion control algorithm, and uses the GCC congestion control algorithm to accurately identify the network state, improve the encoding bit rate and the transmission bit rate when the network is congested, and prevent the bit rate from dropping too much to cause a decrease in video quality when the network is not congested; the present invention provides a high-priority, low-latency, and weak-network-resistant transmission mechanism for video data;

[0117] When the data type of the data to be transmitted is non-video data, the present invention includes: encapsulating the acquired non-video data to obtain an original data packet and an FEC redundant packet; using a sliding window to send the original data packet and the FEC redundant packet to a receiving end; based on a retransmission request sent by the receiving end, using a sliding window to send the original data packet and the FEC redundant packet specified by the retransmission request to the receiving end; wherein the sending window and the sending rate of the sliding window are obtained by performing congestion window control and sending rate adjustment by a BBR congestion control algorithm; the present invention provides a non-video data transmission protocol, and uses the BBR congestion control algorithm to provide a reliable transmission mechanism with low latency and resistance to weak networks for non-video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 This is a flowchart executed by a sending end in a data transmission method in Embodiment 1 of the present invention;

[0119] Figure 2 It is a structural diagram of a network prediction model in a data transmission method in Example 1 of the present invention;

[0120] Figure 3 This is a flowchart executed by a receiving end in a data transmission method in Embodiment 2 of the present invention;

[0121] Figure 4 is a flowchart of video data transmission in a data transmission method in embodiment 3 of the present invention;

[0122] Figure 5 This is a flowchart of non-video data transmission in a data transmission method in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0123] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0124] Embodiment 1:

[0125] like Figure 1 As shown, this embodiment provides a data transmission method, which is executed by a sending end and includes:

[0126] Get the data to be transmitted and its data type;

[0127] In response to the data type of the data to be transmitted being video data, encoding processing and encapsulation processing are sequentially performed on the video data to obtain an original data packet and an FEC redundant packet;

[0128] Send the original data packet and FEC redundant packet to the receiving end;

[0129] Based on the retransmission request sent by the receiving end, the original data packet specified by the retransmission request is sent to the receiving end;

[0130] The encoding bit rate and the sending bit rate to the receiving end are obtained by adjusting the bit rate using the GCC congestion control algorithm based on the prediction results output by the pre-trained network prediction model, the actual delay sent by the receiving end, and the bandwidth detection results.

[0131] In this embodiment, the input of the pre-trained network prediction model is the actual delay sent by the receiving end in the current time period, and the output result is the predicted delay in the next time period.

[0132] As a further improvement, the input of the pre-trained network prediction model is the air interface data of the CPE at the sending end in the current time period, and the output result is the predicted air interface data for the next time period. The input of the pre-trained network prediction model is the actual packet loss rate sent by the receiving end in the current time period, and the output result is the predicted packet loss rate for the next time period.

[0133] The GCC congestion control algorithm includes an overload detector and a code rate controller, wherein the code rate controller includes a finite state machine and a code rate outputter.

[0134] The encoding bit rate and the sending bit rate to the receiving end are obtained by adjusting the bit rate based on the prediction results output by the pre-trained network prediction model and the bandwidth detection results by the GCC congestion control algorithm, including:

[0135] Step 1: The overload detector determines the network status according to the predicted delay of the next time period output by the pre-trained network prediction model, and outputs the network status of the next time period.

[0136] The overload detector performs congestion judgment, and the network state obtained by the judgment includes an overload state, an underload state or a normal state.

[0137] Specifically, the predicted air interface data and predicted packet loss rate in the next time period can provide a basis for judging the network status.

[0138] As a further improvement, RSRP and SINR are used for air interface data to establish an air interface quality evaluation model: v=(4-SINR)+a(80-RSRP)+b, where v is the air interface quality evaluation result, and a and b are coefficients; the network status is judged based on the air interface quality evaluation result.

[0139] Step 2: In response to the network state in the next time period being different from the network state in the current time period, a finite state machine is triggered.

[0140] If the network state at the previous moment is the same as the network state at the current moment, the finite state machine will not be triggered.

[0141] Step 3: The finite state machine adjusts the bandwidth according to the network status in the next time period and outputs the estimated bandwidth.

[0142] Step 3.1: Obtain the maximum bandwidth and current bandwidth of the bandwidth detection result, and calculate the receiving bit rate based on the current bandwidth.

[0143] Step 3.2: Calculate the first bandwidth threshold = maximum bandwidth + 3 * maximum bandwidth standard deviation, and calculate the second bandwidth threshold = maximum bandwidth - 3 * maximum bandwidth standard deviation.

[0144] Step 3.3-1: When the state of the finite state machine in the next time period is an ascending state, increase the bandwidth, including:

[0145] If the received bit rate is greater than the first bandwidth threshold or the received bit rate is less than the second bandwidth threshold, the predicted bandwidth is increased by multiplication, and the estimated bandwidth is the current bandwidth increasing by 8% per second;

[0146] If the second bandwidth threshold ≤ the receiving bit rate ≤ the first bandwidth threshold, the predicted bandwidth is increased using additive increase, and the estimated bandwidth is the current bandwidth that will increase by half the size of the original data packet every RTT time.

[0147] When the state of the finite state machine in the next time period is a decreasing state, the bandwidth is reduced, and the estimated bandwidth is 0.85*receiving bit rate;

[0148] When the state of the finite state machine in the next time period is normal, the estimated bandwidth is the current bandwidth;

[0149] The code rate outputter outputs the corresponding code rate according to the estimated bandwidth output by the finite state machine.

[0150] Step 3.3-2: When the state of the finite state machine in the next time period is a decreasing state, the bandwidth is reduced, and the estimated bandwidth is 0.85*receiving bit rate.

[0151] Step 3.3-3: When the state of the finite state machine in the next time period is normal, the estimated bandwidth is the current bandwidth. When the state of the finite state machine in the next time period is normal, the estimated bandwidth is the current bandwidth.

[0152] Step 3.4: The code rate output device outputs the corresponding code rate according to the estimated bandwidth output by the finite state machine.

[0153] As a further improvement, the network prediction model determines the cause of packet loss based on the predicted packet loss rate in the next time period, and uses a decision tree to distinguish between random packet loss and congestion packet loss. For example, if the error between the actual packet loss rate and the predicted packet loss rate is within the threshold obtained by expert experience, it is random packet loss. If it is random packet loss, packet loss compensation is performed. If it is congestion packet loss, congestion control is required to adjust the encoding bit rate and the sending bit rate.

[0154] Specifically, the encoding bit rate and the sending bit rate to the receiving end are obtained by the GCC congestion control algorithm based on the predicted packet loss rate and bandwidth detection results output by the pre-trained network prediction model, and then the bit rate is adjusted (the encoding bit rate and sending bit rate based on packet loss).

[0155] Based on further improvements, the minimum value of the delay-based and packet loss-based encoding bitrate and the sending bitrate is output.

[0156] like Figure 2 As shown in Figure 1, the pre-trained network prediction model is obtained through the following steps:

[0157] Step 1: Get the raw data.

[0158] Initialize the original sequence for:

[0159] ,

[0160] in, The first data.

[0161] Step 2: Wavelet decomposition and reconstruction.

[0162] The original sequence is smoothed and the smoothed sequence is obtained after wavelet decomposition and reconstruction. for:

[0163] ,

[0164] in, is the first data.

[0165] The accumulated sequence obtained by accumulating the smoothed sequence once for:

[0166] ,

[0167] in, is the first The data are expressed by the following formula:

[0168] ,

[0169] in, is the data in the smoothed series.

[0170] Step 3: Establish a grey prediction model.

[0171] Initialize neighbor value generation sequence for:

[0172] ,

[0173] in, Generate the first The data are expressed by the following formula:

[0174] ,

[0175] in, For taking Indicates the first The coefficient of the data;

[0176] According to the definition of cumulative sequence, we get:

[0177] ;

[0178] After bringing in It is expressed as:

[0179] ;

[0180] Before use Forecast No. Item, then is known, after adjustment, we get the grey differential equation:

[0181] ,

[0182] in, is the gray action, is the development coefficient, For time;

[0183] Pick ,but:

[0184] ;

[0185] The matrix representation is:

[0186] ,

[0187] in:

[0188] ;

[0189] Use the least squares method to get the parameters and Estimated value:

[0190] ,

[0191] in, For the parameter and The matrix of estimated values ​​of ;

[0192] Pick ,but:

[0193] ;

[0194] According to Leibniz formula and the geometric meaning of definite integral, we get:

[0195] ;

[0196] Then the whitening equation is:

[0197] ,

[0198] Solving the differential equations yields the predictive cumulative model:

[0199] ,

[0200] in, is the base of natural logarithms;

[0201] Use the cumulative reduction formula to restore the predicted value:

[0202] .

[0203] Step 4: Establish the grey prediction model of residual.

[0204] Compensate the residuals and establish the GM(1,1) model to obtain the predicted values. The residual sequence is:

[0205] ,

[0206] in:

[0207] ;

[0208] right Taking the absolute value, we get the sequence:

[0209] ,

[0210] in:

[0211] ,

[0212] right Establish a grey model and get the residual prediction value:

[0213] ,

[0214] in, is the development coefficient of the residual value, is the grey action of the residual value;

[0215] For the predicted value After correction, we get:

[0216] .

[0217] Step 5: Use the Markov chain to correct the prediction results.

[0218] Use Markov chain to correct the prediction results and improve the prediction accuracy. According to the deviation, it is divided into states, establish the state transfer matrix :

[0219] ,

[0220] in, Indicates from the state To status The one-step transition probability is expressed as follows:

[0221] ,

[0222] in, Indicates status Transfer to state The number of times Indicates status The total number of .

[0223] Step 6: Use the sample input to adjust the parameters and obtain the pre-trained network prediction model.

[0224] The upper limit of the residual value is , the lower limit of the residual value is , the prediction model is corrected as follows:

[0225] ,

[0226] in, Status The initial state vector.

[0227] This embodiment provides a video data transmission protocol, adds the prediction result output by the network prediction model as the criterion of the GCC congestion control algorithm, uses the GCC congestion control algorithm to accurately identify the network status, improves the encoding bit rate and the sending bit rate when the network is congested, and prevents the bit rate from dropping too much to cause the video quality to drop when the network is not congested; the present invention provides a high-priority, low-latency, and weak network-resistant transmission mechanism for video data.

[0228] like Figure 1 As shown, when the data type of the data to be transmitted is non-video data, it includes:

[0229] Encapsulate the acquired non-video data to obtain the original data packet and FEC redundant packet;

[0230] Use a sliding window to send the original data packet and FEC redundant packet to the receiving end;

[0231] Based on the retransmission request sent by the receiving end, the original data packet and the FEC redundant packet specified by the retransmission request are sent to the receiving end using a sliding window;

[0232] Among them, the sending window and sending rate of the sliding window are obtained by the BBR congestion control algorithm through congestion window control and sending rate adjustment.

[0233] This embodiment provides a non-video data transmission protocol, and uses the BBR congestion control algorithm to provide a low-latency, reliable transmission mechanism for non-video data that is resistant to weak networks.

[0234] Embodiment 2:

[0235] like Figure 3 As shown, this embodiment provides a data transmission method, which is executed by a receiving end and includes:

[0236] Perform packet loss detection on the acquired original data packets and FEC redundant packets to obtain the packet loss location;

[0237] Send the actual delay to the sender;

[0238] Prioritize the packet loss positions, and use the packet loss positions with a priority higher than a preset first priority threshold as the packet loss positions of the original data packets to be retransmitted in the retransmission request;

[0239] Send a retransmission request to the sender.

[0240] When the data type corresponding to the acquired original data packet and FEC redundant packet is video data, decoding and display are performed, including:

[0241] In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet;

[0242] Decapsulating and assembling the restored data packets and the original data packets to obtain video frames;

[0243] Prioritize the video frames, delete the video frames whose priorities are lower than a preset second priority threshold, and sort and assemble the remaining video frames to obtain a picture group;

[0244] The picture groups are sorted based on the time sequence to obtain a picture sequence. When the sorting reaches a buffer threshold, a decodable picture sequence is output, and the decodable picture sequence is decoded and then displayed.

[0245] When the data types corresponding to the acquired original data packets and FEC redundant packets are non-video data, splicing is performed, including:

[0246] In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet;

[0247] Decapsulating the restored data packet and the original data packet to obtain decapsulated non-video data;

[0248] The decapsulated non-video data is spliced ​​according to preset categories to obtain non-video data that can be used.

[0249] Embodiment 3:

[0250] like Figure 4 As shown, this embodiment provides specific steps for data transmission when the data type of the data to be transmitted is video data.

[0251] Step 1: The sending end performs encoding and encapsulation on the video data in sequence to obtain the original data packet and FEC redundant packet.

[0252] Specifically, the video data is first encoded by an encoder, and then encapsulated according to the protocol to obtain an original data packet and an FEC redundant packet. The original data packet and the FEC redundant packet are cached.

[0253] Specifically, the encoding bit rate is adjusted by the GCC congestion control algorithm based on the prediction results output by the pre-trained network prediction model, the actual delay sent by the receiving end, and the bandwidth detection results.

[0254] Step 2: Executed by the sender, the original data packet and FEC redundant packet are sent to the receiver.

[0255] Specifically, the original data packet and the FEC redundant packet are sent to the receiving end through the video slicing channel.

[0256] Step 3: The receiving end performs packet loss detection on the original data packet and FEC redundant packet to obtain the packet loss location; performs weak network indicator detection on the original data packet and FEC redundant packet to obtain the actual delay, and sends the actual delay to the sending end.

[0257] Step 4: The receiving end performs priority sorting on the packet loss positions, uses the packet loss positions with a priority higher than the preset first priority threshold as the packet loss positions of the original data packets to be retransmitted in the retransmission request, and sends the retransmission request to the sending end.

[0258] Step 5: Executed by the receiving end, decoded and displayed.

[0259] Step 5.1: In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet.

[0260] Step 5.2: Decapsulate and assemble the restored data packets and the original data packets to obtain video frames.

[0261] Step 5.3: Prioritize the video frames, delete the video frames whose priorities are lower than the preset second priority threshold, and sort and assemble the remaining video frames to obtain a picture group.

[0262] Step 5.4: Sort the picture groups based on the time sequence to obtain a picture sequence. When the sorting reaches the buffer threshold, output a decodable picture sequence, and decode the decodable picture sequence for display.

[0263] Step 6: Executed by the sending end, based on the retransmission request sent by the receiving end, the original data packet specified by the retransmission request is sent to the receiving end.

[0264] Specifically, the sending bit rate of the sender in step 2 and step 6 is obtained by adjusting the bit rate using the GCC congestion control algorithm according to the prediction results output by the pre-trained network prediction model, the actual delay sent by the receiver, and the bandwidth detection results.

[0265] Embodiment 4:

[0266] like Figure 4 As shown, this embodiment provides specific steps for data transmission when the data type of the data to be transmitted is non-video data.

[0267] Step 1: The sending end performs encapsulation processing on the acquired non-video data to obtain the original data packet and FEC redundant packet.

[0268] Specifically, the non-video data includes but is not limited to force feedback data, vibration data and whole vehicle data collected by the sensor group.

[0269] Step 2: Executed by the sender, which uses a sliding window to send the original data packet and FEC redundant packet to the receiver.

[0270] Specifically, the sending window and sending rate of the sliding window are obtained by the BBR congestion control algorithm to control the congestion window and adjust the sending rate. The original data packet and FEC redundant packet are sent to the receiving end through the IoT slice channel.

[0271] Step 3: The receiving end performs packet loss detection on the acquired original data packets and FEC redundant packets to obtain the packet loss location.

[0272] Step 4: The receiving end performs priority sorting on the packet loss positions, uses the packet loss positions with a priority higher than the preset first priority threshold as the packet loss positions of the original data packets to be retransmitted in the retransmission request, and sends the retransmission request to the sending end.

[0273] Step 5: Executed by the receiving end, data splicing.

[0274] Step 5.1: In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet.

[0275] Step 5.2: Decapsulate the restored data packet and the original data packet to obtain decapsulated non-video data.

[0276] Step 5.3: splice the decapsulated non-video data according to preset categories to obtain non-video data that can be used.

[0277] Step 6: Executed by the sender, based on the retransmission request sent by the receiver, the original data packet and FEC redundant packet specified by the retransmission request are sent to the receiver using a sliding window.

[0278] Specifically, the sending window and sending rate of the sliding window are obtained by the BBR congestion control algorithm to control the congestion window and adjust the sending rate. The original data packet and FEC redundant packet are sent to the receiving end through the IoT slice channel.

[0279] Embodiment 5:

[0280] This embodiment provides a data transmission system for remote takeover of unmanned mining trucks, including a vehicle, a remote cab, and a 5G base station.

[0281] The vehicle includes a transmitting computing unit and a CPE, a camera and a sensor component that are communicatively connected to the transmitting computing unit.

[0282] The sending calculation unit is configured to execute the data transmission method described in Example 1.

[0283] Specifically, an encoder for encoding video data is arranged in a sending calculation unit.

[0284] CPE is used to send original data packets and FEC redundant packets, receive retransmission requests and the actual delay sent by the receiving end, and perform bandwidth detection. It should be noted that the air interface data of CPE (such as SINR, RSRP) can provide a basis for judging the network status.

[0285] The camera is used to collect video data.

[0286] The sensor assembly is used to collect non-video data. The sensor group includes a vehicle data collection sensor, a vehicle vibration sensor, and a force feedback sensor, which respectively collects vehicle data such as vehicle speed and status, vehicle vibration, and force feedback data of action mechanisms such as steering wheels and pedals as non-video data.

[0287] The vehicle also includes a controller and a vehicle execution unit for vehicle operation control, video encoding and data processing, and the collaboration of various units. The vehicle execution unit executes the control signals sent by the remote cockpit, including but not limited to the control signals of the steering device, the braking device, the gear shifting device, and the throttle device.

[0288] The remote cockpit includes a receiving computing unit and a MEC and a remote takeover platform that are communicatively connected to the receiving computing unit.

[0289] The receiving and calculating unit is configured to execute the data transmission method described in Embodiment 2. It should be noted that the receiving and calculating unit decodes and displays the video data and performs data splicing on the non-video data.

[0290] MEC is used to send retransmission requests and actual delays and receive original data packets and FEC redundant packets.

[0291] Specifically, the MEC unit is located at the edge of the network, close to remotely taking over user-plane applications, providing storage and forwarding of video and non-video data, and reducing the latency of service access.

[0292] The remote takeover platform is used to remotely take over the unmanned mining card based on video data and non-video data.

[0293] Specifically, the CPE and MEC are connected in communication through a 5G base station to implement the data transmission method provided in Example 3 and Example 4.

[0294] The 5G base station also includes a network slicing unit, which is used to divide the network slices according to signaling and data (including remotely taken over video data, IoT data, control data, etc.). The remotely taken over data is divided into video slices, IoT slices and control slices to meet differentiated network performance requirements and improve network transmission efficiency and quality.

[0295] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0296] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0297] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0298] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0299] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A data transmission method, characterized in that: Executed by the sender, including: Get the data to be transmitted and its data type; In response to the data type of the data to be transmitted being video data, encoding processing and encapsulation processing are sequentially performed on the video data to obtain an original data packet and an FEC redundant packet; Send the original data packet and FEC redundant packet to the receiving end; Based on the retransmission request sent by the receiving end, the original data packet specified by the retransmission request is sent to the receiving end; The encoding bit rate and the sending bit rate to the receiving end are obtained by adjusting the bit rate using the GCC congestion control algorithm based on the prediction results output by the pre-trained network prediction model, the actual delay sent by the receiving end, and the bandwidth detection results.

2. The data transmission method according to claim 1, characterized in that: The input of the pre-trained network prediction model is the actual delay sent by the receiving end in the current time period, and the output result is the predicted delay in the next time period.

3. The data transmission method according to claim 2, characterized in that: The GCC congestion control algorithm includes an overload detector and a rate controller, and the rate controller includes a finite state machine and a rate outputter; The encoding bit rate and the transmission bit rate sent to the receiving end are obtained by adjusting the bit rate according to the prediction result output by the pre-trained network prediction model and the bandwidth detection result by the GCC congestion control algorithm, including: The overload detector determines the network status according to the predicted delay of the next time period output by the pre-trained network prediction model, and outputs the network status of the next time period; the network status is an overload state, an underload state or a normal state; In response to the network state of the next time period being different from the network state of the current time period, triggering the finite state machine; The finite state machine adjusts the bandwidth according to the network status in the next time period and outputs the estimated bandwidth, including: Get the maximum bandwidth and current bandwidth of the bandwidth detection result, and calculate the receiving bit rate based on the current bandwidth; Calculate the first bandwidth threshold = maximum bandwidth + 3 * maximum bandwidth standard deviation, and calculate the second bandwidth threshold = maximum bandwidth - 3 * maximum bandwidth standard deviation; When the state of the finite state machine in the next time period is an ascending state, the bandwidth is increased, including: If the received bit rate is greater than the first bandwidth threshold or the received bit rate is less than the second bandwidth threshold, the predicted bandwidth is increased by multiplication, and the estimated bandwidth is the current bandwidth increasing by 8% per second; If the second bandwidth threshold ≤ the receiving bit rate ≤ the first bandwidth threshold, the predicted bandwidth is increased by additive increase, and the estimated bandwidth is the current bandwidth plus half the size of the original data packet every RTT time; When the state of the finite state machine in the next time period is a decreasing state, the bandwidth is reduced, and the estimated bandwidth is 0.85*receiving bit rate; When the state of the finite state machine in the next time period is normal, the estimated bandwidth is the current bandwidth; The code rate outputter outputs the corresponding code rate according to the estimated bandwidth output by the finite state machine.

4. The data transmission method according to claim 1, characterized in that: The pre-trained network prediction model is obtained by the following steps: Initialize the original sequence for: ， in, The first individual data; The original sequence is smoothed and the smoothed sequence is obtained after wavelet decomposition and reconstruction. for: ， in, is the first individual data; The accumulated sequence obtained by accumulating the smoothed sequence once for: ， in, is the first The data are expressed by the following formula: ， in, is the data in the smoothed series; Initialize neighbor value generation sequence for: ， in, Generate the first The data are expressed by the following formula: ， in, For taking Indicates the first The coefficient of the data; According to the definition of cumulative sequence, we get: ； After bringing in It is expressed as: ； Before use Prediction No. Item, then is known, after adjustment, we get the grey differential equation: ， in, is the grey action, is the development coefficient, For time; Pick ,but: ； The matrix representation is: ， in: ； Use the least squares method to get the parameters and Estimated value: ， in, For the parameter and The matrix of estimated values ​​of ; Pick ,but: ； According to Leibniz formula and the geometric meaning of definite integral, we get: ； Then the whitening equation is: ， Solving the differential equations yields the predictive cumulative model: ， in, is the base of natural logarithms; Use the cumulative reduction formula to restore the predicted value: ； Compensate the residuals and establish the GM(1,1) model to obtain the predicted values. The residual sequence is: ， in: ； right Taking the absolute value, we get the sequence: ， in: ， right Establish a grey model and get the residual prediction value: ， in, is the development coefficient of the residual value, is the grey action of the residual value; For the predicted value After correction, we get: ； Use Markov chain to correct the prediction results and improve the prediction accuracy. According to the deviation, it is divided into states, establish the state transfer matrix : ， in, Indicates from the state To status The one-step transition probability is expressed as follows: ， in, Indicates status Transfer to state The number of times Indicates status Total number of; The upper limit of the residual value is , the lower limit of the residual value is , the prediction model is corrected as follows: ， in, Status The initial state vector.

5. The data transmission method according to claim 1, characterized in that: When the data type of the data to be transmitted is non-video data, it includes: Encapsulate the acquired non-video data to obtain the original data packet and FEC redundant packet; Use a sliding window to send the original data packet and FEC redundant packet to the receiving end; Based on the retransmission request sent by the receiving end, the original data packet and the FEC redundant packet specified by the retransmission request are sent to the receiving end using a sliding window; Among them, the sending window and sending rate of the sliding window are obtained by the BBR congestion control algorithm through congestion window control and sending rate adjustment.

6. A data transmission method, characterized in that: Executed by the receiving end, including: Perform packet loss detection on the acquired original data packets and FEC redundant packets to obtain the packet loss location; Send the actual delay to the sender; Prioritize the packet loss positions, and use the packet loss positions with a priority higher than a preset first priority threshold as the packet loss positions of the original data packets to be retransmitted in the retransmission request; Send a retransmission request to the sender.

7. The data transmission method according to claim 6, characterized in that: When the data type corresponding to the acquired original data packet and FEC redundant packet is video data, decoding and display are performed, including: In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet; Decapsulating and assembling the restored data packets and the original data packets to obtain video frames; Prioritize the video frames, delete the video frames whose priorities are lower than a preset second priority threshold, and sort and assemble the remaining video frames to obtain a picture group; The picture groups are sorted based on the time sequence to obtain a picture sequence. When the sorting reaches a buffer threshold, a decodable picture sequence is output, and the decodable picture sequence is decoded and then displayed.

8. The data transmission method according to claim 6, characterized in that: When the data types corresponding to the acquired original data packets and FEC redundant packets are non-video data, splicing is performed, including: In response to the existence of a packet loss position in the packet loss detection, the FEC redundant packet is restored using an FEC algorithm to obtain a restored data packet; Decapsulating the restored data packet and the original data packet to obtain decapsulated non-video data; The decapsulated non-video data is spliced ​​according to preset categories to obtain non-video data that can be used.

9. A data transmission system for remote takeover of unmanned mining trucks, characterized in that: Includes vehicle and remote cab; The vehicle comprises a sending computing unit and a CPE, a camera and a sensor component which are communicatively connected to the sending computing unit, the sending computing unit is configured to execute the method according to any one of claims 1 to 5, the CPE is used to send original data packets and FEC redundant packets, receive retransmission requests and actual delays sent by the receiving end, and perform bandwidth detection, the camera is used to collect video data, and the sensor component is used to collect non-video data; The remote cab includes a receiving computing unit and an MEC and a remote takeover platform to which the receiving computing unit is communicatively connected. The receiving computing unit is configured to execute the method described in any one of claims 6 to 8. The MEC is used to send retransmission requests and actual delays and receive original data packets and FEC redundant packets. The remote takeover platform is used to remotely take over the unmanned mining truck based on video data and non-video data.

10. The data transmission system for remote takeover of unmanned mining trucks according to claim 9 is characterized in that: It also includes a 5G base station, and the CPE and the MEC are communicatively connected through the 5G base station.

Images