Video transmission control method and remote driving system

By adjusting the encoding code rate in the remote driving system according to network conditions and vehicle speed, and encoding and transmitting the pictures around the vehicle, the problem of poor video transmission quality in weak network environments is solved, and the video transmission speed and real-time control are improved.

CN119011784APending Publication Date: 2024-11-22TANGSHAN PORT IND GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411296065.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In remote driving scenarios, the video transmission quality of the vehicle in a weak network environment is poor, resulting in video delay and playback stuttering, affecting the real-time and security of control.

Method used

The picture collected by the on-board camera is encoded through the predetermined encoding code rate value, and the encoding code rate is adjusted according to the network performance parameters and vehicle speed. The video data packet is packaged using the streaming media transmission protocol, and transmitted to the remote control terminal in real time, and according to the reception The network performance parameters redetermine the encoded code rate value to adapt to network conditions and vehicle speed.

Benefits of technology

In a weak network environment, video transmission speed and reliability are improved, video delay is reduced, remote control is timely and the transmission of details around the vehicle is ensured, and remote video quality and video transmission quality are balanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011784A_ABST
    Figure CN119011784A_ABST
Patent Text Reader

Abstract

The invention discloses a video transmission control method and a remote driving system, and relates to the technical field of remote control, and the method comprises the steps: a vehicle encodes a vehicle surrounding image collected by a vehicle-mounted camera through employing a predetermined encoding rate value, and obtains encoded video stream data; and packaging the coded video stream data according to a streaming media transmission protocol to obtain a video data packet, transmitting the video data packet to a remote control end for processing by the remote control end, and re-determining a coding rate value to be utilized after receiving a parameter value of a network performance parameter determined by the remote control end according to the video data packet. According to the invention, the vehicle can carry out coding by using the coding rate adaptive to the real-time network condition and the vehicle speed, thereby realizing the balance between the remote video quality and the video transmission quality, and improving the video transmission quality in a weak network environment to a certain extent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of remote control technology, and in particular to a video transmission control method and a remote driving system. Background Art

[0002] The development of Internet technology has provided technical and facility support for the application of remote control technology. Remote control technology is a technology that uses the network to control or operate remote devices. It is used in scenarios such as remote driving, intelligent robots, telemedicine, aerospace military, and industrial production. In the remote control system, remote devices, such as vehicles, robots, and operating table instruments, as the controlled end, need to transmit live images to the remote control end in real time to ensure the real-time control of the remote device by the remote control end. Especially in remote driving scenarios, in order to ensure vehicle driving safety, higher requirements are placed on video delay and playback smoothness.

[0003] Although the 5G network, with its high bandwidth and low latency, provides a better network environment for remote driving scenarios and improves the quality of video transmission to a certain extent, in actual applications, whether it is the 5G network provided by the network operator's 5G public network base station or the 5G private network base station, the signal strength will change with the distance from the vehicle to the base station, and the network coverage area will be interfered with to varying degrees by other signals. In other words, due to the influence of vehicle movement and signal interference, poor network quality often occurs, increasing the possibility of increased video transmission delay and video playback jamming, resulting in poor video transmission quality.

[0004] Therefore, how to improve the video transmission quality of vehicles in a weak network environment has become an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0005] In view of the above problems, the present application provides a video transmission control method and a remote driving system to improve the video transmission quality of the vehicle in a weak network environment.

[0006] The specific plan is as follows:

[0007] A first aspect of the present application provides a video transmission control method, which is applied to a vehicle, and the method comprises:

[0008] The vehicle surrounding images captured by the vehicle-mounted camera are encoded using a predetermined encoding bit rate value to obtain encoded video stream data; wherein the encoding bit rate value is a parameter value determined by the vehicle according to a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the parameter value of the network performance parameter is a parameter value determined by a remote control terminal connected to the vehicle for communication according to data packet information of a received historical video data packet of the vehicle, the network performance parameters include network transmission bandwidth, jitter and total delay, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the vehicle speed, the encoding bit rate is negatively correlated with the safety factor, the encoding bit rate is positively correlated with a first ratio, the first ratio is a ratio of the received jitter to the received total delay; the data packet information includes a data packet size and a data packet timestamp;

[0009] Packing the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, transmitting the video data packet to the remote control terminal for decapsulation processing by the remote control terminal, storing the encoded video stream data obtained by the decapsulation processing into a video playback cache queue, and determining and transmitting a parameter value of a new network performance parameter based on data packet information of the video data packet obtained by the decapsulation processing;

[0010] Upon receiving the parameter value of the network performance parameter transmitted by the remote control terminal, the encoding bit rate value is determined.

[0011] A second aspect of the present application provides a video transmission control method, which is applied to a remote control terminal, and the method includes:

[0012] receiving a video data packet; the video data packet is a message obtained by packaging the encoded video stream data according to a specified streaming media transmission protocol by a vehicle in communication connection with the remote control terminal, the encoded video stream data being obtained by encoding the surrounding images of the vehicle captured by the vehicle-mounted camera using a predetermined encoding bit rate value by the vehicle;

[0013] Decapsulating the video data packet to obtain the encoded video stream data and data packet information of the video data packet; the data packet information includes a data packet size and a data packet timestamp;

[0014] Storing the encoded video stream data in a video playback cache queue;

[0015] Determining parameter values ​​of network performance parameters according to the data packet information of the video data packet, wherein the network performance parameters include network transmission bandwidth, jitter and total delay;

[0016] The parameter value of the network performance parameter is sent to the vehicle so that the vehicle can redetermine the encoding rate value based on a first preset relationship and the parameter value of the received network performance parameter, wherein the first preset relationship is a mapping relationship between the encoding rate and a first independent variable, the first independent variable includes the network performance parameter and a safety factor, the encoding rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the speed of the vehicle, the encoding rate is negatively correlated with the safety factor, and the encoding rate is positively correlated with a first ratio, which is a ratio of the received jitter to the received total delay.

[0017] A third aspect of the present application provides a remote driving system, comprising: a vehicle, and a remote control terminal communicatively connected to the vehicle;

[0018] The vehicle encodes the surrounding images of the vehicle collected by the vehicle-mounted camera using a predetermined encoding bit rate value to obtain encoded video stream data; wherein the encoding bit rate value is a parameter value determined by the vehicle according to a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the parameter value of the network performance parameter is a parameter value determined by the remote control terminal according to data packet information of a historical video data packet received from the vehicle, the network performance parameter includes network transmission bandwidth, jitter and total delay, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the speed of the vehicle, the encoding bit rate is negatively correlated with the safety factor, the encoding bit rate is positively correlated with a first ratio, the first ratio is a ratio of the received jitter to the received total delay; the data packet information includes a data packet size and a data packet timestamp;

[0019] The vehicle packages the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, and transmits the video data packet to the remote control terminal;

[0020] The remote control terminal decapsulates the received video data packet to obtain the encoded video stream data and data packet information of the video data packet;

[0021] The remote control terminal stores the encoded video stream data into a video playback cache queue;

[0022] The remote control terminal re-determines the parameter value of the network performance parameter according to the data packet information of the video data packet and sends the parameter value of the network performance parameter to the vehicle;

[0023] The vehicle redetermines the encoding bit rate value based on the first preset relationship and the parameter value of the received network performance parameter.

[0024] By means of the above technical solution, the remote driving vehicle first uses a predetermined encoding bit rate value to encode the surrounding images of the vehicle collected by the vehicle-mounted camera to obtain encoded video stream data; then the encoded video stream data is packaged according to the streaming media transmission protocol to obtain a video data packet, and the video data packet is transmitted to the remote control terminal for processing by the remote control terminal, and after receiving the parameter value of the network performance parameter determined by the remote control terminal based on the video data packet, the encoding bit rate value to be used is re-determined. Since the encoding bit rate value used by the vehicle decreases with the increase of the vehicle speed, under the same network conditions, the higher the vehicle speed, the smaller the encoding bit rate value, by reducing the amount of transmitted data, the video transmission speed and transmission reliability of the vehicle when the vehicle is driving at high speed are improved, and the video transmission delay is reduced, which provides a basis for ensuring the timeliness of the remote control terminal's control of the vehicle; correspondingly, since the accuracy and safety requirements for vehicle control are relatively low when the vehicle is driving at a low speed, based on this, the control delay of the remote control terminal to the vehicle under the influence of video transmission delay is acceptable. In this case, a higher encoding bit rate value can be used for encoding to provide more details around the vehicle to the remote control terminal. Moreover, the encoding bit rate value used by the vehicle increases with the increase of the network transmission bandwidth, so that when the network conditions are good, the vehicle can transmit video data with richer details and higher quality around the vehicle to the remote control terminal; when the network conditions are poor, the encoding bit rate value used by the vehicle decreases with the decrease of the network transmission bandwidth, which effectively reduces the amount of data transmitted by the vehicle, and to a certain extent improves the transmission speed of video data packets under weak network conditions, providing a basis for ensuring the smoothness of remote video. Based on this, by means of the video transmission control scheme provided by the present application, the vehicle can use the encoding bit rate that is compatible with the real-time network conditions and vehicle speed for encoding, achieving a balance between remote video quality and video transmission quality, and to a certain extent improving the video transmission quality under weak network conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0026] Figure 1An example of a structural schematic diagram of a remote driving system is provided;

[0027] Figure 2 is a flow chart of a video transmission control method applied to a vehicle according to an embodiment of the present application;

[0028] Figure 3 is a structural schematic diagram of a video transmission control device applied to a vehicle according to an embodiment of the present application;

[0029] Figure 4 is a structural schematic diagram of an electronic device according to an embodiment of the present application;

[0030] Figure 5 is a flow chart of a video transmission control method applied to a remote control terminal according to an embodiment of the present application;

[0031] Figure 6 It is a structural schematic diagram of a video transmission control device applied to a remote control terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation mode of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0033] The applicant in this case has found through research that under weak network conditions with insufficient network transmission bandwidth and excessive delay, the consistency and stability of video playback can be improved by adding video frame cache on the remote control end to avoid freezes, frame drops, etc., thereby ensuring smooth video playback. However, as the number of video frames cached on the remote control end increases, the playback delay of the video frames increases, making it impossible for the remote control end to obtain the on-site situation in a timely manner, which poses a safety hazard to remote driving.

[0034] To solve the above problems, the present application provides a video transmission control method and a remote driving system, which can be used to solve the problem of poor video transmission quality of remote driving vehicles under weak network conditions.

[0035] The present application is applied to a remote driving scenario, in which the vehicle acts as a remote device and transmits live images to a remote control terminal in real time so that the remote control terminal can remotely control the vehicle.

[0036] In one possible implementation, the vehicle may include: a control unit, and an image acquisition unit and a communication unit connected to the control unit. Each unit is described in turn below. Among them, the image acquisition unit is connected to the control unit to capture and transmit back the image around the vehicle for processing by the control unit; the type, specification and quantity of the image acquisition device included in the image acquisition unit can be set according to actual needs, and this application does not limit it. Optionally, the image acquisition unit may include 8 Gigabit Multimedia Serial Link (GMSL) vehicle-grade cameras, specifically including: 4 wide-angle cameras and 4 fisheye cameras; wherein the 4 wide-angle cameras are used to capture the front and rear view images of the vehicle respectively, and the specifications may be 1920×1080; the 4 fisheye cameras are used to capture the live images of the front blind area and the rear blind area on the left and right sides of the vehicle respectively, and the specifications may be 1280×720. The control unit may be called a vehicle-mounted remote control unit (RCU), referred to as a control unit RCU, which may be used to receive and process the images around the vehicle captured by the image acquisition unit for transmission. The communication unit may include: 5G CPE (Customer Premise Equipment, CPE, customer premise equipment), wherein the 5G CPE is inserted with a traffic SIM card that can access the Internet (such as a public network or private network SIM card), and the 5G CPE is connected to the control unit RCU using industrial Ethernet, which can be called a multi-network 5G CPE, and is used to provide an Internet access channel for the control unit RCU to communicate with the remote control end.

[0037] Correspondingly, the remote control terminal can be a terminal device that is connected to the vehicle in communication and has data processing capabilities, such as a computer, an industrial computer, etc. Exemplarily, the remote control terminal may include: a high-performance host with an independent graphics card, multiple displays connected to the host, and a remote driving console; wherein the network of the host is connected to the vehicle network, and the host is used to receive video data transmitted by the vehicle and perform video processing and data output; multiple displays are used to display video and vehicle instrument information in real time; the remote driving console is provided with a simulated driving kit for receiving the operation of the remote operator and outputting control instructions, and the control instructions are transmitted to the vehicle through the network to realize remote control of the vehicle. It should be noted that the above-mentioned network connectivity can be direct connectivity or indirect connectivity, which is not limited in this application.

[0038] On the basis of the above, optionally, the vehicle can communicate with the remote control end through the server end, wherein the server end can be deployed locally or in the cloud, such as a public network cloud or a private cloud, for forwarding and processing received data (such as video data sent by the vehicle). Specifically, the vehicle and the server end can be connected through a wireless network, and the communication unit of the vehicle provides an Internet access channel between the control unit RCU and the server end; the remote control end can be connected to the server end through wired broadband.

[0039] Based on the above, Figure 1 A schematic diagram of the structure of a remote driving system is shown in FIG. Figure 1 As shown, on the vehicle side, the camera installed on the vehicle is connected to the control unit RCU, and the control unit RCU is connected to the multi-network 5G CPE, so that the vehicle can communicate with the server side through the 5G network; on the remote control side, the industrial computer is connected to the wired broadband to communicate with the server side, and the industrial computer is connected to multiple display screens for displaying remote video images and a remote driving console for receiving operator operations.

[0040] Figure 2 FIG. 1 is a flow chart of a video transmission control method according to an embodiment of the present application, and the method is applied to a vehicle. Figure 2 As shown, the method may include the following steps:

[0041] Step S101: Encode the vehicle surrounding images captured by the vehicle-mounted camera using a predetermined encoding bit rate value to obtain encoded video stream data.

[0042] Among them, the encoding bit rate value is a parameter value determined by the vehicle based on a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the network performance parameter includes a network transmission bandwidth, jitter, and total delay, the parameter value of the network performance parameter is a parameter value determined by a remote control terminal connected to the vehicle in communication based on the packet information of the received historical video data packet of the vehicle, the packet information includes a packet size and a packet timestamp. The following is an exemplary description of each network performance parameter and its calculation method in turn.

[0043] The network transmission bandwidth is the bandwidth when the vehicle transmits data, which is expressed as W. The parameter value of the network transmission bandwidth can be determined according to the packet size of the historical video data packets received by the remote control terminal, such as the ratio of the total packet size received in the preset period before the current moment to the duration of the preset period. For example, assuming that the size of each video data packet is the same, or using the size of a video data packet to represent the size of each video data packet, the calculation formula of the network transmission bandwidth can be expressed as: W = P r ×S packet ÷T interval Where S packet Indicates the size of a single video data packet in bytes (B); P r Indicates that the time before the current moment is T interval The number of video packets received within a preset period of time; or, P r Indicates the number of received video packets, T interval Indicates that P is received r The time interval between video data packets is measured in seconds (s); the unit of network transmission bandwidth is generally kilobytes per second (KB / s), which can be converted to bits per second (bit / s).

[0044] The total delay represents the transmission delay from the vehicle to the remote control end, denoted as T total , in milliseconds (ms). Taking the transmission process of a video data packet as an example, the total delay calculation formula can be expressed as: T total =T recv -T packet , where T recv Indicates the time when the remote control terminal receives the video data packet, T packet Represents the packet timestamp of the video data packet. On this basis, in order to avoid interference from accidental factors, the average value of multiple total delays can be used as the final total delay parameter value, that is, the total delay can be determined based on the reception time when the remote control terminal receives multiple historical video data packets of the vehicle and the packet timestamps of the multiple historical video data packets. It should be noted that the total delay can also include the time taken by the image acquisition unit to capture the image and the control unit to encode the image. Although this part of the time will also cause delay, it is limited by the computing and processing capabilities of the vehicle-side hardware. This part of the time is usually fixed and can be ignored; if it is not ignored, a dedicated measuring instrument can be used in advance to obtain this part of the time.

[0045] Jitter refers to the deviation between the total delays of two adjacent video data packets, expressed as J, in milliseconds. Taking two adjacent video data packets packet_1 and packet_2 as an example, the calculation formula of jitter can be expressed as: J = |(T recv_1 -Tpacket_1 )-(T recv_2 -T packet_2 )|, where T recv_1 Indicates the receiving time of the video data packet packet_1, T packet_1 Indicates the packet timestamp of the video packet packet_1, T recv_2 Indicates the receiving time of the video data packet packet_2, T packet_2 Indicates the packet timestamp of the video packet packet_2. On this basis, in order to avoid the interference of accidental factors, the average value of multiple jitters can be used as the final jitter parameter value, that is, the jitter can be determined according to the receiving time when the remote control terminal receives multiple historical video packets of the vehicle and the packet timestamps of the multiple historical video packets.

[0046] On the basis of the above, the encoding bit rate is positively correlated with the network transmission bandwidth; the safety factor may include a speed safety index that is positively correlated with the speed of the vehicle, and the encoding bit rate is negatively correlated with the safety factor; the encoding bit rate is positively correlated with a first ratio, and the first ratio is the ratio of the received jitter to the received total delay. It should be noted that the speed safety index is closely related to the delay. Under the same network conditions, the increase in vehicle speed will reduce the encoding bit rate value to a certain extent, thereby reducing the data volume of the video frame, which helps to reduce the video transmission delay from the vehicle end to the remote control end, ensure the real-time nature of the remote video picture under weak network conditions, and provide a basis for ensuring driving safety. In addition, the high-performance encoding chip encoding of the core board of the vehicle control unit can be used to encode the above encoding process to compress the surrounding images of the vehicle captured by the on-board camera. The encoded video stream data can be a data stream in H.264 or H.265 format for transmission.

[0047] Step S102: Pack the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, and transmit the video data packet to the remote control terminal for processing by the remote control terminal.

[0048] The streaming media transmission protocol may be a real-time transport protocol (RTP), or a protocol related to the RTP protocol, such as a network transmission protocol obtained by redefining and optimizing the header and payload of the data packet defined in the RTP protocol. Based on the above content, transmitting the video data packet to the remote control terminal may include: the vehicle sends the video data packet to the server via the streaming media transmission protocol, so that the server can forward it to the remote control terminal.

[0049] Specifically, the above-mentioned processing may include: the remote control terminal performs decapsulation processing, and stores the encoded video stream data obtained by the decapsulation processing into a video playback cache queue, and determines and transmits the parameter value of the new network performance parameter based on the data packet information of the video data packet obtained by the decapsulation processing. Among them, the encoded video stream data flows through the video playback cache queue, and will be displayed after hard decoding in the graphics card. By means of the video playback cache queue, the smoothness of remote video playback can be guaranteed to a certain extent; the process of the remote control terminal determining the parameter value of the network performance parameter can refer to the above description, and will not be repeated here. In addition, the protocol used by the remote control terminal when transmitting the parameter value of the network performance parameter to the vehicle can be the Real-time Control Protocol (RTCP). Specifically, the remote control terminal can send the parameter value of the network performance parameter to the server end for the server end to forward to the vehicle.

[0050] In a possible implementation, initially, the vehicle may respond to a streaming instruction sent by the remote control end and start executing step S102 to transmit the encoded video stream data to the remote control end through a streaming media transmission protocol, wherein the streaming instruction is a streaming instruction sent by the remote control end to the vehicle through the MQTT (Message Queuing Telemetry Transport, MQTT, message queue telemetry transmission) protocol when the software and hardware of each end involved in remote driving (including the vehicle end, remote control end, etc.) are started normally and the vehicle can communicate with the remote control end.

[0051] Step S103: upon receiving the parameter value of the network performance parameter transmitted by the remote control terminal, determining the encoding bit rate value.

[0052] It should be noted that the encoding bit rate value determined in step S103 is a parameter value determined by the vehicle based on the first preset relationship and the parameter value of the network performance parameter transmitted by the remote control terminal, and matches the current speed of the vehicle. When the subsequent vehicle returns to execute step S101, it will use the encoding bit rate value determined when executing step S103 last time for encoding, thereby realizing the update of the encoding bit rate. The method for determining the encoding bit rate value can refer to the above description, which will not be repeated here.

[0053] With the above scheme, the remote driving vehicle first uses a predetermined encoding bit rate value to encode the vehicle's surrounding images captured by the on-board camera to obtain encoded video stream data; then the encoded video stream data is packaged according to the streaming media transmission protocol to obtain a video data packet, and the video data packet is transmitted to the remote control end for processing by the remote control end. After receiving the parameter value of the network performance parameter determined by the remote control end based on the video data packet, the encoding bit rate value to be used is re-determined.

[0054] Since the encoding bit rate value used by the vehicle decreases with the increase of vehicle speed, under the same network conditions, the higher the vehicle speed, the smaller the encoding bit rate value. By reducing the amount of transmitted data, the video transmission speed and transmission reliability of the vehicle when it is driving at high speed are improved, and the video transmission delay is reduced, which provides a basis for ensuring the timeliness of the remote control end's control of the vehicle; correspondingly, since the accuracy and safety requirements for vehicle control are relatively low when the vehicle is driving at a low speed, based on this, the control hysteresis of the remote control end under the influence of video transmission delay is acceptable. In this case, a higher encoding bit rate value can be used for encoding to provide more details around the vehicle to the remote control end. In addition, the encoding bit rate value used by the vehicle increases with the increase of network transmission bandwidth, so that under good network conditions, the vehicle can transmit more detailed and high-quality video data around the vehicle to the remote control end; under poor network conditions, the encoding bit rate value used by the vehicle decreases with the decrease of network transmission bandwidth, which effectively reduces the amount of transmission data of the vehicle, and to a certain extent improves the transmission speed of video data packets under weak network conditions, which provides a basis for ensuring the smoothness of remote video. Based on this, through the video transmission control scheme provided by this application, the vehicle can encode using a coding rate that is suitable for the real-time network conditions and vehicle speed, thereby achieving a balance between remote video quality and video transmission quality, and to a certain extent improving the video transmission quality in a weak network environment.

[0055] The following describes the process of determining the encoding rate value, and explains the encoding rate value used by the vehicle described in this application during encoding.

[0056] In one or more embodiments provided in the present application, the data packet information may further include a data packet sequence number, the network performance parameter may further include a packet loss rate, and the encoding bit rate is positively correlated with the packet loss rate.

[0057] The packet loss rate is the ratio of the number of lost data packets to the total number of sent data packets. The lost data packets can be determined by detecting the data packet sequence number of the video data packet received by the remote control terminal. For example, the packet loss rate calculation formula can be expressed as l=(P s -P r ) / P s, where l (%) represents the packet loss rate, P s represents the number of video data packets sent by the vehicle, which can be obtained by calculating the difference between the data packet sequence numbers at two time points; P r The number of video data packets received by the remote control terminal can be determined by counting the data packet sequence numbers of the video data packets received between the above two time points, and the above two time points can be the current moment and the time interval between the current moment and the preset time length T interval moment.

[0058] On the basis of the above, determining the encoding rate value may include the following steps:

[0059] Step A: determining a parameter value of the safety factor corresponding to the vehicle speed value, and determining a parameter value of the bandwidth correction coefficient corresponding to the parameter value of the network transmission bandwidth.

[0060] The bandwidth correction coefficient is negatively correlated with the network transmission bandwidth and is used to correct the influence of fluctuation of the network transmission bandwidth on the encoding bit rate. The encoding bit rate is positively correlated with the bandwidth correction coefficient.

[0061] Step B: calculating a parameter value of the first ratio according to the received parameter value of the jitter and the received parameter value of the total delay.

[0062] Step C: Calculate the ratio of the received parameter value of the network transmission bandwidth to the parameter value of the safety factor, and calculate the product of the ratio and the parameter value of the first ratio as the initial value of the encoding bit rate.

[0063] Step D: correct the initial value of the encoding rate using the parameter value of the bandwidth correction coefficient, and calculate the product of the corrected initial value of the encoding rate and the packet loss rate to obtain the encoding rate value.

[0064] The above scheme takes into account the impact of packet loss rate on encoding bit rate, and can avoid the situation where the encoding bit rate is too low due to poor network conditions; at the same time, the bandwidth correction coefficient that is negatively correlated with bandwidth is used to correct the impact of wireless network bandwidth fluctuations on the encoding bit rate, which can reduce the increase in encoding bit rate caused by the increase in network transmission bandwidth to a certain extent, and limit the amount of transmission data in a high-bandwidth environment. Even if bandwidth fluctuations occur in a high-bandwidth environment, the possibility of transmission abnormalities caused by excessive encoding bit rate is low, which provides a basis for improving video transmission quality.

[0065] In one or more embodiments provided in the present application, the safety factor also includes a jitter safety factor that is negatively correlated with the vehicle speed, and the jitter safety factor is used to correct the effect of the vehicle speed on the jitter.

[0066] On the basis of the above, determining the parameter value of the safety factor corresponding to the speed value of the vehicle may include:

[0067] Step E: determine the parameter value of the vehicle speed safety index and the parameter value of the jitter safety factor corresponding to the vehicle speed value.

[0068] Step F, calculating the sum of the parameter values ​​of the vehicle speed safety index and the jitter safety factor as the parameter value of the safety factor.

[0069] The safety factor is positively correlated with the vehicle speed. That is, although the vehicle speed safety index increases and the jitter safety factor decreases with the increase of vehicle speed, the overall safety factor increases.

[0070] Based on the above embodiment, illustratively, the first preset relationship of the present application can be expressed as:

[0071] ;

[0072] Where B is the encoding bit rate, in bit / s; W is the network transmission bandwidth, in KB / s; S is the speed safety index, k is the jitter safety factor; γ is the bandwidth correction factor, l is the packet loss rate (%); J is the jitter, T total Indicates the total delay, J / T total Represents the first ratio. For example, after a large number of experiments and road tests, the corresponding relationship between the vehicle speed safety index and the vehicle speed can be expressed as: ; The corresponding relationship between the jitter safety factor and the vehicle speed can be expressed as: ; The corresponding relationship between the bandwidth correction coefficient and the network transmission bandwidth can be expressed as: It should be noted that, when the network transmission bandwidth does not fall within the above value range, a pre-set initial encoding bit rate can be used for encoding. This bit rate is the encoding bit rate value used by the vehicle before receiving the parameter value of the network performance parameter during this video transmission process.

[0073] In one or more embodiments provided in the present application, the cache frame value based on which the remote control terminal stores the encoded video stream data into the video playback cache queue is a cache frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter determined by the remote control terminal based on a second preset relationship.

[0074] Among them, the second preset relationship is a mapping relationship between the number of cached frames and the second independent variable. The number of cached frames described in the present application is the number of cached frames per second, which can be expressed as F, with a unit of fps; the second independent variable includes the network performance parameter and the inter-frame delay. The parameter value of the inter-frame delay is determined based on the data packet timestamps of two video data packets with adjacent data packet sequence numbers. Optionally, the parameter value of the inter-frame delay can be a preset time length T before the current moment. interval The maximum value of the difference between the packet timestamps of each two video packets with adjacent packet sequence numbers received within the period of time, exemplarily, the calculation formula for the inter-frame delay can be expressed as: T frame =max(T i+1 -T i ), where T i Indicates the packet timestamp of the video packet with packet sequence number i, T i+1 Indicates the packet timestamp of the video packet with packet sequence number i+1, T frame Indicates the inter-frame delay in ms.

[0075] On this basis, the number of cached frames is positively correlated with the second ratio and positively correlated with the third ratio.

[0076] The second ratio is the ratio of the network transmission bandwidth to the preset size of a single frame of video data. The size of a single frame of video data is related to factors such as the resolution of the camera and the image encoding format used. For example, if the vehicle-mounted camera is a 1080p camera and the image encoding format used is the YUV420 format, the size of a single frame of video data is about 3.11MB; when determining the cache frame value, the remote control terminal can use a pre-acquired constant value that can characterize the size of a single frame of video data for calculation.

[0077] The third ratio is the ratio of the corrected jitter to time; the corrected jitter is obtained by correcting the jitter using a jitter index that is negatively correlated with the jitter, and the jitter index is used to correct the impact of network jitter on the number of cached frames; the time and is the sum of the round-trip time and the inter-frame delay, the round-trip time is a preset ratio of the total delay, and the preset ratio is negatively correlated with the total delay. It should be noted that the communication connection between the vehicle and the remote control terminal is composed of a wireless communication connection between the vehicle and the server terminal, and a communication connection between the server terminal and the remote control terminal. Based on this, the preset ratio is the ratio of the network round-trip time (RTT) between the remote control terminal and the server terminal to the total delay.

[0078] Optionally, when the network performance parameter also includes a packet loss rate, the number of cached frames is positively correlated with the packet loss rate.

[0079] Based on the above content, the second preset relationship can be expressed as:

[0080] ;

[0081] Where F is the number of cached frames, in fps; W is the network transmission bandwidth, l is the packet loss rate, f is the size of a single frame of video data; λ is the jitter index, J is the jitter, and λJ is the corrected jitter; τ is the preset ratio, T is total represents the total delay, τ·T total represents the round trip time, T frame Indicates the inter-frame delay. For example, after a large number of experiments, the corresponding relationship between the jitter index and the jitter can be expressed as: ; The corresponding relationship between the preset ratio and the total delay can be expressed as: .

[0082] On the remote control end, the more video frames cached in the video playback cache queue, the smoother the video rendering display, and the better the weak network resistance effect, but it will increase the video display delay, that is, the greater the delay from the scene being captured by the vehicle camera to being displayed on the remote control end. This application can ensure that in a weak network environment, the delay is minimized to the greatest extent by caching the number of frames that is adapted to the real-time network performance parameters, while improving the video smoothness as much as possible, providing a basis for ensuring the safety of remote driving vehicles.

[0083] The present application comprehensively considers the vehicle speed and the network conditions between the vehicle and the remote control end, specifically including bandwidth, delay, jitter and the like. After actual vehicle testing of remote driving, the quantitative relationship between the parameters such as the vehicle speed and the uplink speed of the vehicle-side network and the encoding bit rate when the vehicle-side sends the video and the number of cached frames of the remote control end is determined. Under the premise of ensuring the real-time safety of remote driving, the video smoothness can be maximized, the video transmission delay can be reduced, and the video transmission quality can be optimized. Exemplarily, in a poor network environment, the above-mentioned first preset relationship and the second preset relationship can achieve: the encoding bit rate value of the vehicle is not less than 128 kB / s, the cached frame value of the control end per second is not less than 20 fps, and the delay from the vehicle to the remote control end is not higher than 300 ms.

[0084] The video transmission control device provided in an embodiment of the present application is described below. The device is applied to a vehicle. The video transmission control device described below and the video transmission control method described above can be referenced to each other.

[0085] Figure 3 FIG. 1 is a schematic diagram of the structure of a video transmission control device disclosed in an embodiment of the present application. Figure 3 As shown, the device may include:

[0086] The encoding unit 11 is used to encode the surrounding images of the vehicle collected by the vehicle-mounted camera using a predetermined encoding bit rate value to obtain encoded video stream data; wherein the encoding bit rate value is a parameter value determined by the vehicle according to a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the parameter value of the network performance parameter is a parameter value determined by a remote control terminal connected to the vehicle for communication according to data packet information of a received historical video data packet of the vehicle, the network performance parameter includes network transmission bandwidth, jitter and total delay, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the speed of the vehicle, the encoding bit rate is negatively correlated with the safety factor, and the encoding bit rate is positively correlated with a first ratio, the first ratio being a ratio of the received jitter to the received total delay; the data packet information includes a data packet size and a data packet timestamp;

[0087] The video transmission unit 12 is used to package the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, and transmit the video data packet to the remote control terminal for decapsulation processing by the remote control terminal, and store the encoded video stream data obtained by the decapsulation processing into a video playback cache queue, and determine and transmit a parameter value of a new network performance parameter according to the data packet information of the video data packet obtained by the decapsulation processing;

[0088] The bit rate determination unit 13, which may also be called an anti-weak network adaptive module AWNM (Anti-week-network Module), is used to determine the encoding bit rate value when receiving the parameter value of the network performance parameter transmitted by the remote control terminal.

[0089] In one or more embodiments provided in the present application, the data packet information further includes a data packet sequence number, the network performance parameter further includes a packet loss rate, and the encoding bit rate is positively correlated with the packet loss rate;

[0090] On the basis of the above, the process of the bit rate determination unit 13 determining the encoding bit rate value may include:

[0091] Determining a parameter value of the safety factor corresponding to a speed value of the vehicle;

[0092] Calculating a parameter value of the first ratio according to the received parameter value of the jitter and the received parameter value of the total delay;

[0093] Calculating a ratio of the received parameter value of the network transmission bandwidth to the parameter value of the safety factor, and calculating a product of the ratio and the parameter value of the first ratio as an initial value of the encoding bit rate;

[0094] Determine a parameter value of a bandwidth correction coefficient corresponding to the parameter value of the network transmission bandwidth; the bandwidth correction coefficient is negatively correlated with the network transmission bandwidth and is used to correct the influence of fluctuations in the network transmission bandwidth on the encoding bit rate, and the encoding bit rate is positively correlated with the bandwidth correction coefficient;

[0095] The initial value of the encoding rate is corrected using the parameter value of the bandwidth correction coefficient, and the product of the corrected initial value of the encoding rate and the packet loss rate is calculated to obtain the encoding rate value.

[0096] In one or more embodiments provided in the present application, the safety factor may further include a jitter safety factor that is negatively correlated with the vehicle speed of the vehicle, and the jitter safety factor is used to correct the influence of the vehicle speed on the jitter;

[0097] The process of the code rate determination unit 13 determining the parameter value of the safety factor corresponding to the vehicle speed value of the vehicle may include:

[0098] Determining a parameter value of the vehicle speed safety index and a parameter value of the jitter safety factor corresponding to a vehicle speed value of the vehicle;

[0099] The sum of the parameter values ​​of the vehicle speed safety index and the jitter safety factor is calculated as the parameter value of the safety factor; wherein the safety factor is positively correlated with the vehicle speed.

[0100] In one or more embodiments provided in the present application, the cache frame value based on which the remote control terminal stores the encoded video stream data into the video playback cache queue is the cache frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter determined by the remote control terminal according to the second preset relationship;

[0101] Among them, the second preset relationship is a mapping relationship between the number of cached frames and the second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the parameter value of the inter-frame delay is determined according to the data packet timestamp of two video data packets with adjacent data packet sequence numbers; the number of cached frames is positively correlated with the second ratio and positively correlated with the third ratio, the second ratio is the ratio of the network transmission bandwidth to the preset size of a single frame of video data, the third ratio is the ratio of the corrected jitter to time, the corrected jitter is obtained by correcting the jitter using a jitter index negatively correlated with the jitter, the time is the sum of the round-trip time and the inter-frame delay, the round-trip time is the total delay of the preset ratio, and the preset ratio is negatively correlated with the total delay.

[0102] The video transmission control device provided in the embodiment of the present application can be applied to electronic devices, such as a vehicle-mounted remote control unit RCU, etc. Optionally, Figure 4 The hardware structure diagram of the electronic device is shown in FIG. Figure 4 , the hardware structure of the electronic device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0103] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0104] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0105] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory;

[0106] The memory is used to store computer programs, and the processor is used to execute the computer programs, so that the electronic device can implement any of the above-mentioned video transmission control methods.

[0107] Next, the video transmission control scheme provided by the present application is described from the perspective of the remote control terminal. The video transmission control schemes applied to the remote control terminal and the vehicle can refer to each other, and the same parts will not be repeated.

[0108] Figure 5 A flowchart of a video transmission control method applied to a remote control terminal provided in an embodiment of the present application. Figure 5 As shown, the method may include:

[0109] Step S201: Receive a video data packet.

[0110] Among them, the video data packet is a message obtained by packaging the encoded video stream data according to the specified streaming media transmission protocol by the vehicle that is communicatively connected to the remote control terminal. The encoded video stream data is obtained by the vehicle encoding the vehicle surrounding images captured by the on-board camera using a predetermined encoding bit rate value.

[0111] Step S202: Decapsulate the video data packet to obtain the encoded video stream data and data packet information of the video data packet.

[0112] The data packet information includes the data packet size and the data packet timestamp.

[0113] Step S203: storing the encoded video stream data into a video playback cache queue.

[0114] Step S204: Determine a parameter value of a network performance parameter according to the data packet information of the video data packet.

[0115] Wherein, the network performance parameters include network transmission bandwidth, jitter and total delay;

[0116] Step S205: Send the parameter value of the network performance parameter to the vehicle, so that the vehicle can re-determine the encoding bit rate value based on the first preset relationship and the received parameter value of the network performance parameter.

[0117] Among them, the first preset relationship is a mapping relationship between the encoding bit rate and the first independent variable, the first independent variable includes the network performance parameter and the safety factor, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the vehicle speed, the encoding bit rate is negatively correlated with the safety factor, the encoding bit rate is positively correlated with a first ratio, and the first ratio is the ratio of the received jitter to the received total delay.

[0118] In one possible implementation, the safety factor may also include a jitter safety factor that is negatively correlated with the speed of the vehicle, and the jitter safety factor is used to correct the influence of the speed on jitter; the parameter value of the safety factor is the sum of the parameter values ​​of the speed safety index and the jitter safety factor, and the safety factor is positively correlated with the speed of the vehicle. Optionally, the network performance parameters may also include a packet loss rate, and the encoding bit rate is positively correlated with the packet loss rate. Optionally, the network transmission bandwidth may be a network transmission bandwidth corrected by a bandwidth correction coefficient, and the bandwidth correction coefficient is negatively correlated with the network transmission bandwidth, and is used to correct the influence of network transmission bandwidth fluctuations on the encoding bit rate.

[0119] It should be noted that the description of the vehicle transmitting video data packets, the parameter values ​​of the remote control terminal determining the network performance parameters, and the refinement and expansion process of the vehicle determining the encoding bit rate value can be referred to the above description and will not be repeated here.

[0120] In one or more embodiments provided in the present application, the data packet information may further include a data packet sequence number.

[0121] On this basis, the method may further include:

[0122] Step S206: Determine a parameter value of an inter-frame delay according to the received data packet timestamps of two video data packets with adjacent data packet sequence numbers.

[0123] Step S207: Determine a cache frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter according to a second preset relationship.

[0124] Among them, the second preset relationship is a mapping relationship between the number of cached frames and the second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the number of cached frames is positively correlated with the second ratio and the third ratio, the second ratio is the ratio of the network transmission bandwidth to the preset size of a single frame of video data, the third ratio is the ratio of the corrected jitter to time, the corrected jitter is obtained by correcting the jitter using a jitter index that is negatively correlated with the jitter, the time and is the sum of the round-trip time and the inter-frame delay, the round-trip time is the total delay of the preset ratio, and the preset ratio is negatively correlated with the total delay.

[0125] Step S208: configuring the cache frame value to the video play cache queue, so that the remote control terminal stores the encoded video stream data into the video play cache queue according to the cache frame value.

[0126] The above steps S204, S206-S208 implement the task of updating and adjusting the cache frame value of the video playback cache queue by the remote control end. By using the cache frame value adapted to the network conditions, a basis is provided for improving the smoothness of video playback and reducing video delay, thereby improving the video transmission quality in a weak network environment.

[0127] In one or more embodiments provided in the present application, the network performance parameter may also include a packet loss rate.

[0128] Based on the above, the process of determining the cache frame value may include:

[0129] Step G, determining a jitter index value corresponding to the parameter value of the jitter, and a preset ratio value corresponding to the parameter value of the total delay;

[0130] Step H, calculating the parameter value ratio of the network transmission bandwidth and the size of the single frame of video data as a second ratio value;

[0131] Step I: using the jitter index value to correct the parameter value of the jitter to obtain a corrected jitter value;

[0132] Step J, calculating the product of the parameter value of the total delay and the preset ratio value as the parameter value of the round trip time; calculating the sum of the parameter value of the round trip time and the parameter value of the inter-frame delay as the parameter value of the time;

[0133] Step K, calculating the ratio of the corrected jitter value to the time and parameter value as a third ratio value;

[0134] Step L, calculating the product of the second ratio value and the third ratio value as the initial value of the cache frame number;

[0135] Step M: Calculate the product of the packet loss rate and the initial value of the cached frame number as the cached frame value.

[0136] The video transmission control device provided in an embodiment of the present application is described below. The device is applied to a remote control terminal. The video transmission control device described below and the video transmission control method described above can be referenced to each other.

[0137] Figure 6 FIG. 1 is a schematic diagram of the structure of a video transmission control device applied to a remote control terminal disclosed in an embodiment of the present application. Figure 6 As shown, the device may include:

[0138] The receiving unit 21 is used to receive a video data packet; the video data packet is a message obtained by packaging the encoded video stream data according to a specified streaming media transmission protocol by the vehicle in communication connection with the remote control terminal, and the encoded video stream data is obtained by encoding the surrounding images of the vehicle captured by the vehicle camera using a predetermined encoding bit rate value by the vehicle;

[0139] The decapsulation unit 22 is used to decapsulate the video data packet to obtain the encoded video stream data and data packet information of the video data packet; the data packet information includes a data packet size and a data packet timestamp;

[0140] The display unit 23 is used to store the encoded video stream data into a video playback cache queue;

[0141] A calculation unit 24, configured to determine parameter values ​​of network performance parameters according to the data packet information of the video data packet, wherein the network performance parameters include network transmission bandwidth, jitter and total delay;

[0142] A parameter transmission unit 25 is used to send the parameter value of the network performance parameter to the vehicle, so that the vehicle can redetermine the encoding rate value based on a first preset relationship and the parameter value of the received network performance parameter, wherein the first preset relationship is a mapping relationship between the encoding rate and a first independent variable, the first independent variable includes the network performance parameter and a safety factor, the encoding rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index that is positively correlated with the vehicle speed, the encoding rate is negatively correlated with the safety factor, and the encoding rate is positively correlated with a first ratio, which is the ratio of the received jitter to the received total delay.

[0143] In one or more embodiments provided in the present application, the data packet information may further include a data packet sequence number; the calculation unit 24 may further be used to determine a parameter value of an inter-frame delay based on data packet timestamps of two adjacent video data packets received with data packet sequence numbers.

[0144] On the basis of the above, the device may further include a cache control unit, which may also be referred to as a low-latency-high-fluency cache queue module LLHFQ (Low-latency-high-fluency Queue), which is used to determine a cache frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter according to a second preset relationship; and configure the cache frame value to the video playback cache queue, so that the remote control terminal stores the encoded video stream data in the video playback cache queue according to the cache frame value; wherein the second preset relationship is a mapping relationship between the number of cache frames and a second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the number of cache frames is positively correlated with the second ratio and positively correlated with the third ratio, the second ratio is the ratio of the network transmission bandwidth to the preset size of a single frame of video data, the third ratio is the ratio of the corrected jitter to the time, the corrected jitter is obtained by correcting the jitter using a jitter index negatively correlated with the jitter, the time and is the sum of the round-trip time and the inter-frame delay, the round-trip time is the total delay of the preset ratio, and the preset ratio is negatively correlated with the total delay.

[0145] In one or more embodiments provided in the present application, the network performance parameter may also include a packet loss rate;

[0146] On the basis of the above, the process of the cache control unit determining the cache frame value may include:

[0147] Determining a jitter index value corresponding to the parameter value of the jitter, and a preset ratio value corresponding to the parameter value of the total delay;

[0148] Calculating a parameter value ratio of the network transmission bandwidth and the size of the single-frame video data as a second ratio value;

[0149] Using the jitter index value to correct the parameter value of the jitter to obtain a corrected jitter value;

[0150] Calculate the product of the parameter value of the total delay and the preset ratio value as the parameter value of the round-trip time; calculate the sum of the parameter value of the round-trip time and the parameter value of the inter-frame delay as the parameter value of the time;

[0151] Calculate the ratio of the corrected jitter value to the time and parameter value as a third ratio value;

[0152] Calculate the product of the second ratio value and the third ratio value as the initial value of the cache frame number;

[0153] The product of the packet loss rate and the initial value of the cached frame number is calculated as the cached frame value.

[0154] In one or more embodiments provided in the present application, the safety factor may further include a jitter safety factor negatively correlated with the vehicle speed, and the jitter safety factor is used to correct the influence of the vehicle speed on the jitter; the parameter value of the safety factor is the sum of the parameter values ​​of the vehicle speed safety index and the jitter safety factor, and the safety factor is positively correlated with the vehicle speed;

[0155] The network performance parameter may further include a packet loss rate, and the encoding bit rate is positively correlated with the packet loss rate;

[0156] The network transmission bandwidth may be a network transmission bandwidth corrected by a bandwidth correction coefficient, wherein the bandwidth correction coefficient is negatively correlated with the network transmission bandwidth and is used to correct the influence of fluctuations in the network transmission bandwidth on the encoding bit rate.

[0157] The present application also provides a remote driving system, which includes: a vehicle, and a remote control terminal connected to the vehicle for communication. Specifically:

[0158] The vehicle encodes the surrounding images of the vehicle collected by the vehicle-mounted camera using a predetermined encoding bit rate value to obtain encoded video stream data; wherein the encoding bit rate value is a parameter value determined by the vehicle according to a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the parameter value of the network performance parameter is a parameter value determined by the remote control terminal according to data packet information of a historical video data packet received from the vehicle, the network performance parameter includes network transmission bandwidth, jitter and total delay, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the speed of the vehicle, the encoding bit rate is negatively correlated with the safety factor, the encoding bit rate is positively correlated with a first ratio, the first ratio is a ratio of the received jitter to the received total delay; the data packet information includes a data packet size and a data packet timestamp;

[0159] The vehicle packages the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, and transmits the video data packet to the remote control terminal;

[0160] The remote control terminal decapsulates the received video data packet to obtain the encoded video stream data and data packet information of the video data packet;

[0161] The remote control terminal stores the encoded video stream data into a video playback cache queue;

[0162] The remote control terminal re-determines the parameter value of the network performance parameter according to the data packet information of the video data packet and sends the parameter value of the network performance parameter to the vehicle;

[0163] The vehicle redetermines the encoding bit rate value based on the first preset relationship and the parameter value of the received network performance parameter.

[0164] In one or more embodiments provided in the present application, the data packet information may further include a data packet sequence number; and the remote control terminal may further be used to:

[0165] Determining a parameter value of an inter-frame delay according to the data packet timestamps of two received video data packets having adjacent data packet sequence numbers;

[0166] Determine a cached frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter according to a second preset relationship; wherein the second preset relationship is a mapping relationship between the number of cached frames and a second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the number of cached frames is positively correlated with a second ratio and a third ratio, the second ratio is a ratio of the network transmission bandwidth to a preset size of a single-frame video data, the third ratio is a ratio of the corrected jitter to time, the corrected jitter is obtained by correcting the jitter using a jitter index negatively correlated with the jitter, the time is the sum of the round-trip time and the inter-frame delay, the round-trip time is a preset ratio of the total delay, and the preset ratio is negatively correlated with the total delay;

[0167] And, the cache frame value is configured to the video play cache queue, so that the remote control end stores the encoded video stream data into the video play cache queue according to the cache frame value.

[0168] Optionally, other descriptions of the vehicle and the remote control terminal may refer to the above description.

[0169] A storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any video transmission control method provided in the embodiment of the present application.

[0170] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the video transmission control methods provided in the embodiments of the present application.

[0171] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0172] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0173] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video transmission control method, characterized in that: Applied to a vehicle, the method comprises: The vehicle surrounding images captured by the vehicle-mounted camera are encoded using a predetermined encoding bit rate value to obtain encoded video stream data; wherein the encoding bit rate value is a parameter value determined by the vehicle according to a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the parameter value of the network performance parameter is a parameter value determined by a remote control terminal connected to the vehicle for communication according to data packet information of a received historical video data packet of the vehicle, the network performance parameters include network transmission bandwidth, jitter and total delay, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the vehicle speed, the encoding bit rate is negatively correlated with the safety factor, the encoding bit rate is positively correlated with a first ratio, the first ratio is a ratio of the received jitter to the received total delay; the data packet information includes a data packet size and a data packet timestamp; Packing the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, transmitting the video data packet to the remote control terminal for decapsulation processing by the remote control terminal, storing the encoded video stream data obtained by the decapsulation processing into a video playback cache queue, and determining and transmitting a parameter value of a new network performance parameter based on data packet information of the video data packet obtained by the decapsulation processing; Upon receiving the parameter value of the network performance parameter transmitted by the remote control terminal, the encoding bit rate value is determined.

2. The video transmission control method according to claim 1, characterized in that: The data packet information further includes a data packet sequence number, the network performance parameter further includes a packet loss rate, and the encoding bit rate is positively correlated with the packet loss rate; Determining the encoding bit rate value includes: Determining a parameter value of the safety factor corresponding to a speed value of the vehicle; Calculating a parameter value of the first ratio according to the received parameter value of the jitter and the received parameter value of the total delay; Calculating a ratio of the received parameter value of the network transmission bandwidth to the parameter value of the safety factor, and calculating a product of the ratio and the parameter value of the first ratio as an initial value of the encoding bit rate; Determine a parameter value of a bandwidth correction coefficient corresponding to the parameter value of the network transmission bandwidth; the bandwidth correction coefficient is negatively correlated with the network transmission bandwidth and is used to correct the influence of fluctuations in the network transmission bandwidth on the encoding bit rate, and the encoding bit rate is positively correlated with the bandwidth correction coefficient; The initial value of the encoding rate is corrected using the parameter value of the bandwidth correction coefficient, and the product of the corrected initial value of the encoding rate and the packet loss rate is calculated to obtain the encoding rate value.

3. The video transmission control method according to claim 2, characterized in that: The safety factor also includes a jitter safety factor that is negatively correlated with the vehicle speed, and the jitter safety factor is used to correct the effect of the vehicle speed on the jitter; Determining a parameter value of the safety factor corresponding to a speed value of the vehicle includes: Determining a parameter value of the vehicle speed safety index and a parameter value of the jitter safety factor corresponding to a vehicle speed value of the vehicle; The sum of the parameter values ​​of the vehicle speed safety index and the jitter safety factor is calculated as the parameter value of the safety factor; wherein the safety factor is positively correlated with the vehicle speed.

4. The video transmission control method according to any one of claims 1 to 3, characterized in that: The cache frame value based on which the remote control terminal stores the encoded video stream data into the video playback cache queue is the cache frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter determined by the remote control terminal according to the second preset relationship; Among them, the second preset relationship is a mapping relationship between the number of cached frames and the second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the parameter value of the inter-frame delay is determined according to the data packet timestamp of two video data packets with adjacent data packet sequence numbers; the number of cached frames is positively correlated with the second ratio and positively correlated with the third ratio, the second ratio is the ratio of the network transmission bandwidth to the preset size of a single frame of video data, the third ratio is the ratio of the corrected jitter to time, the corrected jitter is obtained by correcting the jitter using a jitter index negatively correlated with the jitter, the time is the sum of the round-trip time and the inter-frame delay, the round-trip time is the total delay of the preset ratio, and the preset ratio is negatively correlated with the total delay.

5. A video transmission control method, characterized in that: Applied to the remote control terminal, the method includes: receiving a video data packet; the video data packet is a message obtained by packaging the encoded video stream data according to a specified streaming media transmission protocol by a vehicle in communication connection with the remote control terminal, the encoded video stream data being obtained by encoding the surrounding images of the vehicle captured by the vehicle-mounted camera using a predetermined encoding bit rate value by the vehicle; Decapsulating the video data packet to obtain the encoded video stream data and data packet information of the video data packet; the data packet information includes a data packet size and a data packet timestamp; Storing the encoded video stream data in a video playback cache queue; Determining parameter values ​​of network performance parameters according to the data packet information of the video data packet, wherein the network performance parameters include network transmission bandwidth, jitter and total delay; The parameter value of the network performance parameter is sent to the vehicle so that the vehicle can redetermine the encoding rate value based on a first preset relationship and the parameter value of the received network performance parameter, wherein the first preset relationship is a mapping relationship between the encoding rate and a first independent variable, the first independent variable includes the network performance parameter and a safety factor, the encoding rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the speed of the vehicle, the encoding rate is negatively correlated with the safety factor, and the encoding rate is positively correlated with a first ratio, which is a ratio of the received jitter to the received total delay.

6. The video transmission control method according to claim 5, characterized in that: The data packet information also includes a data packet sequence number; The method further includes: Determining a parameter value of an inter-frame delay according to the data packet timestamps of two received video data packets having adjacent data packet sequence numbers; Determine a cached frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter according to a second preset relationship; wherein the second preset relationship is a mapping relationship between the number of cached frames and a second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the number of cached frames is positively correlated with a second ratio and a third ratio, the second ratio is a ratio of the network transmission bandwidth to a preset size of a single-frame video data, the third ratio is a ratio of the corrected jitter to time, the corrected jitter is obtained by correcting the jitter using a jitter index that is negatively correlated with the jitter, the time is the sum of the round-trip time and the inter-frame delay, the round-trip time is a preset ratio of the total delay, and the preset ratio is negatively correlated with the total delay; The cache frame value is configured to the video play cache queue, so that the remote control end stores the encoded video stream data into the video play cache queue according to the cache frame value.

7. The video transmission control method according to claim 6, characterized in that: The network performance parameters also include packet loss rate; The process of determining the cache frame value includes: Determining a jitter index value corresponding to the parameter value of the jitter, and a preset ratio value corresponding to the parameter value of the total delay; Calculating a parameter value ratio of the network transmission bandwidth and the size of the single-frame video data as a second ratio value; Using the jitter index value to correct the parameter value of the jitter to obtain a corrected jitter value; Calculate the product of the parameter value of the total delay and the preset ratio value as the parameter value of the round trip time; calculate the sum of the parameter value of the round trip time and the parameter value of the inter-frame delay as the parameter value of the time; Calculate the ratio of the corrected jitter value to the time and parameter value as a third ratio value; Calculate the product of the second ratio value and the third ratio value as the initial value of the cache frame number; The product of the packet loss rate and the initial value of the cached frame number is calculated as the cached frame value.

8. The video transmission control method according to claim 7, characterized in that: The safety factor also includes a jitter safety factor that is negatively correlated with the vehicle speed, and the jitter safety factor is used to correct the influence of the vehicle speed on the jitter; the parameter value of the safety factor is the sum of the parameter values ​​of the vehicle speed safety index and the jitter safety factor, and the safety factor is positively correlated with the vehicle speed; The network performance parameter also includes a packet loss rate, and the encoding bit rate is positively correlated with the packet loss rate; The network transmission bandwidth is the network transmission bandwidth corrected by using a bandwidth correction coefficient. The bandwidth correction coefficient is negatively correlated with the network transmission bandwidth and is used to correct the impact of network transmission bandwidth fluctuations on the encoding bit rate.

9. A remote driving system, characterized in that: include: A vehicle, and a remote control terminal communicatively connected to the vehicle; The vehicle encodes the surrounding images of the vehicle collected by the vehicle-mounted camera using a predetermined encoding bit rate value to obtain encoded video stream data; wherein the encoding bit rate value is a parameter value determined by the vehicle according to a first preset relationship and a parameter value of a received network performance parameter, the first preset relationship is a mapping relationship between the encoding bit rate and a first independent variable, the first independent variable includes a network performance parameter and a safety factor, the parameter value of the network performance parameter is a parameter value determined by the remote control terminal according to data packet information of a historical video data packet received from the vehicle, the network performance parameter includes network transmission bandwidth, jitter and total delay, the encoding bit rate is positively correlated with the network transmission bandwidth, the safety factor includes a speed safety index positively correlated with the speed of the vehicle, the encoding bit rate is negatively correlated with the safety factor, the encoding bit rate is positively correlated with a first ratio, the first ratio is a ratio of the received jitter to the received total delay; the data packet information includes a data packet size and a data packet timestamp; The vehicle packages the encoded video stream data according to a pre-specified streaming media transmission protocol to obtain a video data packet, and transmits the video data packet to the remote control terminal; The remote control terminal decapsulates the received video data packet to obtain the encoded video stream data and data packet information of the video data packet; The remote control terminal stores the encoded video stream data into a video playback cache queue; The remote control terminal re-determines the parameter value of the network performance parameter according to the data packet information of the video data packet and sends the parameter value of the network performance parameter to the vehicle; The vehicle redetermines the encoding bit rate value based on the first preset relationship and the parameter value of the received network performance parameter.

10. The remote driving system according to claim 9, characterized in that: The data packet information also includes a data packet sequence number; The remote control terminal is also used for: Determining a parameter value of an inter-frame delay according to the data packet timestamps of two received video data packets having adjacent data packet sequence numbers; Determine a cached frame value corresponding to the inter-frame delay and the parameter value of the network performance parameter according to a second preset relationship; wherein the second preset relationship is a mapping relationship between the number of cached frames and a second independent variable, the second independent variable includes the network performance parameter and the inter-frame delay, the number of cached frames is positively correlated with a second ratio and a third ratio, the second ratio is a ratio of the network transmission bandwidth to a preset size of a single-frame video data, the third ratio is a ratio of the corrected jitter to time, the corrected jitter is obtained by correcting the jitter using a jitter index negatively correlated with the jitter, the time is the sum of the round-trip time and the inter-frame delay, the round-trip time is a preset ratio of the total delay, and the preset ratio is negatively correlated with the total delay; And, the cache frame value is configured to the video play cache queue, so that the remote control end stores the encoded video stream data into the video play cache queue according to the cache frame value.