Video data processing method and device for remote control driving

Abstract

The invention provides a video data processing method and device for remote control driving. The method comprises the following steps: acquiring a current index value of a reference index corresponding to an unmanned vehicle; wherein the reference index is the reference index for adjusting the video quality of the video acquired by the unmanned vehicle; under the condition that the current index value meets a specified video quality adjustment condition, adopting a target video adjustment mode corresponding to the current index value to adjust the currently acquired video data of the unmanned vehicle to obtain target video data; the target video data is sent to a remote control driving terminal, and the remote control driving terminal is used for outputting the target video data so as to guide a remote control driver to carry out remote control driving on the unmanned vehicle. According to the invention, the bandwidth utilization efficiency is improved, and the transmission stability and real-time performance are enhanced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of unmanned driving and remote control driving, and in particular to a video data processing method and device for remote control driving. Background Art

[0002] Remote control driving technology currently mainly uses the vehicle-network-remote control driving terminal architecture to achieve real-time monitoring and control of vehicle status. On the vehicle side, sensors collect various information, such as position, speed, acceleration, steering angle, etc., especially image sensors collect video information while the vehicle is driving. These data are transmitted to the remote control driving terminal in real time through the network (radio or satellite signals). The remote control driving terminal obtains the vehicle status through the received sensor data and video stream, determines the current driving environment of the vehicle, and sends control instructions to the vehicle as needed (for example: continue driving or avoid obstacles, etc.), thereby achieving real-time control of the vehicle.

[0003] The above architecture has high requirements for the real-time nature of the information collected by the vehicle side. However, currently, due to the influence of network transmission signals or network bandwidth, the remote control driving terminal often fails to receive data in a timely manner, resulting in untimely control of the vehicle and increasing the safety risks of the vehicle. Summary of the invention

[0004] The embodiments of the present disclosure provide a video data processing method and device for remote control driving, so as to solve the related problems existing in the existing solutions.

[0005] Based on the above problems, in a first aspect, a video data processing method for remote control driving is provided, comprising:

[0006] Obtaining a current index value of a reference index corresponding to the unmanned vehicle; wherein the reference index is an index for reference in adjusting the video quality of the video collected by the unmanned vehicle;

[0007] When the current index value satisfies the specified video quality adjustment condition, the target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain the target video data;

[0008] The target video data is sent to a remote control driving terminal, wherein the remote control driving terminal is used to output the target video data to guide a remote control driver to remotely control the unmanned vehicle.

[0009] In combination with the first aspect, in a possible implementation manner, the reference indicator includes at least one of the following: vehicle speed, environmental complexity of the driving environment, and network transmission quality.

[0010] In combination with the first aspect, in a possible implementation, when the current index value satisfies a specified video quality adjustment condition, a target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain target video data, including:

[0011] In a case where the current indicator value satisfies a specified video quality adjustment condition, determining a target video encoding parameter corresponding to the current indicator value;

[0012] Based on the target video encoding parameters, the encoding of the currently acquired video data is adjusted to obtain the target video data.

[0013] In combination with the first aspect, in a possible implementation manner, adjusting the encoding of the currently acquired video data based on the target video encoding parameter to obtain the target video data includes:

[0014] Pre-establishing the corresponding relationship between different indicator values ​​and different target video encoding parameters;

[0015] Determine the target video encoding parameter according to the corresponding relationship and the current indicator value;

[0016] Acquire current video encoding parameters, and determine encoding adjustment parameters according to the target video encoding parameters and the current video encoding parameters;

[0017] The encoding adjustment parameters are used to adjust the encoding of the currently collected video data of the unmanned vehicle to obtain the target video data.

[0018] In combination with the first aspect, in a possible implementation, when the reference index includes vehicle speed, the lower the vehicle speed, the worse the quality of target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or

[0019] In the case where the reference index includes the environmental complexity of the driving environment, the lower the environmental complexity of the driving environment, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or

[0020] In the case where the reference indicator includes network transmission quality, the worse the network transmission quality is, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current indicator value is.

[0021] In combination with the first aspect, in a possible implementation manner, adjusting the encoding of the currently acquired video data based on the target video encoding parameter to obtain the target video data includes:

[0022] Determining encoding adjustment parameters corresponding to the target video encoding parameters;

[0023] The encoding of the currently acquired video data is adjusted using the encoding adjustment parameters to obtain the target video data; wherein the target video encoding parameters include a target bit rate, and the encoding adjustment parameters corresponding to the target bit rate include a first target compression parameter.

[0024] In combination with the first aspect, in a possible implementation manner, the using the encoding adjustment parameter to adjust the encoding of the currently acquired video data to obtain the target video data includes:

[0025] Determine target continuous frames from the video currently collected by the unmanned vehicle according to the first frame position of the continuous frames to be compressed represented by the first target compression parameter and the number of video frames contained in each continuous frame;

[0026] Dividing each video frame included in the target continuous frames into a plurality of pixel blocks according to a preset pixel block division method;

[0027] For each target continuous frame, determine the pixel blocks corresponding to the target position in each frame of the target continuous frame;

[0028] For each pixel block corresponding to the target position, respectively determine the difference between the pixel values ​​of the pixel blocks corresponding to the target position of other frames except the first frame in the target continuous frames and the pixel block corresponding to the target position of the first frame, and obtain the difference pixel blocks corresponding to the other frames respectively;

[0029] Based on each difference pixel block, determining the difference video frames corresponding to the other frames respectively;

[0030] For each target continuous frame, the target video data is obtained based on the difference video frames corresponding to the first frame corresponding to the target continuous frame and the other frames respectively.

[0031] In combination with the first aspect, in a possible implementation manner, determining the difference video frames corresponding to the other frames respectively based on the difference pixel blocks includes:

[0032] performing pixel quantization processing on the obtained difference pixel block according to the quantization parameter represented by the first target compression parameter to obtain a processed difference pixel block;

[0033] Based on the processed difference pixel blocks respectively corresponding to each pixel block of the other frames, the difference video frames respectively corresponding to the other frames are determined.

[0034] In combination with the first aspect, in a possible implementation manner, using the encoding adjustment parameter to adjust the encoding of the currently acquired video data to obtain the target video data includes:

[0035] Determining a target video frame from the video currently collected by the unmanned vehicle according to the first target compression parameter;

[0036] Dividing each target video frame into a plurality of pixel blocks according to a preset pixel block division method;

[0037] Determine a reference pixel block in each target video frame according to a position and / or a number of reference pixel blocks in each video frame represented by the first target compression parameter;

[0038] For each reference pixel block, determine a first pixel block having a preset positional relationship with the reference pixel block, and respectively determine the difference between each first pixel block and the corresponding pixel in the reference pixel block to obtain a difference pixel block corresponding to each first pixel block;

[0039] The target video data is obtained based on the difference pixel blocks respectively corresponding to each reference pixel block and each first pixel block.

[0040] In combination with the first aspect, in a possible implementation, when the current index value satisfies a specified video quality adjustment condition, a target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain target video data, including:

[0041] In the case where the current indicator value satisfies the specified video quality adjustment condition, determining a target video adjustment parameter corresponding to the current indicator value; wherein the target video adjustment parameter includes a target frame rate and / or a target resolution;

[0042] Based on the target video adjustment parameters, the currently collected video data is adjusted accordingly to obtain the target video data.

[0043] In combination with the first aspect, in a possible implementation, when the target video adjustment parameter includes a target frame rate, the corresponding adjustment of the current acquired video data based on the target video adjustment parameter includes: determining a frame extraction position corresponding to the target frame rate, and deleting a video frame corresponding to the frame extraction position in the current acquired video data; or,

[0044] In the case where the target video adjustment parameters include a target resolution, the corresponding adjustment to the currently acquired video data is performed based on the target video adjustment parameters, including: determining a pixel set adjustment position corresponding to the target resolution, and deleting a pixel set at the pixel set adjustment position in each video frame of the currently acquired video data, wherein the pixel set includes pixel rows and / or pixel columns.

[0045] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0046] Receiving video quality feedback information sent by the remote control driving terminal;

[0047] In a case where the video quality feedback information indicates that the video quality needs to be improved, determining a target indicator value based on the current indicator value, and determining a first video adjustment method corresponding to the target indicator value;

[0048] The currently collected video data is adjusted by using the first video adjustment method to obtain target video data with improved video quality;

[0049] Among them, after the collected video data is adjusted by using the first video adjustment method corresponding to the target indicator value, the quality of the obtained video data is higher than the video quality corresponding to the current indicator value.

[0050] In combination with the first aspect, in a possible implementation manner, the environmental complexity of the driving environment of the unmanned vehicle is determined by at least one of the following methods:

[0051] Determining the scene in which the unmanned vehicle is located according to the current operating state of the unmanned vehicle, and determining the environmental complexity according to the scene;

[0052] Determining the environmental complexity according to terrain data corresponding to the current location of the unmanned vehicle;

[0053] Determining the environmental complexity according to the perception result of the surrounding environment by the perception device of the unmanned vehicle;

[0054] Receive surrounding vehicle information sent by other vehicles and / or a cloud control platform, and determine the environmental complexity based on the surrounding vehicle information.

[0055] In a second aspect, a video data processing device for remote control driving is provided, comprising:

[0056] An index value acquisition module, used to obtain the current index value of the reference index corresponding to the unmanned vehicle; wherein the reference index is an index for reference to adjust the video quality of the video collected by the unmanned vehicle;

[0057] A video adjustment module, configured to adjust the currently collected video data of the unmanned vehicle by adopting a target video adjustment method corresponding to the current index value to obtain target video data when the current index value satisfies a specified video quality adjustment condition;

[0058] The video transmission module is used to send the target video data to the remote control driving terminal, wherein the remote control driving terminal is used to output the target video data to guide the remote control driver to remotely control the unmanned vehicle.

[0059] The beneficial effects of the embodiments of the present disclosure include:

[0060] The disclosed embodiment provides a video data processing method and device for remote control driving, which is applied to the remote control driving of an unmanned vehicle, and determines whether it is necessary to adjust the quality of the video data collected by the unmanned vehicle based on multiple reference indicators to obtain the adapted target video quality. The disclosed embodiment first obtains the current index value corresponding to the reference index that will affect the video quality requirement, and then determines the target video adjustment method corresponding to the current index value when the current index value meets the specified video quality adjustment condition, and adjusts the video data according to the target video adjustment method to obtain the target video data, which can be applied to the requirements of variable driving scenes and match the most suitable video quality for different scenes. Finally, the target video data is sent to the remote control driving terminal. Since the higher the video quality, the greater the bandwidth occupied by the video data, the long-term high-quality video data transmission causes the network to be overwhelmed, and the data cannot be transmitted to the remote control driving terminal in time, which makes the vehicle operation have safety hazards. The disclosed embodiment determines the current video quality requirement based on the current index value of the reference index, adjusts the video data collected by the unmanned vehicle according to the required video quality, and then transmits it to the remote control driving terminal. By using the bandwidth that the network can bear, video data that meets the video quality requirements can be sent to the remote control driving terminal in a timely manner, so that the remote control driving terminal can remotely control the unmanned vehicle in a timely manner based on the acquired video that meets the current quality requirements, thereby enhancing the safety of the unmanned vehicle operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flow chart of a video data processing method for remote control driving provided in an embodiment of the present disclosure;

[0062] Figure 2 One of the structural schematic diagrams of a video data processing device for remote control driving provided by an embodiment of the present disclosure;

[0063] Figure 3 The second structural schematic diagram of a video data processing device for remote control driving provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0064] The embodiments of the present disclosure provide a video data processing method and device for remote control driving. The preferred embodiments of the present disclosure are described below in conjunction with the drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0065] The present disclosure provides a video data processing method for remote control driving. Figure 1 As shown, including:

[0066] S101, obtaining a current index value of a reference index corresponding to the unmanned vehicle; wherein the reference index is an index for reference to adjust the video quality of the video collected by the unmanned vehicle;

[0067] S102, when the current index value satisfies the specified video quality adjustment condition, adjusting the current collected video data of the unmanned vehicle using a target video adjustment method corresponding to the current index value to obtain target video data;

[0068] S103: Send the target video data to the remote control driving terminal, wherein the remote control driving terminal is used to output the target video data to guide the remote control driver to remotely control the unmanned vehicle.

[0069] In the disclosed embodiment, the unmanned vehicle can be controlled by a remote control driver through a remote control driving terminal. The unmanned vehicle collects video data during driving and sends it to the remote control driving terminal in real time. The remote control driving terminal determines the current driving environment of the unmanned vehicle based on the received video data, and sends control instructions to the unmanned vehicle in a timely manner according to the driving environment, such as: continue driving or avoid obstacles, etc. This puts higher requirements on the timeliness of video data transmission. Once the amount of data is too large and the network bandwidth cannot bear it, the timeliness of data transmission will be affected, thereby affecting the decision of the remote control driver. However, if the video quality is reduced blindly in order to adapt to the network bandwidth, the video quality is too poor to make the remote control driver unable to determine the current driving environment of the unmanned vehicle through the received video, which will also affect the judgment of the remote control driver, and the correct control instructions cannot be sent in time, affecting the driving safety of the unmanned vehicle.

[0070] In the disclosed embodiment, at least one reference indicator can be determined, and the indicator value of the reference indicator can be used to determine the video quality requirement of the remote control driving terminal, thereby determining the current video quality requirement of the remote control driving terminal by determining the current indicator value of the reference indicator. When the current indicator value meets the specified video quality adjustment condition, the target video adjustment method corresponding to the current indicator value is adopted to adjust the current collected video data of the unmanned vehicle. The target video data obtained in this way can adapt to the current bandwidth conditions and meet the current video quality requirement of the remote control driving terminal, so that the remote control driver can obtain video data that meets the video quality requirement in a timely manner, thereby controlling the unmanned vehicle in a timely manner and improving the safety of the unmanned vehicle operation.

[0071] During implementation, the current index values ​​of the reference indexes corresponding to the driverless vehicle can be obtained first. When the current index values ​​corresponding to these reference indexes are different, the driver will have different perceptions of the received video data, and even affect the driver's driving operation. Therefore, when the current index value changes, the driver has different requirements for video quality, and it is necessary to match the appropriate video quality for different current index values. Therefore, first determine whether the current index value meets the specified video quality adjustment conditions.

[0072] If the current index value meets the specified video quality adjustment condition, the system will use the target video adjustment method corresponding to the current index value to adjust the current collected video data of the unmanned vehicle to obtain the target video data. This adjustment process may include compression, encoding, filtering, enhancement and other video processing technologies to optimize or reduce the current video quality.

[0073] In the disclosed embodiments, video quality adjustment conditions can be preset, for example: vehicle speed increase or decrease, terrain changes, road conditions changes, and even network status changes can all be used to set video quality adjustment conditions. For example, a vehicle's driving speed on a highway is usually faster than in a city, requiring a more timely response from the remote control driver, so the video quality requirement becomes higher; when a vehicle enters an area with complex terrain or a congested road, obstacles increase and careful driving is required, so the video quality requirement becomes higher; when the network transmission quality deteriorates, the video quality requirement becomes lower, etc. Real-time monitoring of reference indicators ensures that the video data can meet the needs of subsequent processing and analysis.

[0074] Finally, the system sends the adjusted target video data to the remote control driving terminal, which is responsible for receiving the target video data, decoding it and outputting it to the visualization device for the remote control driver to perform driving operations. In this way, the remote control driver can observe the environment around the unmanned vehicle in real time and make accurate remote control driving decisions. It effectively supports remote control driving operations, ensures that the remote control driver can obtain appropriate video quality display, and improves the safety and reliability of remote control driving.

[0075] In the disclosed embodiments, the remote control driving terminal may be in various forms, including but not limited to a tablet computer, a smart phone, a professional remote control driving console, etc.

[0076] In another embodiment provided by the present disclosure, the reference index includes at least one of the following: vehicle speed, environmental complexity of the driving environment, and network transmission quality.

[0077] In the disclosed embodiment, the reference index includes at least one of the following: vehicle speed, environmental complexity of the driving environment, and network transmission quality. Specifically, vehicle speed refers to the speed of the vehicle on the road, which can be an instantaneous speed or an average speed. For example, if the current vehicle speed is fast and higher than the vehicle speed corresponding to the preset video quality adjustment condition, the target video adjustment method corresponding to the current index value vehicle speed can be used to adjust the video to improve the video quality, so as to ensure that the image is clear and visible, reduce image blur and delay, and avoid affecting the driving operation of the remote control driver.

[0078] The environmental complexity of the driving environment is a comprehensive indicator to measure the difficulty of the driving environment. It includes but is not limited to factors such as traffic density, road type, weather conditions, and lighting conditions. For example, a busy intersection has a higher environmental complexity than an empty country road. The assessment of environmental complexity is particularly important for autonomous vehicles because it can help the vehicle identify potential risks and take appropriate safety measures. Usually, when the environmental complexity of the driving environment is high, higher video quality is required so that the remote control driver can clearly observe the surrounding environment and send correct control instructions in a timely manner.

[0079] The quality of network transmission determines the efficiency and stability of data exchange between the unmanned vehicle and the remote control driving terminal. It is evaluated through parameters such as data transmission rate, delay, and packet loss rate. In vehicle-to-everything (V2X) technology, network transmission quality is crucial to ensure that vehicles can receive and send key information in a timely manner. For example, when a vehicle exchanges information with other vehicles or traffic infrastructure through the vehicle communication system, the quality of network transmission directly affects the accuracy and real-time nature of the information, which in turn affects driving decisions and driving safety. Therefore, network transmission quality can be determined as a reference indicator.

[0080] In summary, the comprehensive consideration of these reference indicators determines the target video data, which improves the vehicle's driving safety and driving experience. By real-time monitoring and analysis of the current indicator values ​​corresponding to these reference indicators, it can better adapt to different driving scenarios and provide auxiliary decision-making for the driver.

[0081] In another embodiment provided by the present disclosure, the above S102, when the current index value satisfies the specified video quality adjustment condition, uses the target video adjustment method corresponding to the current index value to adjust the current collected video data of the unmanned vehicle to obtain the target video data, which can be implemented as follows:

[0082] Step 1: when the current index value satisfies the specified video quality adjustment condition, determine the target video encoding parameter corresponding to the current index value;

[0083] Step 2: Based on the target video encoding parameters, the encoding of the currently acquired video data is adjusted to obtain the target video data.

[0084] In the disclosed embodiment, when the current index value satisfies the specified video quality adjustment condition, the target video adjustment method corresponding to the current index value is used to adjust the current collected video data of the unmanned vehicle. The adjustment here may include adjustment of video encoding, for example, adjustment of video bit rate. Therefore, the target video adjustment method corresponding to the current index value may be a target video encoding parameter corresponding to the current index value, and the encoding of the current collected video data is adjusted based on the target video encoding parameter to obtain the target video data.

[0085] Taking bit rate as an example, if the reference indicator includes speed, and the current indicator value indicates the need to improve video quality, then the target video encoding parameters may include a higher bit rate setting; if the reference indicator includes the environmental complexity of the driving environment, and the current indicator value indicates the need to reduce video quality, then the target video encoding parameters may include a lower bit rate setting.

[0086] In another embodiment provided by the present disclosure, the above step 2, adjusting the encoding of the currently acquired video data based on the target video encoding parameter to obtain the target video data, can be implemented as follows:

[0087] Step 1: pre-establishing the corresponding relationship between different index values ​​and different target video encoding parameters;

[0088] Step 2: Determine the target video encoding parameters according to the corresponding relationship and the current indicator value;

[0089] Step 3: Obtain current video encoding parameters, and determine encoding adjustment parameters according to target video encoding parameters and current video encoding parameters;

[0090] Step 4: Use the encoding adjustment parameters to adjust the encoding of the current collected video data of the unmanned vehicle to obtain the target video data.

[0091] In the disclosed embodiment, in order to facilitate the unmanned vehicle to determine the adjustment method for the video quality according to the current index value in a timely manner, the correspondence between different index values ​​and different target video encoding parameters can be established in advance. For example, a table can be established, which contains a variety of different reference indicators, such as vehicle speed, environmental complexity of the driving environment, and network transmission quality, as well as corresponding video encoding parameters. These parameters may include but are not limited to bit rate, compression rate, frame rate, resolution, etc. For example, when the current index value shows that the network transmission quality is poor, the correspondence table may recommend the use of a lower bit rate and a lower compression rate.

[0092] In actual operation, the current vehicle speed, environmental complexity of the driving environment, and network transmission quality and other index values ​​can be monitored in real time, so as to determine the target video encoding parameters at any time according to the corresponding relationship and the current index value. Then, it is also necessary to obtain the current video encoding parameters, and determine the encoding adjustment parameters based on the target video encoding parameters and the current video encoding parameters. If the current video encoding parameters are close to the target video encoding parameters (for example: within a preset range), the encoding adjustment parameters corresponding to the current video encoding parameters can be determined. If the difference between the two is large, the encoding adjustment parameters corresponding to the target video encoding parameters can be determined, and the encoding of the current collected video data of the unmanned vehicle can be adjusted based on the determined encoding adjustment parameters.

[0093] The encoding adjustment parameter may be an adjustment parameter used when encoding a video using a video encoding parameter. Taking the video encoding parameter as the bit rate as an example, the video bit rate is the number of bits output per second of the video data (video color quantity, brightness quantity, pixel quantity), generally in kbps. The video encoder (for example: H.264) provided in the related art can automatically encode the video by setting a specific value of the bit rate, and output the target video data that meets the set bit rate. However, this encoding method is like a black box for the user, and the user does not know the specific adjustment method. If the video needs to be encoded more carefully and accurately, it is necessary to adjust the video through more specific adjustment parameters, that is, encoding adjustment parameters. Continuing to take the bit rate as an example, in the process of adjusting the bit rate, a series of compression processes are actually performed on the video data, so the encoding adjustment parameters of the bit rate can include compression parameters, such as: the first frame position of the continuous frame during inter-frame compression, the length of the continuous frame, etc. The compression parameters provided by the present disclosure will be described in detail in the subsequent embodiments. It can be seen that in the embodiments of the present disclosure, the corresponding encoding adjustment parameters can be determined according to the target video encoding parameters, so that the video can be encoded more carefully and accurately.

[0094] In another embodiment provided by the present disclosure, when the reference index includes vehicle speed, the lower the vehicle speed, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or

[0095] In the case where the reference index includes the environmental complexity of the driving environment, the lower the environmental complexity of the driving environment, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or

[0096] When the reference indicator includes network transmission quality, the worse the network transmission quality is, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current indicator value is.

[0097] In the embodiment of the present disclosure, in actual operation, when the reference index includes vehicle speed, a decrease in vehicle speed will correspondingly reduce the quality of video data. This is because when the unmanned vehicle is traveling at a slower speed, the remote control driver does not need to respond too quickly to meet the driving needs of the unmanned vehicle.

[0098] When the reference indicator includes the environmental complexity of the driving environment, the reduction of environmental complexity will also correspond to the reduction of video data quality. Low environmental complexity means that the unmanned vehicle may be driving in a relatively simple and open environment, such as a highway or an empty country road. In this case, the remote control driver does not need too high video quality, so the video quality can be reduced through the corresponding target video adjustment method.

[0099] In the case where the reference indicator includes network transmission quality, the decline in network transmission quality will also correspond to the decline in video data quality. When network conditions are poor, such as in areas with poor signal coverage or network congestion, video data may experience packet loss or delay during transmission, which will directly affect the smoothness and clarity of the video. In this case, if the system continues to use the target video adjustment method corresponding to the current indicator value, it may cause the video data to further deteriorate during transmission, thereby affecting the judgment of the remote control driver. Appropriately reducing the video quality can reduce the size of the transmitted video. On the basis of not affecting the remote control driver's ability to see the video clearly, it can adapt to the bandwidth so that the video is transmitted intact to the remote control driving terminal.

[0100] In summary, this embodiment matches the corresponding target video adjustment method by considering at least one of the three key reference indicators: vehicle speed, environmental complexity of the driving environment, and network transmission quality. Then the quality of the target video is adjusted. In this way, the system can better adapt to different driving conditions and network conditions, ensure clear and smooth video output in all situations, and there will be no waste of resources, providing strong support for the safe driving and remote monitoring of unmanned vehicles.

[0101] In another embodiment provided by the present disclosure, the above step 2, adjusting the encoding of the currently acquired video data based on the target video encoding parameter to obtain the target video data, can be implemented as follows:

[0102] Step 1: Determine the encoding adjustment parameters corresponding to the target video encoding parameters;

[0103] Step 2: Use the encoding adjustment parameters to adjust the encoding of the currently collected video data to obtain target video data; wherein the target video encoding parameters include a target bit rate, and the encoding adjustment parameters corresponding to the target bit rate include a first target compression parameter.

[0104] As described in the above embodiment, the target video encoding parameter may include a target bit rate, and the encoding adjustment parameter corresponding to the target bit rate may include a first target compression parameter. If the target bit rate is low and the corresponding video size is small, the compression rate of the video represented by the first target compression parameter is high, which usually means sacrificing a certain video quality to reduce the amount of data. On the contrary, if the target bit rate is high and the corresponding video size is large, the compression rate of the video represented by the first target compression parameter is low to maintain a high video quality.

[0105] In another embodiment provided by the present disclosure, the above step 2, using the encoding adjustment parameter to adjust the encoding of the currently acquired video data to obtain the target video data, can be implemented as follows:

[0106] Step a, determining the target continuous frames from the video currently collected by the unmanned vehicle according to the first frame position of the continuous frames to be compressed represented by the first target compression parameter and the number of video frames contained in each continuous frame;

[0107] Step b, dividing each video frame included in the target continuous frame into a plurality of pixel blocks according to a preset pixel block division method;

[0108] Step c: for each target continuous frame, determine the pixel block corresponding to the target position in each frame of the target continuous frame;

[0109] Step d, for each pixel block corresponding to the target position, respectively determine the difference between the pixel values ​​of the pixel blocks corresponding to the target position of other frames except the first frame in the target continuous frames and the pixel blocks corresponding to the target position of the first frame, and obtain the difference pixel blocks corresponding to other frames respectively;

[0110] Step e: based on each difference pixel block, determining the difference video frames corresponding to other frames respectively;

[0111] Step f: for each target continuous frame, based on the difference video frames corresponding to the first frame of the target continuous frame and the other frames respectively, the target video data is obtained.

[0112] The video adjustment method provided in the embodiments of the present disclosure is mainly aimed at inter-frame compression. The first target compression parameter represents the adjustment parameter corresponding to the inter-frame compression, for example: the first frame position of the continuous frames to be compressed and the number of video frames contained in each continuous frame. Different video qualities correspond to different first target compression parameters. When performing inter-frame compression, different target video data sizes correspond to different compression methods. In the case where the target video data needs to be large, the first frame position of the continuous frames to be compressed can be selected to be more dispersed, and the number of video frames contained in each continuous frame is small. In this way, the number of video frames to be compressed is correspondingly reduced, and the obtained target video is larger and the target video quality is higher.

[0113] During implementation, the first target compression parameter can be quantized, that is, the first target compression parameter corresponding to the target video encoding parameter (target bit rate) corresponding to different video qualities is determined respectively, and a corresponding relationship is established in advance, so that the corresponding first target compression parameter can be obtained through the target bit rate, and the video compression task can be completed efficiently. The corresponding relationship can be obtained based on actual experience, for example: if the target bit rate is 2M, then the first frame position of the continuous frames to be compressed in the first target compression parameter is determined as one first frame every ten frames, the number of video frames contained in each continuous frame can be five frames, and the pixel block division method can be a division method using 8x8 pixel blocks, then each video frame will be divided into multiple 8x8 pixel blocks.

[0114] Furthermore, for each target continuous frame, the pixel blocks corresponding to the target position of each frame in the target continuous frame are determined. Assuming that the target continuous frames are the 10th (the first frame in the target continuous frames), 12th, 14th, 16th, 18th, and 20th frames (the continuous video frames can also be selected at intervals, and the selection method is not limited here), the pixel blocks are divided in such a way that one frame includes 100 pixel blocks. Since the pixel blocks of each video frame are divided in the same way, then for each position where the pixel blocks are located in a video frame, each video frame has a corresponding pixel block at that position, and the pixel blocks at the target position can be determined first (for example: when the video frame is divided in units of pixel blocks, the pixel blocks located in the first row and first column of the video frame). Then, respectively determine the difference between the pixel values ​​of the pixel blocks corresponding to the target positions (pixel blocks in the first row and first column) of the other frames (12th, 14th, 16th, 18th, and 20th frames) except the first frame (10th frame) in the target continuous frames and the pixel blocks corresponding to the target positions (pixel blocks in the first row and first column) of the first frame, and obtain the difference pixel blocks corresponding to the other frames.

[0115] In this way, for each other frame, the difference pixel blocks contained in the frame can be used to form the difference video frame corresponding to the frame, and compression is achieved compared with the original video frame. Then, based on the original video frame corresponding to the first frame corresponding to the target continuous frame and the difference video frames corresponding to the other frames, the target video data is obtained. For example, by combining the difference video frames of the 12th, 14th, 16th, 18th, and 20th frames with the original video frame of the 10th frame, the target video frames after inter-frame compression of the 10th, 12th, 14th, 16th, 18th, and 20th frames can be obtained. In summary, the compression processing of the original video data is completed, and the target video data is obtained, while maintaining the key information and visual quality of the video content.

[0116] In another embodiment provided by the present disclosure, the above step e determines the difference video frames corresponding to other frames based on the difference pixel blocks, which can be implemented as follows:

[0117] Step 1: performing pixel quantization processing on the obtained difference pixel block according to the quantization parameter represented by the first target compression parameter to obtain a processed difference pixel block;

[0118] Step 2: Based on the processed difference pixel blocks corresponding to each pixel block of other frames, determine the difference video frames corresponding to the other frames.

[0119] In order to increase the compression degree of the video, the first target compression parameter may also include a quantization parameter, and the obtained difference pixel block is subjected to pixel quantization processing by the quantization parameter to obtain the difference pixel block after quantization processing. Specifically, the quantization parameter may be pre-set according to the compression standard, such as the quantization table in the JPEG or H.264 standard. In a possible implementation, the quantization process involves dividing each pixel value in the difference pixel block by the quantization parameter and then rounding to the nearest integer, thereby reducing the amount of data. For example, if the quantization parameter is 8, then the pixel value 123 will be quantized to 15 (123 divided by 8 is equal to 15.375, which is rounded to 15). This process can reduce the accuracy of the data in the video frame, thereby achieving the purpose of compression. In order to optimize the compression effect, the quantization parameter can be dynamically adjusted according to the complexity of the image content to balance the compression rate and image quality. Other quantization methods may also be used during implementation, such as: pixel value minus quantization parameter, etc., which will not be repeated here. Then, based on the processed difference pixel blocks corresponding to each pixel block of other frames, the difference video frames corresponding to the other frames are determined.

[0120] In another embodiment provided by the present disclosure, the above step 2, using the encoding adjustment parameter to adjust the encoding of the currently acquired video data to obtain the target video data, can be implemented as follows:

[0121] Step a: determining a target video frame from the video currently collected by the unmanned vehicle according to the first target compression parameter;

[0122] Step b, dividing each target video frame into a plurality of pixel blocks according to a preset pixel block division method;

[0123] Step c, determining the reference pixel blocks in each target video frame according to the position and / or number of the reference pixel blocks in each video frame represented by the first target compression parameter;

[0124] Step d: for each reference pixel block, determine a first pixel block having a preset positional relationship with the reference pixel block, and respectively determine the difference between each first pixel block and the corresponding pixel in the reference pixel block to obtain a difference pixel block corresponding to each first pixel block;

[0125] Step e: obtaining target video data based on the difference pixel blocks corresponding to each reference pixel block and each first pixel block.

[0126] Similarly, the video adjustment method provided in the embodiments of the present disclosure is mainly aimed at intra-frame compression. The first target compression parameter characterizes the adjustment parameter corresponding to the intra-frame compression, for example: the target video frame determined from the video currently collected by the unmanned vehicle, the position and / or number of reference pixel blocks in each video frame, etc. When performing intra-frame compression, the sizes of different target video data correspond to different compression methods. In the case where the target video data needs to be larger, the number of acquired target video frames can be reduced, the number of reference pixel blocks in each video frame can be reduced, or the position of the reference pixel blocks in each video frame can be dispersed, etc., so that the video frame is less compressed, the obtained target video is larger, and the target video quality is higher.

[0127] During implementation, the first target compression parameter can be quantified, that is, the corresponding first target compression parameter is determined for the target video encoding parameters (target bit rate) corresponding to different video qualities, and a corresponding relationship is established in advance, so that the corresponding first target compression parameter can be obtained through the target bit rate, and the video compression task can be completed efficiently. This corresponding relationship can be obtained based on actual experience. For example, if the target bit rate is 2M, the target video frame determined from the current collected video of the unmanned vehicle in the corresponding first target compression parameter is a target video frame determined every ten frames, and the reference pixel blocks in each video frame are 50, etc. The pixel block division method can be a division method using 8x8 pixel blocks.

[0128] Further, according to the position and / or number of the reference pixel blocks in each video frame characterized by the first target compression parameter, the reference pixel blocks in each target video frame are determined. For example, the algorithm identifies the reference pixel blocks in each video frame according to the rules specified by the compression parameters, and these pixel blocks are used as reference points in the compression process. The selection of the reference pixel blocks may be based on their importance in the image, such as edges or areas with rich textures, to ensure that the details of these areas are preserved during the compression process. For each reference pixel block, a first pixel block having a preset positional relationship with the reference pixel block is determined, and the difference between each first pixel block and the corresponding pixel in the reference pixel block is determined respectively, so as to obtain the difference pixel blocks corresponding to each first pixel block. For example, the processing unit will find pixel blocks adjacent to the reference pixel block or in a specific mode as the first pixel blocks. Then, the pixel differences between these pixel blocks and the reference pixel blocks are calculated to generate difference pixel blocks. These difference pixel blocks represent areas with less changes in the image and can be used for further compression processing. The processing unit recombines all the reference pixel blocks and the difference pixel blocks to form compressed video frames. These video frames are then encoded and stored or transmitted. In this way, the original video data is compressed to maintain the necessary image quality while reducing the data volume, thereby improving the efficiency of storage and transmission.

[0129] In another embodiment provided by the present disclosure, the above step e, the difference pixel blocks corresponding to each reference pixel block and each first pixel block respectively, obtain the target video data, which can be implemented as follows:

[0130] Step 1: performing pixel quantization processing on each obtained reference pixel block and difference pixel block according to the quantization parameter represented by the first target compression parameter to obtain a processed difference pixel block;

[0131] Step 2: Based on the processed difference pixel blocks corresponding to each pixel block of other frames, determine the difference video frames corresponding to the other frames.

[0132] In the disclosed embodiment, in order to increase the compression degree of the video, the first target compression parameter may also include a quantization parameter, and the obtained difference pixel block is subjected to pixel quantization processing by the quantization parameter to obtain the difference pixel block after quantization processing. For example, if the quantization parameter indicates the use of 8-bit quantization, then each pixel value will be limited to the range of 0 to 255. During the quantization process, different quantization strategies may be adopted, such as uniform quantization or non-uniform quantization, to meet different compression requirements and quality requirements. The quantized difference pixel block will have fewer bits, thereby achieving the purpose of compressing data.

[0133] In another embodiment provided by the present disclosure, when the current index value satisfies the specified video quality adjustment condition, the target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain the target video data, which can be implemented as follows:

[0134] When the current indicator value satisfies the specified video quality adjustment condition, determining a target video adjustment parameter corresponding to the current indicator value; wherein the target video adjustment parameter includes a target frame rate and / or a target resolution;

[0135] Based on the target video adjustment parameters, the currently collected video data is adjusted accordingly to obtain the target video data.

[0136] In the disclosed embodiment, when it is detected that the current video quality index value meets the preset video quality adjustment condition, the target video adjustment method corresponding to the current index value is determined, and the adjustment here may also include adjustment of the video frame rate and / or resolution. The target video adjustment parameters mainly include the target frame rate and / or the target resolution.

[0137] In another embodiment provided by the present disclosure, when the target video adjustment parameter includes a target frame rate, the corresponding adjustment of the current acquired video data based on the target video adjustment parameter includes: determining a frame extraction position corresponding to the target frame rate, and deleting a video frame corresponding to the frame extraction position in the current acquired video data; or

[0138] In the case where the target video adjustment parameters include a target resolution, the corresponding adjustment to the currently acquired video data is performed based on the target video adjustment parameters, including: determining a pixel set adjustment position corresponding to the target resolution, and deleting a pixel set at the pixel set adjustment position in each video frame of the currently acquired video data, wherein the pixel set includes pixel rows and / or pixel columns.

[0139] In the disclosed embodiment, on the one hand, when the target video adjustment parameters include a target frame rate, the currently acquired video data is adjusted accordingly according to the target frame rate. First, the frame extraction position corresponding to the target frame rate is determined. It can be understood that frame extraction refers to the process of selecting specific frames from a video sequence according to certain rules. For example, if the target frame rate is 30fps and the original video is 60fps, the system will delete one frame every other frame to achieve the target frame rate. This process can accurately calculate the position of the frames to be deleted through an algorithm to ensure a balance between video smoothness and quality.

[0140] For example, assuming that the frame rate of the original video is 60fps and the target frame rate is 24fps, a rule of deleting one frame every two frames is determined, and the corresponding frame extraction position is determined, and the video frame at the corresponding frame extraction position is deleted, so as to ensure the continuity of the video playback. In actual operation, timestamps or frame numbers may be used to identify and delete specific frames. In addition, in order to further optimize the video quality, interpolation processing can be performed before deleting frames to smooth the differences between transition frames.

[0141] On the other hand, when the target video adjustment parameters include the target resolution, the currently collected video data will also be adjusted accordingly according to the target resolution. In this process, the pixel set adjustment position corresponding to the target resolution will be determined. It can be understood that the pixel set adjustment usually involves the deletion of specific pixel rows and / or pixel columns in the video frame. For example, if the target resolution is 1280x720, and the resolution of the original video is 1920x1080, the width and height of the video frame need to be reduced to adapt to the new resolution.

[0142] When executing the deletion of the pixel set, based on the determined pixel set adjustment position, the position of the pixel row and / or column corresponding to the pixel set adjustment position is deleted. In a possible implementation, this process can be implemented by an algorithm to ensure that the deleted pixels will not have too much impact on the overall quality of the video. For example, the system may choose to delete the edge pixel rows and columns because these pixels usually contain less key information and have less impact on the visual effect after deletion. In addition, in order to maintain the aspect ratio of the video, the system may adopt a proportional scaling method when adjusting the resolution to effectively adjust the resolution of the video while maintaining the clarity and integrity of the video content as much as possible.

[0143] During implementation, the frame extraction position corresponding to the frame rate and the pixel set adjustment position corresponding to the resolution can be quantified respectively, that is, the corresponding frame extraction position and pixel set adjustment position are determined for the target frame rate and target resolution corresponding to different video qualities, and corresponding relationships are established respectively, so that the corresponding frame extraction position can be obtained through the target frame rate, and the corresponding pixel set adjustment position can be obtained through the target resolution, and the video compression task can be completed efficiently. The corresponding relationship can be obtained based on actual experience and will not be repeated here.

[0144] In addition, the frame rate and / or resolution may be adjusted before the encoding is adjusted (ie, the bit rate is adjusted), that is, the bit rate of the video is adjusted after the frame rate and / or resolution are adjusted.

[0145] In another embodiment provided by the present disclosure, receiving video quality feedback information sent by the remote control driving terminal;

[0146] In a case where the video quality feedback information indicates that the video quality needs to be improved, determining a target indicator value based on the current indicator value, and determining a first video adjustment method corresponding to the target indicator value;

[0147] The currently collected video data is adjusted by using the first video adjustment method to obtain target video data with improved video quality;

[0148] Among them, after the collected video data is adjusted by using the first video adjustment method corresponding to the target indicator value, the quality of the obtained video data is higher than the video quality corresponding to the current indicator value.

[0149] In the disclosed embodiment, the remote-controlled unmanned vehicle receives video quality feedback information sent from the remote-controlled driving terminal. This information can reflect whether the quality of the currently transmitted video is appropriate. For example, if the remote-controlled driver inputs an evaluation of the video quality through a user interface when using the remote-controlled driving function, or the system automatically detects problems such as blur, delay, or frame loss in the video stream, these will be fed back to the unmanned vehicle as feedback information. In an emergency, an operation button can also be set so that the remote-controlled driver can directly feedback that the current video quality is unqualified by pressing a button.

[0150] When the unmanned vehicle receives this video quality feedback information and analyzes it to find that the feedback information indicates that the video quality needs to be improved, the system can determine a target indicator value based on the current indicator value. For example, if the current video resolution is 720p, and the feedback information indicates that the driver needs a clearer image to better identify obstacles, the system may be able to set the target indicator value to 1080p full HD resolution.

[0151] In order to achieve this target index value, it is necessary to determine the corresponding first video adjustment method. This adjustment method may include one or a series of parameter adjustments corresponding to the index value, such as increasing the bit rate of video encoding, adjusting the frame rate, optimizing the compression algorithm, or adjusting the resolution, or even changing the transmission protocol of the video stream. The specific adjustment method can also refer to the adjustment method described in the embodiment of the present disclosure, which will not be repeated here. After adjusting the currently collected video data using the above-mentioned first video adjustment method, the system can obtain the target video data after the video quality is improved.

[0152] Finally, after adjusting the collected video data using the first video adjustment method corresponding to the target index value, the quality of the obtained video data will be higher than the video quality corresponding to the current index value. This means that the driver can get clearer and smoother video feedback when driving remotely, so as to make more accurate driving decisions. For example, if the driver needs to bypass a sudden obstacle during remote control operation, high-quality video can help the driver see the size and position of the obstacle more clearly, so as to make a quick and accurate response.

[0153] In another embodiment provided by the present disclosure, the environmental complexity of the driving environment of the unmanned vehicle is determined by at least one of the following methods:

[0154] Determining the scene in which the unmanned vehicle is located according to the current operating state of the unmanned vehicle, and determining the environmental complexity according to the scene;

[0155] Determining the environmental complexity according to terrain data corresponding to the current location of the unmanned vehicle;

[0156] Determining the environmental complexity according to the perception result of the surrounding environment by the perception device of the unmanned vehicle;

[0157] Receive surrounding vehicle information sent by other vehicles and / or a cloud control platform, and determine the environmental complexity based on the surrounding vehicle information.

[0158] In the embodiment of the present disclosure, determining the environmental complexity of the driving environment of the unmanned vehicle can be achieved by at least one of the following methods:

[0159] First, the scene the unmanned vehicle is in can be determined by analyzing its current operating state. For example, if the current state of the unmanned vehicle is loading, it can be determined that the scene the unmanned vehicle is in is the loading area, and the corresponding environmental complexity can be determined based on the loading area.

[0160] Second, terrain data corresponding to the current location of the unmanned vehicle can also be used to determine the complexity of the environment. For example, if the unmanned vehicle is located in a mountainous or hilly area, the terrain data may show that the road is tortuous, the slope is steep, and the field of vision is limited, which will increase the difficulty and complexity of driving. On the contrary, if the unmanned vehicle is driving on a highway in a plain area, the terrain data may indicate that the road is straight, the field of vision is wide, and the environmental complexity is relatively low.

[0161] Third, the perception results of the unmanned vehicle's perception of the surrounding environment can also be used to determine the environmental complexity. These perception devices may include cameras, radars, laser scanners (LiDAR), cameras, etc., which can monitor objects and obstacles in the surrounding environment in real time. For example, if the perception device detects that there are a large number of pedestrians, bicycles, animals or other vehicles around, and the movement trajectories of these objects are complex and changeable, then the environmental complexity will be judged as high. On the contrary, if the surrounding environment is relatively empty and the object's movement trajectory is simple, the environmental complexity is low.

[0162] Fourth, receiving surrounding vehicle information sent by other vehicles and / or the cloud control platform can also determine the complexity of the environment. This information may include data such as the speed, direction, and location of other vehicles, through which a real-time traffic environment model can be constructed. For example, if the surrounding vehicles are dense and the movement status is changeable, it may cause traffic congestion or potential collision risks, thereby increasing the complexity of the environment.

[0163] Unmanned vehicles can determine the environmental complexity of their driving environment by themselves, or they can obtain the environmental complexity of their driving environment from the cloud control platform. The cloud control platform can use big data analysis and artificial intelligence algorithms to process and analyze this information to provide accurate environmental complexity assessments for unmanned vehicles.

[0164] In one possible implementation, the environmental complexity of the unmanned vehicle's driving environment can be more comprehensively and accurately evaluated by comprehensively considering at least one of the unmanned vehicle's operating status, terrain data, perception results of the perception equipment, and surrounding vehicle information, thereby providing strong support for the safe driving of the unmanned vehicle.

[0165] Based on the same disclosed concept, the embodiment of the present disclosure also provides a video data processing device for remote control driving. Since the principle of the problem solved by the device is similar to the aforementioned video data processing method for remote control driving, the implementation of the device can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0166] With the above Figure 1 Corresponding to the method shown, the embodiment of the present disclosure also provides a video data processing device for remote control driving, such as Figure 2 As shown, including:

[0167] The index value acquisition module 201 is used to obtain the current index value of the reference index corresponding to the unmanned vehicle; wherein the reference index is an index for reference to adjust the video quality of the video collected by the unmanned vehicle;

[0168] The video adjustment module 202 is used to adjust the current collected video data of the unmanned vehicle by adopting a target video adjustment method corresponding to the current index value to obtain target video data when the current index value meets the specified video quality adjustment condition;

[0169] The video transmission module 203 is used to send the target video data to the remote control driving terminal, wherein the remote control driving terminal is used to output the target video data to guide the remote control driver to remotely control the unmanned vehicle.

[0170] In another embodiment provided by the present disclosure, in the indicator value acquisition module 201, the reference indicator includes at least one of the following: vehicle speed, environmental complexity of the driving environment, and network transmission quality.

[0171] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: determine the target video encoding parameters corresponding to the current indicator value when the current indicator value meets the specified video quality adjustment conditions; based on the target video encoding parameters, adjust the encoding of the currently acquired video data to obtain the target video data.

[0172] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: pre-establish a correspondence between different indicator values ​​and different target video encoding parameters; determine the target video encoding parameters according to the correspondence and the current indicator values; obtain the current video encoding parameters, and determine the encoding adjustment parameters according to the target video encoding parameters and the current video encoding parameters; use the encoding adjustment parameters to adjust the encoding of the current collected video data of the unmanned vehicle to obtain the target video data.

[0173] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: when the reference index includes the vehicle speed, the lower the vehicle speed, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or

[0174] In the case where the reference index includes the environmental complexity of the driving environment, the lower the environmental complexity of the driving environment, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or

[0175] When the reference indicator includes network transmission quality, the worse the network transmission quality is, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current indicator value is.

[0176] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: determine the encoding adjustment parameters corresponding to the target video encoding parameters; use the encoding adjustment parameters to adjust the encoding of the currently acquired video data to obtain the target video data; wherein the target video encoding parameters include a target bit rate, and the encoding adjustment parameters corresponding to the target bit rate include a first target compression parameter.

[0177] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: determine the target continuous frame from the video currently collected by the unmanned vehicle according to the first frame position of the continuous frame to be compressed represented by the first target compression parameter and the number of video frames contained in each continuous frame; divide each video frame contained in the target continuous frame into multiple pixel blocks according to a preset pixel block division method; for each target continuous frame, determine the pixel blocks corresponding to the target position of each frame in the target continuous frame; for each pixel block corresponding to the target position, respectively determine the difference between the pixel values ​​of the pixel blocks corresponding to the target positions of other frames except the first frame in the target continuous frame and the pixel blocks corresponding to the target position of the first frame, and obtain the difference pixel blocks corresponding to the other frames respectively; based on the difference pixel blocks, determine the difference video frames corresponding to the other frames respectively; for each target continuous frame, obtain the target video data based on the difference video frames corresponding to the first frame corresponding to the target continuous frame and the other frames respectively.

[0178] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: perform pixel quantization processing on the obtained difference pixel block according to the quantization parameter represented by the first target compression parameter to obtain the processed difference pixel block; and determine the difference video frames corresponding to the other frames based on the processed difference pixel blocks corresponding to each pixel block of the other frames.

[0179] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: determine a target video frame from a video currently collected by the unmanned vehicle according to a first target compression parameter; divide each target video frame into multiple pixel blocks according to a preset pixel block division method; determine the reference pixel block in each target video frame according to the position and / or number of the reference pixel blocks in each video frame represented by the first target compression parameter; for each reference pixel block, determine a first pixel block having a preset positional relationship with the reference pixel block, and respectively determine the difference between each first pixel block and the corresponding pixel in the reference pixel block to obtain the difference pixel blocks corresponding to each first pixel block; obtain the target video data based on the difference pixel blocks corresponding to each reference pixel block and each first pixel block.

[0180] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: determine the target video adjustment parameters corresponding to the current indicator value when the current indicator value meets the specified video quality adjustment conditions; wherein the target video adjustment parameters include the target frame rate and / or the target resolution; based on the target video adjustment parameters, make corresponding adjustments to the currently acquired video data to obtain the target video data.

[0181] In another embodiment provided by the present disclosure, the video adjustment module 202 is used to: when the target video adjustment parameter includes a target frame rate, based on the target video adjustment parameter, make corresponding adjustments to the current collected video data, including: determining a frame extraction position corresponding to the target frame rate, and deleting a video frame corresponding to the frame extraction position in the current collected video data; or,

[0182] In the case where the target video adjustment parameters include a target resolution, corresponding adjustments are made to the currently acquired video data based on the target video adjustment parameters, including: determining a pixel set adjustment position corresponding to the target resolution, and deleting a pixel set at the pixel set adjustment position in each video frame of the currently acquired video data, wherein the pixel set includes pixel rows and / or pixel columns.

[0183] In another embodiment provided by the present disclosure, Figure 3 As shown, the above device also includes:

[0184] The feedback adjustment module 204 is used to receive video quality feedback information sent by the remote control driving terminal; when the video quality feedback information indicates that the video quality needs to be improved, a target indicator value is determined based on the current indicator value, and a first video adjustment method corresponding to the target indicator value is determined; the first video adjustment method is used to adjust the current collected video data to obtain target video data with improved video quality; wherein, after the collected video data is adjusted using the first video adjustment method corresponding to the target indicator value, the quality of the obtained video data is higher than the video quality corresponding to the current indicator value.

[0185] In another embodiment provided by the present disclosure, the indicator value acquisition module 201 is used to determine the environmental complexity of the driving environment of the unmanned vehicle by using at least one of the following methods:

[0186] Determine the scene where the unmanned vehicle is located based on the current operating status of the unmanned vehicle, and determine the complexity of the environment based on the scene;

[0187] Determine the complexity of the environment based on the terrain data corresponding to the current location of the unmanned vehicle;

[0188] Determine the complexity of the environment based on the perception results of the unmanned vehicle's perception equipment on the surrounding environment;

[0189] Receive surrounding vehicle information sent by other vehicles and / or the cloud control platform, and determine the complexity of the environment based on the surrounding vehicle information.

[0190] Through the description of the above implementation methods, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0191] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present disclosure.

[0192] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the present embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0193] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0194] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A video data processing method for remote control driving, characterized in that: include: Obtaining a current index value of a reference index corresponding to the unmanned vehicle; wherein the reference index is an index for reference in adjusting the video quality of the video collected by the unmanned vehicle; When the current index value satisfies the specified video quality adjustment condition, the target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain the target video data; The target video data is sent to a remote control driving terminal, wherein the remote control driving terminal is used to output the target video data to guide a remote control driver to remotely control the unmanned vehicle.

2. The method according to claim 1, characterized in that The reference index includes at least one of the following: vehicle speed, environmental complexity of the driving environment, and network transmission quality.

3. The method according to claim 1 or 2, characterized in that When the current index value satisfies the specified video quality adjustment condition, the target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain the target video data, including: In the case where the current indicator value satisfies the specified video quality adjustment condition, determining a target video encoding parameter corresponding to the current indicator value; Based on the target video encoding parameters, the encoding of the currently acquired video data is adjusted to obtain the target video data.

4. The method according to claim 3, characterized in that The step of adjusting the encoding of the currently acquired video data based on the target video encoding parameter to obtain the target video data comprises: Pre-establishing the corresponding relationship between different indicator values ​​and different target video encoding parameters; Determine the target video encoding parameter according to the corresponding relationship and the current indicator value; Acquire current video encoding parameters, and determine encoding adjustment parameters according to the target video encoding parameters and the current video encoding parameters; The encoding adjustment parameters are used to adjust the encoding of the currently collected video data of the unmanned vehicle to obtain the target video data.

5. The method according to claim 2, characterized in that In the case where the reference index includes vehicle speed, the lower the vehicle speed, the worse the quality of target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or In the case where the reference index includes the environmental complexity of the driving environment, the lower the environmental complexity of the driving environment, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current index value; and / or In the case where the reference indicator includes network transmission quality, the worse the network transmission quality is, the worse the quality of the target video data obtained after adjusting the current collected video data of the unmanned vehicle using the target video adjustment method corresponding to the current indicator value is.

6. The method according to claim 3, characterized in that Based on the target video encoding parameter, the encoding of the currently acquired video data is adjusted to obtain the target video data, including: Determining encoding adjustment parameters corresponding to the target video encoding parameters; The encoding of the currently acquired video data is adjusted using the encoding adjustment parameters to obtain the target video data; wherein the target video encoding parameters include a target bit rate, and the encoding adjustment parameters corresponding to the target bit rate include a first target compression parameter.

7. The method according to claim 6, characterized in that The step of adjusting the encoding of the currently acquired video data by using the encoding adjustment parameter to obtain the target video data includes: Determine target continuous frames from the video currently collected by the unmanned vehicle according to the first frame position of the continuous frames to be compressed represented by the first target compression parameter and the number of video frames contained in each continuous frame; Dividing each video frame included in the target continuous frames into a plurality of pixel blocks according to a preset pixel block division method; For each target continuous frame, determine the pixel blocks corresponding to the target position in each frame of the target continuous frame; For each pixel block corresponding to the target position, respectively determine the difference between the pixel values ​​of the pixel blocks corresponding to the target position of other frames except the first frame in the target continuous frames and the pixel block corresponding to the target position of the first frame, and obtain the difference pixel blocks corresponding to the other frames respectively; Based on each difference pixel block, determining the difference video frames corresponding to the other frames respectively; For each target continuous frame, the target video data is obtained based on the difference video frames corresponding to the first frame corresponding to the target continuous frame and the other frames respectively.

8. The method according to claim 7, characterized in that The determining, based on each difference pixel block, difference video frames corresponding to the other frames respectively comprises: performing pixel quantization processing on the obtained difference pixel block according to the quantization parameter represented by the first target compression parameter to obtain a processed difference pixel block; Based on the processed difference pixel blocks respectively corresponding to each pixel block of the other frames, the difference video frames respectively corresponding to the other frames are determined.

9. The method according to claim 6, characterized in that The encoding of the currently acquired video data is adjusted by using the encoding adjustment parameter to obtain the target video data, including: Determining a target video frame from the video currently collected by the unmanned vehicle according to the first target compression parameter; Dividing each target video frame into a plurality of pixel blocks according to a preset pixel block division method; Determine a reference pixel block in each target video frame according to a position and / or a number of reference pixel blocks in each video frame represented by the first target compression parameter; For each reference pixel block, determine a first pixel block having a preset positional relationship with the reference pixel block, and respectively determine the difference between each first pixel block and the corresponding pixel in the reference pixel block to obtain a difference pixel block corresponding to each first pixel block; The target video data is obtained based on the difference pixel blocks respectively corresponding to each reference pixel block and each first pixel block.

10. The method according to claim 1 or 3, characterized in that: When the current index value satisfies the specified video quality adjustment condition, the target video adjustment method corresponding to the current index value is adopted to adjust the current collected video data of the unmanned vehicle to obtain the target video data, including: In the case where the current indicator value satisfies the specified video quality adjustment condition, determining a target video adjustment parameter corresponding to the current indicator value; wherein the target video adjustment parameter includes a target frame rate and / or a target resolution; Based on the target video adjustment parameters, the currently collected video data is adjusted accordingly to obtain the target video data.

11. The method according to claim 10, characterized in that In the case where the target video adjustment parameter includes a target frame rate, the corresponding adjustment of the current acquired video data based on the target video adjustment parameter includes: determining a frame extraction position corresponding to the target frame rate, and deleting a video frame corresponding to the frame extraction position in the current acquired video data; or In the case where the target video adjustment parameters include a target resolution, the corresponding adjustment to the currently acquired video data is performed based on the target video adjustment parameters, including: determining a pixel set adjustment position corresponding to the target resolution, and deleting a pixel set at the pixel set adjustment position in each video frame of the currently acquired video data, wherein the pixel set includes pixel rows and / or pixel columns.

12. The method according to claim 1, characterized in that Also includes: Receiving video quality feedback information sent by the remote control driving terminal; In a case where the video quality feedback information indicates that the video quality needs to be improved, determining a target indicator value based on the current indicator value, and determining a first video adjustment method corresponding to the target indicator value; The currently collected video data is adjusted by using the first video adjustment method to obtain target video data with improved video quality; Among them, after the collected video data is adjusted by using the first video adjustment method corresponding to the target indicator value, the quality of the obtained video data is higher than the video quality corresponding to the current indicator value.

13. The method according to claim 1, characterized in that The environmental complexity of the driving environment of the unmanned vehicle is determined by at least one of the following methods: Determining the scene in which the unmanned vehicle is located according to the current operating state of the unmanned vehicle, and determining the environmental complexity according to the scene; Determining the environmental complexity according to terrain data corresponding to the current location of the unmanned vehicle; Determining the environmental complexity according to the perception result of the surrounding environment by the perception device of the unmanned vehicle; Receive surrounding vehicle information sent by other vehicles and / or a cloud control platform, and determine the environmental complexity based on the surrounding vehicle information.

14. A video data processing device for remote control driving, characterized in that: include: An index value acquisition module, used to obtain the current index value of the reference index corresponding to the unmanned vehicle; wherein the reference index is an index for reference to adjust the video quality of the video collected by the unmanned vehicle; A video adjustment module, configured to adjust the currently collected video data of the unmanned vehicle by adopting a target video adjustment method corresponding to the current index value to obtain target video data when the current index value satisfies a specified video quality adjustment condition; The video transmission module is used to send the target video data to the remote control driving terminal, wherein the remote control driving terminal is used to output the target video data to guide the remote control driver to remotely control the unmanned vehicle.

Images