Vehicle cloud collaborative perception decision-making high-speed automatic auxiliary navigation driving method

Abstract

The invention relates to a vehicle cloud collaborative perception decision-making high-speed automatic auxiliary navigation driving method. The method comprises the following steps: S1, a vehicle end sends collected vehicle end data to a cloud end; s2, the cloud generates a first decision instruction based on the vehicle end data of all vehicles and other sensor data; s3, after the vehicle receives the first decision instruction, the current moment is recorded as a second timestamp, and accumulated time delay is calculated; s4, judging whether the accumulated time delay is smaller than a pre-configured time interval threshold value or not, and if yes, executing the step S5; s5, generating a first speed constraint boundary condition according to the instruction content; and S6, the vehicle end takes the first speed constraint boundary condition as one of constraint conditions of the decision model to generate a second decision instruction. Compared with the prior art, the method has the advantages that the problem of low driving safety of high-speed automatic auxiliary navigation caused by limited single vehicle detection distance is solved under the conditions that the vehicle end computing power requirement is hardly increased, the network condition is poor, and particularly the visibility is poor.

Description

Technical Field

[0001] The present invention relates to the field of intelligent assisted driving, and particularly to a vehicle-cloud collaborative perception and decision-making high-speed automated assisted navigation driving method. Background Art

[0002] An intelligent driving or autonomous driving system can be understood as a computer system that observes the environment through its own sensors, makes judgments based on the observations, and then controls the vehicle to perform driving tasks. In recent years, with the popularization of machine learning (deep learning) and the rapid development of computer vision technology, inexpensive intelligent driving systems have become standard equipment for passenger cars.

[0003] In the further development route of intelligent driving systems, the performance of vehicle sensors plays a crucial role. To solve the detection problems in scenarios such as at night, most current solutions are equipped with lidar. Lidar needs to actively emit laser beams into the environment. Based on the safety requirements of the human eye and other devices, the power of the laser is strictly controlled. However, based on the basic principle that the power of light decays with the square of the distance in space, the detection distance of the laser is physically limited and is currently only several hundred meters, which is not enough for driving.

[0004] In addition, in terms of pilot assistance, according to the traffic management notice requirements of the Ministry of Public Security under low visibility conditions, when the visibility is less than 500 meters, the speed should not exceed 80 kilometers per hour, and the dipped headlights should be turned on, and a distance of more than 150 meters should be maintained from the vehicle in front in the same lane. And this performance requirement itself exceeds the performance boundary of current assisted driving. In sharp contrast, the perception distance of intelligent driving systems is less than 500 meters. In other words, for all lidar solutions, their perception distances are below the visibility range of 500 meters. According to the relevant regulations, their cruising speeds during pilot assistance should not exceed 80 kilometers per hour. However, the speed limits on most highways in China are 100 kilometers per hour or 120 kilometers per hour. If cruising at a speed of 80 kilometers per hour, there will undoubtedly be a large speed difference from the surrounding vehicles, thus generating relatively large risks.

[0005] In this regard, some existing technologies gather more data through the Internet of Vehicles. For example, Chinese patent CN117985045A provides a method for gathering sensor data of surrounding vehicles, so as to have more sensor data sources to improve the visual range of the vehicle. However, in this method, due to the large amount of data and limited computing power on the vehicle side, it is difficult to be applied in the industry. In addition, Chinese patent CN118466291A discloses a vehicle data collaborative processing method, system, computer equipment and medium. The vehicle system encodes and desensitizes the vehicle driving video of the vehicle, thereby generating a video to be processed, and then sends the video to be processed to the road side system for driving decision-making, and sends the finally generated vehicle operation suggestion back to the vehicle system, so that the driver can assist the vehicle in automatic driving according to the vehicle operation suggestion. Through the above method, the driving decision that consumes the most system computing power is transferred from the vehicle system to the road side system, thereby achieving the effect of reducing the computing pressure of the vehicle system, solving the problem that the road side system in the prior art only serves as a communication bridge between the vehicle system and the cloud, resulting in the vehicle system can only rely on its own decision-making ability to handle complex conditions, resulting in excessive computing pressure on the vehicle system, and reducing the computing pressure of the vehicle system.

[0006] However, although the above method increases the detection distance in disguise through vehicle-cloud collaboration, the driving suggestions given in the cloud do not directly interact with the assisted driving decisions on the vehicle side. Instead, they are displayed in a language form to inform the driver whether he needs to exit the pilot assistance. In reality, it still requires the driver to be highly concentrated and has limited effectiveness in reducing driver fatigue.

[0007] In addition, the above solution relies on good network conditions on the vehicle side and the cloud side, but most highways are actually located in some remote areas, which do not have good network conditions. When the network conditions are poor, the cloud side will frequently give wrong driving suggestions, resulting in more interference information, which will increase the driver's fatigue.

[0008] Therefore, how to solve the problem of low safety of high-speed automatic assisted navigation driving caused by the limited detection distance of a single vehicle without almost increasing the computing power requirements on the vehicle side and under poor network conditions, especially poor visibility. Summary of the invention

[0009] The purpose of the present invention is to solve the defects of the above-mentioned prior art and provide a vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method.

[0010] The purpose of the present invention can be achieved by the following technical solutions: A vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method, comprising: Step S1: When establishing a vehicle-cloud collaboration task, the vehicle terminal sends the collected vehicle terminal data to the cloud. Step S2: The cloud generates a first decision instruction based on the vehicle terminal data of all vehicles and other sensor data, where the first decision instruction includes an instruction type, instruction content, and a first timestamp. The instruction type includes a pilot assist survival instruction and a pilot speed boundary constraint instruction. The first timestamp is generated based on the collection times of all vehicle terminal data and the collection times of other sensor data. Step S3: The cloud sends the first decision instruction to the vehicle. After receiving the first decision instruction, the vehicle records the current time as the second timestamp and calculates the time interval between the second timestamp and the first timestamp as the cumulative delay of the RSU full link. Step S4: Determine whether the cumulative delay of the RSU full link is less than a pre-configured time interval threshold. If so, execute Step S5. Step S5: Determine the instruction type of the first decision instruction. If the instruction type is a pilot assist survival instruction, generate display content according to the instruction content. If the instruction type is a pilot speed boundary constraint instruction, generate a first speed constraint boundary condition according to the instruction content and execute Step S6. Step S6: The vehicle terminal uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction.

[0011] The other sensor data includes roadside sensor data and / or drone sensor data.

[0012] The generation process of the first timestamp includes: Obtain the sampling frequencies of all vehicle terminal data and other sensor data used to generate the first decision instruction. Select the data type with the highest sampling frequency and obtain the collection time of this data type. Use the obtained collection time as the first timestamp.

[0013] The vehicle terminal data includes image data, which at least includes a full-frame low-resolution image, a fixation high-resolution image, and optionally a full-frame high-resolution low-frame-rate image.

[0014] The vehicle terminal data is uploaded to the cloud after being desensitized. The desensitization calculation of the image data is based on the full-frame low resolution, and the desensitization calculation results are synchronously applied to the fixation high-resolution image and the full-frame high-resolution low-frame-rate image.

[0015] The pre-configured time interval threshold is 8 seconds.

[0016] The specific content of Step S6 includes: The vehicle end generates a first vehicle control instruction according to its own decision-making model, where the first vehicle control instruction includes assisted driving survival information, guided vehicle speed information, and lane change decision information; Extract the guided vehicle speed information based on the first vehicle control instruction, and determine whether the guided vehicle speed information meets the first speed constraint boundary condition. If so, use the first vehicle control instruction as the second decision instruction.

[0017] The specific steps of step S6 include: Obtain the decision-making model constructed by the vehicle end, and extract the vehicle speed decision sub-model, where the vehicle speed decision sub-model includes an optimization target and original constraint conditions; Extract the speed guidance constraint in the original constraint conditions, find the intersection of the first speed constraint boundary condition and the speed guidance constraint, and replace the speed guidance constraint in the original constraint conditions with the obtained intersection to obtain an updated vehicle speed decision sub-model; Generate a second vehicle control instruction based on the updated vehicle speed decision sub-model as the second decision instruction, where the second vehicle control instruction guides the vehicle speed information.

[0018] A vehicle-cloud collaborative perception decision high-speed automatic assisted navigation driving device includes a memory, a processor, and a program stored in the memory. When the processor executes the program, the above-mentioned method is implemented.

[0019] A storage medium stores a program, and when the program is executed, the above-mentioned method is implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The first decision instruction generated by the cloud is not only a prompt text, but also includes specific speed constraints. On the one hand, by judging the cumulative delay of the entire RSU link, it is determined whether the first decision instruction of the cloud has reference value. If it has no reference value, it is directly discarded, thus filtering the first decision instruction of the cloud once, avoiding the interference problem caused by generating incorrect first decision instructions due to poor network conditions. On the other hand, the first decision instruction of the cloud generates a constraint on the speed of the vehicle end's local decision-making, forming a collaborative decision-making mechanism between the vehicle end and the cloud, rather than a simple patchwork, improving the automation degree, reducing the interference to the user, and alleviating the user's fatigue. In addition, the first decision instruction of the cloud only acts on the speed of the local decision-making and does not include lane changes. The lane change decision is still obtained from the local decision-making of the vehicle end. The cloud decision focuses on distant obstacles, and the local decision focuses on the nearby road conditions. More importantly, the first decision instruction of the cloud generates a constraint on the speed of the vehicle end's local decision-making, which can be directly substituted into the speed constraint conditions, so that it is not necessary to make major modifications to the local decision-making model and there will be no large computing power requirements.

[0021] 2. Use the acquisition time of the sensor data with the highest sampling frequency as the first timestamp, thereby helping the second decision instruction determine its actual effectiveness and ensuring that the second decision instruction meets the first speed constraint boundary condition.

[0022] 3. Replace the speed guidance constraint in the original constraint condition with the obtained intersection. Only a simple intersection operation is required to replace the speed guidance constraint, with low computing power requirements and improved response speed. Brief Description of the Drawings

[0023] Figure 1 It is a schematic diagram of the main step flow of the method of the present invention. Detailed Embodiment

[0024] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0025] Pilot assisted driving, especially pilot assisted driving on highways, is also known as highway automated lane keeping. Since the speed on highways is relatively fast, the general vehicle driving speed is 120 kilometers per hour, and the forward distance per second is about 33 m / s. And braking from 120 to 0, under a comfortable deceleration (-1.5 m / s 2 ) the required distance d = v 2 / (2×a) is about 363 m. This means that the vehicle needs to execute actions on a stationary target event at least about 370 m away. This distance exceeds 50% of the typical effective distance (about 250 m) of the current intelligent driving perception system.

[0026] Generally, traffic management announcements under low visibility conditions require that when the visibility is less than 500 meters, the speed should not exceed 80 kilometers per hour, and the low beam lights should be turned on, and a distance of more than 150 meters should be maintained from the vehicle in front in the same lane. And this performance requirement itself exceeds the performance boundary of the current assisted driving. (The perception distance of the current intelligent driving system is less than 500 meters). Therefore, there are no objective conditions to further upgrade from assisted driving to autonomous driving.

[0027] Such performance limitations are not due to the current front-end of machine vision sensors. The currently common 8-megapixel camera has an optical resolution close to the fovea of the human eye retina (reaching 60 ppd at 60-degree fov, with each pixel covering 1 arc minute), but because the vehicle cannot process a large amount of information, the image resolution has to be reduced to 1 / 4 to 16 ppd. While the computing power drops to 1 / 16, it is equivalent to the visual acuity on the logarithmic visual acuity chart dropping from 1.5 to 0.2.

[0028] For the vehicle decision-making part, since the vehicle's moving speed is 33 meters per second, there is a running distance of 290 meters between the expected comfortable movement distance of 370 meters and the approximately 80 meters for emergency braking response. Through simple calculation, if the cloud decision-making loop delay does not exceed 8 seconds, it will bring positive improvement to the vehicle's functional experience. The cloud decision-making focuses on long-distance obstacles, while the local decision-making focuses on the road conditions in the vicinity.

[0029] Based on the above analysis, the present application provides a vehicle-cloud collaborative perception and decision-making high-speed automatic assisted navigation driving method, as Figure 1 shown, including: Step S1: When establishing a vehicle-cloud collaborative task, the vehicle terminal sends the vehicle terminal data collected to the cloud; Specifically, when the vehicle activates the pilot function, that is, the high-speed automatic assisted navigation driving function, the vehicle will attempt to establish a connection with the cloud. After this connection is established, the video encoding system will start and wait for the cloud to send a control signaling.

[0030] After receiving the connection from the vehicle terminal, the cloud will start the service processing module for the video stream. After the service processing module finishes working, it sends a control signaling to the vehicle terminal, requesting it to start uploading the video stream data.

[0031] After receiving the cloud start signaling, the vehicle terminal will continuously upload the encoding upload task of the camera perception image according to the cloud signaling requirements until the control signaling changes or a termination instruction is received.

[0032] The vehicle terminal data includes image data, and the image data includes at least a full-frame low-resolution image and a fixation point high-resolution image, as well as optionally a full-frame high-resolution low-frame-rate image.

[0033] The vehicle terminal data is uploaded to the cloud after being desensitized. The desensitization calculation of the image data is based on the full-frame low resolution, and the desensitization calculation result is synchronously applied to the fixation point high-resolution image and the full-frame high-resolution low-frame-rate image.

[0034] Similar to the related prior art, the part of video encoding and transmission is based on the webRTC technology. There is a function named Simulcast in webRTC, which allows encoding a single information source into multiple data layers with different spatial and temporal flexibilities to improve the scalability of the system. In this solution, the most important vehicle forward image information is divided into three layers: a full-frame low-resolution layer, a fixation point high-resolution layer, and an optionally full-frame high-resolution low-frame-rate layer. The first two are encoded using 720p / 10fps, and the latter uses the 2160p original resolution but the frame rate is reduced to 1fps.

[0035] The time-consuming desensitization calculation can be reused on the same-viewpoint images at different resolutions. Therefore, the calculation process is carried out on the images at a lower resolution, and the results are synchronously applied to the high-resolution images.

[0036] For the 8-megapixel camera of the vehicle applied in this solution, through reasonable content selection, the video encoding part makes full use of the available bandwidth. At a bitrate of 0.1 bpp, the full bandwidth is about 3 Mbps. When not transmitting the optional full-frame data, the bandwidth is 2 Mbps. It is also possible to completely stop data transmission or further reduce the transmission resolution and bandwidth of the reduced image until the transmission is completely paused.

[0037] Step S2: The cloud generates a first decision instruction based on the vehicle-end data and other sensor data of all vehicles. The first decision instruction includes an instruction type, instruction content, and a first timestamp. The instruction type includes a pilot assist survival instruction and a pilot speed boundary constraint instruction. The first timestamp is generated based on the collection times of all vehicle-end data and other sensor data. In most embodiments, the other sensor data includes roadside sensor data and / or drone sensor data.

[0038] In terms of cloud computing (RSU computing), since the video frame intervals transmitted through the cloud are not stable, this solution divides the cloud processing into two stages: alignment and processing. The first stage processes the multi-viewpoint alignment and spatial coordinate decomposition at the video information generation moment, and the second stage processes the specific driving task strategies. The connection between the two is through the aligned image sequence or feature vector (Vector) and the neural network token (Token). The algorithm details of the cloud at each stage do not belong to the content of this solution and can be fine-tuned as the business develops.

[0039] However, particularly in this embodiment, the generation process of the first timestamp includes: Obtain the sampling frequencies of all vehicle-end data and other sensor data for generating the first decision instruction; Select the data type with the highest sampling frequency and obtain the collection time of this data type; Use the obtained collection time as the first timestamp.

[0040] In this way, through the sensor timestamp of the earliest reading in the entire data packet, the timeliness of the first decision calculated in the future is ensured. Even if there are inevitable transmission delays in the processing link, it can also ensure that the delay compensation when calculating the second decision does not increase additional risks.

[0041] Step S3: The cloud sends the first decision instruction to the vehicle. After receiving the first decision instruction, the vehicle records the current moment as the second timestamp and calculates the time interval between the second timestamp and the first timestamp as the cumulative delay of the RSU full link. Step S4: Determine whether the cumulative delay of the entire RSU link is less than the pre-configured time interval threshold. If so, execute Step S5; In this embodiment, according to calculations, the pre-configured time interval threshold is set to 8 seconds.

[0042] Step S5: Determine the instruction type of the first decision instruction. If the instruction type is the pilot assist survival instruction, generate display content according to the instruction content. If the instruction type is the pilot speed boundary constraint instruction, generate the first speed constraint boundary condition according to the instruction content, and execute Step S6; Step S6: The vehicle terminal uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction.

[0043] In a certain embodiment, Step S6 specifically includes: The vehicle terminal generates a first vehicle control instruction according to its own decision model, where the first vehicle control instruction includes assisted driving survival information, guided vehicle speed information, and lane change decision information; Extract the guided vehicle speed information based on the first vehicle control instruction, and determine whether the guided vehicle speed information meets the first speed constraint boundary condition. If so, use the first vehicle control instruction as the second decision instruction. Otherwise, perform a screen display reminder. This combination method is relatively primitive and has a low implementation difficulty.

[0044] Of course, in this embodiment, Step S6 specifically includes: Obtain the decision model constructed by the vehicle terminal, and extract the vehicle speed decision sub-model, where the vehicle speed decision sub-model includes an optimization target and original constraint conditions; Extract the speed guidance constraint in the original constraint conditions, find the intersection of the first speed constraint boundary condition and the speed guidance constraint, and replace the speed guidance constraint in the original constraint conditions with the obtained intersection to obtain an updated vehicle speed decision sub-model; For example, the optimization target of the vehicle speed decision sub-model is: min v J =∑ i=1 30 ( v i - v i-1 ) 2 Where: J To minimize speed changes to ensure a smooth experience, v is the vehicle speed, v i is i the vehicle speed at time vi-1 The vehicle speed at i time - 1; The original constraint condition is: ∣ v i − v i−1 ∣≤ a max ·Δ t Where: a max is the maximum acceleration, and Δ t is the i time and i the time interval between time - 1; The speed change should meet the comfort requirement: v min,i ≤ v i ≤ v max,i ,∀ i = 1,…,30 Wherein, v min,i is the i lower limit of the original vehicle speed at time v max,i is the i upper limit of the original vehicle speed at time. The original constraint condition ensures that the vehicle conforms to the upper and lower limits of the speed under the safety boundary. The updated constraint condition is: min( v min,i , v ’ min,i ) ≤ v i ≤ min(v max,i , v ’ max,i ), ∀ i = 1,…,30 Where: v ’ min,i and v ’ max,i are the upper and lower limits obtained from the first - speed constraint.

[0045] Generate a second vehicle control instruction as the second decision instruction based on the updated vehicle - speed decision sub - model. Among them, the second vehicle control instruction guides the vehicle - speed information.

[0046] Verify the present application through simulation. The simulation scenario is as follows: Taking a 6-kilometer-long one-way 4-lane and two-way 8-lane highway as an example, there is lane construction at the 3-kilometer mark, and it is necessary to divert to the first lane in the oncoming direction. Moreover, there is a lack of speed limit signs on the road, but there are cone barrels placed. Starting from the starting point, the vehicle of this embodiment successfully decelerates. When there are still 300 meters to the construction site, the speed has dropped to 30 kilometers per hour.

[0047] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

Claims

1. A vehicle-cloud collaborative perception and decision-making high-speed automated assisted navigation driving method, characterized in that Including: Step S1: When establishing a vehicle-cloud collaboration task, the vehicle terminal sends the collected vehicle-terminal data to the cloud. Step S2: The cloud generates a first decision instruction based on the vehicle-terminal data of all vehicles and other sensor data, where the first decision instruction includes an instruction type, instruction content, and a first timestamp. The instruction type includes a pilot-assist survival instruction and a pilot-speed boundary constraint instruction. The first timestamp is generated based on the collection times of all vehicle-terminal data and other sensor data. Step S3: The cloud sends the first decision instruction to the vehicle. After receiving the first decision instruction, the vehicle records the current time as the second timestamp and calculates the time interval between the second timestamp and the first timestamp as the cumulative delay of the RSU full link. Step S4: Determine whether the cumulative delay of the RSU full link is less than a pre-configured time interval threshold. If so, execute Step S5. Step S5: Determine the instruction type of the first decision instruction. If the instruction type is a pilot-assist survival instruction, generate display content according to the instruction content. If the instruction type is a pilot-speed boundary constraint instruction, generate a first speed constraint boundary condition according to the instruction content and execute Step S6. Step S6: The vehicle terminal uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction.

2. The vehicle-cloud collaborative perception and decision-making high-speed automatic assisted navigation driving method according to claim 1, wherein, The other sensor data includes roadside sensor data and / or UAV sensor data.

3. The vehicle-cloud collaborative perception and decision-making high-speed automated assisted navigation driving method according to claim 1, wherein The generation process of the first timestamp includes: Obtain the sampling frequencies of all vehicle-terminal data and other sensor data used to generate the first decision instruction. Select the data type with the highest sampling frequency and obtain the collection time of this data type. Use the obtained collection time as the first timestamp.

4. A vehicle-cloud collaborative perception and decision-making high-speed automatic assisted navigation driving method according to claim 1, characterized in that, The vehicle-terminal data includes image data, which at least includes a full-frame low-resolution image, a fixation high-resolution image, and optionally a full-frame high-resolution low-frame-rate image.

5. The vehicle-cloud collaborative perception and decision-making high-speed automatic assisted navigation driving method according to claim 4, wherein, The vehicle-terminal data is uploaded to the cloud after being desensitized. The desensitization calculation of the image data is based on the full-frame low resolution, and the desensitization calculation result is synchronously applied to the fixation high-resolution image and the full-frame high-resolution low-frame-rate image.

6. The vehicle-cloud collaborative perception and decision-making high-speed automatic assisted navigation driving method according to claim 1, wherein The pre-configured time interval threshold is 8 seconds.

7. A vehicle-cloud collaborative perception and decision-making high-speed automated assisted navigation driving method according to claim 1, characterized in that The specific content of Step S6 includes: The vehicle terminal generates a first vehicle control instruction according to its own decision model. Among them, the first vehicle control instruction includes assistive driving survival information, guiding vehicle speed information, and lane-changing decision information. Extract the guiding vehicle speed information based on the first vehicle control instruction and determine whether the guiding vehicle speed information meets the first speed constraint boundary condition. If so, use the first vehicle control instruction as the second decision instruction.

8. A vehicle-cloud collaborative perception and decision-making high-speed automated assisted navigation driving method according to claim 1, characterized in that, The specific content of Step S6 includes: Obtain the decision model constructed by the vehicle terminal and extract the vehicle speed decision sub-model. Among them, the vehicle speed decision sub-model includes an optimization target and original constraint conditions. Extract the speed guidance constraint in the original constraint conditions, find the intersection of the first speed constraint boundary condition and the speed guidance constraint, and replace the speed guidance constraint in the original constraint conditions with the obtained intersection to obtain an updated vehicle speed decision sub-model. Generate a second vehicle control instruction as a second decision instruction based on the updated vehicle speed decision sub-model, wherein the second vehicle control instruction guides vehicle speed information.

9. A vehicle-cloud collaborative perception and decision-making high-speed automatic assisted navigation driving device, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1-8.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method according to any one of claims 1-8.

Images