A high-speed automatic assisted navigation driving method based on vehicle-cloud collaborative perception and decision-making

Abstract

The present invention relates to a vehicle-cloud collaborative perception and decision-making method for high-speed automatic assisted navigation driving, comprising: step S1: the vehicle sends the collected vehicle-side data to the cloud; step S2: the cloud generates a first decision instruction based on the vehicle-side data and other sensor data of all vehicles; step S3: after receiving the first decision instruction, the vehicle records the current time as a second timestamp and calculates the cumulative delay; step S4: determines whether the cumulative delay is less than a preconfigured time interval threshold, and if so, executes step S5; step S5: generates a first speed constraint boundary condition based on the instruction content; step S6: the vehicle uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction. Compared with the existing technology, the present invention solves the problem of low safety of high-speed automatic assisted navigation driving caused by the limited detection range of a single vehicle, without almost increasing the computing power requirements of the vehicle-side and under poor network conditions, especially poor visibility.

Description

Technical Field

[0001] The present invention relates to the field of intelligent assisted driving, and in particular to a vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method. Background Art

[0002] An intelligent driving or autonomous driving system can be understood as a computer system that uses its sensors to observe the environment, makes judgments based on these observations, and then controls the vehicle to perform its tasks. In recent years, with the widespread adoption of machine learning (deep learning) and the rapid development of computer vision technology, affordable intelligent driving systems have become standard equipment in passenger cars.

[0003] The performance of vehicle sensors plays a crucial role in the further development of intelligent driving systems. To address detection challenges in scenarios like nighttime, most current solutions utilize lidar (LiDAR). LiDAR actively emits laser beams into the environment. To protect the human eye and other devices, the laser power is strictly controlled. However, due to the fundamental principle that light power decays with the square of its distance, laser detection range is physically limited to only a few hundred meters, which is insufficient for driving.

[0004] Furthermore, regarding navigation assistance, according to the Ministry of Public Security's traffic management notice for low-visibility conditions, when visibility is less than 500 meters, the speed should not exceed 80 kilometers per hour, low-beam headlights must be engaged, and a distance of at least 150 meters must be maintained from the vehicle ahead in the same lane. This performance requirement exceeds the current performance limits of assisted driving. In stark contrast, the perception range of intelligent driving systems is less than 500 meters. In other words, all LiDAR solutions have a perception range below 500 meters. According to relevant regulations, the cruising speed during navigation assistance should not exceed 80 kilometers per hour. However, the speed limit on most domestic highways is 100 or 120 kilometers per hour. Cruising at 80 kilometers per hour would undoubtedly create a significant speed difference with surrounding vehicles, creating a relatively high risk.

[0005] To address this, some existing technologies utilize the Internet of Vehicles (IoV) to aggregate more data. For example, Chinese patent CN117985045A proposes a method for aggregating sensor data from surrounding vehicles, thereby increasing the vehicle's visibility. However, this approach suffers from the large volume of data and limited computing power on the vehicle side, making it difficult to apply in industry. Furthermore, Chinese patent CN118466291A discloses a method, system, computer device, and medium for collaborative vehicle data processing. In this method, a vehicle system encodes and desensitizes a vehicle's driving video to generate a processed video. This video is then sent to a roadside system for driving decision-making. The resulting vehicle operation recommendations are then sent back to the vehicle system, allowing the driver to assist in autonomous driving based on the vehicle operation recommendations. This method shifts the most computationally intensive driving decisions from the vehicle system to the roadside system, thereby reducing the vehicle system's computational burden. This addresses the existing problem in which the roadside system serves solely as a communication bridge between the vehicle system and the cloud, resulting in the vehicle system relying solely on its own decision-making capabilities to handle complex situations, resulting in excessive computational pressure on the vehicle system.

[0006] However, although the above method increases the detection distance in disguise through vehicle-cloud collaboration, the driving suggestions given by the cloud do not directly interact with the assisted driving decisions on the vehicle side. Instead, they are displayed in a language form to inform the driver whether he needs to exit the navigation assistance. In reality, it still requires the driver to be highly concentrated and has limited effectiveness in reducing driver fatigue.

[0007] In addition, the above solution relies on good network conditions on the vehicle side and the cloud side. However, most highways are actually located in some remote areas, which do not have good network conditions. When the network conditions are poor, the cloud side will frequently give incorrect driving suggestions, resulting in more interference information and increasing driver fatigue.

[0008] Therefore, how to solve the problem of low safety of high-speed automatic assisted navigation driving caused by the limited detection distance of a single vehicle without almost increasing the computing power requirements on the vehicle side and under poor network conditions, especially poor visibility. Summary of the Invention

[0009] The purpose of the present invention is to provide a vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method in order to solve the defects of the above-mentioned prior art.

[0010] The purpose of the present invention can be achieved by the following technical solutions:

[0011] A vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method, comprising:

[0012] Step S1: When establishing a vehicle-cloud collaborative task, the vehicle sends the collected vehicle-side data to the cloud;

[0013] Step S2: The cloud generates a first decision instruction based on all vehicle-side data and other sensor data, wherein the first decision instruction includes an instruction type, instruction content, and a first timestamp. The instruction types include a pilot assistance continuation instruction and a pilot speed limit constraint instruction. The first timestamp is generated based on the collection time of all vehicle-side data and other sensor data.

[0014] Step S3: The cloud sends the first decision instruction to the vehicle. After receiving the first decision instruction, the vehicle records the current time as the second timestamp and calculates the time interval between the second timestamp and the first timestamp as the cumulative delay of the RSU full link;

[0015] Step S4: Determine whether the cumulative delay of the RSU full link is less than the pre-configured time interval threshold. If yes, execute step S5;

[0016] Step S5: Determine the instruction type of the first decision instruction.

[0017] If the instruction type is a pilot assistance survival instruction, the display content is generated according to the instruction content.

[0018] If the instruction type is a pilot speed boundary constraint instruction, a first speed constraint boundary condition is generated according to the instruction content, and step S6 is executed;

[0019] Step S6: The vehicle side uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction.

[0020] The other sensor data includes roadside sensor data and / or drone sensor data.

[0021] The process of generating the first timestamp includes:

[0022] Obtaining the sampling frequency of all vehicle-side data and other sensor data that generate the first decision instruction;

[0023] Select the data type with the highest sampling frequency and obtain the sampling time of the data type;

[0024] The acquired acquisition moment is used as the first timestamp.

[0025] The vehicle-side data includes image data, which includes at least full-frame low-resolution images and gaze point high-resolution images, as well as optional full-frame high-resolution low-frame rate images.

[0026] The vehicle-side data is uploaded to the cloud after desensitization, and the desensitization calculation of the image data is implemented based on the full-frame low resolution, and the desensitization calculation results are synchronously applied to the gaze point high-resolution image and the full-frame high-resolution low-frame rate image.

[0027] The preconfigured time interval threshold is 8 seconds.

[0028] The step S6 specifically includes:

[0029] The vehicle side generates a first vehicle control instruction based on its own decision model, wherein the first vehicle control instruction includes assisted driving existence information, guidance vehicle speed information, and lane change decision information;

[0030] The guiding vehicle speed information is extracted based on the first vehicle control instruction, and it is determined whether the guiding vehicle speed information meets the first speed constraint boundary condition. If yes, the first vehicle control instruction is used as the second decision instruction.

[0031] The step S6 specifically includes:

[0032] Obtaining the decision model constructed on the vehicle side and extracting the vehicle speed decision sub-model, wherein the vehicle speed decision sub-model includes the optimization objective and the original constraint conditions;

[0033] Extract the speed guidance constraint from the original constraint conditions, calculate the intersection of the first speed constraint boundary condition and the speed guidance constraint, and replace the speed guidance constraint in the original constraint conditions with the obtained intersection to obtain an updated vehicle speed decision sub-model;

[0034] A second vehicle control instruction is generated based on the updated vehicle speed decision sub-model as a second decision instruction, wherein the second vehicle control instruction guides vehicle speed information.

[0035] A vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving device includes a memory, a processor, and a program stored in the memory. When the processor executes the program, the method as described above is implemented.

[0036] A storage medium stores a program, which implements the above method when executed.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The first decision instruction generated by the cloud is not just a prompt text, but also includes specific speed constraints. On the one hand, the cumulative delay of the entire RSU link is used to determine whether the first decision instruction on the cloud has reference value. If it has no reference value, it is directly discarded, thereby filtering the first decision instruction on the cloud to avoid interference caused by erroneous first decision instructions generated due to poor network conditions. On the other hand, the first decision instruction on the cloud forms a collaborative decision-making mechanism between the vehicle and the cloud by constraining the speed of local decisions on the vehicle side, rather than simply patching together. This improves the degree of automation, reduces interference to users, and alleviates user fatigue. In addition, the first decision instruction on the cloud only affects the speed of local decisions and does not include lane changes. Lane change decisions are still made by local decisions on the vehicle side. Cloud decisions focus on long-distance obstacles, while local decisions focus on short-range road conditions. More importantly, the first decision instruction on the cloud can directly incorporate speed constraints by constraining the speed of local decisions on the vehicle side, eliminating the need for major modifications to the local decision model and not generating high computing power requirements.

[0039] 2. The acquisition time of the sensor data with the highest sampling frequency is used as the first timestamp to help the second decision instruction determine its actual validity and ensure that the second decision instruction meets the first speed constraint boundary condition.

[0040] 3. Replace the speed-guided constraint in the original constraint condition with the obtained intersection. Only a simple intersection operation is required to replace the speed-guided constraint, which requires low computing power and improves response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the main steps of the method of the present invention. DETAILED DESCRIPTION

[0042] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0043] Pilot-assisted driving, especially pilot-assisted driving on highways, is also known as high-speed automatic assisted navigation driving. Due to the high speed of highways, the average vehicle speed is 120 kilometers per hour, and the forward distance per second is about 33m / s. Braking from 120 to 0 at a comfortable deceleration (-1.5m / s 2 ) Required distance d = v 2 / (2×a) is approximately 363 meters. This means the vehicle needs to be at least 370 meters away to execute an action for a stationary target event. This distance exceeds the typical effective range of current intelligent driving perception systems (approximately 250 meters) by 50%.

[0044] Generally, traffic regulations for low-visibility conditions require that when visibility is less than 500 meters, the speed limit should not exceed 80 kilometers per hour, low-beam headlights should be engaged, and a distance of at least 150 meters should be maintained from the vehicle ahead in the same lane. This performance requirement alone exceeds the current performance limits of assisted driving (current intelligent driving systems have a perception range of less than 500 meters). Therefore, the objective conditions for upgrading from assisted driving to autonomous driving are not met.

[0045] This performance limitation isn't due to limitations in current machine vision sensor front-ends. Common 8-megapixel cameras currently have an optical resolution close to that of the human fovea (60 ppd at a 60-degree field of view, with each pixel covering 1 arc minute). Rather, vehicles are unable to process this large amount of information, forcing them to reduce image resolution to 1 / 4, or 16 ppd. This reduction in computational complexity is equivalent to a decrease in visual acuity from 1.5 to 0.2 on a logarithmic eye chart.

[0046] Regarding vehicle decision-making, since the vehicle moves at a speed of 33 meters per second, there is a 290-meter operating distance between the desired comfortable maneuvering distance of 370 meters and the approximately 80-meter emergency braking response. A simple calculation shows that if the cloud-based decision-making loop latency does not exceed 8 seconds, it will bring positive improvements to the vehicle's functional experience. Cloud-based decision-making focuses on long-range obstacles, while local decision-making focuses on short-range road conditions.

[0047] Based on the above analysis, this application provides a vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method, such as Figure 1 As shown, including:

[0048] Step S1: When establishing a vehicle-cloud collaborative task, the vehicle sends the collected vehicle-side data to the cloud;

[0049] Specifically, when the vehicle starts the navigation function, that is, the high-speed automatic assisted navigation driving function, the vehicle will attempt to establish a connection with the cloud. When this connection is established, the video encoding system will start and wait for the cloud to send control signals.

[0050] After receiving the connection from the vehicle, the cloud will start the business processing module of the video stream. After the business processing module completes its work, it will send a control signal to the vehicle, requiring it to start uploading video stream data.

[0051] After the vehicle receives the cloud start signaling, it will continue to upload the encoding upload task of the camera perception image according to the cloud signaling requirements until the control signaling changes or receives a termination command.

[0052] The vehicle-side data includes image data, which includes at least full-frame low-resolution images and gaze point high-resolution images, as well as optional full-frame high-resolution low-frame rate images.

[0053] The vehicle-side data is uploaded to the cloud after desensitization. The desensitization calculation of the image data is implemented based on the full-frame low resolution, and the desensitization calculation results are synchronously applied to the gaze point high-resolution image and the full-frame high-resolution low-frame rate image.

[0054] Similar to existing technologies, the video encoding and delivery is based on webRTC technology. Simulcast, a feature in webRTC, allows a single source of information to be encoded into multiple data layers with varying spatial and temporal elasticity to improve system scalability. In this solution, the most important vehicle forward image information is divided into three layers: a full-frame low-resolution layer, a gaze-based high-resolution layer, and an optional full-frame high-resolution low-frame-rate layer. The first two are encoded at 720p / 10fps, while the latter uses the original 2160p resolution but with a reduced frame rate of 1fps.

[0055] The more time-consuming desensitization calculation can be reused on the same-viewing angle pictures of different resolutions, so the calculation processing is performed on the lower-resolution picture and the results are applied to the high-resolution picture at the same time.

[0056] The 8-megapixel camera in the vehicle used in this solution utilizes the available bandwidth for video encoding through appropriate content trade-offs. Using a 0.1bpp bitrate, the full bandwidth is approximately 3Mbps, and when not transmitting the optional full-frame data, the bandwidth is 2Mbps. Data transmission can also be completely eliminated, or the resolution and bandwidth can be further reduced by reducing the image size, ultimately suspending transmission altogether.

[0057] Step S2: The cloud generates a first decision instruction based on all vehicle-side data and other sensor data, where the first decision instruction includes an instruction type, instruction content, and a first timestamp. The instruction types include a pilot assistance continuation instruction and a pilot speed limit constraint instruction. The first timestamp is generated based on the collection time of all vehicle-side data and other sensor data.

[0058] In most embodiments, the other sensor data includes roadside sensor data and / or drone sensor data.

[0059] Regarding cloud computing (RSU computing), due to the erratic intervals between video frames transmitted via the cloud, this solution divides cloud processing into two phases: alignment and processing. The first phase addresses multi-view alignment and spatial coordinate decomposition at the moment the video information is generated, while the second phase handles the specific driving task strategy. The two phases are connected through aligned image sequences or feature vectors and neural network tokens. The details of the cloud-based algorithms for each phase are not part of this solution and can be fine-tuned as the business evolves.

[0060] However, in particular, in this embodiment, the process of generating the first timestamp includes:

[0061] Obtaining the sampling frequency of all vehicle-side data and other sensor data that generate the first decision instruction;

[0062] Select the data type with the highest sampling frequency and obtain the sampling time of the data type;

[0063] The acquired acquisition moment is used as the first timestamp.

[0064] This ensures the timeliness of the first decision calculated in the future, using the sensor timestamp of the earliest reading in the entire data packet. Even if there are inevitable transmission delays in the processing link, delay compensation when calculating the second decision can be guaranteed without adding additional risk.

[0065] Step S3: The cloud sends the first decision instruction to the vehicle. After receiving the first decision instruction, the vehicle records the current time as the second timestamp and calculates the time interval between the second timestamp and the first timestamp as the cumulative delay of the RSU full link;

[0066] Step S4: Determine whether the cumulative delay of the RSU full link is less than the pre-configured time interval threshold. If yes, execute step S5;

[0067] In this embodiment, according to calculation, the preconfigured time interval threshold is set to 8 seconds.

[0068] Step S5: Determine the instruction type of the first decision instruction.

[0069] If the instruction type is a pilot assistance survival instruction, the display content is generated according to the instruction content.

[0070] If the instruction type is a pilot speed boundary constraint instruction, a first speed constraint boundary condition is generated according to the instruction content, and step S6 is executed;

[0071] Step S6: The vehicle side uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction.

[0072] In one embodiment, step S6 specifically includes:

[0073] The vehicle side generates a first vehicle control instruction based on its own decision model, wherein the first vehicle control instruction includes assisted driving existence information, guidance vehicle speed information, and lane change decision information;

[0074] Based on the first vehicle control instruction, the guiding vehicle speed information is extracted, and it is determined whether the guiding vehicle speed information meets the first speed constraint boundary condition. If so, the first vehicle control instruction is used as the second decision instruction. Otherwise, a reminder is displayed on the screen. This combination method is relatively primitive and has little difficulty in implementation.

[0075] Of course, in this embodiment, step S6 specifically includes:

[0076] Obtain the decision model built on the vehicle side and extract the vehicle speed decision sub-model, where the vehicle speed decision sub-model includes the optimization objective and the original constraint conditions;

[0077] Extract the speed guidance constraint from the original constraint conditions, calculate the intersection of the first speed constraint boundary condition and the speed guidance constraint, and replace the speed guidance constraint in the original constraint conditions with the obtained intersection to obtain an updated vehicle speed decision sub-model;

[0078] For example, the optimization objective of the vehicle speed decision sub-model is:

[0079] min v J =∑ i=1 30 ( v i - v i-1 ) 2

[0080] in: J To minimize speed changes and ensure a smooth experience, v is the vehicle speed, v i for i The speed of the car at the moment, v i-1 for i -1 moment's speed;

[0081] The original constraints are:

[0082] ∣ v i − v i−1 ∣≤ a max ·Δ t

[0083] in: a maxis the maximum acceleration, Δ t for i Moment and i -1 moment interval;

[0084] Speed ​​changes should meet comfort requirements:

[0085] v min,i ≤ v i ≤ v max,i ,∀ i =1,…,30

[0086] in, v min,i for i The lower limit of the original vehicle speed at the moment, v max,i for i The original vehicle speed limit at the moment, the original constraint conditions ensure that the vehicle meets the upper and lower speed limits under the safety boundary,

[0087] The updated constraints are:

[0088] min( v min,i , v ' min,i )≤v i ≤min(v max,i , v ' max,i ),∀ i =1,…,30

[0089] in: v ' min,i and v ' max,i The upper and lower limits obtained for the first velocity constraint.

[0090] A second vehicle control instruction is generated based on the updated vehicle speed decision sub-model as a second decision instruction, wherein the second vehicle control instruction guides the vehicle speed information.

[0091] This application was verified by simulation, and the simulation scenario is as follows: Taking a 6-kilometer-long one-way 4-lane and two-way 8-lane highway as an example, there is lane construction at 3 kilometers, and it is necessary to connect to the opposite first lane for passage. The road lacks speed limit signs, but cones are placed. Starting from the starting point, the vehicle using this embodiment successfully slows down, and the speed has dropped to 30 kilometers 300 meters away from the construction site.

[0092] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

Claims

1. A vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method, characterized in that: include: Step S1: When establishing a vehicle-cloud collaborative task, the vehicle sends the collected vehicle-side data to the cloud; Step S2: The cloud generates a first decision instruction based on all vehicle-side data and other sensor data, wherein the first decision instruction includes an instruction type, instruction content, and a first timestamp. The instruction types include a pilot assistance continuation instruction and a pilot speed limit constraint instruction. The first timestamp is generated based on the collection time of all vehicle-side data and other sensor data. Step S3: The cloud sends the first decision instruction to the vehicle. After receiving the first decision instruction, the vehicle records the current time as the second timestamp and calculates the time interval between the second timestamp and the first timestamp as the cumulative delay of the RSU full link; Step S4: Determine whether the cumulative delay of the RSU full link is less than the pre-configured time interval threshold. If yes, execute step S5; Step S5: Determine the instruction type of the first decision instruction. If the instruction type is a pilot assistance survival instruction, the display content is generated according to the instruction content. If the instruction type is a pilot speed boundary constraint instruction, a first speed constraint boundary condition is generated according to the instruction content, and step S6 is executed; Step S6: The vehicle side uses the first speed constraint boundary condition as one of the constraint conditions of the decision model to generate a second decision instruction; The process of generating the first timestamp includes: Obtaining the sampling frequency of all vehicle-side data and other sensor data that generate the first decision instruction; Select the data type with the highest sampling frequency and obtain the sampling time of the data type; The acquired acquisition moment is used as the first timestamp.

2. The vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method according to claim 1 is characterized in that: The other sensor data includes roadside sensor data and / or drone sensor data.

3. The vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method according to claim 1 is characterized in that: The vehicle-side data includes image data, which includes at least full-frame low-resolution images and gaze point high-resolution images, as well as optional full-frame high-resolution low-frame rate images.

4. The vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method according to claim 3 is characterized in that: The vehicle-side data is uploaded to the cloud after desensitization, and the desensitization calculation of the image data is implemented based on the full-frame low resolution, and the desensitization calculation results are synchronously applied to the gaze point high-resolution image and the full-frame high-resolution low-frame rate image.

5. The vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method according to claim 1 is characterized in that: The preconfigured time interval threshold is 8 seconds.

6. The vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method according to claim 1 is characterized in that: The step S6 specifically includes: The vehicle side generates a first vehicle control instruction based on its own decision model, wherein the first vehicle control instruction includes assisted driving existence information, guidance vehicle speed information, and lane change decision information; The guiding vehicle speed information is extracted based on the first vehicle control instruction, and it is determined whether the guiding vehicle speed information meets the first speed constraint boundary condition. If yes, the first vehicle control instruction is used as the second decision instruction.

7. The vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving method according to claim 1 is characterized in that: The step S6 specifically includes: Obtaining the decision model constructed on the vehicle side and extracting the vehicle speed decision sub-model, wherein the vehicle speed decision sub-model includes the optimization objective and the original constraint conditions; Extract the speed guidance constraint from the original constraint conditions, calculate the intersection of the first speed constraint boundary condition and the speed guidance constraint, and replace the speed guidance constraint in the original constraint conditions with the obtained intersection to obtain an updated vehicle speed decision sub-model; A second vehicle control instruction is generated based on the updated vehicle speed decision sub-model as a second decision instruction, wherein the second vehicle control instruction guides vehicle speed information.

8. A vehicle-cloud collaborative perception decision-making high-speed automatic assisted navigation driving device, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

9. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 7 is implemented.

Images