Vehicle-mounted unmanned aerial vehicle auxiliary automatic driving system and method

Through the on-board drone-assisted autonomous driving system, drones are used to obtain environmental information at high altitudes and integrate them with on-board sensor data, solving problems such as limited viewing angle and limited perception range of the autonomous driving system, improving the safety of complex road conditions and overall performance.

CN119065302APending Publication Date: 2024-12-03CHINA FAW CO LTD +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411200262.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing autonomous driving systems have limitations in terms of limited perspective, limited perception range, insufficient environmental adaptability and lack of road conditions ahead, resulting in challenges in safety and reliability under complex road conditions.

Method used

Introduced a vehicle-mounted drone-assisted autonomous driving system, which obtains environmental information at high altitudes and integrates it with vehicle-mounted sensor data to generate comprehensive road conditions for path planning and vehicle control.

Benefits of technology

It significantly improves the environmental perception capability of the autonomous driving system and its ability to deal with complex road conditions, improves overall performance and safety, and has high flexibility and real-time capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065302A_ABST
    Figure CN119065302A_ABST
Patent Text Reader

Abstract

The invention relates to the field of automatic driving, and discloses a system and method for assisting automatic driving of a vehicle-mounted unmanned aerial vehicle, and the method comprises the steps: controlling the unmanned aerial vehicle to take off to a preset height when a shielding object or a complex road condition is detected in front of the vehicle in a low-altitude visual angle environment information obtaining process; the unmanned aerial vehicle obtains high-altitude view angle information of roads in front of and around the vehicle in a high-altitude view angle when in the air; the high-altitude view angle information comprises traffic flow, weather, road conditions and potential obstacles; receiving and processing data returned by the unmanned aerial vehicle to the locomotive end, and fusing the data with pre-acquired vehicle-mounted sensor data to generate comprehensive road condition information; executing tasks of each stage in different scenes, and performing path planning and vehicle control on the vehicle terminal based on the comprehensive road condition information; and after the task is completed, the unmanned aerial vehicle returns and lands to enter a standby state. According to the scheme, the high-altitude view angle and the flexible deployment capability of the unmanned aerial vehicle are combined, so that the unmanned aerial vehicle can provide effective auxiliary support under various road conditions and environments, and the overall performance of automatic driving is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a system, method, electronic device, storage medium and vehicle for vehicle-mounted drone-assisted autonomous driving. Background Art

[0002] A major highway collapse accident occurred not long ago, which occurred on the Meilong Expressway at around 2 a.m. on May 1, 2024. When the accident occurred, a 17.9-meter-long road surface suddenly collapsed, forming a deep crack, causing several moving vehicles to fall into it.

[0003] With the rapid development of autonomous driving technology, the ability of autonomous vehicles to perceive the surrounding environment and deal with complex road conditions is constantly improving. However, due to the limited field of view of the vehicle's own sensors, especially in some complex road conditions and under obstructed vision, the existing autonomous driving system still has some shortcomings in perception and prediction.

[0004] In the existing technology, self-driving cars mainly rely on on-board sensors (such as lidar, cameras and radar) to obtain information about the surrounding environment. However, these sensors can only provide environmental information within a limited range around the car, and there are certain limitations in obtaining information about the road conditions ahead, traffic congestion, accidents, etc.

[0005] Introduction to existing technology:

[0006] Autonomous driving technology is one of the important directions of the development of the modern automobile industry. It mainly realizes autonomous driving of vehicles in different environments by carrying various sensors and computing devices. At present, the autonomous driving system mainly relies on the following sensors to obtain environmental information:

[0007] Camera: used to obtain visual information of the surrounding environment and identify traffic signals, road signs, pedestrians and other obstacles.

[0008] LiDAR: It generates a high-precision three-dimensional environmental map by emitting laser beams and receiving reflected signals, providing detailed information around the vehicle.

[0009] Millimeter-wave radar: uses radio wave ranging technology to detect obstacles around the vehicle, especially in bad weather conditions.

[0010] Ultrasonic sensor: Mainly used for close-range measurement, assisted parking and obstacle detection during low-speed driving.

[0011] GPS and inertial navigation system: Provides vehicle location information and motion status to help achieve precise positioning and navigation.

[0012] However, although the above-mentioned autonomous driving technologies have made significant progress, the existing sensor systems still have some obvious problems and pain points, which are mainly reflected in the following aspects:

[0013] Limited viewing angle: The installation position and angle of the on-board sensors limit their viewing angle, especially when there are large vehicles or buildings blocking the view ahead, so they cannot obtain comprehensive road condition information. This limited viewing angle may prevent the autonomous driving system from obtaining key data in a timely manner when dealing with complex road conditions, increasing the risk of accidents.

[0014] Limited sensing range: Although LiDAR and millimeter-wave radar can provide high-precision environmental information, their sensing range is still limited, usually only covering a range of tens to hundreds of meters. Such a sensing range is insufficient when driving at high speeds or when advance judgment is required.

[0015] Insufficient environmental adaptability: The performance of existing cameras and lidars will significantly decline in adverse weather conditions such as rain, snow, fog and haze, affecting the accuracy and reliability of environmental perception. For example, rain or snow may interfere with the camera's field of view, and radar signals are easily attenuated in haze.

[0016] Lack of information about the road ahead: Current autonomous driving systems rely mainly on their own sensors to obtain real-time data about the surrounding environment, and lack the ability to perceive the road ahead in advance. This shortens the system's response time when faced with emergencies (such as traffic accidents, temporary construction, etc.), increasing the difficulty of coping.

[0017] High data processing pressure: The autonomous driving system needs to process a large amount of sensor data, including images, point clouds, radar signals, etc. This places high demands on the performance of on-board computing devices and increases the complexity and cost of the system.

[0018] In summary, existing autonomous driving technologies have certain limitations in terms of viewing angle, perception range, environmental adaptability, and advance perception of road conditions ahead. These problems not only limit the overall performance of the autonomous driving system, but also pose challenges to its safety and reliability under complex road conditions. Therefore, a new technology that can break through these bottlenecks is urgently needed to comprehensively improve the autonomous driving system's environmental perception capabilities and its ability to cope with complex road conditions. Summary of the invention

[0019] Based on this, it is necessary to provide a system and method for vehicle-mounted drone-assisted automatic driving in response to the above technical problems. By introducing a vehicle-mounted drone-assisted automobile automatic driving system, several key problems existing in existing automatic driving technology are solved: limited viewing angle, limited perception range, insufficient environmental adaptability, lack of information about road conditions ahead, and high data processing pressure.

[0020] In a first aspect, the present application provides a vehicle-mounted drone-assisted autonomous driving system, comprising: a drone body mounted on the top of a vehicle and a vehicle-mounted drone control module; and also comprising a drone sensor module, a vehicle-mounted information processing module, and an autonomous driving control module connected to the vehicle-mounted drone control module via a wireless communication module;

[0021] The vehicle-mounted drone control module is used to control the drone body to take off from the top of the vehicle to a predetermined height when an obstruction or complex road condition is detected in front of the vehicle during the low-altitude environmental information acquisition process; and after the mission is completed, the drone body is descended and returned, and landed on the top of the vehicle to enter a standby state;

[0022] The wireless communication module is used to establish data transmission between the drone body and the locomotive end;

[0023] The drone sensor module is used to obtain high-altitude perspective information of the road in front of the vehicle and the surrounding road at a high-altitude perspective when the drone body is in the air, and send it to the locomotive end through the wireless communication module; wherein the high-altitude perspective information includes traffic flow, weather conditions, road conditions, and potential obstacles;

[0024] The vehicle-mounted information processing module is used to receive and process the data transmitted back by the drone body to the locomotive end, and fuse it with the pre-acquired vehicle-mounted sensor data to generate comprehensive road condition information;

[0025] The autonomous driving control module is used to execute tasks at various stages in different scenarios, and to perform path planning and vehicle control on the vehicle side based on comprehensive road condition information.

[0026] Optionally, the drone sensor module includes a collection unit arranged on the drone body; the collection unit may be one or more of the camera, lidar, and millimeter-wave radar sensors.

[0027] Optionally, the pre-acquired on-board sensor data is data obtained by an on-board sensor at the locomotive end;

[0028] The vehicle-mounted sensor is used to obtain environmental information and obstacle time sequence information of the road ahead and around the vehicle from a low-altitude perspective.

[0029] Optionally, the vehicle-mounted sensor includes:

[0030] A perception module is used to obtain environmental information and obstacle time sequence information of the road ahead and around the vehicle from a low-altitude perspective;

[0031] The prediction module is used to input the environmental information of the road ahead and around the vehicle from the low-altitude perspective and the obstacle time series information into a pre-established deep learning model to predict the trajectory corresponding to the obstacle, including position, speed, and acceleration information.

[0032] Optionally, the automatic driving control module includes:

[0033] An acquisition unit, configured to acquire a predefined automatic driving function; the automatic driving function includes a driving function and a parking function;

[0034] A configuration unit, configured to allocate a scene, a stage, and an execution task to each autonomous driving function; wherein the autonomous driving function corresponds to the scene one by one, a scene includes multiple stages, each stage includes multiple tasks, and each task includes a planner; wherein the tasks include: identifying comprehensive road condition information, generating a path plan based on the comprehensive road condition information, and making control decisions to adjust the vehicle's driving route and driving speed;

[0035] A definition unit, used to define parameters and parameter acquisition functions for each scenario, and each stage and task under different scenarios; wherein the parameter acquisition function is used to call the corresponding scenario according to the driver's requirements, and the tasks of each stage under different scenarios;

[0036] The scenarios include: urban road scenarios and highway scenarios.

[0037] Optionally, the definition unit includes:

[0038] The scenario design unit is used to implement the scheduling of scenarios, stages and tasks under different scenarios through the registration mechanism function;

[0039] Parameter configuration design unit, used to set scene configuration parameters;

[0040] The task design unit is used to provide task scheduling services.

[0041] Optionally, the automatic driving control module further includes: a navigation signal receiving unit and a control processing unit;

[0042] The navigation signal receiving unit is used to obtain the navigation signal of the current vehicle in real time after the automatic driving function is enabled;

[0043] The navigation signal receiving unit is connected to the input end of the control processing unit and is used to transmit the navigation signal of the current vehicle, as well as the driving speed and position of the vehicle in the navigation coordinate system to the control processing unit for vehicle control.

[0044] Optionally, the control processing unit includes: an acquisition unit, a generation unit and a transmission unit;

[0045] The acquisition unit is used to acquire the navigation destination location, the vehicle's current driving route and longitude and latitude through the navigation signal receiving module;

[0046] The generating unit is used to compare the navigation destination position sequence with the vehicle's current driving route, longitude and latitude, and generate an offset and a driving speed control amount;

[0047] The transmission unit is used to send the offset and the driving speed control amount to the driving side.

[0048] Optionally, the parameter configuration design unit includes:

[0049] The first definition unit is used to define the stage parameters and execution task parameters under the scene according to the scene configuration parameters, and obtain the parameters required by the scene management module;

[0050] The second definition unit is used to define the parameters required for the decision module to implement the decision logic function according to the decision maker configuration parameters;

[0051] The third definition unit is used to define the planning module implementation method and the required parameters of the quality planning function according to the planner configuration parameters.

[0052] In a second aspect, the present application provides a method for vehicle-mounted drone-assisted autonomous driving, the method comprising:

[0053] When obtaining environmental information from a low-altitude perspective, if an obstruction or complex road conditions are detected in front of the vehicle, the drone is controlled to take off from the top of the vehicle to a predetermined height;

[0054] When the drone is in the air, it obtains high-altitude perspective information of the road ahead of the vehicle and the surrounding road from a high-altitude perspective, and sends it to the locomotive end by wireless means; wherein the high-altitude perspective information includes traffic flow, weather conditions, road conditions, and potential obstacles;

[0055] Establish data transmission between the UAV and the locomotive; receive and process the data sent back by the UAV to the locomotive, and fuse it with the pre-acquired vehicle sensor data to generate comprehensive road condition information;

[0056] Perform tasks at different stages in different scenarios, and perform path planning and vehicle control on the vehicle side based on comprehensive road condition information;

[0057] After the mission is completed, the drone descends, returns and lands on top of the vehicle and enters standby mode.

[0058] Compared with the closest prior art, the beneficial effects of the present invention are:

[0059] The above-mentioned vehicle-mounted drone-assisted automobile automatic driving system and method, when an obstruction or complex road condition is detected in front of the vehicle during the process of obtaining environmental information from a low-altitude perspective, the drone is controlled to take off to a predetermined height; when the drone is in the air, it obtains high-altitude perspective information of the road in front of the vehicle and the surrounding road at a high-altitude perspective; the high-altitude perspective information includes traffic flow, road conditions, and potential obstacles; receives and processes the data sent back by the drone to the locomotive end, and fuses it with the pre-acquired vehicle-mounted sensor data to generate comprehensive road condition information; performs tasks at various stages in different scenarios, and performs path planning and vehicle control on the vehicle end based on the comprehensive road condition information; after the task is completed, the drone returns and lands to enter the standby state. In actual working conditions, it has extremely high flexibility, real-time and wide applicability. Based on the high-altitude perspective and flexible deployment capabilities of the drone, its target vehicle can provide effective auxiliary support in a variety of road conditions and environments, significantly improving the overall performance and safety of the automatic driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of the system structure of a vehicle-mounted drone assisting in the automatic driving of a vehicle in one embodiment;

[0061] Figure 2 This is a flow chart of a method for a vehicle-mounted drone to assist in the automatic driving of a vehicle in one embodiment;

[0062] Figure 3 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] The embodiments of the present application provide a system and method for vehicle-mounted drone-assisted autonomous driving, which can be adapted to the application environment of all types of scenarios of all vehicle autonomous driving functions.

[0065] The method can be applied to a terminal, a server, or a system including a terminal and a server, and implemented through the interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc.

[0066] In one embodiment, Figure 1As shown, a decision-making and planning system for vehicle automatic driving is provided, including: a drone body 110 and a vehicle-mounted drone control module 120 installed on the top of the vehicle; and also including a drone sensor module 140, a vehicle-mounted information processing module 150 and an automatic driving control module 160 connected to the vehicle-mounted drone control module 120 through a wireless communication module 130. Among them, the drone body 110 and the vehicle-mounted drone control module 120 are installed on the car and can take off and land when needed. The drone has the ability to fly autonomously and collect data from a high altitude perspective.

[0067] The vehicle-mounted drone control module 120 is used to control the drone body to take off from the top of the vehicle to a predetermined height when an obstruction or complex road condition is detected in front of the vehicle during the process of acquiring environmental information from a low-altitude perspective; and after the mission is completed, the drone body is made to descend and return, and land on the top of the vehicle to enter a standby state;

[0068] Wireless communication module 130, used to establish data transmission between the drone body and the locomotive end;

[0069] The drone sensor module 140 is used to obtain high-altitude perspective information of the road in front of the vehicle and the surrounding road at a high-altitude perspective when the drone body is in the air, and send it to the locomotive end through the wireless communication module; wherein the high-altitude perspective information includes traffic flow, weather conditions, road conditions, and potential obstacles;

[0070] The vehicle information processing module 150 is used to receive and process the data transmitted by the drone body to the locomotive end, and fuse it with the pre-acquired vehicle sensor data to generate comprehensive road condition information;

[0071] The automatic driving control module 160 is used to execute tasks at various stages in different scenarios and perform path planning and vehicle control on the vehicle side based on comprehensive road condition information.

[0072] In the above embodiment, the drone body 110: the drone has high flight stability and flexibility, and can fly stably at different altitudes and in complex environments.

[0073] In the above embodiment, the vehicle-mounted drone control module 120 includes a drone take-off and landing mechanism, which can realize the control method of drone take-off and landing from the car and its autonomous flight capability. When the weather conditions contained in the acquired high-altitude perspective information are extreme weather, and the weather does not meet the current driving conditions, it can be changed in real time during driving to make the best route recommendation. Weather conditions include: wind speed, wind direction, temperature, relative humidity, light brightness, air pressure and irradiance, etc. All of them will affect the driving safety of the vehicle. For example, in the process of driving at night, when the cloud density contained in the acquired high-altitude perspective information is high, combined with the driving time of the vehicle-mounted terminal, it can be judged whether there will be a situation that affects the driving side in the future period. Such as extreme weather such as rain and hail. At this time, adjustments can be made based on the obtained forecast information.

[0074] In the above embodiment, the drone sensor module 140 can be equipped with a variety of sensors such as high-resolution cameras, laser radars, millimeter-wave radars, etc. to ensure comprehensive acquisition of road condition information from a high-altitude perspective.

[0075] Wireless communication module: The wireless communication module between the drone and the car should have high bandwidth and low latency characteristics to ensure real-time transmission and information synchronization.

[0076] In the above embodiment, the drone sensor module 140 includes a collection unit arranged on the drone body; the collection unit can be one or more of the camera, laser radar, and millimeter wave radar sensor.

[0077] Drone Sensor Module 140:

[0078] It is equipped with multiple sensors such as cameras, lidar, millimeter-wave radar, etc. to obtain road condition information from a high-altitude perspective.

[0079] In addition, sensor data can be collected and transmitted in real time.

[0080] In the above embodiment, the pre-acquired vehicle-mounted sensor data is data obtained by the vehicle-mounted sensor at the locomotive end;

[0081] The vehicle-mounted sensor is used to obtain environmental information and obstacle time sequence information of the road ahead and around the vehicle from a low-altitude perspective.

[0082] Further, the vehicle-mounted sensor includes: a perception module for acquiring environmental information and obstacle time sequence information of the road ahead of and around the vehicle from a low-altitude perspective of the vehicle;

[0083] The prediction module is used to input the environmental information of the road ahead and around the vehicle from the low-altitude perspective and the obstacle time series information into a pre-established deep learning model to predict the trajectory corresponding to the obstacle, including position, speed, and acceleration information.

[0084] In the above embodiment, in the vehicle information processing module 150, data fusion algorithm: the vehicle information processing system should adopt advanced data fusion algorithm to fuse the data collected by the drone with the vehicle sensor data to generate accurate comprehensive road condition information. Computing power: the information processing system should have strong computing power to quickly process and analyze a large amount of sensor data to ensure real-time performance and accuracy.

[0085] In the above embodiment, the automatic driving control module 160 is embodied in the following two aspects for the path planning and vehicle control of the vehicle:

[0086] Path planning algorithm: The autonomous driving control system should adopt an efficient path planning algorithm to plan the optimal path based on comprehensive road conditions information to avoid potential risks.

[0087] Control strategy: The control system should have a flexible control strategy that can adjust the vehicle's route and speed according to real-time traffic information to ensure driving safety and comfort.

[0088] In the above embodiment, the automatic driving control module 160 includes:

[0089] An acquisition unit, configured to acquire a predefined automatic driving function; the automatic driving function includes a driving function and a parking function;

[0090] A configuration unit, configured to allocate a scene, a stage, and an execution task to each autonomous driving function; wherein the autonomous driving function corresponds to the scene one by one, a scene includes multiple stages, each stage includes multiple tasks, and each task includes a planner; wherein the tasks include: identifying comprehensive road condition information, generating a path plan based on the comprehensive road condition information, and making control decisions to adjust the vehicle's driving route and driving speed;

[0091] A definition unit, used to define parameters and parameter acquisition functions for each scenario, and each stage and task under different scenarios; wherein the parameter acquisition function is used to call the corresponding scenario according to the driver's requirements, and the tasks of each stage under different scenarios;

[0092] The scenes include: urban road scenes and highway scenes. The division of scenes can improve the accuracy and recognition ability of autonomous driving.

[0093] In the above embodiment, the definition unit includes:

[0094] The scenario design unit is used to implement the scheduling of scenarios, stages and tasks under different scenarios through the registration mechanism function;

[0095] Parameter configuration design unit, used to set scene configuration parameters;

[0096] The task design unit is used to provide task scheduling services.

[0097] In the above embodiment, the automatic driving control module 160 further includes: a navigation signal receiving unit and a control processing unit;

[0098] The navigation signal receiving unit is used to obtain the navigation signal of the current vehicle in real time after the automatic driving function is enabled;

[0099] The navigation signal receiving unit is connected to the input end of the control processing unit and is used to transmit the navigation signal of the current vehicle, as well as the driving speed and position of the vehicle in the navigation coordinate system to the control processing unit for vehicle control.

[0100] In the above embodiment, the control processing unit includes: an acquisition unit, a generation unit and a transmission unit;

[0101] The acquisition unit is used to acquire the navigation destination location, the vehicle's current driving route and longitude and latitude through the navigation signal receiving module;

[0102] The generating unit is used to compare the navigation destination position sequence with the vehicle's current driving route, longitude and latitude, and generate an offset and a driving speed control amount;

[0103] The transmission unit is used to send the offset and the driving speed control amount to the driving side.

[0104] In the above embodiment, the parameter configuration design unit includes:

[0105] The first definition unit is used to define the stage parameters and execution task parameters under the scene according to the scene configuration parameters, and obtain the parameters required by the scene management module;

[0106] The second definition unit is used to define the parameters required for the decision module to implement the decision logic function according to the decision maker configuration parameters;

[0107] The third definition unit is used to define the planning module implementation method and the required parameters of the quality planning function according to the planner configuration parameters.

[0108] Based on the above embodiment, in practical application, the effect of the system can be tested by the following steps:

[0109] 1. Simulation test: Test the performance of the system in a simulation environment, including drone takeoff, data collection and transmission, information processing and fusion, path planning and control, etc.

[0110] 2. Field testing: Field testing is carried out in different scenarios (such as highways, urban roads, etc.) to evaluate the actual effects and performance of the system, including perception capabilities, safety, driving efficiency, etc.

[0111] 3. Data analysis: By analyzing the test data, evaluate the system's performance indicators, optimize the algorithm and system configuration, and ensure the stability and reliability of the system.

[0112] Through the above-mentioned specific embodiments and auxiliary materials, we can better understand and apply this technology and improve the overall performance and safety of the autonomous driving system.

[0113] Based on the same inventive concept, the embodiment of the present application also provides a method for implementing the vehicle-mounted drone-assisted vehicle automatic driving system involved in the vehicle-mounted drone-assisted vehicle automatic driving system. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme recorded in the above-mentioned system, so the specific limitations in the embodiments of one or more vehicle-mounted drone-assisted vehicle automatic driving methods provided below can refer to the limitations of the vehicle-mounted drone-assisted vehicle automatic driving system above, and will not be repeated here.

[0114] In one embodiment, Figure 2 As shown, a method for assisting a vehicle in automatic driving by using a vehicle-mounted drone is provided, and the method is described by taking the application of the method to a terminal as an example, including the following steps:

[0115] Step S101, when an obstruction or complex road condition is detected in front of the vehicle during the low-altitude environmental information acquisition process, the drone is controlled to take off from the top of the vehicle to a predetermined height;

[0116] Step S102, when the drone is in the air, obtains high-altitude view information of the road in front of the vehicle and the surrounding road at a high-altitude view, and sends it to the locomotive end by wireless means; wherein the high-altitude view information includes traffic flow, weather conditions, road conditions, and potential obstacles;

[0117] Step S103, establishing data transmission between the UAV and the locomotive; receiving and processing the data transmitted back by the UAV to the locomotive, fusing it with the pre-acquired vehicle sensor data, and generating comprehensive road condition information;

[0118] Step S104, executing tasks at each stage under different scenarios, and performing path planning and vehicle control on the vehicle terminal based on comprehensive road condition information;

[0119] Step S105, after the mission is completed, the drone descends, returns and lands on the top of the vehicle and enters a standby state.

[0120] Based on the inventive concept of the above steps S101 to S105, the following application embodiments are proposed in specific scenarios:

[0121] Example 1: Highway scenario

[0122] On highways, autonomous vehicles face problems with limited viewing angles and limited perception range, especially when there are large trucks or complex road conditions ahead. At this time, vehicle-mounted drones can play an important role.

[0123] Drone takeoff: When a large truck is detected ahead or the line of sight is obstructed, the vehicle-mounted drone system is activated and the drone takes off from the top of the car and flies to a predetermined height (e.g., more than 50 meters).

[0124] Data collection: Drones use cameras and lidar at high altitudes to obtain information about road conditions ahead, including traffic flow, lane conditions, road safety, and the location of large vehicles.

[0125] Data transmission: The drone transmits the collected road condition information to the vehicle information processing system in real time through the wireless communication module.

[0126] Information processing and fusion: The on-board information processing system integrates the data sent back by the drone with the data collected by on-board sensors (such as lidar, millimeter-wave radar, etc.) to generate comprehensive road condition information.

[0127] Path planning and vehicle control: The autonomous driving control system performs path planning and control based on comprehensive road condition information, adjusts the vehicle's route and speed, avoids potential risks, and ensures safe driving.

[0128] Drone recovery: After the mission is completed, the drone returns and lands on top of the car and enters standby mode.

[0129] Example 2: Urban road scene

[0130] On urban roads, self-driving cars may face complex traffic environments, including traffic signals, pedestrians, non-motor vehicles, etc. Drones can assist in obtaining more comprehensive road condition information.

[0131] Drone takeoff: On urban roads, when complex traffic or line of sight obstruction is detected, the drone takes off from the top of the car and flies to a predetermined height; for example, 30-50 meters.

[0132] Data collection: Drones use cameras and lidar at high altitudes to collect information about road conditions ahead, including traffic signal status, pedestrians, non-motor vehicles, traffic congestion, etc.

[0133] Data transmission: The drone transmits the collected road condition information to the vehicle information processing system in real time through the wireless communication module.

[0134] Information processing and fusion: The on-board information processing system integrates the data sent back by the drone with the data collected by on-board sensors (such as cameras, millimeter-wave radars, etc.) to generate comprehensive road condition information.

[0135] Path planning and vehicle control: The autonomous driving control system performs path planning and control based on comprehensive road condition information, adjusts the vehicle's route and speed, and ensures safe driving.

[0136] Drone recovery: After the mission is completed, the drone returns and lands on top of the car and enters standby mode.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for assisting a vehicle in automatic driving by a vehicle-mounted drone is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0139] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method described in the second aspect are implemented.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the second aspect are implemented.

[0142] The present invention also provides a vehicle, which is provided with a vehicle body, and a system for assisting the automatic driving of a vehicle with a vehicle-mounted drone as described above, which is arranged in the vehicle body.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0144] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A vehicle-mounted drone-assisted automatic driving system, characterized in that: include: The drone body and the vehicle-mounted drone control module are installed on top of the vehicle; It also includes a drone sensor module, a vehicle information processing module and an automatic driving control module that are connected to the vehicle-mounted drone control module through a wireless communication module; The vehicle-mounted drone control module is used to control the drone body to take off from the top of the vehicle to a predetermined height when an obstruction or complex road condition is detected in front of the vehicle during the low-altitude perspective environmental information acquisition process; After the mission is completed, the drone body descends and returns, landing on the top of the vehicle and entering standby mode; The wireless communication module is used to establish data transmission between the drone body and the locomotive end; The drone sensor module is used to obtain high-altitude perspective information of the road in front of the vehicle and the surrounding road at a high-altitude perspective when the drone body is in the air, and send it to the locomotive end through the wireless communication module; wherein the high-altitude perspective information includes traffic flow, weather conditions, road conditions, and potential obstacles; The vehicle-mounted information processing module is used to receive and process the data transmitted back by the drone body to the locomotive end, and fuse it with the pre-acquired vehicle-mounted sensor data to generate comprehensive road condition information; The autonomous driving control module is used to execute tasks at various stages in different scenarios, and to perform path planning and vehicle control on the vehicle side based on comprehensive road condition information.

2. The system according to claim 1, characterized in that The drone sensor module includes a collection unit arranged on the drone body; the collection unit can be one or more of the camera, laser radar, and millimeter wave radar sensor.

3. The system according to claim 1, characterized in that The pre-acquired on-board sensor data is data obtained by the on-board sensor at the locomotive end; The vehicle-mounted sensor is used to obtain environmental information and obstacle time sequence information of the road ahead and around the vehicle from a low-altitude perspective.

4. The system according to claim 3, characterized in that The vehicle-mounted sensor comprises: A perception module is used to obtain environmental information and obstacle time sequence information of the road ahead and around the vehicle from a low-altitude perspective; The prediction module is used to input the environmental information of the road ahead and around the vehicle from the low-altitude perspective and the obstacle time series information into a pre-established deep learning model to predict the trajectory corresponding to the obstacle, including position, speed, and acceleration information.

5. The system according to claim 1, characterized in that The automatic driving control module comprises: An acquisition unit, configured to acquire a predefined automatic driving function; the automatic driving function includes a driving function and a parking function; A configuration unit, configured to allocate a scene, a stage, and an execution task to each autonomous driving function; wherein the autonomous driving function corresponds to the scene one by one, a scene includes multiple stages, each stage includes multiple tasks, and each task includes a planner; wherein the tasks include: identifying comprehensive road condition information, generating a path plan based on the comprehensive road condition information, and making a control decision to adjust the vehicle's driving route and driving speed; A definition unit, used to define parameters and parameter acquisition functions for each scenario, and each stage and task under different scenarios; wherein the parameter acquisition function is used to call the corresponding scenario according to the driver's requirements, and the tasks of each stage under different scenarios; The scenarios include: urban road scenarios and highway scenarios.

6. The system according to claim 5, characterized in that The definition unit includes: The scenario design unit is used to implement the scheduling of scenarios, stages and tasks under different scenarios through the registration mechanism function; Parameter configuration design unit, used to set scene configuration parameters; The task design unit is used to provide task scheduling services.

7. The system according to claim 6, characterized in that The automatic driving control module further includes: a navigation signal receiving unit and a control processing unit; The navigation signal receiving unit is used to obtain the navigation signal of the current vehicle in real time after the automatic driving function is enabled; The navigation signal receiving unit is connected to the input end of the control processing unit and is used to transmit the navigation signal of the current vehicle, as well as the driving speed and position of the vehicle in the navigation coordinate system to the control processing unit for vehicle control.

8. The system according to claim 7, characterized in that The control processing unit includes: an acquisition unit, a generation unit and a transmission unit; The acquisition unit is used to acquire the navigation destination location, the vehicle's current driving route and longitude and latitude through the navigation signal receiving module; The generating unit is used to compare the navigation destination position sequence with the vehicle's current driving route, longitude and latitude, and generate an offset and a driving speed control amount; The transmission unit is used to send the offset and the driving speed control amount to the driving side.

9. The system according to claim 6, characterized in that The parameter configuration design unit comprises: The first definition unit is used to define the stage parameters and execution task parameters under the scene according to the scene configuration parameters, and obtain the parameters required by the scene management module; The second definition unit is used to define the parameters required for the decision module to implement the decision logic function according to the decision maker configuration parameters; The third definition unit is used to define the planning module implementation method and the required parameters of the quality planning function according to the planner configuration parameters.

10. A method for vehicle-mounted drone-assisted automatic driving, characterized in that: The method comprises: When obtaining environmental information from a low-altitude perspective, if an obstruction or complex road conditions are detected in front of the vehicle, the drone is controlled to take off from the top of the vehicle to a predetermined height; When the drone is in the air, it obtains high-altitude perspective information of the road in front of the vehicle and the surrounding road from a high-altitude perspective, and sends it to the locomotive end by wireless means; wherein the high-altitude perspective information includes traffic flow, weather conditions, road conditions, and potential obstacles; Establish data transmission between the UAV and the locomotive; receive and process the data sent back by the UAV to the locomotive, and fuse it with the pre-acquired vehicle sensor data to generate comprehensive road condition information; Perform tasks at different stages in different scenarios, and perform path planning and vehicle control on the vehicle side based on comprehensive road condition information; After the mission is completed, the drone descends, returns and lands on top of the vehicle and enters standby mode.

Citation Information

Cited By

  • Unmanned aerial vehicle emergency control system based on artificial intelligence

    CN120161777A

  • Vehicle perception optimization method and device based on unmanned aerial vehicle, vehicle and storage medium

    CN121626112A