Automobile all-terrain recognition and unmanned aerial vehicle path finding system and method

Abstract

The invention discloses an automobile all-terrain recognition and unmanned aerial vehicle path-finding system, and the system comprises a front-view camera recognition module which is used for obtaining the image information of a front road condition, and analyzing and outputting the obtained front road condition information to a perception fusion module; the unmanned aerial vehicle path-finding module is used for obtaining road condition image information obtained based on an unmanned aerial vehicle path-finding function, analyzing and outputting the obtained environment road condition information to the perception fusion module; the perception fusion module is used for carrying out perception fusion on the front road condition information and the environment road condition information, analyzing and outputting an obtained vehicle driving mode to the control module; and the control module obtains the driving mode entered by the vehicle and controls the vehicle to enter the determined driving mode. The method is an accurate perception fusion scheme, foresight camera recognition and unmanned aerial vehicle path finding are combined, the all-terrain mode can be automatically switched when the environment is complex, and the stability of user cross-country is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle control, and particularly relates to a system and method combining all-terrain mode recognition and unmanned aerial vehicle (UAV) pathfinding. Background Art

[0002] Automobiles have various driving modes such as comfort, economy, and sport. These driving modes are required by users because they have different vehicle adjustment functions. Different from these driving modes, off-road vehicle models and pickup trucks with a higher chassis and off-road performance have additional off-road modes, and the all-terrain mode is a general term for off-road modes.

[0003] For example, a published document with the publication number CN118478887A, publication date August 13, 2024, and patent name "An all-terrain driving mode switching system and switching method based on a PHEV four-wheel drive vehicle" discloses a switching system including a wire control braking system, a vehicle-mounted system, a hybrid control system, an electric power steering system, a driving mode switching switch, a body domain control, and an instrument system; the driving mode switching switch is used to send a driving mode request signal to the vehicle-mounted system; the vehicle-mounted system is used to forward the driving mode request signal to the wire control braking system, the hybrid control system, and the electric power steering system, and the wire control braking system, the hybrid control system, and the electric power steering system execute corresponding MAPs.

[0004] In the switching process of the all-terrain mode of similar off-road vehicle models, the driver needs to manually select and switch to the required off-road modes such as snow, mud, grassland, and rock according to the current road conditions. There is currently no function to automatically switch the all-terrain mode according to the road surface type. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to meet the needs of different off-road players to switch the all-terrain mode, improve the switching method of the all-terrain mode, and implement a method combining all-terrain mode recognition and UAV pathfinding to achieve automatic switching of all-terrain recognition.

[0006] To achieve the above object, the technical solution adopted by the present invention is: an all-terrain recognition and UAV pathfinding system for an automobile:

[0007] A front-view camera recognition module, which is used to obtain image information of the road conditions ahead, analyze and output the obtained road conditions information ahead to the perception fusion module;

[0008] A UAV pathfinding module, which is used to obtain road condition image information obtained based on the UAV pathfinding function, analyze and output the obtained environmental road conditions information to the perception fusion module;

[0009] A perception fusion module, which is used to perform perception fusion on the road conditions information ahead and the environmental road conditions information, analyze and output the obtained vehicle driving mode to the control module;

[0010] A control module, which obtains the driving mode entered by the vehicle and controls the vehicle to enter the determined driving mode.

[0011] The front view camera recognition module is used to obtain the image recognition result obtained by the vehicle's vision system and the confidence corresponding to the image recognition result; the image recognition result is obtained by comparing the collected ground image information with a variety of pre-stored characteristic road surface images, and the obtained image recognition result is used as the road conditions information ahead;

[0012] The front view camera module consists of a wide-angle camera and a telephoto camera. The telephoto camera has a longer recognition distance and is used for long-distance pre-scanning of road surface type recognition. The wide-angle camera has a shorter recognition distance and is used for precise road surface type recognition at close range.

[0013] The UAV pathfinding module is used to obtain the characteristic road surface recognition result obtained based on the UAV pathfinding function and the corresponding confidence. The characteristic road surface recognition result is obtained by comparing the UAV pathfinding result with a variety of pre-stored characteristic road surface parameter information, and the obtained characteristic road surface recognition result is used as the environmental road conditions information;

[0014] The UAV pathfinding module is equipped with a UAV, and the UAV detects the road surface characteristics 0 meters to 350 meters in front of the vehicle.

[0015] The front view camera recognition module and the UAV pathfinding module transmit the road conditions information ahead and the environmental road conditions information to the perception fusion module through TBOX for perception fusion.

[0016] A method for all-terrain recognition of a vehicle and UAV pathfinding includes the following steps:

[0017] Step 1, obtain the road conditions information ahead obtained by the vehicle's vision system and the confidence corresponding to the road conditions information ahead;

[0018] Step 2, obtain the environmental road conditions information obtained based on the UAV pathfinding function and the confidence corresponding to the environmental road conditions information;

[0019] Step 3, analyze the road conditions information ahead and the environmental road conditions information, and determine the driving mode

[0020] Step 4, control the vehicle to enter the determined mode.

[0021] In step 1, the confidence level of the forward road condition information is judged according to the visibility in front of the vehicle. The higher the visibility, the higher the confidence level, and the lower the visibility, the lower the confidence level.

[0022] In step 2, during the operation of the drone, it communicates with the vehicle in real time, obtains the forward direction of the vehicle according to the vehicle position, and obtains the forward road condition information of the vehicle based on the map information. The confidence level of the environmental road condition information is defined according to the coincidence degree between the path exploration result of the drone and multiple pre-stored characteristic road surface parameter information. The higher the matching coincidence degree, the higher the confidence level.

[0023] In step 3, judge the consistency of the forward road condition information and the environmental road condition information. If they are consistent, output the confirmed driving mode. If they are inconsistent, based on the confidence levels of the forward road condition information and the environmental road condition information, use the road condition information with a higher confidence level as the confirmed output driving mode.

[0024] The all-terrain recognition method for vehicles and the path exploration method of drones are started under the condition of meeting all the following conditions:

[0025] Condition 1: The vehicle speed is lower than the set value; the vehicle can only switch to the off-road mode when driving at a low speed;

[0026] Condition 2: The four-wheel drive system is turned on;

[0027] Condition 3: The battery power of the drone is higher than 20% of the insurance power;

[0028] Condition 4: The visibility is higher than the set value;

[0029] Condition 5: Meet the preset minimum weather conditions.

[0030] The present invention is a method combining the all-terrain mode and the path exploration of drones, which provides users with the option of automatically switching the all-terrain mode, broadens the driving experience of users, makes off-roading easier, and enables novice players to experience the fun of off-roading. It is an accurate perception fusion solution that combines the recognition of the front camera and the path exploration of drones, and can also realize the automatic switching of the all-terrain mode in a more complex environment, ensuring the stability of users' off-roading. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The following briefly describes the content expressed in each drawing in the specification of the present invention:

[0032] Figure 1 It is a schematic flowchart of the automatic switching method of the all-terrain mode in the embodiment of the present application;

[0033] Figure 2 For Figure 1 It is a schematic flowchart of the implementation process of the front camera system included in the automatic switching method of the all-terrain mode in

[0034] Figure 3 For Figure 1 The schematic diagram of the implementation process of the automatic switching method for the all-terrain mode in Figure 1 includes the implementation process of using a drone to explore the road;

[0035] Figure 4 For Figure 1 The specific process diagram of the automatic switching method for the all-terrain mode in Figure 1 combined with the GPS positioning system;

[0036] Figure 5 The module schematic diagram of the automatic switching device for the all-terrain mode. Specific implementation manners

[0037] The following is a more detailed description with reference to the accompanying drawings through the description of the embodiments. The specific implementation manners of the present invention, such as the shapes, structures of the various components involved, the mutual positions and connection relationships between the various parts, the functions and working principles of the various parts, the manufacturing processes, and the operation and use methods, are further described in detail to help those skilled in the art have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0038] A system and method for combining the all-terrain mode with drone pathfinding to help pick-up / truck users automatically adjust the all-terrain mode in different scenarios, realizing the automatic recognition and switching of the all-terrain mode. The system includes:

[0039] A front-view camera recognition module for obtaining the image recognition result obtained by the vehicle's vision system and the corresponding confidence level of the image recognition result; the image recognition result is obtained by comparing and recognizing the collected ground image information with the image information corresponding to a variety of pre-stored characteristic road surface images; the front-view camera module consists of a wide-angle camera and a telephoto camera; the telephoto camera has a longer recognition distance and is used for long-distance pre-scanning of road surface type recognition, and the wide-angle camera has a shorter recognition distance and is used for precise road surface type recognition at close range.

[0040] A drone pathfinding module for obtaining the characteristic road surface recognition result obtained based on the drone pathfinding function and the corresponding confidence level, and transmitting it to the domain controller through TBOX for perception fusion; the characteristic road surface recognition result is obtained by comparing and recognizing the drone pathfinding result with a variety of pre-stored characteristic road surface parameter information. The drone scans the road surface 0 meters to 350 meters in front of the vehicle, and the scanning range is the road surface characteristic information between 0 meters and 350 meters in front of the vehicle.

[0041] The perception fusion module outputs the confirmed driving mode if the recognition result of the front camera system is the same as the current characteristic road surface represented by the characteristic road surface type obtained by the UAV pathfinding; if the recognition result of the front camera system is inconsistent with the current characteristic road surface represented by the characteristic road surface type obtained by the UAV pathfinding, the perception fusion of the front camera recognition result and the UAV pathfinding recognition result is performed according to the confidence of the front camera system recognition result and the confidence of the UAV pathfinding recognition result, and the determined driving mode is output; the control module is associated with the opponent part and controls the vehicle to enter the determined mode.

[0042] The control module is used to control the vehicle to enter the determined mode.

[0043] The control method of the control module includes:

[0044] step1, Obtain the image recognition result obtained by the vehicle's vision system and the corresponding confidence of the image recognition result;

[0045] step2, Obtain the characteristic road surface recognition result obtained based on the UAV pathfinding function and the corresponding confidence, and transmit it to the domain controller through TBOX for perception fusion;

[0046] step3, Compare the recognition result of the front camera system with the current characteristic road surface represented by the characteristic road surface type obtained by the UAV pathfinding;

[0047] step4, Control the vehicle to enter the determined mode.

[0048] The corresponding confidence of the image recognition result and the corresponding confidence of the road surface feature recognition result are obtained in the following manner:

[0049] When the road surface information can be clearly recognized normally through the front camera system or the driver's visibility is greater than 50 meters, at this time, the overall recognition accuracy of the camera is high and the accuracy rate is high. Therefore, the confidence of the front camera system is increased to increase the influence of the front camera system on the entire recognition perception fusion. At this time, the accuracy of the subsequent perception fusion can be improved based on the road surface recognition result obtained by the front camera system.

[0050] When the road surface information is difficult to recognize through the front camera system or the driver's visibility is less than 50 meters but is relatively clear at a closer distance, including but not limited to: rough sections, slight fog, rain curtain, dust, and snowy days, etc., at this time, the overall accuracy rate of the UAV to recognize the road surface type is high and the distance is far. The confidence of the front camera system is reduced to reduce the influence of the front camera system on the entire recognition perception fusion. At this time, the accuracy of the subsequent perception fusion can be improved based on the road surface recognition result obtained by the UAV pathfinding.

[0051] The enabling conditions for the automatic all-terrain mode are as follows:

[0052] Condition 1: The vehicle can only switch to the off-road mode when driving at a low speed to prevent the drone from being unable to keep up with the vehicle when driving at a high speed;

[0053] Condition 2: If the vehicle is equipped with a four-wheel drive system, ensure that the four-wheel drive system is turned on before enabling the off-road mode;

[0054] Condition 3: The battery power of the drone should be higher than 20% of the insurance power. The automatic all-terrain mode cannot be enabled when it is lower than 20%;

[0055] Condition 4: The visibility is higher than the set value. The all-terrain mode is not allowed to be enabled in bad weather and road conditions with a visibility lower than 30 meters.

[0056] Condition 5: Meet the preset minimum weather conditions. The weather information of the location can be obtained through the navigation system or by using the sensors on the vehicle, mainly including wind speed data and the conditions of rain, snow, and fog. It is necessary to pre-design the minimum standards that can be used to avoid damaging the drone.

[0057] Condition 6: Before entering the automatic all-terrain mode, it is also recommended to conduct a comprehensive inspection of the vehicle, including tires, brakes, suspension, fuel quantity, etc., to ensure the vehicle condition when entering this state, that is, to conduct a self-check;

[0058] As Figure 1 shown, the automatic switching method of this all-terrain mode can include the following:

[0059] (1) Obtain the image recognition result and the corresponding confidence level obtained from the front-view camera image system of the vehicle; the recognition result is obtained by comparing the collected ground image information with the image information corresponding to multiple pre-stored characteristic road surface images; the ground image information is used to reflect the road surface conditions where the vehicle is driving.

[0060] (2) Obtain the road surface recognition result and the corresponding confidence level obtained based on the drone's path exploration; the drone recognition result is obtained by comparing the image information corresponding to multiple pre-stored characteristic road surface images;

[0061] (3) If the recognition result of the front camera system is the same as the current characteristic road surface represented by the characteristic road surface type obtained by the UAV pathfinding, the confirmed driving mode is output; if the recognition result of the front camera system is inconsistent with the current characteristic road surface represented by the characteristic road surface type obtained by the UAV pathfinding, the recognition result of the front camera and the recognition result of the UAV pathfinding are perceptually fused based on the confidence level of the recognition result of the front camera system and the confidence level of the recognition result of the UAV pathfinding, and the determined driving mode is output; the control module associates with the counterparty and controls the vehicle to enter the determined mode.

[0062] As Figure 2 shown, the camera module of the front camera system consists of a wide-angle camera with a horizontal FOV of 120° and a telephoto camera with a horizontal FOV of 30°; the telephoto camera has a longer recognition distance and is used for long-distance preview recognition, and the wide-angle camera has a shorter recognition distance and is used for close-range precise recognition. The road surface type is recognized by the front camera and output to the perception module to sense the recognition result: road surface type. Each module inputs to the perception post-processing and optimization module: (1) the recognition result of the perception module; (2) the dynamic information of the vehicle chassis: IMU data (characterizing the bumps and slip rate of the vehicle), ESP information (slip rate of the four wheels, etc.); (3) other auxiliary information: the navigation map provides road grade information. For example, if the road grade is a highway, there must be no sandy road surface, muddy road surface, or rock climbing road surface; when it is within the ODD area of the high-precision map, there should be no sandy road surface, rock climbing road surface, and the probability of muddy road surface is also relatively small.

[0063] The perception post-processing and optimization module sends the recognition result to the analysis module through the CAN signal, and the analysis module makes a confidence analysis. The CAN output module: is responsible for outputting the perception result to the perception fusion module to support the perception fusion module to perceive and fuse to switch the driving mode.

[0064] As Figure 3 shown, the automatic switching method of the all-terrain mode includes a schematic diagram of the implementation process of UAV pathfinding; the UAV pathfinding system collects road surface features by the UAV camera; the UAV video system conducts video analysis, outputs the confidence level after road surface feature perception and recognition, and then outputs the confidence level to the perception fusion module.

[0065] As Figure 4Schematic diagram of the specific process of the automatic switching method of the all-terrain mode in combination with the positioning system. The automatic switching method of the all-terrain mode in the embodiment of the present application further includes the vehicle controller automatically controlling the controlled subsystem to achieve automatic switching by completing the automatic switching of the all-terrain driving mode. Among them, the controlled subsystem may include, but is not limited to, BCM (Brake Control Module, braking control system), EMS (Engine Management System, engine management system), ECU (Electrical Control Unit, electronic control unit), EPS (Electrical Power Steering, electric power steering), CDC (Continuous Damping Control, automatic adjustment and uninterrupted shock absorption control system), AWD (All Wheel Drive, all-wheel drive system) and HMI (Human Machine Interaction, human-machine interaction).

[0066] As Figure 5 Schematic diagram of the modules of the automatic switching device of the all-terrain mode shown in the figure, including the following modules:

[0067] The front view camera image information acquisition module is used to acquire the image recognition result obtained by the vehicle's vision system and the confidence corresponding to the image recognition result; the image recognition result is obtained by comparing and recognizing the collected ground image information with the image information corresponding to a variety of pre-stored characteristic road surface images.

[0068] The drone road surface feature acquisition module is used to acquire the feature road surface recognition result obtained based on the drone pathfinding function and the corresponding confidence; the feature road surface recognition result is obtained by comparing and recognizing the drone pathfinding result with a variety of pre-stored characteristic road surface parameter information.

[0069] The perception fusion module is used to, if the current characteristic road surface represented by the image system recognition result and the drone recognition result is inconsistent, perform perception fusion on the two recognition results according to the confidence of the image system recognition result and the confidence of the drone recognition result, and output the mode determined from the all-terrain driving modes of the vehicle. The mode control module is used to control the vehicle to enter the determined mode.

[0070] The above system and method constitute a brand-new method of combining the all-terrain mode with drone pathfinding, providing users with the option of automatically switching the all-terrain mode, broadening the user's driving experience, making off-roading easier, and allowing novice players to also experience the fun of off-roading. It can accurately perform perception fusion, combining the front view camera recognition with drone pathfinding, and can also achieve automatic switching of the all-terrain mode in a more complex environment, ensuring the stability of the user's off-roading.

[0071] The present invention has been described by way of example in conjunction with the accompanying drawings. Obviously, the specific implementation of the present invention is not limited by the above methods. As long as various non-substantive improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A vehicle all-terrain recognition and drone pathfinding system, characterized by: The front camera recognition module is used to obtain image information of the road condition ahead, analyze and output the obtained road condition information ahead to the perception fusion module; The UAV pathfinding module is used to obtain the road condition image information based on the UAV pathfinding function, analyze and output the obtained environmental road condition information to the perception fusion module; A perception fusion module is used to perceive and fuse the front road condition information and the environmental road condition information, analyze and output the obtained vehicle driving mode to the control module; The control module obtains the driving mode entered by the vehicle and controls the vehicle to enter the determined driving mode.

2. The vehicle all-terrain recognition and drone pathfinding system according to claim 1 is characterized by: The front camera recognition module is used to obtain the image recognition result obtained by the visual system of the vehicle and the corresponding confidence of the image recognition result; the image recognition result is obtained by comparing the collected ground image information with a plurality of pre-stored characteristic road surface images, and the image recognition result is used as the road condition information ahead; The front-view camera module consists of a wide-angle camera and a telephoto camera. The telephoto camera has a longer recognition distance and is used for long-distance preview of road type recognition. The wide-angle camera has a shorter recognition distance and is used for close-range accurate road type recognition.

3. The vehicle all-terrain recognition and drone pathfinding system according to claim 1 is characterized by: The drone pathfinding module is used to obtain a characteristic road surface recognition result and a corresponding confidence level based on the drone pathfinding function, wherein the characteristic road surface recognition result is obtained by comparing the drone pathfinding result with a plurality of pre-stored characteristic road surface parameter information, identifying the obtained characteristic road surface recognition result, and using the characteristic road surface recognition result as environmental road condition information; The drone pathfinding module is provided with a drone, and the drone detects road features at a distance from 0 meters to 350 meters in front of the vehicle.

4. The vehicle all-terrain recognition and drone pathfinding system according to claim 1 is characterized by: The front-view camera recognition module and the UAV pathfinding module transmit the front road condition information and the environmental road condition information to the perception fusion module for perception fusion through TBOX.

5. A method for all-terrain recognition of automobiles and path finding by unmanned aerial vehicles, characterized in that: The following steps are involved: Step 1: obtaining the front road condition information obtained by the visual system of the vehicle and the corresponding confidence level of the front road condition information; Step 2: Obtaining environmental road condition information obtained based on the drone's pathfinding function and the corresponding confidence level of the environmental road condition information; Step 3: Analyze the road conditions ahead and the surrounding road conditions, and determine the driving mode Step 4: Control the vehicle to enter the determined mode.

6. The method for all-terrain recognition of automobiles and path finding by unmanned aerial vehicles according to claim 5 is characterized in that: In the step 1, the confidence of the road condition information ahead is judged according to the visibility in front of the vehicle. The higher the visibility, the higher the confidence, and the lower the visibility, the lower the confidence.

7. The method for all-terrain recognition of automobiles and path finding by unmanned aerial vehicles according to claim 6 is characterized in that: In step 2, the UAV communicates with the vehicle in real time during operation, obtains the vehicle's forward direction based on the vehicle's position, and obtains road condition information in front of the vehicle based on map information. The confidence level of the environmental road condition information is defined according to the degree of overlap between the UAV's pathfinding results and a plurality of pre-stored characteristic road surface parameter information. The higher the matching overlap, the higher the confidence level.

8. The method for all-terrain recognition of automobiles and path finding by unmanned aerial vehicles according to claim 7 is characterized in that: In step 3, the consistency between the road condition information ahead and the environmental road condition information is determined. If they are consistent, the confirmed driving mode is output; if they are inconsistent, the road condition information with higher confidence is used as the confirmed driving mode output based on the confidence levels of the road condition information ahead and the environmental road condition information.

9. The method for all-terrain recognition of automobiles and path finding by unmanned aerial vehicles according to claim 8, characterized in that: The vehicle all-terrain recognition and drone pathfinding methods are activated when all conditions are met: Condition 1: The vehicle speed is lower than the set value; the vehicle can only switch to off-road mode when it is driving at a low speed; Condition 2: The four-wheel drive system is turned on; Condition 3: The drone’s battery level is higher than 20% of the insurance level; Condition 4: Visibility is higher than the set value; Condition 5: Meet the preset minimum weather conditions.

Images