Off-road course generation system, method, electronic device, and storage medium

Abstract

The application relates to an off-road route generation system and method, an electronic device and a storage medium, which comprise a take-off and landing platform configured to carry a vehicle-mounted unmanned aerial vehicle and send a flight task instruction to the vehicle-mounted unmanned aerial vehicle; a sensor assembly configured to collect laser radar point cloud data and camera image perception data of a target wild terrain based on the vehicle-mounted unmanned aerial vehicle; and a computing module configured to fuse the laser radar point cloud data and the camera image perception data to generate fused data, perform semantic recognition on the fused data, construct a three-dimensional semantic model of the target wild terrain according to a recognition result, generate an optimal off-road route of a target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters and a preset path planning algorithm, and obtain driving suggestions according to the optimal off-road route. Thus, the problem that the unmanned aerial vehicle has weak perception and map construction capability, cannot construct a precise three-dimensional terrain model and cannot plan a reliable driving route for an off-road vehicle is solved.

Description

Technical Field

[0001] This application relates to the field of path planning technology, and in particular to an off-road route generation system, method, electronic device and storage medium. Background Technology

[0002] Off-road environments are significantly more complex than paved roads for regular cars. Off-road environments involve driving on various materials and with different coverings. The ground is uneven, and obstacles include different materials (soft grass, hard trees, rocks, etc.). The ground coverings include water, sand, mud, etc. Especially for novices without off-road experience, the lack of terrain information can easily lead to dangerous situations such as getting lost or stuck.

[0003] Currently, related technologies can use vehicle-mounted drones with sensors and communication modules to collect sensor data and send it to the associated vehicle. At the same time, the vehicle with a processing unit can be used to generate and update a digital map of the area. In addition, related technologies can also use drones to photograph target off-road areas, create semantic segmentation maps, and import them into the QGIS system to provide auxiliary navigation for outdoor off-road equipment.

[0004] However, existing technologies only generate vehicle maps with the assistance of drones, and rely on existing maps to assist drone perception. In the complex outdoor conditions where off-road vehicles are located, the environment is prone to change or there are uncovered areas. When existing maps cannot provide enough road information, they may even interfere with the route results. In addition, although existing technologies can also use drones to perceive and build maps mainly through cameras, the camera's perception capability is limited in complex terrain conditions in the wild (such as when dense forests cover the road surface from the drone's perspective).

[0005] In summary, existing technologies for drones have weak perception and mapping capabilities, making it difficult to build accurate 3D terrain models and plan reliable routes for off-road vehicles, which greatly affects the user's driving experience and urgently needs to be addressed. Summary of the Invention

[0006] This application provides an off-road route generation system, method, electronic device, and storage medium to solve the problems in the prior art, such as the weak perception and map building capabilities of UAVs, the difficulty in building accurate three-dimensional terrain models, and the inability to plan reliable driving routes for off-road vehicles.

[0007] The first aspect of this application provides an off-road route generation system, comprising: a take-off and landing platform for carrying at least one vehicle-mounted drone and sending flight mission commands to the at least one vehicle-mounted drone; a sensor component for collecting lidar point cloud data and camera image perception data of a target wilderness terrain based on the at least one vehicle-mounted drone; and a computing module for fusing the lidar point cloud data and the camera image perception data to generate fused data, performing semantic recognition on the fused data, constructing a three-dimensional semantic model of the target wilderness terrain based on the recognition results, generating an optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route.

[0008] Optionally, in one embodiment of this application, it further includes: a communication module, used to transmit the lidar point cloud data and the camera image perception data to the computing module, and to transmit the flight mission command, flight trajectory data and / or aerial photography mission data to the at least one vehicle-mounted UAV; and a display component, used to convert the format of the three-dimensional semantic model and the optimal off-road route to obtain three-dimensional model format data, and to display the three-dimensional semantic model and the optimal off-road route based on a preset demonstration device and the three-dimensional model format data.

[0009] Optionally, in one embodiment of this application, the calculation module includes: a semantic map component, used to generate a three-dimensional semantic grid map of the target wilderness terrain based on the three-dimensional semantic model, and to solve the Euclidean distance between each grid and its neighboring grids in the three-dimensional semantic grid map according to a preset Euclidean distance transformation strategy, so as to determine the distance between each grid and the nearest obstacle; a path planning component, used to calculate the travel time of the target vehicle using the Euclidean distance, a preset speed function and the vehicle geometric passability parameters, to construct a travel time map of the target wilderness terrain based on the travel time, and to search backwards in the travel time map for the target direction with the fastest decrease in travel time gradient between the target point and the starting point, so as to obtain the optimal off-road route of the target vehicle based on the target direction; and a driving guidance component, used to calculate the driving parameters of the target vehicle based on the optimal off-road route, and to generate driving behavior suggestions matching the optimal off-road route based on the driving parameters.

[0010] Optionally, in one embodiment of this application, the take-off and landing platform includes a power module for supplying power to the at least one vehicle-mounted drone.

[0011] Optionally, in one embodiment of this application, the vehicle geometric passability parameters include at least one of vehicle kinematic constraints, vehicle dynamic performance, vehicle departure angle, vehicle approach angle, and tire adhesion conditions.

[0012] A second aspect of this application provides a method for generating off-road routes, comprising the following steps: collecting lidar point cloud data and camera image perception data of a target wilderness terrain based on at least one vehicle-mounted drone; fusing the lidar point cloud data and the camera image perception data to generate fused data, performing semantic recognition on the fused data, constructing a three-dimensional semantic model of the target wilderness terrain based on the recognition results, generating an optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route.

[0013] Optionally, in one embodiment of this application, after generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, the method further includes: converting the three-dimensional semantic model and the optimal off-road route into a format to obtain three-dimensional model format data, and displaying the three-dimensional semantic model and the optimal off-road route based on a preset demonstration device and the three-dimensional model format data.

[0014] Optionally, in one embodiment of this application, the step of generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route, includes: generating a three-dimensional semantic grid map of the target wilderness terrain based on the three-dimensional semantic model, and solving the Euclidean distance between each grid and its neighboring grids in the three-dimensional semantic grid map according to a preset Euclidean distance transformation strategy to determine the distance between each grid and the nearest obstacle; calculating the travel time of the target vehicle using the Euclidean distance, a preset speed function, and the vehicle geometric passability parameters; constructing a travel time map of the target wilderness terrain based on the travel time; and searching backwards in the travel time map for the target direction with the fastest decrease in travel time gradient between the target point and the starting point to obtain the optimal off-road route for the target vehicle based on the target direction; calculating the driving parameters of the target vehicle based on the optimal off-road route, and generating driving behavior suggestions matching the optimal off-road route based on the driving parameters.

[0015] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the off-road route generation method as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described off-road route generation method.

[0017] Therefore, the embodiments of this application have the following beneficial effects:

[0018] Embodiments of this application include a take-off and landing platform for carrying at least one vehicle-mounted drone and sending flight mission commands to the at least one vehicle-mounted drone; a sensor component for collecting lidar point cloud data and camera image perception data of the target wilderness terrain based on the at least one vehicle-mounted drone; and a computing module for fusing the lidar point cloud data and camera image perception data to generate fused data, performing semantic recognition on the fused data, constructing a three-dimensional semantic model of the target wilderness terrain based on the recognition results, and generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route. This application uses sensors such as lidar mounted on the drone to acquire three-dimensional information of the surrounding terrain, constructing an accurate three-dimensional terrain model, and planning routes for off-road vehicles based on the constructed terrain model, thus achieving safe and efficient off-road crossing without relying on existing maps. This solves the problems of weak perception and map construction capabilities of drones in the prior art, the difficulty in constructing accurate three-dimensional terrain models, and the inability to plan reliable driving routes for off-road vehicles.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 This is an example diagram of an off-road route generation system according to an embodiment of this application;

[0022] Figure 2 A schematic diagram of the logical architecture of an off-road route generation system provided in one embodiment of this application;

[0023] Figure 3 A schematic diagram of the execution logic of an off-road route generation system provided in one embodiment of this application;

[0024] Figure 4 This is a flowchart of a method for generating off-road routes according to an embodiment of this application;

[0025] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0026] Among them, 10-off-road route generation system, 100-lifting platform, 200-sensor assembly, 300-computing module, 501-memory, 502-processor, and 503-communication interface. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0028] The following description, with reference to the accompanying drawings, describes an off-road route generation system, method, electronic device, and storage medium according to embodiments of this application. Addressing the problems mentioned in the background section, this application provides an off-road route generation system, comprising: a take-off and landing platform for carrying at least one vehicle-mounted drone and sending flight mission commands to the at least one vehicle-mounted drone; a sensor assembly for collecting lidar point cloud data and camera image perception data of the target terrain based on the at least one vehicle-mounted drone; and a computing module for fusing the lidar point cloud data and camera image perception data to generate fused data, performing semantic recognition on the fused data, constructing a three-dimensional semantic model of the target terrain based on the recognition results, generating an optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route. This application uses sensors such as lidar mounted on a drone to acquire three-dimensional information of the surrounding terrain, constructing an accurate three-dimensional terrain model, and planning routes for off-road vehicles based on the constructed terrain model, thus achieving safe and efficient off-road crossing without relying on existing maps. This solves the problems in existing technologies, such as the weak perception and mapping capabilities of drones, the difficulty in building accurate 3D terrain models, and the inability to plan reliable driving routes for off-road vehicles.

[0029] Specifically, Figure 1 This is a block diagram of the off-road route generation system according to an embodiment of this application.

[0030] like Figure 1 As shown, the off-road route generation system 10 includes: a take-off and landing platform 100, a sensor assembly 200, and a computing module 300.

[0031] Among them, the take-off and landing platform 100 is used to carry at least one vehicle-mounted drone and send flight mission commands to at least one vehicle-mounted drone.

[0032] In the embodiments of this application, the aforementioned take-off and landing platform can be installed on the top of an off-road vehicle, enabling the vehicle-mounted drone (i.e., a mapping drone) to take off and land on the roof. The platform can also be used to fix and transport the vehicle-mounted drone while the vehicle is in motion. In addition, the take-off and landing platform can receive off-road route generation instructions such as flight mission instructions sent by the vehicle to the vehicle-mounted drone, thereby controlling the take-off and landing status of the vehicle-mounted drone and providing a basis for obtaining reliable off-road routes in the future.

[0033] Furthermore, the vehicle-mounted drone in this embodiment can also accurately identify its own vehicle and precisely land on the vehicle's take-off and landing platform through methods such as visual identification, positioning identification, or communication pairing identification, effectively improving the intelligence level and reliability of the vehicle-mounted drone.

[0034] Optionally, in one embodiment of this application, the take-off and landing platform 100 includes a power module for powering at least one vehicle-mounted drone.

[0035] It should be noted that the power module in the take-off and landing platform of this application embodiment can charge the vehicle-mounted drone by setting the charging time and charging power through the vehicle-mounted power supply, thereby effectively ensuring the endurance of the vehicle-mounted drone and the stability of the off-road route generated by the vehicle-mounted drone.

[0036] Sensor component 200 is used to collect lidar point cloud data and camera image perception data of the target field terrain based on at least one vehicle-mounted UAV.

[0037] In embodiments of this application, the aforementioned vehicle-mounted drone may include a variety of sensor devices:

[0038] 1. Gimbal-mounted mapping camera and lidar: to acquire information about the surrounding environment of the UAV;

[0039] 2. RTK receiver: Acquires the location information of the UAV;

[0040] 3. Inertial Measurement Unit: Acquires the attitude information of the UAV.

[0041] Therefore, the vehicle-mounted drone in this embodiment of the application uses LiDAR and camera fusion perception. Compared with visible light sensors, LiDAR point cloud data processing speed is faster. In vegetation-covered areas, laser penetration capability is stronger. Through multiple laser reflection data from LiDAR, laser reflection can directly obtain terrain data information through vegetation, thereby providing detailed and reliable data support for the subsequent construction of three-dimensional semantic models and the generation of optimal off-road routes.

[0042] The computing module 300 is used to fuse lidar point cloud data and camera image perception data to generate fused data, perform semantic recognition on the fused data, construct a three-dimensional semantic model of the target field terrain based on the recognition results, and generate the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters and preset path planning algorithm, and obtain driving suggestions based on the optimal off-road route.

[0043] After collecting LiDAR point cloud and camera image perception data of the field terrain through the vehicle-mounted UAV sensor components, the embodiments of this application can further process the data collected by the sensor components through the computing module in the vehicle-mounted UAV, that is, fuse the LiDAR point cloud data and camera image perception data to generate colored point cloud data, and obtain the positional relationship based on the position and timestamp information of each collection by the sensor components, thereby constructing a complete three-dimensional terrain of the surrounding environment through data splicing and synthesis.

[0044] Subsequently, embodiments of this application can use deep learning to train neural network models to perform semantic recognition on the constructed three-dimensional terrain model, so as to assign semantic information to terrain objects such as trees, rocks, and rivers, thereby constructing a semantic terrain with rich information.

[0045] Therefore, the embodiments of this application fuse the data collected by the sensor components through the computing module and perform semantic recognition to obtain three-dimensional information of the surrounding terrain. Thus, without relying on existing maps, an accurate three-dimensional semantic model of the field terrain can be constructed, providing a reliable theoretical basis for the subsequent planning of off-road routes.

[0046] Optionally, in one embodiment of this application, the computing module 300 includes: a semantic map component, a path planning component, and a driving guidance component.

[0047] The semantic map component is used to generate a three-dimensional semantic grid map of the target field terrain based on a three-dimensional semantic model, and to solve the Euclidean distance between each grid and its neighboring grids in the three-dimensional semantic grid map according to a preset Euclidean distance transformation strategy, so as to determine the distance between each grid and the nearest obstacle.

[0048] The path planning component is used to calculate the travel time of a target vehicle using Euclidean distance, a preset speed function, and vehicle geometric passability parameters. Based on the travel time, a travel time map of the target wilderness terrain is constructed. In the travel time map, the target direction with the fastest decrease in travel time gradient between the target point and the starting point is searched in reverse to obtain the optimal off-road route for the target vehicle based on the target direction.

[0049] The driving guidance component is used to calculate the driving parameters of the target vehicle based on the optimal off-road route, and generate driving behavior suggestions that match the optimal off-road route based on the driving parameters.

[0050] It should be noted that the embodiments of this application can determine a three-dimensional semantic grid map by using a semantic map component, based on terrain information obtained after fusing lidar point cloud data and camera image perception data, and by using Euclidean distance transformation, solve the Euclidean distance for each grid cell in the three-dimensional semantic grid map to solve for the adjacent grid cells around each grid cell, thereby gradually updating and determining the distance between each grid cell and the nearest obstacle, and treating terrain and objects with a certain volume as obstacles.

[0051] Furthermore, embodiments of this application can utilize a path planning component to calculate the travel time of off-road vehicles by using a speed function f(d) containing semantic information about the environment (speed map), geometric passability parameters of the off-road vehicle, etc., and construct a travel time map based on the travel time. This comprehensively considers issues such as obstacle avoidance, obstacle crossing ability, terrain traversal difficulty, and danger of off-road vehicles. The mathematical expression of the speed function is as follows:

[0052]

[0053] In the formula, c i c is the time cost factor for terrain i, which reflects the time cost of traversing different terrains. i The larger the value, the longer the time required and the greater the difficulty of traversing the terrain; d is the distance from this location to the nearest obstacle; e is the Euler number; v m This represents the maximum operating speed of off-road vehicles in this environment.

[0054] Secondly, embodiments of this application can reverse-search the direction of the fastest decrease in travel time gradient between the target point and the starting point in the travel time map, thereby obtaining a fast and safe off-road route.

[0055] After obtaining the optimal off-road route, embodiments of this application can also calculate vehicle driving parameters such as the required power, braking force, and adhesion of each tire based on the three-dimensional terrain model, the optimal off-road route, slope information, road conditions, and the vehicle's passability. Based on the vehicle driving parameters, the application can generate the optimal safe passage route suggestion, including information such as steering, throttle, brake, gear, four-wheel drive settings, and chassis settings.

[0056] For example, after obtaining the optimal off-road route, the embodiments of this application can provide vehicle gear shifting suggestions at a position 10 meters from the optimal off-road route, and can provide steering wheel angle suggestions throughout the entire optimal off-road route, thereby controlling the vehicle to drive according to the route suggestions and the optimal off-road route, greatly ensuring the safety and reliability of the vehicle and improving the user's driving experience.

[0057] Optionally, in one embodiment of this application, the vehicle geometric passability parameters include at least one of vehicle kinematic constraints, vehicle dynamic performance, vehicle departure angle, vehicle approach angle, and tire adhesion conditions.

[0058] It should be noted that, in the embodiments of this application, the reliability of the obtained off-road route can be effectively guaranteed by combining vehicle geometric passability parameters, such as vehicle kinematic constraints, vehicle dynamic performance, vehicle departure angle, vehicle approach angle, tire adhesion conditions, etc., with path planning algorithms.

[0059] Optionally, in one embodiment of this application, the off-road route generation system 10 of this application embodiment further includes: a communication module and a display component.

[0060] The communication module is used to transmit lidar point cloud data and camera image perception data to the computing module, and to transmit flight mission instructions, flight trajectory data and / or aerial photography mission data to at least one vehicle-mounted drone.

[0061] The display component is used to convert the 3D semantic model and the optimal off-road route into a 3D model format data, and then display the 3D semantic model and the optimal off-road route based on the preset demonstration device and the 3D model format data.

[0062] In the embodiments of this application, the communication module mainly includes communication module 1 and communication module 2, such as... Figure 2 As shown, its main functions are as follows:

[0063] 1. Communication module 1 can transmit flight mission commands sent from the take-off and landing platform to the vehicle-mounted UAV;

[0064] 2. Communication module 2 can receive off-road route information data generated by the calculation module;

[0065] 3. Communication module 2 can transmit the flight trajectory and aerial photography mission information sent by the computing module to the vehicle-mounted drone;

[0066] 4. Communication module 2 can transmit laser point cloud or image data sent from the transmission sensor component to the computing module in real time.

[0067] Furthermore, after obtaining the optimal off-road route, embodiments of this application can further convert the terrain 3D semantic model and the optimal off-road route into 3D model format data that supports display on the display component, so as to display them through AR, VR, electronic screen display, or holographic projection, etc. Figure 2 As shown, this allows for a more intuitive display of the terrain's 3D semantic model and the optimal off-road route to the user, making the vehicle and the function more user-friendly and intelligent.

[0068] The execution logic of the off-road route generation system of this application will be described below with reference to the accompanying drawings.

[0069] Figure 3 This is a schematic diagram of the execution logic of the off-road route generation system of this application, as shown below. Figure 3 As shown, the execution logic of the off-road route generation system of this application is as follows:

[0070] S301: Uses a vehicle-mounted drone for terrain mapping. The drone is equipped with multiple sensors to collect sensor data.

[0071] S302: The collected data is sent to the computing module via the communication module;

[0072] S303: The computing module integrates lidar point cloud data and camera image perception data, and performs semantic recognition on the integrated data to construct a three-dimensional semantic model of the field terrain;

[0073] S304: Based on a three-dimensional semantic model, the optimal off-road route is generated by using a path planning algorithm that comprehensively considers information such as vehicle kinematic constraints, dynamic performance, and tire adhesion conditions.

[0074] S305: Displays a 3D semantic model of the terrain and the best off-road routes through the display component.

[0075] The off-road route generation system proposed in this application includes a take-off and landing platform for carrying at least one vehicle-mounted drone and sending flight mission commands to the drone; a sensor component for collecting lidar point cloud data and camera image perception data of the target terrain based on the drone; and a computing module for fusing the lidar point cloud data and camera image perception data to generate fused data, performing semantic recognition on the fused data, constructing a three-dimensional semantic model of the target terrain based on the recognition results, generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route. This application uses sensors such as lidar mounted on the drone to acquire three-dimensional information of the surrounding terrain, constructing an accurate three-dimensional terrain model, and planning routes for off-road vehicles based on the constructed terrain model, thus enabling safe and efficient off-road crossing without relying on existing maps.

[0076] Secondly, the method for generating off-road routes according to embodiments of this application is described with reference to the accompanying drawings.

[0077] Figure 4 This is a flowchart of a method for generating off-road routes provided in an embodiment of this application.

[0078] like Figure 4 As shown, the method for generating off-road routes includes the following steps:

[0079] In step S401, based on at least one vehicle-mounted UAV, lidar point cloud data and camera image perception data of the target field terrain are collected.

[0080] In step S402, lidar point cloud data and camera image perception data are fused to generate fused data. Semantic recognition is performed on the fused data, and a three-dimensional semantic model of the target field terrain is constructed based on the recognition results. The optimal off-road route for the target vehicle is generated based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm. Driving suggestions are obtained based on the optimal off-road route.

[0081] Optionally, in one embodiment of this application, generating the optimal off-road route for the target vehicle based on a three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm includes: generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and the preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route, including: generating a three-dimensional semantic grid map of the target wilderness terrain based on the three-dimensional semantic model, and solving the Euclidean distance between each grid and its neighboring grids in the three-dimensional semantic grid map according to a preset Euclidean distance transformation strategy to determine the distance between each grid and the nearest obstacle; calculating the travel time of the target vehicle using the Euclidean distance, a preset speed function, and vehicle geometric passability parameters; constructing a travel time map of the target wilderness terrain based on the travel time; and searching backwards in the travel time map for the target direction with the fastest decrease in travel time gradient between the target point and the starting point to obtain the optimal off-road route for the target vehicle based on the target direction; calculating the driving parameters of the target vehicle based on the optimal off-road route, and generating driving behavior suggestions matching the optimal off-road route based on the driving parameters.

[0082] Optionally, in one embodiment of this application, after generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, the method further includes: converting the three-dimensional semantic model and the optimal off-road route into a format to obtain three-dimensional model format data, and displaying the three-dimensional semantic model and the optimal off-road route based on a preset demonstration device and the three-dimensional model format data.

[0083] It should be noted that the foregoing explanation of the off-road route generation system embodiment also applies to the off-road route generation method of this embodiment, and will not be repeated here.

[0084] The off-road route generation method proposed in this application involves collecting LiDAR point cloud data and camera image perception data of the target wilderness terrain using at least one vehicle-mounted UAV. The LiDAR point cloud data and camera image perception data are then fused to generate fused data. Semantic recognition is performed on the fused data, and a three-dimensional semantic model of the target wilderness terrain is constructed based on the recognition results. Finally, the optimal off-road route for the target vehicle is generated based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm. This application utilizes sensors such as LiDAR mounted on a UAV to acquire three-dimensional information of the surrounding terrain, constructing an accurate three-dimensional terrain model. This model is then used to plan routes for off-road vehicles, enabling safe and efficient off-road traversal without relying on existing maps.

[0085] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0086] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0087] When processor 502 executes the program, it implements the off-road route generation method provided in the above embodiments.

[0088] Furthermore, electronic devices also include:

[0089] Communication interface 503 is used for communication between memory 501 and processor 502.

[0090] The memory 501 is used to store computer programs that can run on the processor 502.

[0091] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0092] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0093] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0094] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0095] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described off-road route generation method.

[0096] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0098] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0099] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0100] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0103] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A cross-country route generation system, characterized in that, include: A take-off and landing platform for carrying at least one vehicle-mounted drone and sending flight mission commands to the at least one vehicle-mounted drone; Sensor components are used to collect lidar point cloud data and camera image perception data of the target field terrain based on the at least one vehicle-mounted UAV. The computing module is used to fuse the lidar point cloud data and the camera image perception data to generate fused data, perform semantic recognition on the fused data, construct a three-dimensional semantic model of the target field terrain based on the recognition results, and generate the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters and preset path planning algorithm, and obtain driving suggestions based on the optimal off-road route. The off-road route generation system also includes: A communication module is used to transmit the lidar point cloud data and the camera image perception data to the computing module, and to transmit the flight mission command, flight trajectory data and / or aerial photography mission data to the at least one vehicle-mounted UAV. The display component is used to convert the format of the three-dimensional semantic model and the optimal off-road route to obtain three-dimensional model format data, and display the three-dimensional semantic model and the optimal off-road route based on the preset demonstration device and the three-dimensional model format data. The computing module includes: A semantic map component is used to generate a three-dimensional semantic grid map of the target field terrain based on the three-dimensional semantic model, and to solve the Euclidean distance between each grid and its neighboring grids in the three-dimensional semantic grid map according to a preset Euclidean distance transformation strategy, so as to determine the distance between each grid and the nearest obstacle. The path planning component is used to calculate the travel time of the target vehicle using the Euclidean distance, the preset speed function and the vehicle's geometric passability parameters. Based on the travel time, a travel time map of the target wilderness terrain is constructed. In the travel time map, the target direction with the fastest decrease in travel time gradient between the target point and the starting point is searched in reverse to obtain the optimal off-road route for the target vehicle based on the target direction. A driving guidance component is used to calculate the driving parameters of the target vehicle based on the optimal off-road route, and generate driving behavior suggestions that match the optimal off-road route based on the driving parameters. The take-off and landing platform includes: a power module for supplying power to the at least one vehicle-mounted drone; The vehicle geometric passability parameters include at least one of the following: vehicle kinematic constraints, vehicle dynamic performance, vehicle departure angle, vehicle approach angle, and tire adhesion conditions.

2. A method for generating off-road routes, characterized in that, Includes the following steps: Based on at least one vehicle-mounted drone, collect lidar point cloud data and camera image perception data of the target's outdoor terrain; The point cloud data from the lidar and the image perception data from the camera are fused to generate fused data. Semantic recognition is performed on the fused data. A three-dimensional semantic model of the target terrain is constructed based on the recognition results. The optimal off-road route for the target vehicle is generated based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm. Driving suggestions are obtained based on the optimal off-road route. The process, after generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, further includes: The three-dimensional semantic model and the optimal off-road route are converted into a format to obtain three-dimensional model format data, and the three-dimensional semantic model and the optimal off-road route are displayed based on a preset demonstration device and the three-dimensional model format data. The process of generating the optimal off-road route for the target vehicle based on the three-dimensional semantic model, vehicle geometric passability parameters, and a preset path planning algorithm, and obtaining driving suggestions based on the optimal off-road route, includes: Based on the three-dimensional semantic model, a three-dimensional semantic grid map of the target field terrain is generated, and according to a preset Euclidean distance transformation strategy, the Euclidean distance between each grid and its neighboring grids in the three-dimensional semantic grid map is solved to determine the distance between each grid and the nearest obstacle. Using the Euclidean distance, the preset speed function, and the vehicle's geometric passability parameters, the travel time of the target vehicle is calculated. Based on the travel time, a travel time map of the target wilderness terrain is constructed. In the travel time map, the target direction with the fastest decrease in travel time gradient between the target point and the starting point is searched in reverse to obtain the optimal off-road route for the target vehicle based on the target direction. Based on the optimal off-road route, the driving parameters of the target vehicle are calculated, and driving behavior suggestions matching the optimal off-road route are generated according to the driving parameters.

3. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the off-road route generation method as described in claim 2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the off-road route generation method as described in claim 2.

Images