Vehicle formation cooperative obstacle avoidance method, device, equipment and storage medium

Abstract

The invention relates to the technical field of automatic driving, discloses a vehicle formation cooperative obstacle avoidance method, device and equipment and a storage medium, and aims to improve the cooperative obstacle avoidance efficiency and safety of formation vehicles to obstacles. The vehicle formation cooperative obstacle avoidance method comprises the following steps: collecting multi-modal environment data through a plurality of sensors carried on each vehicle; inputting the multi-modal environment data into a preset end-to-end deep learning model for obstacle detection, and when an obstacle is detected, outputting a first obstacle avoidance control instruction set to guide the vehicle to run in an obstacle avoidance manner; packaging the first obstacle avoidance control instruction set and the detected obstacle information, and carrying out front vehicle obstacle avoidance data packet sharing through a C-V2X inter-vehicle communication module; and when the following vehicle does not detect an obstacle and receives the obstacle avoidance data packet of the front vehicle, performing cooperative path planning based on the obstacle state packet to obtain a second obstacle avoidance control instruction set so as to guide the following vehicle to travel in an obstacle avoidance manner.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a vehicle platoon cooperative obstacle avoidance method, device, equipment and storage medium. BACKGROUND

[0002] In the application of automatic driving system in complex mine environments such as open-pit mine, underground mine and quarry, one special scenario is that multiple automatic driving trucks perform platoon operation on narrow mine road. In this case, the vehicles need to detect obstacles and avoid obstacles in limited space.

[0003] However, due to the detection blind area of single sensor, and the following vehicle in the vehicle platoon cannot follow the leading vehicle to avoid obstacles in time due to perception delay or information loss, which easily causes risks and seriously restricts the safety and stability of vehicle platoon in complex mine environments. SUMMARY

[0004] The present application provides a vehicle platoon cooperative obstacle avoidance method, device, equipment and storage medium to solve the traditional problems.

[0005] The first aspect of the present application provides a vehicle platoon cooperative obstacle avoidance method, comprising: applied to a vehicle platoon cooperative system, the vehicle platoon cooperative system is used for cooperative control of multiple automatic driving vehicles following driving, the vehicle platoon cooperative obstacle avoidance method comprises: collecting multi-modal environment data through multiple sensors carried on each vehicle; inputting the multi-modal environment data into a preset end-to-end deep learning model for obstacle detection, and outputting a first obstacle avoidance control instruction set when an obstacle is detected, to guide the vehicle to avoid obstacle driving; encapsulating the first obstacle avoidance control instruction set and the detected obstacle information, and sharing the leading vehicle obstacle avoidance data packet through a C-V2X inter-vehicle communication module; when the following vehicle does not detect an obstacle and receives the leading vehicle obstacle avoidance data packet, cooperative path planning is performed based on the obstacle state packet to obtain a second obstacle avoidance control instruction set, to guide the following vehicle to avoid obstacle driving.

[0006] Further, the application also proposes that the end-to-end deep learning model comprises an input layer, a feature extraction layer, and a decision output layer; the multi-modal environment data is input into the preset end-to-end deep learning model for obstacle detection, and a first obstacle avoidance control instruction set is output when an obstacle is detected, comprising: the multi-modal environment data is preprocessed through the input layer to obtain corresponding analysis data of each mode; the corresponding analysis data of each mode is respectively subjected to feature extraction through the feature extraction layer, and an environment feature matrix is obtained through cross-modal feature fusion; the environment feature matrix is input into the decision output layer for obstacle avoidance scene classification, and a control instruction mapping is performed based on a target obstacle avoidance scene label to obtain the first obstacle avoidance control instruction set, wherein the target obstacle avoidance scene label is any one of a cross-row scene label, a detour scene label, and a rolling scene label.

[0007] Further, the application also proposes that the multi-modal environment data at least comprises point cloud data collected by a laser radar, image data collected by a binocular camera, and radar data collected by a millimeter wave radar; the multi-modal environment data is preprocessed through the input layer to obtain corresponding analysis data of each mode, comprising: the point cloud data collected by the laser radar is subjected to coordinate conversion and downsampling processing to obtain bird's eye view point cloud analysis data; the image data collected by the binocular camera is subjected to binocular parallax calculation to obtain binocular parallax analysis data; the radar data collected by the millimeter wave radar is subjected to noise filtering and target clustering to obtain radar point cluster analysis data.

[0008] Further, the application also proposes that the feature extraction layer comprises a visual branch, a point cloud branch, a radar branch, and a feature pyramid network; the corresponding analysis data of each mode is respectively subjected to feature extraction through the feature extraction layer, and an environment feature matrix is obtained through cross-modal feature fusion, comprising: the binocular parallax analysis data is subjected to convolution operation and activation processing through the visual branch to obtain obstacle visual semantic features; the bird's eye view point cloud analysis data is subjected to voxelization and feature encoding through the point cloud branch to obtain obstacle point cloud geometric features; the radar point cluster analysis data is subjected to distance transformation and velocity estimation through the radar branch to obtain obstacle radar distance features; the obstacle visual semantic features, the obstacle point cloud geometric features, and the obstacle radar distance features are input into the feature pyramid network to obtain an environment feature matrix fused with multi-modal information through multi-level feature splicing and dimension normalization processing.

[0009] Further, the application also proposes that, when the following vehicle does not detect obstacles and receives the front vehicle obstacle avoidance data packet, cooperative path planning is performed based on the obstacle state packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid obstacles and travel, including: when the following vehicle does not detect obstacles and receives the front vehicle obstacle avoidance data packet, determining the obstacle height in the front vehicle obstacle avoidance data packet; if the obstacle height is less than a preset height threshold, generating a suspension parameter adjustment instruction and a straight travel instruction to guide the following vehicle to cross low obstacles; if the obstacle height is greater than or equal to the preset height threshold, generating a detour control instruction according to the obstacle information and the first obstacle avoidance control instruction set in combination with the vehicle dynamics model of the following vehicle to guide the following vehicle to detour obstacles.

[0010] Further, the application also proposes that, after cooperative path planning based on the obstacle state packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid obstacles and travel when the following vehicle does not detect obstacles and receives the obstacle state packet, the application further includes: performing conflict detection processing on the obstacle avoidance path requests generated by multiple vehicles at the same time; and performing traffic right allocation according to a preset load priority principle and a near distance priority principle to obtain a cooperative traffic sequence.

[0011] Further, the application also proposes that, the application further includes: performing threshold comparison on the confidence of the output of the end-to-end deep learning model; and when the confidence is less than a preset confidence threshold, switching to a millimeter wave radar point cloud clustering algorithm to detect obstacles and generating a third obstacle avoidance control instruction set through a preset multi-rule constraint algorithm.

[0012] The second aspect of the application provides a vehicle platoon cooperative obstacle avoidance device, including: a collection module configured to collect multi-modal environment data through a plurality of sensors mounted on each vehicle; a first processing module configured to input the multi-modal environment data into a preset end-to-end deep learning model to detect obstacles and output a first obstacle avoidance control instruction set to guide the vehicle to avoid obstacles and travel when obstacles are detected; a communication module configured to encapsulate the first obstacle avoidance control instruction set and the detected obstacle information and share a front vehicle obstacle avoidance data packet through a C-V2X inter-vehicle communication module; a second processing module configured to, when the following vehicle does not detect obstacles and receives the front vehicle obstacle avoidance data packet, perform cooperative path planning based on the obstacle state packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid obstacles and travel.

[0013] The third aspect of the present application provides a vehicle platoon cooperative obstacle avoidance device, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to enable the vehicle platoon cooperative obstacle avoidance device to perform the vehicle platoon cooperative obstacle avoidance method described above.

[0014] The fourth aspect of the present application provides a computer-readable storage medium having instructions stored therein, which, when executed on a computer, causes the computer to perform the vehicle platoon cooperative obstacle avoidance method described above.

[0015] In the technical solutions provided by the present application, by constructing a multi-vehicle cooperative perception and decision-making system, the problem of cooperative obstacle avoidance of platoon vehicles in complex mine scenes is solved. By collecting multi-modal environment data through multiple sensors, the detection blind area of a single sensor is overcome, and the comprehensiveness of obstacle detection is improved; an end-to-end deep learning model is used for obstacle detection to realize real-time decision-making under multi-source data fusion, directly generate a first obstacle avoidance control instruction set, and ensure that the vehicle timely performs accurate obstacle avoidance actions, reducing the delay of the traditional decision-making link; through the C-V2X communication module, the front vehicle obstacle avoidance data packet is packaged and shared, an information synchronization mechanism between the platoon vehicles is established, and the obstacle avoidance lag problem caused by the information asynchronization of the following vehicle is solved; when the following vehicle does not detect an obstacle, cooperative path planning is performed based on the received front vehicle obstacle avoidance data to generate a second obstacle avoidance control instruction set, obstacle avoidance action cooperation between the platoon vehicles is realized, and the obstacle avoidance safety and cooperation of the entire vehicle platoon are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 Fig. 1 is a schematic diagram of the vehicle platoon cooperative obstacle avoidance method in the present application; Figure 2 Fig. 2 is a schematic diagram of the vehicle platoon cooperative obstacle avoidance device in the present application; Figure 3 Fig. 3 is a schematic diagram of the vehicle platoon cooperative obstacle avoidance device in the present application. DETAILED DESCRIPTION

[0017] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0018] Also, the term "comprising" or "having" and its variations is meant to encompass the inclusion of one or more steps or units and that not all of the steps or units are necessarily included in the process, method, system, product or apparatus. Furthermore, the term "comprising" or "having" and its variations is meant to encompass the inclusion of one or more steps or units and that not all of the steps or units are necessarily included in the process, method, system, product or apparatus.

[0019] In the related art, the automatic driving vehicle formation operation in a complex mine environment faces many challenges. The narrow road, uneven distribution of low obstacles and the complexity of multi-vehicle cooperative control lead to detection blind area of traditional single sensor scheme, and the following vehicles in the formation cannot respond to the obstacle avoidance action of the front vehicle in time due to perception delay or information loss, which easily causes collision risk. Especially in the open pit or underground mine scene, environmental interference such as dust and light changes further reduces the reliability of obstacle detection, and the existing system is difficult to realize real-time cooperative obstacle avoidance between formation vehicles.

[0020] To solve the above problems, the specific process of the present application is described below, referring to Figure 1 For the first embodiment of the vehicle formation cooperative obstacle avoidance method of the present application, it is applied to a vehicle formation cooperative system for cooperative control of multiple automatic driving vehicles following driving, and the vehicle formation cooperative obstacle avoidance method comprises the following steps: 101. Collecting multi-modal environment data through multiple sensors mounted on each vehicle; 102. Inputting the multi-modal environment data into a preset end-to-end deep learning model for obstacle detection, and outputting a first obstacle avoidance control instruction set when an obstacle is detected to guide the vehicle to avoid obstacle driving; 103. Packaging the first obstacle avoidance control instruction set and the detected obstacle information, and sharing the front vehicle obstacle data packet through the C-V2X inter-vehicle communication module; 104. When the following vehicle does not detect an obstacle and receives the front vehicle obstacle data packet, cooperative path planning is performed based on the obstacle state packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid obstacle driving.

[0021] The multi-modal environment data refers to complementary environment information obtained through different types of sensors, and can be implemented by combining a laser radar, a binocular camera and a millimeter wave radar. The combination can obtain three-dimensional space information and detect moving targets. The end-to-end deep learning model refers to a neural network architecture directly connecting sensor input and control output, and can be implemented by using a multi-branch feature extraction network and a decision level cascade structure to realize end-to-end mapping from raw data to control instructions. The C-V2X inter-vehicle communication module refers to a special hardware unit supporting a vehicle networking communication protocol, and can be implemented by using a 5G NR-V2X module to ensure a stable inter-vehicle communication link in a complex environment. The cooperative path planning refers to generating an obstacle avoidance trajectory based on shared information and coordinating the overall motion of the formation, and can be implemented by using a model predictive control algorithm combined with vehicle dynamics constraints to ensure the spatiotemporal consistency of multi-vehicle obstacle avoidance actions.

[0022] Specifically, when the lead vehicle of the formation detects an obstacle, the multi-modal sensors carried by the lead vehicle synchronously collect environment data. The laser radar generates three-dimensional point cloud data, the binocular camera obtains stereo vision information, and the millimeter wave radar captures moving target features. After normalization, these heterogeneous data are input into the end-to-end model. The model processes different modal data through parallel feature extraction branches, performs cross-modal fusion at the feature level, and finally outputs the first obstacle avoidance control instruction containing parameters such as steering angle and vehicle speed adjustment. The instruction set and information such as obstacle coordinates and size are packaged into a standardized data packet and broadcast to the following vehicles in the formation through a low-latency C-V2X link. When the following vehicles do not detect obstacles through their own sensors, they analyze the received data packets and call the path planning algorithm to generate matched obstacle avoidance control instructions in combination with the vehicle position and state, thereby achieving cooperative coordination with the lead vehicle.

[0023] Compared with related technologies, the traditional formation control method relies on single-vehicle independent perception and decision-making, which is prone to missed detection due to sensor limitations. The existing cooperative scheme mostly uses a hierarchical architecture, and there is cumulative delay in the data processing link. The present scheme integrates perception and decision-making through an end-to-end model, shortening the key path processing time. The multi-modal data fusion enhances the robustness of obstacle detection, the C-V2X communication ensures the timeliness of formation information synchronization, and the model predictive control algorithm ensures the coordination of multi-vehicle actions.

[0024] Through the above technical solutions, the problem of cooperative obstacle avoidance lag caused by the perception blind area of the formation vehicles is effectively solved. Multi-modal data fusion improves the detection accuracy of low obstacles, end-to-end models shorten the decision response time, inter-vehicle communication realizes real-time sharing of obstacle avoidance information, and cooperative planning ensures the consistency of the overall obstacle avoidance actions of the formation. In the mine field test, this method shortens the average braking distance of the formation vehicles when encountering sudden obstacles, reduces the inter-vehicle spacing control error, and improves the overall operation efficiency.

[0025] The application further provides a vehicle platoon cooperative obstacle avoidance method, comprising the following steps: preprocessing multi-modal environment data through an input layer to obtain corresponding to-be-analyzed data of each mode; performing feature extraction on the to-be-analyzed data corresponding to each mode through a feature extraction layer, and obtaining an environment feature matrix through cross-modal feature fusion; inputting the environment feature matrix into a decision output layer for obstacle avoidance scene classification, and performing control instruction mapping based on a target obstacle avoidance scene label to obtain a first obstacle avoidance control instruction set, the target obstacle avoidance scene label being any one of a cross-row scene label, a detour scene label and a rolling scene label.

[0026] The input layer is a processing module for format conversion and standardization of multi-source heterogeneous sensor data, and specifically can use a coordinate transformation algorithm to project laser radar point cloud data into an aerial view, use a parallax calculation algorithm to process binocular image data, and use a clustering algorithm to process millimeter wave radar data, so as to realize unified representation of different modal data. The feature extraction layer is a structure for extracting features of each sensor data through a sub-modal neural network and performing fusion, and specifically can use a convolutional neural network to process visual data, use a three-dimensional convolutional network to process point cloud data, and use a fully connected network to process radar data, so as to realize multi-scale feature fusion through a feature pyramid network. The decision output layer is a classifier for mapping fused features into control instructions, and specifically can use a fully connected layer and a softmax function to realize obstacle avoidance scene classification, and generate vehicle steering, braking or suspension adjustment instructions through a preset instruction mapping table.

[0027] Specifically, the original point cloud collected by the laser radar is converted into a two-dimensional grid map of a bird's-eye view, the binocular camera image is processed through parallax calculation to generate a depth information map, and the millimeter wave radar data is clustered to form a target point cluster. The visual branch extracts obstacle edge and texture features in the image through a convolutional neural network, the point cloud branch extracts obstacle geometric shape features through a three-dimensional convolutional network, and the radar branch extracts target distance and speed features through a fully connected network. The feature pyramid network up-samples and splices feature maps of different resolutions to generate an environment feature matrix containing multi-scale spatial information. The decision output layer matches a cross-row, detour or rolling scene label according to the height, shape and motion state of the obstacle in the feature matrix, and calls a corresponding control instruction set to realize vehicle action control.

[0028] Compared with the related art, the traditional method uses a single sensor for obstacle detection, which is prone to missed detection due to the sensor blind area, and multiple coordinate alignment and time synchronization are required when fusing multi-sensor data, resulting in processing delay. In the related art, a rule-based obstacle avoidance decision logic is used, which is difficult to adapt to complex and variable obstacle shapes. The present scheme realizes parallel processing and automatic feature fusion of multi-modal data through an end-to-end deep learning model, eliminates the efficiency bottleneck caused by manually designed fusion rules, automatically learns the feature expression of different obstacle avoidance scenarios using a neural network, and directly outputs a control instruction set, reducing the intermediate processing link.

[0029] Through the above technical solution, the present application effectively solves the problem of obstacle avoidance decision delay caused by low efficiency of multi-sensor data fusion, improves the accuracy of obstacle detection through cross-modal feature fusion, and avoids the risk of missed detection caused by single sensor blind area. The end-to-end model is used to realize the direct mapping from raw data to control instructions, shorten the data processing link, and ensure the real-time obstacle avoidance ability of vehicle platoon in complex mine environment. The instruction mapping mechanism based on scene classification can automatically select the optimal obstacle avoidance strategy for different obstacle types, improving the ability of vehicles to respond to special scenarios such as low obstacles and moving obstacles.

[0030] The present application further proposes that the multi-modal environment data at least includes point cloud data collected by a laser radar, image data collected by a binocular camera, and radar data collected by a millimeter wave radar; the multi-modal environment data is preprocessed through an input layer to obtain corresponding analysis data of each mode, including: coordinate conversion and downsampling processing of the point cloud data collected by the laser radar to obtain bird's eye view point cloud analysis data; binocular disparity calculation of the image data collected by the binocular camera to obtain binocular disparity analysis data; noise filtering and target clustering of the radar data collected by the millimeter wave radar to obtain radar point cluster analysis data.

[0031] The coordinate conversion and downsampling processing refers to converting the three-dimensional point cloud data collected by the laser radar from the polar coordinate system to the bird's eye projection in the vehicle coordinate system, and specifically can use the voxel grid method for downsampling to retain the obstacle contour features while reducing data redundancy. Binocular disparity calculation refers to using the images collected by the left and right cameras to perform stereo matching, and specifically can generate a disparity map through a semi-global matching algorithm to obtain the depth information of the obstacle. Noise filtering and target clustering refer to dynamic threshold filtering of the original reflection signal of the millimeter wave radar, and specifically can use a density-based clustering algorithm to separate effective obstacle point clusters and eliminate environmental noise interference.

[0032] Specifically, the laser radar point cloud forms a two-dimensional structured data of a bird's eye view after coordinate conversion, which can intuitively reflect the planar geometric distribution of the obstacle; the binocular disparity calculation extracts the spatial position information of the obstacle through the principle of stereo vision, making up for the defect that the monocular camera cannot directly obtain the depth; after noise filtering of the millimeter wave radar data, the effective reflection point cluster forms a discrete target set through clustering, providing a basis for subsequent motion trajectory prediction. After targeted preprocessing of different modal data, standardized and high signal-to-noise ratio data to be analyzed are formed, providing a consistent input format for cross-modal feature fusion.

[0033] Compared with related technologies, the existing method usually uses single sensor data or unoptimized multi-modal fusion mode, such as directly superimposing original point cloud and image pixels, resulting in mismatch of data dimensions or serious noise interference. The present scheme designs differentiated preprocessing procedures for the characteristics of each sensor, such as retaining key geometric features through downsampling, enhancing depth perception through disparity calculation, and eliminating radar clutter through clustering, so that multi-modal data are effectively aligned and complementary at the feature level.

[0034] Through the above technical solutions, the present application can eliminate redundant information and noise interference in the original multi-modal data, and improve the effectiveness of cross-modal feature fusion. The standardized preprocessed data retains the advantage features of each sensor while reducing the computational complexity of subsequent processing, so that the obstacle detection model can more accurately identify the shape and position information of low obstacles, thereby providing a reliable environmental perception basis for vehicle formation cooperative obstacle avoidance.

[0035] The present application further proposes that the feature extraction layer includes a visual branch, a point cloud branch, a radar branch, and a feature pyramid network; the binocular disparity data to be analyzed is subjected to convolution operation and activation processing through the visual branch to obtain obstacle visual semantic features; the bird's eye view point cloud data to be analyzed is voxelized and feature encoded through the point cloud branch to obtain obstacle point cloud geometric features; the radar point cluster data to be analyzed is subjected to distance transformation and velocity estimation through the radar branch to obtain obstacle radar distance features; the obstacle visual semantic features, the obstacle point cloud geometric features, and the obstacle radar distance features are input into the feature pyramid network, and through multi-level feature splicing and dimension normalization processing, an environmental feature matrix fused with multi-modal information is obtained.

[0036] The visual branch refers to a neural network branch for processing binocular disparity data, which can be specifically implemented by combining multiple convolution layers and activation functions, and extracts obstacle surface texture and shape features through convolution operation. The point cloud branch refers to a neural network branch for processing bird's eye view point cloud data, which can be specifically implemented by voxelizing grid division and three-dimensional convolution layer, and retains the three-dimensional geometric structure of the obstacle through voxelization processing. The radar branch refers to a neural network branch for processing radar point cluster data, which can be specifically implemented by distance transformation algorithm and velocity estimation algorithm, and extracts spatial position features of the obstacle through distance transformation. The feature pyramid network refers to a neural network structure for multi-scale feature fusion, which can be specifically implemented by cross-level feature splicing and normalization layer, and realizes dimension alignment and spatial correlation of different modal data through multi-level feature fusion.

[0037] Specifically, the visual branch performs convolution operation on binocular disparity data, extracts obstacle edge and surface texture information using a convolution kernel, and enhances non-linear feature expression capability through an activation function to generate visual features containing obstacle shape semantics. The point cloud branch divides the point cloud data at the bird's eye view angle into a voxel grid, and extracts the spatial distribution features of the point cloud layer by layer through a three-dimensional convolution layer to retain the geometric properties such as height and volume of the obstacle. The radar branch performs distance transformation on the clustered radar point cluster, calculates the distance gradient of each point cluster to the vehicle, and estimates the motion velocity of the obstacle in combination with the Doppler effect to generate radar features containing position and dynamic information. The feature pyramid network receives the feature maps output by the three branches, performs cross-modal splicing at different resolution levels, eliminates the dimensional differences of different modal data through a normalization layer, and finally outputs an environmental feature matrix that fuses visual semantics, geometric structure and dynamic information.

[0038] Compared with related technologies, the traditional multi-modal fusion method usually adopts single-scale feature splicing or simple weighted fusion, resulting in the loss of spatial correlation of geometric features and dynamic features. In related technologies, visual and radar data are often processed independently, lacking a cross-modal feature alignment mechanism, and it is difficult to accurately identify the three-dimensional attributes of low obstacles. The present scheme designs a special processing path for different sensor data characteristics through a multi-modal feature extraction architecture, realizes cross-modal information complementation through a multi-level feature pyramid, and effectively solves the problem of spatial mismatch of multi-source heterogeneous data features.

[0039] Through the above technical solutions, the present application can simultaneously capture the surface texture, three-dimensional geometric structure and motion state information of the obstacle, avoiding the misjudgment problem caused by the missed detection of a single sensor. The multi-level feature fusion mechanism enhances the recognition accuracy of the obstacle contour, especially in the low obstacle detection scene, and can accurately distinguish the ground undulation from the real obstacle, providing multi-dimensional feature data containing spatial position, shape size and motion trend for subsequent obstacle avoidance decision-making.

[0040] The application further proposes that when the following vehicle does not detect obstacles and receives the front vehicle obstacle avoidance data packet, the height of the obstacle in the front vehicle obstacle avoidance data packet is determined; if the height of the obstacle is less than a preset height threshold, a suspension parameter adjustment instruction and a straight line instruction are generated to guide the low obstacle to cross the line; if the height of the obstacle is greater than or equal to the preset height threshold, a detour control instruction is generated according to the obstacle information and the first obstacle avoidance control instruction set in combination with the vehicle dynamics model to guide the obstacle to detour.

[0041] Among them, the front vehicle obstacle avoidance data packet refers to a data set containing obstacle height, position and front vehicle obstacle avoidance control instruction transmitted through the C-V2X communication module, which can be structured and packaged in ASN.1 encoding format to realize obstacle information sharing between platoon vehicles. Among them, the preset height threshold refers to the obstacle judgment threshold value set according to the vehicle chassis ground clearance and suspension stroke, which can be realized by a dynamic adjustment algorithm, for example, the threshold range is calculated in real time based on the current vehicle load state, which is used to distinguish the obstacle avoidance mode of low obstacle and high obstacle. Among them, the suspension parameter adjustment instruction refers to the instruction for controlling the height adjustment of air suspension or hydraulic suspension, which can be realized by sending target suspension stiffness and stroke parameters through CAN bus to improve the chassis ground clearance when the vehicle passes through low obstacles. Among them, the vehicle dynamics model refers to the state space equation containing tire slip rate, yaw rate and mass center side slip angle, which can be realized by a two-degree-of-freedom single-track model to constrain the matching relationship between path curvature and vehicle speed in the detour control.

[0042] Specifically, when the following vehicle in the platoon does not detect obstacles due to the sensing blind area, the height information of the obstacle is obtained by analyzing the obstacle avoidance data packet sent by the front vehicle. If the height of the obstacle is lower than the preset threshold, it indicates that the obstacle can be directly crossed by lifting the chassis, at this time, the suspension lifting instruction is generated and the straight line trajectory is maintained to avoid the fluctuation of the platoon distance caused by the path deviation. If the height of the obstacle exceeds the threshold, it indicates that the obstacle needs to be detoured, at this time, the maximum safe steering angle and vehicle speed are calculated in combination with the front vehicle obstacle avoidance trajectory and the dynamics model to generate a detour path that meets the kinematic constraints of the vehicle to prevent the risk of side slip caused by excessive steering.

[0043] Compared with related technologies, the traditional platoon obstacle avoidance method usually only takes the front vehicle path as the following reference, without considering the influence of obstacle height difference on the obstacle avoidance strategy, resulting in unnecessary path deviation in low obstacle scenarios. While the present scheme maintains the straight line trajectory in low obstacle scenarios and adjusts the suspension parameters to reduce the amount of path planning calculation; in high obstacle scenarios, the detour path is generated in combination with the dynamics model to avoid control instability caused by ignoring the vehicle motion constraints.

[0044] By the technical solution, the application solves the problem that the platoon vehicle cannot respond to the obstacle avoidance information of the preceding vehicle in time due to sensing delay, and realizes hierarchical obstacle avoidance control adaptive to obstacle height. Through cooperative control of suspension adjustment and straight-line instruction, the platoon driving stability is maintained in the low obstacle scenario; through the dynamic model constrained detour path planning, the vehicle control safety in the high obstacle scenario is ensured, and the cooperative obstacle avoidance efficiency and safety of the vehicle platoon in the complex mine environment are effectively improved.

[0045] The application further proposes that after the following vehicle does not detect the obstacle and receives the obstacle state package, the second obstacle avoidance control instruction set is obtained based on the obstacle state package for cooperative path planning to guide the following vehicle to avoid obstacles, and then the conflict detection processing is performed on the obstacle avoidance path requests generated by multiple vehicles at the same time, and the passing right is allocated according to the preset load priority principle and the near distance priority principle to obtain a cooperative passing sequence.

[0046] The conflict detection processing refers to identifying the conflict points that may collide or cross paths by comparing the obstacle avoidance path parameters planned by each vehicle in real time, which can be realized by path space overlap analysis and time window comparison algorithm, and is used to find the potential conflict risk when multiple vehicles cooperatively avoid obstacles. The load priority principle refers to dynamically allocating the passing priority according to the actual load value of the vehicle, which can be realized by acquiring real-time load data through the vehicle-mounted mass sensor and establishing a priority sorting rule, and this principle can ensure the safe passing of heavy vehicles with large inertia. The near distance priority principle refers to dynamically adjusting the passing order according to the relative distance between the vehicle and the obstacle, which can be realized by using the laser radar ranging data combined with the path planning distance parameter calculation, and this principle ensures that the vehicle close to the obstacle obtains an earlier obstacle avoidance opportunity. The cooperative passing sequence refers to the vehicle passing order list formed after the conflict detection and priority sorting, which can be generated by a distributed decision algorithm and broadcast to each vehicle in the platoon for execution, and is used to guide the orderly passing of multiple vehicles through the obstacle area.

[0047] Specifically, after generating the second obstacle avoidance control instruction set, the system collects in real time the obstacle avoidance path request data submitted by each vehicle in the platoon. Through the path space overlap analysis module, the three-dimensional space projection comparison of the path coordinate points planned by each vehicle is performed to identify the conflict area with trajectory intersection or insufficient minimum safety distance. The time window comparison module further calculates the time interval at which each vehicle is expected to arrive at the conflict area, and marks it as a time conflict when the time overlap exceeds the preset threshold. For the detected conflict request, the decision system first extracts the real-time load data of each vehicle, divides the priority level according to the preset load threshold interval, and the vehicle with larger load automatically obtains higher right of way. At the same time, the Euclidean distance between the current position of each vehicle and the boundary of the obstacle is calculated, and the vehicles in the same load level are sorted according to the distance from near to far. The finally generated cooperative passing sequence is broadcast to all vehicles in the platoon through the Internet of Vehicles, and each vehicle adjusts the execution time sequence of its own obstacle avoidance action according to the sequence order.

[0048] Compared with related technologies, the traditional multi-vehicle cooperative obstacle avoidance method usually adopts fixed priority rules or manual scheduling methods, which is difficult to dynamically adapt to changes in vehicle state under complex working conditions. The scheme fuses the dual decision dimensions of load and distance to build a dynamic priority evaluation system, which optimizes the overall passing efficiency of the platoon while ensuring the safe passing of heavy vehicles. The problem of decision-making blind area caused by single sensor path planning in related technologies is effectively solved through the conflict detection mechanism of multi-source data fusion.

[0049] Through the above technical solutions, the application effectively solves the path conflict problem caused by simultaneous obstacle avoidance when multiple vehicles are driving in a platoon, and realizes intelligent allocation of the right of way between vehicles. Through the application of dynamic priority rules, both the safety hazards caused by emergency braking of heavy vehicles are avoided and the overall obstacle avoidance time of the platoon is shortened. The generation mechanism of the cooperative passing sequence significantly improves the orderliness and reliability of multi-vehicle cooperative operation, and is particularly suitable for the automatic driving platoon control requirements in space-limited scenes such as mine tunnels.

[0050] The application further proposes to compare the confidence of the end-to-end deep learning model output with a preset confidence threshold, and when the confidence is less than the preset confidence threshold, switch to the millimeter wave radar point cloud clustering algorithm for obstacle detection, and generate a third obstacle avoidance control instruction set through a preset multi-rule constraint algorithm.

[0051] The confidence threshold comparison refers to quantitative evaluation of the reliability of the detection result output by the deep learning model, which can be realized by, for example, a probability value interval division method. For example, the confidence threshold is set to 0.85, and when the model output probability is lower than the value, the redundant detection mechanism is triggered. The millimeter wave radar point cloud clustering algorithm refers to an obstacle recognition method based on the spatial distribution characteristics of radar reflection points. Specifically, the DBSCAN density clustering algorithm can be used to extract the obstacle contour by calculating the spatial density distribution of the radar point cloud. The multi-rule constraint algorithm refers to a comprehensive decision-making method that integrates vehicle dynamics parameters, path smoothness indicators, and safety distance thresholds. Specifically, a weighted multi-objective optimization algorithm can be used to take the maximum steering angle of the vehicle, acceleration limit, and path curvature constraint as the optimization boundary condition.

[0052] Specifically, when the obstacle detection confidence output by the end-to-end deep learning model is lower than the preset threshold, it indicates that the current environment has abnormal illumination, sensor noise interference, or obstacle shape beyond the model training range. At this time, the independent detection process based on the millimeter wave radar point cloud clustering algorithm is switched to. After the radar point cloud data is divided into an obstacle area by density clustering, the multi-rule constraint algorithm is used to generate an obstacle avoidance control instruction. For example, the maximum executable steering angle is calculated under the constraint of vehicle dynamics, while ensuring that the path curvature change does not exceed the set threshold, and maintaining a safe distance from the vehicle in front. Therefore, when the deep learning model fails, feasible obstacle avoidance instructions can still be generated through the redundant detection mechanism and physical rule constraints.

[0053] Compared with related technologies, the existing scheme usually only relies on a single deep learning model for obstacle detection and lacks effective redundant detection mechanisms when the model output confidence is insufficient, resulting in an increased risk of obstacle detection failure or false detection. The present scheme switches the detection mode by triggering the confidence threshold, combines millimeter wave radar point cloud clustering and multi-rule constraint algorithms, and constructs a multi-level detection and decision system, solving the reliability problem of a single model in complex working conditions.

[0054] Through the above technical solutions, the present application can automatically switch to a feature detection method based on physical signals when the detection result of the deep learning model is unreliable, effectively avoiding the risk of collision of the platoon vehicle caused by model misjudgment. At the same time, by integrating vehicle dynamics constraints and path optimization rules, the generated obstacle avoidance control instructions can meet the actual vehicle control ability, improving the obstacle handling capability of the automatic driving platoon system in complex working conditions.

[0055] Reference Figure 2 The present application further proposes a vehicle platoon cooperative obstacle avoidance device, which comprises a collection module 201, a first processing module 202, a communication module 203, and a second processing module 204.

[0056] The collection module 201 is a hardware combination for collecting multi-modal environment data through multiple automatic driving vehicles equipped with laser radars, binocular cameras and millimeter wave radars. The multi-sensor synchronous triggering and data alignment technology can be used to eliminate the detection blind area of a single sensor and provide complete environment perception input for subsequent processing.

[0057] The first processing module 202 is an embedded computing unit for running an end-to-end deep learning model. The GPU-accelerated neural network inference framework can be used to directly map multi-modal environment data to obstacle avoidance control instructions, avoiding the calculation delay in the traditional step-by-step perception decision-making process. The communication module is a vehicle-to-vehicle communication unit based on cellular vehicle networking technology. The C-V2X protocol stack supporting low-latency and high-reliability transmission can be used to encapsulate the front vehicle obstacle avoidance data packet into a standardized message format, realizing real-time information sharing between formation vehicles.

[0058] The second processing module 204 is a controller with a cooperative path planning function. The trajectory optimization algorithm based on the vehicle dynamics model can be used to analyze the obstacle information shared by the front vehicle and generate obstacle avoidance instructions matching the formation motion state, ensuring that the following vehicle still has cooperative obstacle avoidance ability when the perception is limited.

[0059] Specifically, the collection module 201 synchronously collects point cloud, image and radar data through multiple sensors, eliminating the detection blind area caused by shielding or environmental interference of a single sensor. The first processing module 202 inputs multi-modal data into an end-to-end deep learning model, directly outputs an obstacle avoidance control instruction set containing steering, braking and suspension adjustment through cross-modal feature fusion, and shortens the response time from perception to decision-making. The communication module 203 encapsulates the control instructions and obstacle height and position information into a data packet and broadcasts it to other vehicles in the formation through C-V2X, ensuring that the following vehicle can still obtain the obstacle avoidance information of the front vehicle when its own sensor does not detect the obstacle. The second processing module 204 analyzes the obstacle state according to the received data packet and dynamically generates an obstacle avoidance path combined with the motion parameters of the vehicle, such as triggering a suspension lifting instruction when the obstacle height is below a threshold, or generating a detour trajectory when the obstacle cannot be crossed, realizing the synchronization of cooperative obstacle avoidance logic between formation vehicles.

[0060] Compared with related technologies, the traditional scheme relies on single sensor detection and lacks real-time information sharing mechanism between formations, resulting in that the following vehicle cannot respond to obstacles in time when the perception is limited. The present scheme constructs a cross-vehicle environment perception sharing network through multi-sensor fusion and vehicle-to-vehicle communication technology, directly generates control instructions combined with an end-to-end model, and eliminates the information asymmetry problem in formation cooperation. The decision-making delay caused by the step-by-step processing flow in related technologies is overcome by the parallel computing of the end-to-end model and the low-latency transmission of C-V2X.

[0061] Through the technical solution, the following vehicle can still generate an obstacle avoidance instruction adapting to the motion state of the vehicle based on the obstacle avoidance data packet shared by the front vehicle when the obstacle is not detected by the sensor of the following vehicle, so as to avoid the collision risk caused by the sensing blind area. The multi-sensor fusion mechanism improves the detection reliability of low obstacles, the end-to-end model shortens the processing delay from perception to control, and the C-V2X communication ensures the real-time synchronization of obstacle avoidance information between the platoon vehicles, thereby realizing the cooperative obstacle avoidance function of the vehicle platoon in the complex mine environment.

[0062] The above Figure 2 The vehicle platoon cooperative obstacle avoidance device in the present application is described in detail from the perspective of modular functional entities, and the vehicle platoon cooperative obstacle avoidance device in the present application is described in detail from the perspective of hardware processing.

[0063] Referring to Figure 3 The vehicle platoon cooperative obstacle avoidance device includes a processor 300 and a memory 301, and the memory 301 stores machine executable instructions that can be executed by the processor 300, and the processor 300 executes the machine executable instructions to realize the above-mentioned vehicle platoon cooperative obstacle avoidance method.

[0064] Further, Figure 3 The vehicle platoon cooperative obstacle avoidance device shown in the figure further includes a bus 302 and a communication interface 303, and the processor 300, the communication interface 303 and the memory 301 are connected through the bus 302.

[0065] The memory 301 can include a high-speed random access memory (RAM), and can also include a non-volatile memory, for example, at least one disk memory. The communication between the system network element and at least one other network element is realized through at least one communication interface 303 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used. The bus 302 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 3 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0066] The processor 300 can be an integrated circuit chip with a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 300 or the instruction in the form of software. The processor 300 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block disclosed in the embodiments of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 301, and the processor 300 reads the information in the memory 301 and combines the hardware to complete the method steps of the foregoing embodiments.

[0067] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium, and can also be a volatile computer readable storage medium, and the computer readable storage medium stores instructions, when the instructions run on the computer, the computer executes the steps of the vehicle formation cooperative obstacle avoidance method.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0069] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0070] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle formation collaborative obstacle avoidance method, characterized in that: Applied to a vehicle platooning coordination system, the vehicle platooning coordination system is used for the coordinated control of multiple autonomous driving vehicles following each other. The vehicle platooning coordination obstacle avoidance method includes: Collect multimodal environmental data through multiple sensors on each vehicle; Inputting the multimodal environmental data into a preset end-to-end deep learning model to perform obstacle detection, and outputting a first obstacle avoidance control instruction set when an obstacle is detected to guide the vehicle to avoid the obstacle; Encapsulating the first obstacle avoidance control instruction set and the detected obstacle information, and sharing the obstacle avoidance data packet with the preceding vehicle through the C-V2X inter-vehicle communication module; When the following vehicle does not detect an obstacle and receives the leading vehicle obstacle avoidance data packet, collaborative path planning is performed based on the obstacle status packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid the obstacle.

2. The vehicle formation cooperative obstacle avoidance method according to claim 1, characterized in that: The end-to-end deep learning model includes an input layer, a feature extraction layer, and a decision output layer; Inputting the multimodal environment data into a preset end-to-end deep learning model to perform obstacle detection, and outputting a first obstacle avoidance control instruction set when an obstacle is detected, includes: Preprocessing the multimodal environment data through the input layer to obtain data to be analyzed corresponding to each modality; The feature extraction layer extracts features from the data to be analyzed corresponding to each modality, and obtains an environmental feature matrix through cross-modal feature fusion; The environmental feature matrix is ​​input into the decision output layer for obstacle avoidance scenario classification, and control instruction mapping is performed based on the target obstacle avoidance scenario label to obtain a first obstacle avoidance control instruction set, wherein the target obstacle avoidance scenario label is any one of a cross-row scene label, a bypass scene label, and a crushing scene label.

3. The vehicle formation collaborative obstacle avoidance method according to claim 2, characterized in that: The multimodal environmental data includes at least point cloud data collected by a laser radar, image data collected by a binocular camera, and radar data collected by a millimeter-wave radar; The preprocessing of the multimodal environment data through the input layer to obtain the data to be analyzed corresponding to each modality includes: Performing coordinate conversion and downsampling processing on the point cloud data collected by the laser radar to obtain bird's-eye view point cloud data to be analyzed; Performing binocular disparity calculation on the image data collected by the binocular camera to obtain binocular disparity data to be analyzed; Noise filtering and target clustering are performed on the radar data collected by the millimeter-wave radar to obtain radar point cluster data to be analyzed.

4. The vehicle formation cooperative obstacle avoidance method according to claim 3 is characterized in that: The feature extraction layer includes a visual branch, a point cloud branch, a radar branch and a feature pyramid network; The feature extraction layer extracts features from the data to be analyzed corresponding to each modality, and obtains an environmental feature matrix through cross-modal feature fusion, including: Performing convolution operation and activation processing on the binocular disparity data to be analyzed through the visual branch to obtain visual semantic features containing obstacles; voxelize and feature encode the bird's-eye view point cloud data to be analyzed by the point cloud branch to obtain geometric features of the obstacle point cloud; Performing distance transformation and speed estimation on the radar point cluster data to be analyzed by the radar branch to obtain radar distance features containing obstacles; The obstacle visual semantic features, the obstacle point cloud geometric features and the obstacle radar distance features are input into a feature pyramid network, and an environmental feature matrix integrating multimodal information is obtained through multi-level feature splicing and dimensional normalization processing.

5. The vehicle formation collaborative obstacle avoidance method according to claim 1, characterized in that: When the following vehicle does not detect an obstacle and receives the obstacle avoidance data packet of the leading vehicle, performing collaborative path planning based on the obstacle status packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid the obstacle, including: When the following vehicle does not detect an obstacle and receives the obstacle avoidance data packet of the leading vehicle, determining the obstacle height in the obstacle avoidance data packet of the leading vehicle; If the height of the obstacle is less than a preset height threshold, a suspension parameter adjustment instruction and a straight-ahead instruction are generated to guide the following vehicle to cross the low obstacle; If the obstacle height is greater than or equal to a preset height threshold, a detour control instruction is generated based on the obstacle information and the first obstacle avoidance control instruction set, combined with the vehicle dynamics model of the following vehicle, to guide the following vehicle to detour the obstacle.

6. The vehicle formation collaborative obstacle avoidance method according to claim 1, characterized in that: After performing collaborative path planning based on the obstacle status packet to obtain a second obstacle avoidance control instruction set to guide the following vehicle to avoid the obstacle when the following vehicle does not detect the obstacle and receives the obstacle status packet, the method further includes: Perform conflict detection and processing on obstacle avoidance path requests generated by multiple vehicles simultaneously; The right of way is allocated according to the preset load priority principle and close distance priority principle to obtain a coordinated passage sequence.

7. The vehicle formation cooperative obstacle avoidance method according to claim 1, characterized in that: Also includes: Performing a threshold comparison on the confidence level of the end-to-end deep learning model output; When the confidence level is less than a preset confidence threshold, the system switches to the millimeter-wave radar point cloud clustering algorithm for obstacle detection, and generates a third obstacle avoidance control instruction set through a preset multi-rule constraint algorithm.

8. A vehicle formation cooperative obstacle avoidance device, characterized in that: The vehicle formation collaborative obstacle avoidance device comprises: An acquisition module, used to collect multimodal environmental data through multiple sensors carried by each vehicle; a first processing module, configured to input the multimodal environment data into a preset end-to-end deep learning model to perform obstacle detection, and output a first obstacle avoidance control instruction set when an obstacle is detected to guide the vehicle to avoid the obstacle; a communication module, configured to encapsulate the first obstacle avoidance control instruction set and the detected obstacle information, and share the obstacle avoidance data packet with the preceding vehicle via the C-V2X inter-vehicle communication module; The second processing module performs collaborative path planning based on the obstacle status packet when the following vehicle does not detect an obstacle and receives the leading vehicle obstacle avoidance data packet, and obtains a second obstacle avoidance control instruction set to guide the following vehicle to avoid obstacles.

9. A vehicle formation cooperative obstacle avoidance device, characterized in that: The vehicle formation cooperative obstacle avoidance device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the vehicle formation cooperative obstacle avoidance device to execute the vehicle formation cooperative obstacle avoidance method according to any one of claims 1 to 6.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is read and executed, the vehicle formation collaborative obstacle avoidance method as described in any one of claims 1 to 6 is executed.

Images