Vehicle-road collaborative sensing and parking method and system for mine draw shaft environment

Abstract

The invention discloses a vehicle-road collaborative sensing and parking method and system for a shaft mine draw shaft environment, and relates to the field of unmanned driving, and the method comprises the steps: collecting roadside laser radar data, roadside camera data, roadside laser range finder data, vehicle-mounted laser radar data and vehicle-mounted camera data; respectively calculating roadside obstacle information and vehicle-mounted obstacle information; vehicle-road obstacles are matched through a Hungary algorithm, and a vehicle-road collaborative perception result is generated; performing global path planning by calculating an optimal driving path from the starting point to the unloading point by using the global map to generate a global path; performing local path planning through a path obstacle avoidance algorithm to generate a local path; and according to the global path and the local path, controlling the driving path, the vehicle speed and the parking of the unmanned vehicle. In order to solve the problems that in the prior art, an unmanned vehicle cannot comprehensively sense obstacles and cannot stop in place under the mine shaft environment, comprehensive sensing of the obstacles in the mine shaft environment and accurate parking of the vehicle are achieved.

Description

Technical Field

[0001] The present application relates to the field of unmanned driving, and in particular to a vehicle-road collaborative perception and parking method and system in an underground mine chute environment. Background Art

[0002] Underground mines are a type of mine that mines mineral resources underground. They are characterized by complex working environments, high mining difficulty, and high technical requirements. The main types include coal mines, metal mines, etc. The working environment is usually harsh, with dangerous factors such as high temperature, high humidity, dust, and harmful gases. With the advancement of science and technology, intelligent mining has become the development trend of underground mining technology. As an important part of intelligent mining, underground unmanned driving can effectively solve problems such as shortage of underground manpower, low work efficiency, and poor safety, and reduce the accident rate.

[0003] In the process of intelligent transformation of unmanned driving in underground mines, completing automatic loading and unloading of materials is a relatively important link, which is related to whether the unmanned driving scheduling task can be completed. The precise docking at the unloading point is the basis for realizing automatic unloading of materials. At present, the precise docking of vehicles is mainly achieved in the following ways: Precise docking based on positioning technology mainly relies on the global positioning system (GPS), Beidou system or other indoor positioning systems (such as SLAM, UWB, Wi-Fi positioning, etc.). These systems determine the precise location of the vehicle by receiving satellite or base station signals. Through the pre-set docking point coordinates, the system can calculate the optimal path to reach the point and adjust the speed when approaching the target to achieve precise docking. Laser radar (LiDAR) obtains three-dimensional information of the surrounding environment by emitting laser beams and measuring their reflection time. In precise docking, LiDAR can generate high-precision environmental maps and update the position of the vehicle in real time. By matching real-time scanning data with pre-stored docking point information, the system can accurately control the docking process.

[0004] However, the existing technologies still have the following problems: the above methods mainly rely on vehicle-mounted sensors for environmental perception, lack the use of roadside environmental information, resulting in incomplete obstacle perception, especially limited perception of dynamic obstacles. In terms of path planning, the existing methods do not take static obstacles into consideration sufficiently, and fail to make full use of the vehicle-road collaborative perception results, resulting in the planned path being not accurate enough and unable to adapt to the complex and changing underground environment. Summary of the invention

[0005] In response to the problem in the prior art that unmanned vehicles cannot fully perceive obstacles in an underground mine chute environment, the present application provides a vehicle-road collaborative perception and docking method in an underground mine chute environment. Through vehicle-road collaborative perception, it integrates roadside perception information and vehicle-mounted perception information to achieve comprehensive perception of obstacles in an underground mine chute environment and precise parking of vehicles.

[0006] The purpose of this application is achieved through the following technical solutions.

[0007] One aspect of the present application provides a vehicle-road cooperative perception and parking method for an underground mining chute environment, comprising: S1, collecting roadside laser radar data, roadside camera data, roadside laser rangefinder data, vehicle-mounted laser radar data and vehicle-mounted camera data; S2, obtaining roadside obstacle information through time synchronization, space synchronization and calculation of matching factors based on the collected roadside laser radar data, roadside camera data and roadside laser rangefinder data; S3, obtaining vehicle-mounted obstacle information through time synchronization, space synchronization and calculation of matching factors based on the collected vehicle-mounted laser radar data and vehicle-mounted camera data; S4, matching vehicle-road obstacles through the Hungarian algorithm based on the roadside obstacle information and the vehicle-mounted obstacle information, and generating a vehicle-road cooperative perception result; S5, based on the vehicle-road cooperative perception result, using a global map to calculate the optimal driving path from a starting point to an unloading point for global path planning, and generating a global path; S6, based on the vehicle-road cooperative perception result, performing local path planning through a path obstacle avoidance algorithm, and generating a local path; S7, controlling the driving path, vehicle speed and parking of the unmanned vehicle based on the global path and the local path.

[0008] Furthermore, S2 obtains roadside obstacle information based on the collected roadside lidar data, roadside camera data, and roadside laser rangefinder data through time synchronization, space synchronization, and calculation of matching factors, including: S21, time synchronization of roadside lidar data and roadside camera data using PTP or gPTP; S22, obstacle detection based on the time-synchronized roadside lidar data to obtain the position data and reflection intensity data of the lidar obstacle; S23, target detection using the YOLO network based on the time-synchronized roadside camera data to obtain the position data and category of the camera target obstacle; S24, timestamp alignment of the position data and reflection intensity data of the lidar obstacle, as well as the position data and category of the camera target obstacle; S25, spatial synchronization of the position data of the lidar obstacle after timestamp alignment, as well as the position data of the camera target obstacle; S26, calculation of the laser after spatial synchronization The position data of the radar obstacle and the intersection-and-union ratio and distance between the position data of the camera target obstacle; according to the intersection-and-union ratio and the distance, the first matching factor is calculated; S27, the position data of the laser radar obstacle and the position data of the camera target obstacle after spatial synchronization are optimally matched using the first matching factor by using the Hungarian algorithm; S28, for the obstacles successfully matched in S27, the category of the camera target obstacle is added to the position data and reflection intensity data of the corresponding laser radar obstacle and output; S29, for the laser radar obstacle that failed to match in S27, the reflection intensity data and tracking time of the corresponding laser radar obstacle in S22 are obtained, and when the tracking time is greater than the first preset threshold or the average reflection intensity is greater than the second preset threshold, the position data and reflection intensity data of the corresponding laser radar obstacle are retained, otherwise deleted; the retained laser radar obstacle is output together with the laser radar obstacle data successfully matched in S28 as roadside obstacle information. In this application, the first matching factor, the second matching factor and the third matching factor are calculated in the same way.

[0009] Among them, PTP (Precision Time Protocol) is a protocol for achieving time synchronization in distributed systems. It exchanges time synchronization information through the network so that the clocks between different devices are consistent. gPTP (generalized Precision Time Protocol) is an extended version of the PTP protocol and is suitable for a wider range of application scenarios, such as automobiles, industrial automation and other fields. YOLO (You Only Look Once) is a convolutional neural network architecture for real-time target detection. It divides the image into grids and predicts the bounding box and category probability of the target in each grid cell. Reflection intensity data refers to the intensity value of the return signal measured by the lidar. When the laser beam hits the surface of an object, part of the light energy is reflected back to the lidar receiver, and the reflection intensity data represents the intensity of the reflected signal. Objects of different materials and at different distances have different reflection intensities. By analyzing the reflection intensity data, the attribute information of the object, such as material, roughness, etc., can be obtained. Intersection over Union (IoU) is an indicator that measures the degree of overlap between two bounding boxes. It calculates the ratio of the intersection area to the union area of ​​the two bounding boxes, and the value range is [0, 1]. The larger the IoU ratio, the higher the overlap between the two bounding boxes. Distance refers to the Euclidean distance between the center points of the two bounding boxes. It reflects the distance relationship between the two targets in space. The matching factor is a metric used to evaluate whether two targets match. It takes into account the IoU ratio and distance between the targets and calculates a value through a certain calculation formula. The larger the matching factor, the higher the similarity between the two targets, and the more likely they are to correspond to the same real target. Common matching factor calculation methods include weighted average and geometric average.

[0010] Further, S26, calculating the intersection-and-union ratio and distance between the position data of the laser radar obstacle after spatial synchronization and the position data of the camera target obstacle; calculating the first matching factor according to the intersection-and-union ratio and the distance, including: calculating the intersection-and-union ratio value1 by the following formula: ,in, Refers to the intersection area of ​​the bounding box of the lidar obstacle and the camera target obstacle. A is the bounding box area of ​​the camera target obstacle; B is the bounding box area of ​​the lidar obstacle.

[0011] The distance value2 is calculated using the following formula: ,when hour, , other cases, , where L1 represents the diagonal length of the bounding box of the camera target obstacle, and L2 is the distance between the center points of the bounding boxes of the lidar obstacle and the camera obstacle.

[0012] Calculate the first matching factor , through the following formula: ,in, represents the matching factor, and scale represents the weight factor.

[0013] Furthermore, target obstacles include pedestrians, vehicles, cones, construction signs and retaining walls.

[0014] Further, S3, based on the collected vehicle-mounted laser radar data and vehicle-mounted camera data, obtain the vehicle-mounted obstacle information through time synchronization, space synchronization and calculation of matching factors, including: S31, using PTP or gPTP to time synchronize the vehicle-mounted laser radar data and the vehicle-mounted camera data; S32, according to the time-synchronized vehicle-mounted laser radar data, obstacle detection is performed to obtain the position data and reflection intensity data of the vehicle-mounted laser radar obstacle; S33, according to the time-synchronized vehicle-mounted camera data, target detection is performed using the YOLO network to obtain the position data and category of the vehicle-mounted camera target obstacle; S34, the position data and reflection intensity data of the vehicle-mounted laser radar obstacle, and the position data and category of the vehicle-mounted camera target obstacle are timestamp aligned; S35, the position data of the vehicle-mounted laser radar obstacle after timestamp alignment, and the position data of the vehicle-mounted camera target obstacle are spatially synchronized; S36, the position of the vehicle-mounted laser radar obstacle after spatial synchronization is calculated data, and the intersection-and-union ratio and distance between the position data of the target obstacle of the vehicle-mounted camera; calculate the second matching factor according to the intersection-and-union ratio and the distance; S37, use the Hungarian algorithm to optimally match the position data of the vehicle-mounted laser radar obstacle and the position data of the target obstacle of the vehicle-mounted camera after spatial synchronization using the second matching factor; S38, for the obstacle successfully matched in S37, add the category of the target obstacle of the vehicle-mounted camera to the position data and reflection intensity data of the corresponding vehicle-mounted laser radar obstacle and output it; S39, for the vehicle-mounted laser radar obstacle that failed to match in S37, obtain the reflection intensity data and tracking time of the corresponding vehicle-mounted laser radar obstacle in S32, when the tracking time is greater than the first preset threshold or the average reflection intensity is greater than the second preset threshold, retain the position data and reflection intensity data of the corresponding vehicle-mounted laser radar obstacle, otherwise delete it; output the retained vehicle-mounted laser radar obstacle together with the vehicle-mounted laser radar obstacle data that successfully matched in S38 as vehicle-mounted obstacle information.

[0015] Further, S4, according to the roadside obstacle information and the vehicle-mounted obstacle information, the vehicle-road obstacle is matched by the Hungarian algorithm to generate the vehicle-road cooperative perception result, including: S41, based on the timestamp of the vehicle-mounted obstacle information, the roadside obstacle information and the vehicle-mounted obstacle information are synchronized in time; S42, the roadside obstacle information and the vehicle-mounted obstacle information after time synchronization are synchronized by spatial coordinate conversion, and spatial synchronization is performed; S43, the intersection and union ratio and distance of the roadside obstacle information and the vehicle-mounted obstacle information after spatial synchronization are calculated to obtain the third matching factor; S44, the roadside obstacle information after spatial synchronization is synchronized by the Hungarian algorithm The information and the on-board obstacle information are optimally matched using the third matching factor; S45, for the obstacles successfully matched in S44, the corresponding roadside obstacle information is removed based on the on-board obstacle information; S46, for the roadside obstacle information that failed to match in S45, it is determined whether the corresponding roadside obstacle information is the unmanned vehicle itself, if so, it is deleted, otherwise it is retained; the on-board obstacle information that failed to match in S45 is retained; S47, the on-board obstacle information that successfully matched in S45, the on-board obstacle information that failed to match, and the roadside obstacle information retained in S46 are integrated to generate the vehicle-road collaborative perception result.

[0016] Furthermore, S5, based on the vehicle-road collaborative perception result, a global path planning is performed by calculating the optimal driving path from the starting point to the unloading point using the global map to generate a global path, including: S51, obtaining a global map of the underground mine chute environment, the global map containing the underground mine chute road network topology structure; S52, obtaining the current position of the unmanned vehicle as the path planning starting point, and obtaining the chute unloading point position as the path planning target point; S53, based on the vehicle-road collaborative perception result of S4, obtaining the static obstacle position, category and bounding box; S54, adding the static obstacle position to the global map to obtain an updated global map; S55, based on the updated global map, constructing a graph structure for path search, in which the nodes in the graph are feature points in the road network, and the edges are connections between feature points; S56, using the A* or Dijkstra algorithm to search for the optimal path from the starting point to the target point in the graph structure as the global path.

[0017] Furthermore, S6, based on the vehicle-road collaborative perception result, local path planning is performed through a path obstacle avoidance algorithm to generate a local path, including: S61, based on the vehicle-road collaborative perception result of S4, the dynamic obstacle position, category and bounding box are obtained; S62, based on the dynamic obstacle position and the global path, the dynamic obstacle located on the global path or within the preset range of the global path is obtained as the target dynamic obstacle; S63, based on the dynamic obstacle position obtained in S61, a stopping distance threshold is set; S64, using the target dynamic obstacle and the stopping distance threshold as constraints, with the current position of the unmanned vehicle as the starting point, and the point within the preset range of the global path as the local target point, a local graph structure for local path search is constructed; S65, in the local graph structure, the Dijkstra algorithm or the A* algorithm is used to search for the optimal path from the starting point to the local target point as the local path.

[0018] Further, S7 controls the driving path, speed and stopping of the unmanned vehicle according to the global path and the local path, including: S71, controls the driving of the unmanned vehicle through lateral control and longitudinal control according to the global path and the local path, the lateral control adjusts the steering of the unmanned vehicle through the steering system, and the longitudinal control adjusts the speed of the unmanned vehicle through the throttle system and the braking system; S72, calculates the relative position relationship between the unmanned vehicle and the obstacle according to the vehicle-road collaborative perception result and the current position of the unmanned vehicle; S73, according to the relative position relationship of S72, when the obstacle is located in front of the unmanned vehicle, returns to step S71, otherwise executes step S74; S74, according to the relative position relationship of S72, obtains the obstacle with the smallest distance to the unmanned vehicle as the nearest obstacle; S75, according to the vehicle-road collaborative perception result, determines whether the nearest obstacle is a retaining wall, if not, executes step S76, if yes, executes step S77; S76, controls the unmanned vehicle to stop; S77, controls the unmanned vehicle to reverse until the nearest obstacle is a retaining wall.

[0019] Another aspect of the present application also provides a vehicle-road collaborative perception and docking system for an underground mine chute environment, which is used to execute a vehicle-road collaborative perception and docking method for an underground mine chute environment of the present application.

[0020] Compared with the prior art, the advantages of this application are:

[0021] This application adopts the vehicle-road collaborative perception method, comprehensively utilizes the data collected by roadside laser radar, roadside camera, roadside laser rangefinder, vehicle-mounted laser radar and vehicle-mounted camera, obtains roadside obstacle information and vehicle-mounted obstacle information through time synchronization, space synchronization and calculation of matching factors, and generates vehicle-road collaborative perception results through matching and fusion through the Hungarian algorithm. This method makes full use of the roadside and vehicle-mounted multi-source heterogeneous sensor data, makes up for the shortcomings of single-view perception, realizes comprehensive and accurate perception of static obstacles and dynamic obstacles, and improves the perception ability and reliability in complex underground environments.

[0022] This application adds static obstacle information obtained by vehicle-road collaborative perception to the global map, performs global path search through A* or Dijkstra algorithm, and obtains the optimal path from the starting point to the target point. At the same time, based on the dynamic obstacle information obtained by vehicle-road collaborative perception, local path planning is performed through a path obstacle avoidance algorithm. This method fully considers the impact of static obstacles and dynamic obstacles on the driving path, combines global map information and real-time perception information, realizes accurate planning of the driving path, and improves the accuracy and safety of unmanned vehicle navigation.

[0023] This application performs path tracking control on the unmanned vehicle through lateral control and longitudinal control according to the planned global path and local path. At the same time, based on the obstacle position information obtained by vehicle-road collaborative perception, the relative position relationship between the unmanned vehicle and the obstacle is calculated. When the obstacle is behind the unmanned vehicle, the obstacle with the shortest distance is obtained, and different parking strategies are adopted according to the obstacle category information. If the nearest obstacle is a retaining wall, the unmanned vehicle is controlled to reverse slowly until it stops; if it is another obstacle, the unmanned vehicle is controlled to keep a certain distance and stop. This control method comprehensively considers the location and category characteristics of the obstacle, adopts targeted parking strategies, improves the flexibility and accuracy of the unmanned vehicle control, and especially solves the problem of difficulty in parking caused by retaining wall obstacles in the chute environment.

[0024] This application is designed specifically in terms of obstacle perception and matching based on the characteristics of the chute environment. By calculating parameters such as the intersection ratio and distance, the matching factor is constructed, and the Hungarian algorithm is used to match obstacles to improve the matching accuracy; according to the reflection intensity data and tracking time of the lidar obstacle, the obstacles that failed to match are screened and the wrong matches are removed; the complementary characteristics of the lidar and the camera are fully utilized to achieve accurate identification of obstacles of different categories, especially the recognition ability of retaining wall obstacles is greatly improved. The above measures effectively overcome the perception difficulties such as insufficient data in the chute environment and irregular obstacles, and improve the applicability and robustness of the method in actual environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present application will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0026] Figure 1 It is a schematic diagram of an exemplary application scenario of a vehicle-road cooperative perception and parking method in an underground mine chute environment according to some embodiments of the present application;

[0027] Figure 2 It is an exemplary module diagram of a vehicle-road cooperative perception and parking system in an underground mine chute environment according to some embodiments of the present application;

[0028] Figure 3 is an exemplary flow chart of a roadside lidar and camera fusion process according to some embodiments of the present application;

[0029] Figure 4 is a schematic diagram of matching factor calculation according to some embodiments of the present application;

[0030] Figure 5 is a flowchart of exemplary steps of a vehicle-side and roadside data fusion process according to some embodiments of the present application;

[0031] Figure 6 It is an exemplary step flow chart of reversing control in a chute environment according to some embodiments of the present application. DETAILED DESCRIPTION

[0032] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0033] like Figure 1 and Figure 2As shown, the underground positioning unit includes: UWB location service base station: providing UWB positioning service; vehicle-mounted UWB positioning card: vehicle-mounted equipment, receiving UWB positioning signal; UWB and SLAM fusion positioning: integrating UWB and SLAM technology to achieve high-precision vehicle positioning; the mine-used intelligent roadside perception unit includes: mine safety intelligent industrial control computer, mine safety camera, mine safety fill light, mine safety laser radar, mine safety laser rangefinder, V2X roadside equipment RSU. The underground communication unit includes: Wi-Fi: providing wireless communication services. 5G: achieving high-speed and low-latency communication through 5G network. Wired network: using wired connection as communication support. V2X (Vehicle-to-Everything): realizing communication between vehicles and vehicles, vehicles and infrastructure, etc., and enhancing collaboration and safety. The mine-used intelligent vehicle-mounted perception unit includes: vehicle-mounted domain control host, vehicle-mounted switch, mine safety camera, mine safety laser radar, V2X vehicle-mounted equipment OBU. The vehicle planning and control unit includes: global path planning: providing vehicles with driving paths within the global range. Local path planning: real-time adjustment of the vehicle's path in a local area. Braking control: intelligent control of the vehicle's braking. Steering control: adjust the vehicle's steering according to path planning. Braking: perform the vehicle's braking function.

[0034] Specifically, the underground communication function of this application adopts a variety of communication technologies, including Wi-Fi, wired network, 5G and V2X, etc., to realize the real-time transmission and interaction of underground information. The IEEE 802.11ac standard Wi-Fi communication has an operating frequency band of 5GHz and a theoretical transmission rate of up to 1.3Gbps. In the underground environment, multiple Wi-Fi access points are deployed to achieve continuous communication between the vehicle and the underground control center through the coverage and switching of wireless signals. Gigabit Ethernet technology is used to achieve wired network communication, with a transmission rate of up to 1000Mbps. Optical fiber is laid in the main underground lanes, and the network is expanded through repeaters and other equipment to achieve high-speed and stable communication between the vehicle and the underground control center. The 5G communication of the 3GPP Release 15 standard is adopted, working in the Sub-6GHz and mmWave frequency bands, with a peak downlink rate of up to 20Gbps. 5G micro base stations are deployed in key areas underground, and real-time interaction between vehicles and the underground control center is achieved through high-speed, low-latency 5G networks. It adopts 3GPP Release 14 standard V2X communication, supports LTE-V2X technology, and has a transmission delay of less than 100ms. Through V2X communication, direct communication between vehicles and vehicles and between vehicles and infrastructure is realized, which improves vehicle perception and coordination capabilities and enhances the safety of underground operations. Through the collaborative work and redundant backup design of the above-mentioned multiple communication technologies, this application builds a high-speed, stable and reliable underground communication network, providing strong communication guarantee for unmanned vehicles.

[0035] The underground positioning function is constructed by integrating UWB and SLAM. The two work together to achieve high-precision positioning of the vehicle. The UWB positioning system consists of a UWB location service base station and a vehicle-mounted UWB positioning card. Among them, the UWB location service base station adopts the DecaWave DWM1001 module, which works in the 6.5GHz frequency band and has a positioning accuracy of up to 10cm; the vehicle-mounted UWB positioning card adopts the DecaWave DWM1001C module, which realizes positioning through the TWR ranging algorithm, and the refresh rate can reach 10Hz. Through UWB positioning, the vehicle can obtain centimeter-level absolute position information. The SLAM positioning system collects environmental point cloud data through the Velodyne VLP-16 laser radar, uses the LOAM algorithm to build a global map, and realizes real-time matching and updating of the vehicle position, with a positioning accuracy of up to 20cm. SLAM positioning can provide relative position information of the vehicle to ensure the continuity and stability of positioning.

[0036] This application uses the Kalman filter algorithm to fuse the results of UWB positioning and SLAM positioning and output the optimal position estimate of the vehicle. Among them, UWB positioning information can effectively suppress SLAM cumulative errors, and SLAM positioning results can make up for the lack of UWB signals, and the two complement each other.

[0037] First, the vehicle-mounted rear-facing radar collects distance information behind the vehicle, and after time synchronization, it is integrated with the vehicle odometer data to obtain the real-time location information of obstacles around the vehicle. The vehicle-mounted rear-facing camera collects image data behind the vehicle, and performs distortion correction, image enhancement and other operations through image preprocessing algorithms to improve image quality.

[0038] The roadside camera collects image data of the chute environment with the assistance of fill light, and performs distortion correction, image enhancement and other operations through image preprocessing algorithms to improve image quality. The roadside laser radar collects three-dimensional point cloud data of the chute environment, and removes noise and redundant data through preprocessing operations such as point cloud filtering and downsampling. The roadside laser rangefinder collects the distance data between the vehicle and the retaining wall, and smoothes the data through the Kalman filter algorithm to improve the accuracy of distance measurement.

[0039] Accurately calibrate the vehicle-mounted laser radar and the basic coordinates of the vehicle body. Use the calibration plate method or the hand-eye calibration method to calculate the transformation matrix between the laser radar and the vehicle body coordinate system to achieve alignment of the laser radar data with the vehicle coordinate system. Calibrate the intrinsic parameters and distortion coefficients of the vehicle-mounted camera. Use the Zhang calibration method or the nonlinear optimization algorithm to calculate the camera's intrinsic parameter matrix and distortion coefficient to achieve geometric correction of the image. Calibrate the extrinsic parameters of the vehicle-mounted laser radar and camera. Use the rigid body transformation or the ICP algorithm to calculate the transformation matrix between the laser radar and the camera to achieve alignment of the point cloud data with the image data.

[0040] Calibrate the roadside LiDAR according to the global point cloud map, calculate the transformation matrix between the roadside LiDAR and the global coordinate system through the ICP algorithm or NDT algorithm, and realize the alignment of the roadside point cloud data with the global map. Calibrate the intrinsic parameters and distortion coefficients of the roadside camera, calculate the intrinsic parameter matrix and distortion coefficients of the camera through Zhang's calibration method or nonlinear optimization algorithm, and realize the geometric correction of the image. Calibrate the extrinsic parameters of the roadside LiDAR and camera, calculate the transformation matrix between the LiDAR and the camera through rigid body transformation or ICP algorithm, and realize the alignment of the point cloud data and the image data.

[0041] The vehicle-mounted LiDAR data and rear-facing camera data are synchronized in time, and the two data are unified into the vehicle body coordinate system through coordinate transformation. The obstacle detection results of the LiDAR and camera are fused using the Kalman filter algorithm or the particle filter algorithm to obtain the precise position and speed information of the obstacle behind the vehicle.

[0042] The roadside lidar data, camera data and laser rangefinder data are synchronized in time, and the three types of data are unified into a global coordinate system through coordinate transformation. The obstacle detection results of the lidar, camera and laser rangefinder are fused using the Kalman filter algorithm or the particle filter algorithm to obtain the precise location information of obstacles in the chute environment.

[0043] The vehicle-mounted data and roadside data are synchronized in time and space, and the two data are unified into the global coordinate system through coordinate transformation. The perception results of the vehicle-mounted sensors and roadside sensors are fused using the Bayesian fusion algorithm or Dempster-Shafer evidence theory to obtain high-precision position and status information of vehicles and obstacles in the chute environment, providing reliable environmental perception data for the planning and control of unmanned vehicles. Through the precise calibration and data fusion of the vehicle-mounted and roadside sensors, a comprehensive perception and understanding of the chute environment is achieved.

[0044] like Figure 3 As shown, first collect roadside data. Lidar data collection: connect the lidar to the industrial computer through a network cable, run the lidar driver on the roadside intelligent industrial computer, and directly obtain lidar data. Camera data collection: connect the camera to the industrial computer through an aviation plug, run the camera driver on the roadside intelligent industrial computer, and directly obtain camera data. Laser rangefinder data collection: connect the laser rangefinder to the industrial computer through an aviation plug, run the laser rangefinder driver on the roadside intelligent industrial computer, and directly obtain laser rangefinder data.

[0045] The laser rangefinder is mainly used to measure whether the light beam is blocked by an obstacle. When the light beam is not blocked, the measured distance is the distance between the left and right tunnel walls; when the light beam is blocked by an obstacle, the measured distance decreases sharply. Secondly, the roadside lidar and camera data are integrated, and the time is hard synchronized: the clock source is unified using ptp / gptp, with the domain control host as the master and other sensors as slaves to align the sensor time dimension. Lidar perception target output: The algorithm is based on lidar data, and detects and identifies target obstacles through preprocessing, top removal, ground removal, clustering, contour generation and other operations. Camera perception target output: Target detection is mainly performed through the yolo series deep learning network, and the location and category of the target (usually including pedestrians, vehicles, cones, construction signs, retaining walls, etc.) are output.

[0046] Time soft synchronization: For the lidar and camera nodes, cache the detection results of the last five frames, including the timestamp of each frame and the detected target information (such as position, size, category, etc.). Use the lidar timestamp as the reference time because the lidar timestamp is usually more accurate and stable. For each frame of lidar detection result, find the frame with the closest timestamp in the cached camera detection result as the corresponding camera detection result. Calculate the timestamp difference: For the i-th frame detection result of the lidar, calculate its timestamp The timestamp of each camera detection result cached The difference between . Find the minimum difference: the difference between all timestamps Find the minimum value in , the corresponding camera detection result is the result closest to the timestamp of the lidar frame i. Set the timestamp threshold: In order to avoid matching errors caused by large timestamp differences, set a timestamp threshold (such as 50ms), if the minimum difference If the value is greater than the threshold, it is considered that no corresponding camera result is found and no matching is performed. Each frame of lidar detection result and the corresponding camera detection result found form a synchronized data pair as the output result of time soft synchronization.

[0047] Spatial synchronization: Convert the lidar detection results from the lidar coordinate system to the camera coordinate system so that they can be spatially matched with the camera detection results. The external parameter matrix from the lidar coordinate system to the camera coordinate system is obtained through calibration. , which contains the rotation matrix R and translation vector t between the two coordinate systems. For each three-dimensional point of the lidar detection result , convert it to a three-dimensional point in the camera coordinate system , the conversion formula is: .

[0048] The converted lidar 3D points Project it onto the camera’s image plane to get the corresponding two-dimensional pixel coordinates (u, v). According to the camera’s intrinsic parameter matrix K, the three-dimensional point Projected onto the image plane, the projection formula is: , ,in, is the focal length of the camera, are the coordinates of the principal point. These parameters can be obtained through camera calibration. The obtained pixel coordinates (u, v) are used as the coordinates of the lidar detection result in the camera image for subsequent target matching. Generate a corresponding two-dimensional bounding box based on the three-dimensional size of the lidar detection result and the image coordinates obtained by projection. The lidar detection result usually contains the three-dimensional size of the target (l, w, h), which represents the length, width, and height of the target respectively. According to the three-dimensional size and image coordinates of the target, calculate the coordinates of the four corner points of the bounding box, for example, the coordinates of the upper left corner are (u - l / 2, v - h), and the coordinates of the lower right corner are (u + l / 2, v). Generate bounding box: According to the coordinates of the four corner points, generate a rectangular bounding box as the representation of the lidar detection result in the camera image. The two-dimensional bounding box corresponding to each lidar detection result is used as the output result of spatial synchronization for subsequent target matching. Temporal soft synchronization caches the detection results of a certain number of frames and finds a pair of data with the closest timestamps, thus realizing the time alignment of the two sensor data; spatial synchronization maps the lidar detection results to the camera image plane through coordinate system conversion and projection operations, generates the corresponding two-dimensional bounding box, and realizes the spatial alignment of the two sensor data. These synchronization operations can effectively improve the accuracy and efficiency of data fusion.

[0049] like Figure 4 As shown in the figure, the matching factor calculation is: considering that the lidar detection frame may divide an obstacle into multiple objects, the size of the lidar will be smaller than the size of the camera. The IOU is optimized and the original intersection-union ratio is changed to the following formula: ,in, Refers to the intersection area of ​​the lidar and camera bounding boxes. A is the bounding box area of ​​the camera target obstacle; B is the bounding box area of ​​the lidar obstacle; considering that the lidar and camera bounding boxes may not intersect at a distance, the distance factor needs to be added. It cannot be directly calculated using absolute distance, because images are larger near and smaller far away, so a relative quantity must be used to measure. , ,in, Refers to the final matching factor used, L1 refers to the diagonal length of the camera bounding box, L2 is the distance between the center points of the bounding boxes of the lidar obstacle and the camera obstacle; scale is a weight factor, and its value range is between (0, 1). It is close to 1 when the distance is close and close to 0 when the distance is far.

[0050] Multi-source data matching, based on the previously calculated matching factor M, construct the cost matrix C between the lidar target and the camera target. The matrix elements represents the matching cost between the i-th lidar target and the j-th camera target, The cost matrix C is used as input, and the Hungarian algorithm is used to solve the optimal matching problem to obtain the optimal matching relationship between the lidar target and the camera target. The time complexity of the Hungarian algorithm is , where n is the number of targets. For each pair of matching relationships obtained by the Hungarian algorithm, determine whether its matching factor M is greater than the preset threshold (such as 0.5), if it is, the match is considered successful, otherwise it is considered a match failure.

[0051] For lidar targets and camera targets that have been matched successfully, retain the bounding box information of the lidar target, including position, size, orientation, etc. Add the category information of the successfully matched camera target (such as pedestrians, vehicles, cones, etc.) to the corresponding lidar target. Output the fused target information (including lidar bounding box and camera category) as the final obstacle perception result.

[0052] The lidar targets that failed to match are extracted as the unmatched lidar target set U. For each target in the unmatched lidar target set U , determine its tracking time in the historical frame Is it greater than the preset threshold T (such as 5 frames), if so, it is considered that the target may be a real obstacle, otherwise it may be a noise point or a false detection target. , calculate its average reflection intensity in the historical frame ,judge Is it greater than the preset threshold? (such as 50), if it is, it is considered that the target may be a real obstacle, otherwise it is considered to be a noise point or a falsely detected target. For unmatched lidar targets that meet the tracking time and average intensity conditions, they are retained as supplementary obstacle perception results. The successfully matched fusion target and the retained unmatched lidar target are output together as the final obstacle perception result. The Hungarian algorithm is used to achieve the optimal matching of lidar targets and camera targets, and fusion processing and supplementary processing are performed based on the matching results, and finally an obstacle perception result that comprehensively considers the characteristics of the two sensors is obtained. Among them, the successfully matched target integrates the precise position information of the lidar and the semantic category information of the camera, while the lidar target that failed to match is judged by the tracking time and average intensity, and the obstacles that may exist but are not recognized by the camera are retained, which improves the accuracy and completeness of the perception results.

[0053] like Figure 5 As shown, vehicle-mounted unit data acquisition: Lidar data acquisition: connect the lidar to the vehicle-mounted switch through a network cable, and the switch is then connected to the vehicle-mounted domain controller through a network cable. Run the lidar driver on the vehicle-side domain controller to directly obtain lidar data. Camera data acquisition: connect the camera directly to the vehicle-mounted domain controller through an aviation plug. Run the camera driver on the vehicle-side domain controller to directly obtain camera data. Fusion of the vehicle-mounted unit lidar and camera data, refer to the lidar and camera fusion process in step (3), the fusion process of the vehicle-mounted unit is consistent with it, including time hard synchronization, separate perception target output, cache of separate perception results, time soft synchronization, space synchronization, matching factor calculation, multi-source data matching and other steps, and finally obtain the fusion perception result of the vehicle-mounted unit.

[0054] The vehicle receives the roadside detection data. Through V2X (Vehicle-to-Everything) technology, the vehicle-mounted unit can communicate with the roadside equipment and receive the obstacle information sent by the roadside equipment. The obstacle information sent by the roadside equipment includes the location, category, bounding box, etc. of the obstacle, as well as the information output by the laser rangefinder whether there is any obstacle blocking the road. After receiving the data sent by the roadside equipment, the vehicle-mounted unit parses the data and extracts various information about the obstacle for subsequent data fusion.

[0055] Fuse the vehicle-side and roadside data to obtain the perception fusion results from the roadside and vehicle-side, that is, the fused perception results obtained in step 2. Cache the perception results of the roadside and vehicle-side for time synchronization. Based on the latest vehicle-side data, compare the timestamps with the roadside cached data to find the pair of data with the closest timestamps to achieve soft time synchronization. Through the joint calibration module, convert the spatial coordinates of the roadside perception results to the vehicle-side coordinate system to ensure that the two are in the same coordinate system in space.

[0056] The vehicle-side and roadside data after time-space synchronization are matched and fused, and the Hungarian algorithm is used for matching. Specifically, according to the vehicle-side and roadside perception results after time-space synchronization, the cost matrix C is constructed, and the matrix elements Represents the matching cost between the i-th vehicle-side target and the j-th roadside target. The matching cost can take into account factors such as the position distance, size difference, and category similarity between targets to design a suitable cost function. The cost matrix C is used as input and the Hungarian algorithm is used for optimal matching to obtain the optimal matching relationship between the vehicle-side target and the roadside target. The Hungarian algorithm continuously adjusts the values ​​of the matrix elements to find the matching solution that minimizes the total matching cost. For each pair of matching relationships obtained by the Hungarian algorithm, determine whether its matching cost is less than the preset threshold (such as 1.0), if it is, the match is considered successful, otherwise it is considered a match failure.

[0057] For successfully matched vehicle-side targets and roadside targets, the perception results of the vehicle-side targets shall prevail, including the target's position, speed, acceleration, bounding box and other information. The successfully matched roadside targets are removed from the roadside perception results to avoid repeated appearance in subsequent processing. Unmatched roadside targets may include the perception results of the vehicle itself. By comparing the position of the roadside target with the position of the vehicle itself, the roadside targets that are less than a certain threshold (such as 5m) from the center point of the vehicle are regarded as the vehicle itself and removed from the unmatched roadside targets. For the remaining unmatched roadside targets, they are directly retained as supplementary environmental perception information.

[0058] For unmatched vehicle-side targets, directly retain them as targets perceived independently by the vehicle-side. Merge the successfully matched vehicle-side targets, the retained unmatched roadside targets, and the unmatched vehicle-side targets to obtain the final fusion perception results. For successfully matched vehicle-side targets, if the roadside perception results contain attribute information (such as target category) that the vehicle-side targets do not have, these attribute information can be supplemented to the vehicle-side targets to enrich the description of the targets. Add a unified timestamp to all merged targets to indicate the generation time of the fusion perception results. Unify the location information of all targets in the vehicle-side coordinate system to facilitate subsequent decision-making and control. Output the fusion perception results in predefined data formats, such as JSON, Protocol Buffers, etc., to facilitate data interaction with other modules.

[0059] In this application, the successfully matched targets are based on the vehicle-side perception results, ensuring the continuity and consistency of the perception results; the unmatched roadside targets are screened as supplementary information to expand the scope of perception; the unmatched vehicle-side targets reflect the results of independent vehicle-side perception. Finally, through target merging and attribute supplementation, a more comprehensive and accurate fusion perception result is obtained, providing high-quality environmental perception input for the autonomous driving system.

[0060] Global path planning, based on the actual situation of the chute environment, establishes a static global map, including key location information such as loading points, unloading points, and main channels. Add the location and contour information of stationary global obstacles (such as fixed buildings, facilities, etc.) to the global map. Use appropriate path search algorithms, such as A* algorithm, Dijkstra algorithm, etc., to search for the optimal path from the loading point to the unloading point on the global map. Optimize the searched path, such as smoothing, considering the vehicle turning radius limit, etc., to obtain a smooth and feasible global path. Output the planned global path in an appropriate data format, such as waypoints sequence, for use by local path planning and control modules.

[0061] Local path planning: The local path planning module receives real-time obstacle information from the perception module, including the location, outline, and category of obstacles. The obstacle information is converted from the coordinate system of the perception module (such as the lidar coordinate system and the camera coordinate system) to the coordinate system used by the local path planning module (such as the vehicle center coordinate system). The obstacle information is converted into a data format that can be processed by the local path planning algorithm, such as occupancy status in a grid map, polygonal or circular representation in an obstacle map, etc.

[0062] Determine the size and boundaries of the local map based on the vehicle's current location and perception range. Usually, a sufficiently large rectangular area is constructed with the vehicle as the center as the local map. Divide the local map into grids of equal size, each grid representing a small area. According to the obstacle information, mark the grid as occupied (with obstacles) or idle (without obstacles). According to the outline information of the obstacle, construct a polygon or circle with a shape similar to the obstacle in the local map to indicate the location and range of the obstacle.

[0063] According to the obstacle category information provided by the perception module, determine what type of obstacle it is, such as retaining wall, pedestrian, vehicle, cone, construction sign, etc. Set a corresponding stopping distance value for each obstacle category and store it in a mapping table or dictionary. For example, the stopping distance for retaining wall is 0.5 meters, the stopping distance for pedestrian is 2 meters, the stopping distance for vehicle is 3 meters, the stopping distance for cone and construction sign is 1 meter, etc. The stopping distance value can be adjusted dynamically according to the actual situation. For example, when the vehicle speed is low, the stopping distance can be appropriately reduced; when the vehicle speed is high, the stopping distance can be appropriately increased.

[0064] The dynamic window algorithm (DWA) is used to plan the local path, and the state of the vehicle is defined as a combination of position, speed, and direction. In the current state, a set of candidate speeds and steering angles are generated according to the vehicle's dynamic model and converted into a set of feasible trajectories. Each candidate trajectory is evaluated, and a comprehensive evaluation index is calculated considering factors such as the smoothness of the trajectory, the distance to obstacles, and the distance to the target point. The trajectory with the best comprehensive evaluation index is selected as the local path at the current moment.

[0065] Alternatively, use model predictive control (MPC) to plan a local path and define the state of the vehicle as a combination of position, velocity, acceleration, and direction. Based on the vehicle's dynamic model, establish a mathematical model for predicting future states. Define an objective function, including evaluations of factors such as trajectory smoothness, distance to obstacles, and distance to target points. Set constraints on vehicle motion, such as speed limits, acceleration limits, steering angle limits, etc. Use optimization algorithms (such as quadratic programming, gradient descent, etc.) to solve the objective function and obtain an optimal sequence of control quantities. Use the first element of the optimal sequence of control quantities as the control command at the current moment to control vehicle motion.

[0066] Smooth the planned local path, remove sharp corners or mutations, and make the path smoother and more continuous. Commonly used smoothing algorithms include spline interpolation and Bezier curves. Take the vehicle's dynamic characteristics (such as maximum speed, maximum acceleration, minimum turning radius, etc.) as constraints to optimize the path and ensure that the path is within the vehicle's dynamic capabilities. Perform collision checks on the optimized path to ensure that there is a sufficient safe distance between the path and obstacles to avoid collision risks.

[0067] Discretize the optimized local path into a series of trajectory points, each of which contains information such as position, speed, acceleration, direction, etc. Add a timestamp to each trajectory point, indicating that the vehicle should reach the position at that time. Output the trajectory point sequence in an appropriate data format (such as JSON, Protocol Buffers, etc.) to facilitate the control module to read and use.

[0068] The local path planning module can receive perception information in real time, build a local map, and set different stopping distances according to the obstacle category. Then, an optimal path to avoid obstacles is planned on the local map using algorithms such as DWA or MPC, and smooth optimization and collision checks are performed. Finally, the optimized path is output in the form of a trajectory point sequence for use by the control module. In this way, the vehicle can drive safely and smoothly in a dynamic environment and avoid collisions with obstacles.

[0069] For lateral control, a suitable path tracking algorithm, such as the Stanley method and the Pure Pursuit method, is used to calculate the steering control amount of the vehicle based on the planned target path and the current position of the vehicle. The steering control amount output by the path tracking algorithm is used to control the vehicle's steering system, such as the steering motor and steering hydraulic cylinder, to achieve the vehicle's steering movement. The lateral error between the vehicle's current position and the target path is calculated in real time as a feedback signal to adjust the steering control amount. According to the physical characteristics and safety requirements of the vehicle, the steering angle is limited to avoid excessive steering angles that cause vehicle instability or tire damage.

[0070] Longitudinal control plans the target speed curve of the vehicle according to the characteristics of the global path and the local path to ensure that the vehicle travels at an appropriate speed in different sections of the road. According to the target speed and the current speed of the vehicle, the throttle control amount is calculated to control the vehicle's throttle system, such as the electronic throttle, throttle-by-wire system, etc., to achieve the acceleration of the vehicle. According to the target speed and the current speed of the vehicle, the braking control amount is calculated to control the vehicle's braking system, such as the electronic brake, hydraulic brake, etc., to achieve the deceleration or parking of the vehicle. The longitudinal error between the current speed of the vehicle and the target speed is calculated in real time as a feedback signal to adjust the throttle and braking control amounts. Considering the dynamic characteristics of the vehicle, such as the maximum acceleration and maximum deceleration, the throttle and braking control amounts are limited to ensure the smoothness and safety of the vehicle's movement.

[0071] like Figure 6 As shown in the figure, in the case of reverse parking in a well environment, the control module receives the vehicle-road collaborative fusion results output by the perception module, including the location, category and other information of the obstacle. According to the current position of the vehicle and the position of the obstacle, the relative position relationship between the vehicle and the obstacle is calculated. If the obstacle is in front of the vehicle, no processing is done and normal driving continues. If the obstacle is behind the vehicle, the reverse parking process is entered. From the obstacles behind the vehicle, the obstacle closest to the vehicle is selected as the parking reference object. Determine whether the category of the nearest obstacle is a retaining wall. If it is not a retaining wall (such as pedestrians, vehicles, cones, construction signs, etc.), set a longer parking distance threshold. When the distance between the vehicle and the obstacle is less than the threshold, the parking operation is performed. If it is a retaining wall, the precise parking process is entered. The vehicle is controlled to reverse slowly at a lower speed, while monitoring the data of the roadside laser rangefinder. When the laser rangefinder is blocked (that is, the measured distance decreases sharply), it means that the vehicle has approached the retaining wall and the parking operation is performed immediately. After parking, the distance between the vehicle and the retaining wall is close, achieving precise parking. Once the vehicle stops, the control module continues to output the braking control amount to maintain the vehicle's parking state until a new control instruction is received.

Claims

1. A vehicle-road cooperative sensing and parking method in an underground mine chute environment, characterized in that: include: S1, collects roadside lidar data, roadside camera data, roadside laser rangefinder data, vehicle-mounted lidar data and vehicle-mounted camera data; S2, obtaining roadside obstacle information through time synchronization, space synchronization and calculation of matching factors based on the collected roadside lidar data, roadside camera data and roadside laser rangefinder data; S3, based on the collected vehicle-mounted laser radar data and vehicle-mounted camera data, obtain the vehicle-mounted obstacle information through time synchronization, space synchronization and calculation of matching factors; S4, based on the roadside obstacle information and the vehicle-mounted obstacle information, the vehicle-road obstacle is matched through the Hungarian algorithm to generate the vehicle-road collaborative perception result; S5, based on the vehicle-road cooperative perception result, the global map is used to calculate the optimal driving path from the starting point to the unloading point for global path planning, and a global path is generated; S6, based on the vehicle-road cooperative perception results, local path planning is performed through the path obstacle avoidance algorithm to generate a local path; S7, controls the driving path, speed and stopping of the unmanned vehicle according to the global path and local path.

2. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 1 is characterized in that: S2, through time synchronization, space synchronization and calculation of matching factors, obtains roadside obstacle information, including: S21, using PTP or gPTP to synchronize the roadside lidar data and the roadside camera data; S22, performing obstacle detection according to the time-synchronized roadside laser radar data to obtain the laser radar obstacle position data and reflection intensity data; S23, performing target detection using a YOLO network based on the time-synchronized roadside camera data to obtain location data and categories of camera target obstacles; S24, performing timestamp alignment on the position data and reflection intensity data of the laser radar obstacle, and the position data and category of the camera target obstacle; S25, spatially synchronizing the position data of the laser radar obstacle after the timestamps are aligned and the position data of the camera target obstacle; S26, calculating the intersection-over-union ratio and distance between the position data of the laser radar obstacle after spatial synchronization and the position data of the camera target obstacle; Calculate the first matching factor according to the intersection ratio and the distance; S27, using the Hungarian algorithm to optimally match the spatially synchronized laser radar obstacle position data and the camera target obstacle position data using the first matching factor; S28, for the obstacle successfully matched in S27, adding the category of the camera target obstacle to the position data and reflection intensity data of the corresponding laser radar obstacle and outputting it; S29, for the laser radar obstacle that failed to be matched in S27, the reflection intensity data and tracking duration of the corresponding laser radar obstacle in S22 are obtained, and when the tracking duration is greater than the first preset threshold or the average reflection intensity is greater than the second preset threshold, the position data and reflection intensity data of the corresponding laser radar obstacle are retained, otherwise they are deleted; The retained laser radar obstacle is output together with the laser radar obstacle data successfully matched in S28 as roadside obstacle information.

3. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 2 is characterized in that: S26, calculating a first matching factor, including: The intersection-and-union ratio value1 is calculated using the following formula: ; in, Refers to the intersection area of ​​the bounding box of the lidar obstacle and the camera target obstacle; A is the bounding box area of ​​the camera target obstacle; B is the bounding box area of ​​the lidar obstacle; The distance value2 is calculated using the following formula: ; Where L1 represents the diagonal length of the bounding box of the camera target obstacle, and L2 is the distance between the center points of the bounding boxes of the lidar obstacle and the camera obstacle; Calculate the first matching factor , through the following formula: ; in, represents the matching factor, and scale represents the weight factor.

4. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 3 is characterized in that: Target obstacles include pedestrians, vehicles, cones, construction signs and retaining walls.

5. The vehicle-road cooperative sensing and parking method in an underground mine chute environment according to any one of claims 2 to 4, characterized in that: S3, obtain the vehicle obstacle information, including: S31, using PTP or gPTP to synchronize the vehicle-mounted lidar data and the vehicle-mounted camera data; S32, performing obstacle detection according to the time-synchronized vehicle-mounted laser radar data to obtain the position data and reflection intensity data of the vehicle-mounted laser radar obstacle; S33, performing target detection using a YOLO network according to the time-synchronized onboard camera data to obtain the position data and category of the target obstacle of the onboard camera; S34, aligning the time stamps of the position data and reflection intensity data of the vehicle-mounted laser radar obstacle and the position data and category of the vehicle-mounted camera target obstacle; S35, spatially synchronizing the position data of the vehicle-mounted laser radar obstacle after the timestamps are aligned and the position data of the vehicle-mounted camera target obstacle; S36, calculating the intersection-over-union ratio and distance between the position data of the on-board laser radar obstacle after spatial synchronization and the position data of the on-board camera target obstacle; Calculate the second matching factor according to the intersection ratio and the distance; S37, using the Hungarian algorithm to optimally match the position data of the on-board laser radar obstacle and the position data of the on-board camera target obstacle after spatial synchronization using the second matching factor; S38, for the obstacle successfully matched in S37, adding the category of the on-board camera target obstacle to the position data and reflection intensity data of the corresponding on-board laser radar obstacle and outputting it; S39, for the vehicle-mounted laser radar obstacle that failed to be matched in S37, the reflection intensity data and tracking duration of the corresponding vehicle-mounted laser radar obstacle in S32 are obtained, and when the tracking duration is greater than the first preset threshold or the average reflection intensity is greater than the second preset threshold, the position data and reflection intensity data of the corresponding vehicle-mounted laser radar obstacle are retained, otherwise they are deleted; The retained vehicle-mounted laser radar obstacle is output together with the vehicle-mounted laser radar obstacle data successfully matched in S38 as vehicle-mounted obstacle information.

6. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 5 is characterized in that: S4, generates vehicle-road cooperative perception results, including: S41, based on the timestamp of the vehicle-borne obstacle information, time synchronization is performed on the roadside obstacle information and the vehicle-borne obstacle information; S42, performing spatial synchronization on the time-synchronized roadside obstacle information and the vehicle-mounted obstacle information through spatial coordinate conversion; S43, calculating the intersection-and-joint ratio and distance of the spatially synchronized roadside obstacle information and the vehicle-mounted obstacle information to obtain a third matching factor; S44, using the Hungarian algorithm to optimally match the spatially synchronized roadside obstacle information and the vehicle-mounted obstacle information using the third matching factor; S45, for the obstacles successfully matched in S44, the corresponding roadside obstacle information is removed based on the vehicle-borne obstacle information; S46, for the roadside obstacle information that failed to be matched in S45, determine whether the corresponding roadside obstacle information is the unmanned vehicle itself, if yes, delete it, otherwise keep it; Keep the vehicle obstacle information that failed to match S45; S47, the vehicle-mounted obstacle information successfully matched in S45, the vehicle-mounted obstacle information that failed to match, and the roadside obstacle information retained in S46 are integrated to generate a vehicle-road collaborative perception result.

7. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 6 is characterized in that: S5, generate a global path, including: S51, obtaining a global map of the underground mine chute environment, the global map including the underground mine chute road network topology; S52, obtaining the current position of the unmanned vehicle as the starting point of the path planning, and obtaining the position of the chute unloading point as the target point of the path planning; S53, according to the vehicle-road cooperative perception result of S4, obtain the position, category and bounding box of the static obstacle; S54, adding the static obstacle position to the global map to obtain an updated global map S55, constructing a graph structure for path search according to the updated global map, where nodes in the graph are feature points in the road network, and edges are connections between feature points; S56, using A* or Dijkstra algorithm to search for the optimal path from the starting point to the target point in the graph structure as the global path.

8. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 7 is characterized in that: S6, generating a local path, including: S61, based on the S4 vehicle-road cooperative perception result, obtain the position, category and bounding box of the dynamic obstacle; S62, according to the dynamic obstacle position and the global path, obtaining a dynamic obstacle located on the global path or within a preset range of the global path as a target dynamic obstacle; S63, setting a stopping distance threshold according to the dynamic obstacle position obtained in S61; S64, taking the target dynamic obstacle and the stopping distance threshold as constraint conditions, taking the current position of the unmanned vehicle as the starting point, and taking the points within the preset range of the global path as the local target points, to construct a local graph structure for local path search; S65, in the local graph structure, using the Dijkstra algorithm or the A* algorithm to search for the optimal path from the starting point to the local target point as the local path.

9. The vehicle-road cooperative sensing and parking method in a mine chute environment according to claim 8 is characterized in that: S7, based on the global path and local path, controls the driving path, speed and parking of the unmanned vehicle, including: S71, controlling the driving of the unmanned vehicle through lateral control and longitudinal control according to the global path and the local path, wherein the lateral control adjusts the steering of the unmanned vehicle through a steering system, and the longitudinal control adjusts the speed of the unmanned vehicle through a throttle system and a brake system; S72, calculating the relative position relationship between the unmanned vehicle and the obstacle according to the vehicle-road cooperative perception result and the current position of the unmanned vehicle; S73, according to the relative position relationship of S72, when the obstacle is located in front of the unmanned vehicle, return to step S71, otherwise execute step S74; S74, according to the relative position relationship in S72, obtaining the obstacle with the shortest distance from the unmanned vehicle as the nearest obstacle; S75, judging whether the nearest obstacle is a retaining wall according to the vehicle-road cooperative perception result, if not, executing step S76, if yes, executing step S77; S76, controlling the parking of the unmanned vehicle; S77, controls the unmanned vehicle to reverse until the nearest obstacle is a retaining wall.

10. A vehicle-road cooperative sensing and parking system for an underground mine chute environment, characterized in that: include: At least one processing unit; used to execute instructions to implement the vehicle-road collaborative perception and docking method in an underground mine chute environment as described in any one of claims 1 to 9.

Images