Mine trackless vehicle anti-collision method and system based on target identification and structured light ranging

Abstract

The invention relates to the technical field of mine safety, in particular to a mine trackless vehicle anti-collision method and system based on target recognition and structured light ranging, and the method comprises the steps: collecting original point cloud pictures in all directions around a mine trackless vehicle based on a structured light camera; based on a panoramic looking-around method, splicing the original point cloud pictures in all directions to obtain a panoramic point cloud picture; based on the panoramic point cloud picture, pedestrian target detection is carried out by using a 3D point cloud target detection algorithm; and alarming and braking based on the pedestrian target detection result. The accuracy of the detection result can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mine safety, and particularly to an anti-person collision method and system for trackless vehicles in mines based on target recognition and structured light ranging. Background Art

[0002] During mine operations, the working environment is complex and space is limited, the passageways are narrow and crisscrossed, and the visibility is restricted. When a trackless vehicle travels in a narrow roadway, it is difficult to timely detect the personnel in the working area during the operation process, which may thus cause serious potential safety hazards to personnel. Traditional safety protection measures usually rely on the experience and reaction of drivers and operators. However, in such a highly uncertain and complex underground environment, relying solely on the experience and judgment of operators has significant limitations and it is difficult to comprehensively and effectively prevent accidents from occurring. In recent years, there have been many advancements in the method of anti-person collision for mine engineering vehicles, such as:

[0003] A method for predicting anti-collision between workers and machinery based on computer vision. By obtaining the monitoring video sequence of the target construction site, target detection algorithms are used to identify the machinery and workers in the monitoring video sequence; the historical movement trajectories of the machinery and workers are tracked, the subsequent movement trajectories of the machinery and workers are predicted based on the historical movement trajectories, the subsequent movement trajectories are compared, and it is determined whether there is a collision risk between the machinery and the workers according to the comparison result. The method adopted by this solution is to monitor the movement trajectories of the machinery and workers in the working area based on the video monitoring in the working environment, predict the movement trajectories of the machinery and pedestrians based on computer vision, and determine whether to give an alarm or perform emergency braking based on whether there is a possibility of the movement trajectories intersecting. Although this method can predict human behavior and give a prediction, it is greatly affected by the ambient light, there is a risk of unclear images, and due to the strong subjective initiative of people, the trajectory prediction is prone to errors, resulting in failure to brake in time, thus causing casualties. When the video analysis device processes data for the target vehicle, it needs to access the target vehicle through a wireless communication network to establish a video transmission channel, and this communication method has inherent defects of insufficient network stability and high communication latency.

[0004] For another example, a non-scanning automotive anti-collision lidar system based on structured light includes a signal processing component, an infrared CCD, a filter, and a laser emission component. The infrared CCD is equipped with a filter and only receives the wavelength emitted by the laser emission component. The signal processing component collects the output data of the infrared CCD, extracts and calculates the imaging of the emitted laser on the surface of the infrared CCD, and obtains the potential target contour and its distance in the radar irradiation area. This solution is based on structured light technology, uses a laser emitter to emit laser and an infrared CCD to receive the laser to measure the contour of objects in the target area, and measures the distance between obstacles and people and the vehicle to avoid collisions with people and vehicles. However, this technology is only applicable to urban roads and has not yet adapted to the mine application environment. It may produce multiple reflections in narrow spaces or interfere with the normal operation of the infrared camera. The working distance of the lidar is excellent beyond 5m, and the accuracy will decrease within 5m.

[0005] For another example, an omnidirectional intelligent anti-collision system for underground trackless equipment includes an edge computing and processing unit, a signal input / output unit, a handheld positioning card, a ranging radar unit, an infrared camera supplementary lighting unit, and a 360° omnidirectional positioning base station unit. The edge computing and processing unit is respectively connected to the signal input / output unit, the ranging radar unit, the infrared camera supplementary lighting unit, and the 360° omnidirectional positioning base station unit. The handheld positioning card transmits a narrow pulse signal to the 360° omnidirectional positioning base station unit, and the handheld positioning card and the 360° omnidirectional positioning base station unit transmit communication signals to each other. This solution uses UWB technology to implement the anti-collision system for underground trackless vehicles, uses a handheld positioning card to determine the position of personnel, and uses the edge processing and computing unit to measure the distance from the vehicle to achieve personnel anti-collision. However, this technology requires each person to wear a personnel positioning card, and without wearing it, the anti-collision effect cannot be achieved, and the installation cost is high. Moreover, when the positioning card is lost or fails, the system cannot detect the presence of personnel, and it is easy to cause danger. At the same time, in a complex environment, the signal may be reflected or refracted multiple times, resulting in multipath interference and affecting the ranging accuracy.

[0006] Currently, in the methods and systems for anti-collision of underground trackless vehicles against people, the measurement of the person, vehicle, and distance is mainly achieved based on UWB technology and target detection technology. With the emergence and development of structured light technology, real-time depth perception of people within the field of view can be performed through a structured light camera, and more accurate ranging of personnel can be carried out. Then, braking can be performed within a predetermined distance range to achieve the effect of anti-collision against people. Moreover, due to the high precision of structured light technology and the real-time and accuracy of short-distance ranging, the distance between the vehicle and the obstacle can be measured more accurately and quickly. Combining with target detection technology, people can be identified, and the function of stopping when a person is recognized and not stopping when a static obstacle such as a wall is recognized can be realized on the basis of general anti-collision. Currently, there is no method and system that directly applies structured light technology to the anti-collision of underground trackless vehicles under mine conditions. Summary of the invention

[0007] The purpose of the present invention is to propose a method and system for preventing trackless vehicles from hitting people in mines based on target recognition and structured light ranging, so as to solve the problems existing in the above-mentioned prior art and effectively improve the accuracy of detection results.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A method for preventing trackless vehicles from collision with people in mines based on target recognition and structured light ranging, comprising:

[0010] Based on the structured light camera, the original point cloud images in all directions around the mine trackless vehicle are collected;

[0011] Based on the panoramic viewing method, the original point cloud images in all directions are stitched together to obtain the panoramic point cloud image;

[0012] Based on the panoramic point cloud map, pedestrian target detection is performed using a 3D point cloud target detection algorithm;

[0013] Based on the pedestrian target detection results, alarm and braking are performed.

[0014] Optionally, based on a structured light camera, original point cloud images of trackless vehicles in all directions are collected, including:

[0015] After the emitted infrared light is reflected by the reference plane and the actual object, the distance from the actual target point to the camera plane is obtained using the principle of triangulation;

[0016] Based on the distance between the actual point of the target and the camera plane, the target is reconstructed in three dimensions to obtain the point cloud coordinates of the target surface;

[0017] The original point cloud image is constructed based on the target surface point cloud coordinates and the corresponding camera coordinate system parameters.

[0018] Optionally, the distance from the actual target point to the camera plane is:

[0019]

[0020] Where Z represents the distance from the actual target point to the camera plane, b is the distance between the infrared projector and the infrared camera, f is the focal length of the infrared camera, d is the difference in pixel position between the same point in the infrared camera image and the projector image, and Z0 is the depth of the reference plane.

[0021] Optionally, the target is reconstructed in three dimensions and the expression for obtaining the target surface point cloud coordinates is:

[0022]

[0023] Among them, (X, Y, Z) are the coordinates of the target surface point cloud, (u, v) are the pixel coordinates in the camera image, and f x and f y are the focal lengths of the structured light camera in the x-axis and y-axis directions, respectively.

[0024] Optionally, stitching the original point cloud maps in each direction to obtain a panoramic point cloud map includes:

[0025] Performing rough registration on each original point cloud map;

[0026] Based on the point cloud map after rough registration, performing fine registration on the point clouds of adjacent frames through the weighted ICP algorithm to optimize the poses of adjacent frames;

[0027] Performing voxel filtering on the registered point cloud to remove redundant points;

[0028] Based on the point cloud after voxel filtering, generating a seamless panoramic point cloud map using Poisson reconstruction or moving least squares method.

[0029] Optionally, performing rough registration on each original point cloud map includes:

[0030] Extracting the feature descriptors of each local point cloud and determining the initial transformation matrix through feature matching;

[0031] Based on the known spatial positions of the calibration boards, aligning the global coordinate systems of each camera and calculating the initial poses of the overlapping regions.

[0032] Optionally, using a 3D point cloud object detection algorithm for pedestrian object detection includes:

[0033] Preprocessing the panoramic point cloud map;

[0034] Inputting the preprocessed point cloud into a pre-trained deep learning model and outputting pedestrian candidate boxes and their confidence levels;

[0035] If there is a candidate box with a confidence level exceeding the preset threshold, it is determined that there is a pedestrian, and the three-dimensional bounding box coordinates of the pedestrian position are output.

[0036] Optionally, preprocessing the panoramic point cloud map includes:

[0037] First, performing voxel downsampling and statistical filtering on the original point cloud to eliminate noise;

[0038] Using the dynamic threshold RANSAC algorithm to segment the ground plane, and its distance threshold increases linearly with the point cloud height to adapt to terrain undulations;

[0039] For non-ground point clouds, combining the normal vector horizontal deviation angle constraint and the region growing algorithm to accurately extract the wall surface and vertical structures;

[0040] Optimize the ground point cloud by quadratic polynomial surface fitting and filter the residual abnormal points to solve the interference of non-ideal planes;

[0041] Extract dynamic objects using Euclidean clustering and assist with three-dimensional morphological opening operations to remove floating noise points;

[0042] Generate an expanded buffer along the segmentation boundary to ensure the integrity of the edges of dynamic objects and remove the static background point cloud in the scene.

[0043] An anti-collision system for trackless mine vehicles based on object recognition and structured light ranging, the system includes: an image acquisition module, an image processing module, an image display module, a voice alarm module, and a vehicle control module;

[0044] The image acquisition module is used to install structured light cameras around the trackless mine vehicle to collect the original point cloud maps in all directions of the trackless mine vehicle;

[0045] The image processing module is used to splice the original point cloud maps in all directions to obtain a panoramic point cloud map, and based on the panoramic point cloud map, use a 3D point cloud object detection algorithm to detect pedestrian targets;

[0046] The image display module is used to display the panoramic point cloud map and mark the bounding box of the pedestrian target on the screen to help the driver observe the surrounding environment;

[0047] The voice alarm module is used to issue an emergency alarm and prompt the driver of the location of the person when the vehicle is in motion and a pedestrian is detected in the running direction of the vehicle and the distance value is within the first preset threshold;

[0048] The vehicle control module is used to automatically perform an emergency brake on the vehicle when the vehicle is in motion and a pedestrian is detected in the running direction of the vehicle and the distance value is within the second preset threshold.

[0049] The beneficial effects of the present invention are:

[0050] The structured light technology based on infrared light can achieve high-precision and high-resolution distance measurement. By accurately projecting a structured infrared light pattern and analyzing its deformation, the distance between the person and the vehicle can be effectively measured. Due to the unique penetration characteristics of infrared light, the influence of insufficient light and dust occlusion in the mine on the measurement accuracy is effectively avoided, ensuring high-precision and high-stability structured light three-dimensional reconstruction. The entire system has low error and high robustness, can work stably in complex environments, and ensures accurate measurement results can be provided under different lighting conditions. Description of the Drawings

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 A method for preventing trackless vehicles from hitting people in mines based on target recognition and structured light ranging is provided in an embodiment of the present invention;

[0053] Figure 2 The triangulation principle of the embodiment of the present invention;

[0054] Figure 3 This is a flow chart of the vehicle control module effect according to an embodiment of the present invention;

[0055] Figure 4 The present invention provides a trackless vehicle collision avoidance system for mines based on target recognition and structured light ranging. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] Structured light: This technology uses a known light pattern (a dot pattern in this example) to project onto an object's surface, using the deformation of the light to measure the object's three-dimensional shape. This technology projects a specific light pattern onto the object, which deforms the light according to the surface's shape. A camera then captures this deformed light image, and the object's three-dimensional information is then calculated.

[0059] like Figure 1 As shown, this embodiment proposes a method for preventing trackless vehicles from hitting people in mines based on target recognition and structured light ranging, including:

[0060] Based on the structured light camera, the original point cloud images in all directions around the mine trackless vehicle are collected;

[0061] Based on the panoramic viewing method, the original point cloud images in all directions are stitched together to obtain the panoramic point cloud image;

[0062] Based on the panoramic point cloud map, pedestrian target detection is performed using the 3D point cloud target detection algorithm;

[0063] Based on the pedestrian target detection results, alarm and braking are performed.

[0064] Specifically, this embodiment installs structured light cameras with different orientations on the outer shell of a mine trackless vehicle, stitches the captured images into a panoramic surround view image, uses a target detection algorithm to determine whether there are pedestrians, and obtains the distance between pedestrians and vehicles based on structured light projection and triangulation. When the distance is too close, sound and light alarms and emergency braking measures are taken to prevent collisions with pedestrians.

[0065] Furthermore, the original point cloud images of the mine trackless vehicle in all directions are collected, including:

[0066] After the emitted infrared light is reflected by the reference plane and the actual object, the distance from the actual target point to the camera plane is obtained using the principle of triangulation;

[0067] Based on the distance between the actual point of the target and the camera plane, the target is reconstructed in three dimensions to obtain the point cloud coordinates of the target surface;

[0068] The original point cloud image is constructed based on the target surface point cloud coordinates and the corresponding camera coordinate system parameters.

[0069] Specifically, in this embodiment, the original point cloud images of the mine trackless vehicle in various directions are collected. First, structured light cameras with different orientations are installed on the outer shell of the mine trackless vehicle, including:

[0070] Install structured light cameras with a horizontal lens angle of 60° to 90° in locations that do not obstruct the vehicle's field of view, ensuring they can clearly detect all scenes within the current field of view. If using cameras with other field of view angles, calculate 360° / (horizontal field of view angle), round up the value, and install the corresponding number of cameras.

[0071] Secondly, the original point cloud images in all directions are obtained through the structured light camera:

[0072] (1) System calibration:

[0073] Obtain the focal length parameters of the image acquisition module. In this embodiment, the image acquisition module uses an infrared camera in a structured light camera. x 、f y are the focal lengths of the image acquisition module in the x-axis and y-axis directions, respectively.

[0074] Calculate the disparity d:

[0075] Translate along the z-axis within the depth range of 0 - 10m in front of the camera in advance. Taking 0.01m intervals as the standard, establish reference planes, and extract infrared speckle images for each reference plane. Shoot the target speckle image of the actual scene, and then match the target speckle image with the speckle images of the reference planes through a correlation function. In this case, the speckle image of the reference plane obtained at a known distance is called the reference image, and the reference disparity value of the reference image can be calculated according to the following formula:

[0076]

[0077] Among them, d Ref is the reference disparity value of the reference image, b is the baseline length, which is the distance between the infrared projector and the infrared camera, f is the focal length of the camera, and Z Ref is the distance between the reference plane and the camera;

[0078] Then, similarly, for each valid matching point p(x, y) in the target speckle image captured by the camera, the corresponding reference point p Ref (x Ref , y Ref ) with the maximum correlation value can be found in the reference image. The relationship between the coordinate values (x, y) of point p and the coordinate values (x Ref , y Ref ) of p Ref ) is:

[0079] (x, y) = (x Ref + d Rel , y Ref );

[0080] Among them, x Ref is the x-axis coordinate of the corresponding reference point on the reference plane, and y Ref is the y-axis coordinate of the corresponding reference point on the reference plane;

[0081] In the above formula, d Rel is the relative disparity value of the target image with respect to the reference image. Thus, the actual disparity value of each valid target point can be obtained in this way:

[0082] d = d Ref + d Rel .

[0083] (2) Calculate the depth:

[0084] After calibration, use the Figure 2 principle of triangulation to calculate the depth information through the deformation of the camera image and the structured light pattern:

[0085] After the infrared light emitted by the structured light projection module is reflected by the reference plane and the actual object, two points are obtained in the image receiving module. The difference between the positions of the two matching points is used as the parallax d, and the distance from the actual point of the object to the camera plane is called the depth Z. The formula for the depth Z is as follows:

[0086]

[0087] Note: b is the baseline length, which is the distance between the infrared projector and the infrared camera; f is the focal length of the infrared camera; d is the parallax (i.e., the pixel position difference of the same point in the infrared camera image and the projector image); Z0 is the depth of the reference plane.

[0088] (3) 3D reconstruction, obtaining the point cloud coordinates P(X, Y, Z) of the object surface according to the following formula;

[0089]

[0090] (u, v) are the pixel coordinates in the camera image.

[0091] Then, the obtained original point cloud map is subjected to point cloud stitching to obtain the panoramic point cloud map. It includes:

[0092] (1) Acquisition of multi-viewpoint cloud data;

[0093] Local point cloud data is collected through each structured light camera, and each point cloud data contains three-dimensional coordinates (X, Y, Z) and the corresponding camera coordinate system parameters;

[0094] (2) Coarse registration;

[0095] Feature descriptors (such as FPFH, SHOT) of each local point cloud are extracted, and the initial transformation matrix T init =[R|t];

[0096] R represents the rotation matrix, and t is the translation vector.

[0097] Based on the known spatial position of the calibration board, the global coordinate systems of each camera are aligned, and the initial pose of the overlapping area is calculated;

[0098] (3) Fine registration;

[0099] The point clouds of adjacent frames are finely registered through the weighted ICP algorithm to optimize the poses of adjacent frames, realizing the precise stitching between multiple frames of point clouds and obtaining a complete point cloud map. Through dynamic weight adjustment, the registration weights of noise points and non-overlapping areas are reduced;

[0100] (4) Point cloud fusion and optimization;

[0101] Voxel filtering is performed on the registered point cloud to remove redundant points;

[0102] Generate a seamless panoramic point cloud map based on Poisson reconstruction or moving least squares (MLS), and output it in a standardized format.

[0103] Then, use a 3D point cloud object detection algorithm on the panoramic point cloud map to determine whether there are pedestrians; including:

[0104] (1) Point cloud data preprocessing;

[0105] Perform voxel grid downsampling (resolution 50 - 200mm) on the input original point cloud map, and remove outliers through statistical filtering;

[0106] (2) 3D object detection model inference;

[0107] Input the preprocessed point cloud into a pre-trained deep learning model, and output pedestrian candidate boxes and their confidence levels. The model is a neural network based on the PointNet++ or VoxelNet architecture;

[0108] (3) Pedestrian presence determination;

[0109] If there is a candidate box with a confidence level exceeding the threshold (≥0.8), it is determined that there are pedestrians, and the three-dimensional bounding box coordinates (X min , Y min , Z min , X max , Y max , Z max ) of the pedestrian position are output.

[0110] Among them, when using a 3D point cloud object detection algorithm on the panoramic point cloud map to determine whether there are pedestrians, the background point cloud can be removed, and then the 3D point cloud object detection algorithm can be used to determine whether there are pedestrians; including:

[0111] (1) Background point cloud removal;

[0112] First, perform voxel downsampling and statistical filtering on the original point cloud to eliminate noise; subsequently, use the dynamic threshold RANSAC algorithm to segment the ground plane, and its distance threshold increases linearly with the point cloud height to adapt to terrain undulations; for non-ground point clouds, combine the normal vector horizontal deviation angle constraint and the region growing algorithm to accurately extract walls and vertical structures; optimize the ground point cloud through quadratic polynomial surface fitting and filter residual abnormal points to solve the interference of non-ideal planes; use Euclidean clustering to extract dynamic objects, and supplement with three-dimensional morphological opening operations to remove floating noise points; finally, generate an expanded buffer zone along the segmentation boundary to ensure the integrity of the dynamic object edges and remove static background point clouds in the scene.

[0113] (2) Pedestrian target detection;

[0114] Analyze the remaining point cloud through the 3D point cloud target detection algorithm, and output the determination result of whether there are pedestrians and the three-dimensional position information of the pedestrians.

[0115] Finally, make alarm and braking reactions according to the target detection result and the depth detection result. Include:

[0116] Such as Figure 3 shown; when the vehicle is in a moving state, when it is detected that there are people in the running direction of the vehicle and the distance value is between 5 meters and 10 meters, access the voice alarm module, and the voice alarm module issues an emergency alarm and prompts the driver of the location of the person.

[0117] When the vehicle is in a running state, when it is detected that a person is in the running direction of the vehicle and the distance value is less than 5 meters, access the vehicle control module, and the vehicle control module triggers the brake system module to automatically perform an emergency brake on the vehicle to prevent a collision.

[0118] The structured light technology based on infrared light in this embodiment can achieve high-precision and high-resolution distance measurement. By precisely projecting a structured infrared light pattern and analyzing its deformation, the distance between the person and the vehicle can be effectively measured. Due to the unique penetration characteristics of infrared light, the influence of strong environmental light sources in the mine (such as coal dust, searchlights, etc.) on the measurement accuracy is effectively avoided, ensuring high-precision and high-stability structured light three-dimensional reconstruction. The entire system has low error and high robustness, can work stably in complex environments, and can ensure accurate measurement results under different lighting conditions.

[0119] Compared with the anti-collision method using mine personnel positioning, the present invention does not require personnel to wear positioning cards, has a wider range of application scenarios, does not need to consider system compatibility issues, and reduces system investment.

[0120] Adopt panoramic image stitching to monitor the pedestrians around the vehicle in real time, which can detect pedestrians omnidirectionally and without dead angles, and judge whether the person in the real world is in the running direction of the vehicle according to the position of the person in the panoramic image. It can quickly identify and locate the target in a complex dynamic environment and meet the requirements of high timeliness.

[0121] Adopt the point cloud target detection algorithm to perform target detection on the point cloud map obtained by the structured light camera, reduce the image processing steps, improve the processing speed, and effectively avoid the disadvantage of slow operation speed caused by multi-step image conversion.

[0122] The roadway space is narrow, and traditional collision detection methods are not applicable. The optimization control module of the present invention effectively avoids false alarms caused by non-target objects such as walls and equipment by introducing advanced person recognition, thereby preventing unnecessary emergency braking triggers. This control module can comprehensively utilize sensor data for accurate analysis, distinguish real threats from other obstacles, and dynamically adjust the braking strategy. It ensures that emergency braking is only activated in real dangerous situations, thereby improving overall safety and operational fluency and avoiding unnecessary risks caused by false alarms.

[0123] As Figure 4 shown, this embodiment also proposes a mine trackless vehicle anti-collision system based on target recognition and structured light ranging. The system includes: an image acquisition module, an image processing module, an image display module, a voice alarm module, and a vehicle control module;

[0124] The image acquisition module is used to install structured light cameras around the mine trackless vehicle to collect the original point cloud maps in all directions of the mine trackless vehicle;

[0125] The image processing module is used to splice the original point cloud maps in all directions to obtain a panoramic point cloud map, and based on the panoramic point cloud map, use a 3D point cloud target detection algorithm to detect pedestrian targets;

[0126] The image display module is used to display the panoramic point cloud map and mark the bounding box of the pedestrian target on the screen to help the driver observe the surrounding environment;

[0127] The voice alarm module is used to issue an emergency alarm and prompt the driver of the location of the person when the vehicle is in motion and a pedestrian is detected in the running direction of the vehicle and the distance value is within the first preset threshold;

[0128] The vehicle control module is used to automatically perform emergency braking of the vehicle when the vehicle is in motion and a pedestrian is detected in the running direction of the vehicle and the distance value is within the second preset threshold.

[0129] This embodiment is based on structured light technology. Structured light cameras with different orientations are installed on the shell of the mine trackless vehicle. The collected images are spliced into a panoramic view, and a target detection algorithm is used to determine whether there are pedestrians. The distance between the pedestrian and the vehicle is obtained based on structured light projection and point cloud detection. When the distance is too close, sound and light alarms and emergency braking measures are taken to prevent collisions with pedestrians. Specifically:

[0130] 1. Apply structured light ranging to the mine trackless vehicle anti-collision system to monitor the distance between the underground trackless vehicle and pedestrians using structured light.

[0131] 2. Obtain the surrounding images through at least four cameras in the front, back, left, and right directions, perform panoramic view stitching, and determine the orientation of the person.

[0132] 3. Use the panoramic point cloud map for target recognition to obtain personnel presence information and position distance.

[0133] 4. After removing the background point cloud from the panoramic point cloud map, perform target recognition to improve the recognition accuracy.

[0134] 5. According to different vehicle operating states, the control module takes different actions. When a person is detected in the running direction, an alarm is issued or emergency braking is performed. When a person is detected in the opposite direction of the running direction, no action is taken.

[0135] Compared with the method and system for anti-collision prediction of workers and machinery based on computer vision in the background technology, this embodiment collects images on the vehicle and analyzes whether there are personnel in the vehicle running direction, which has stronger adaptability to the mine working environment and higher safety factor. Compared with the anti-collision lidar system for vehicles without scanning and its working method based on structured light in the background technology, this embodiment adapts to the mine working environment. In a narrow roadway, when a person is detected within 0 - 10m, an alarm and emergency braking will be performed, and no braking will be performed when a wall is detected, realizing the function of anti-collision of trackless vehicles in mines in a narrow roadway. The measurement accuracy of the UWB technology used in an omnidirectional intelligent anti-collision system and method for underground trackless equipment in the background technology is lower than the structured light ranging technology used in this embodiment.

[0136] The above-described embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for preventing a trackless vehicle in a mine from hitting people based on target recognition and structured light ranging, characterized in that, include: Based on the structured light camera, the original point cloud images in all directions around the mine trackless vehicle are collected; Based on the panoramic viewing method, the original point cloud images in all directions are stitched together to obtain the panoramic point cloud image; Based on the panoramic point cloud image, pedestrian target detection is performed using a 3D point cloud target detection algorithm; Based on the pedestrian target detection results, alarm and braking are performed.

2. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 1, characterized in that, Based on the structured light camera, the original point cloud images of the mine trackless vehicle in all directions are collected, including: Calibrate the structured light camera and emit infrared light towards the target; After the emitted infrared light is reflected by the reference plane and the actual target, the distance from the actual target point to the camera plane is obtained using the principle of triangulation; Based on the distance between the actual point of the target and the camera plane, the target is reconstructed in three dimensions to obtain the point cloud coordinates of the target surface; The original point cloud image is constructed based on the target surface point cloud coordinates and the corresponding camera coordinate system parameters.

3. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 2, wherein The distance from the actual target point to the camera plane is: Where Z represents the distance from the actual target point to the camera plane, b is the distance between the infrared projector and the infrared camera, f is the focal length of the infrared camera, d is the difference in pixel position between the same point in the infrared camera image and the projector image, and Z0 is the depth of the reference plane.

4. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 2, characterized in that, Perform three-dimensional reconstruction on the target and obtain the expression of the target surface point cloud coordinates: where (X, Y, Z) are the coordinates of the target surface point cloud, (u, v) are the pixel coordinates in the camera image, and f x , f<000002>and f include: are the focal lengths of the structured light camera in the x-axis and y-axis directions, respectively.

5. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 1, characterized in that, The original point cloud images in various directions are stitched together to obtain the panoramic point cloud image, including: Perform coarse registration on each original point cloud image; Based on the roughly registered point cloud image, the weighted ICP algorithm is used to finely register the point clouds of adjacent frames to optimize the poses of adjacent frames. Perform voxel filtering on the registered point cloud to remove redundant points; For the point cloud after voxel filtering, a seamless panoramic point cloud map is generated based on Poisson reconstruction or moving least squares method.

6. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 5, characterized in that The coarse registration of each original point cloud image includes: Extract the feature descriptors of each local point cloud and determine the initial transformation matrix through feature matching; Based on the known spatial position of the calibration plate, the global coordinate systems of each camera are aligned and the initial pose of the overlapping area is calculated.

7. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 1, wherein Pedestrian target detection using 3D point cloud target detection algorithms includes: Preprocessing the panoramic point cloud image; Input the pre-processed point cloud into the pre-trained deep learning model and output the pedestrian candidate box and its confidence score; If there is a candidate box whose confidence exceeds the preset threshold, it is determined that a pedestrian exists, and the three-dimensional bounding box coordinates of the pedestrian's location are output.

8. The anti-collision method for trackless mine vehicles based on target recognition and structured light ranging according to claim 1, characterized in that Preprocessing the panoramic point cloud image includes: First, the original point cloud is subjected to voxel downsampling and statistical filtering to remove noise; The dynamic threshold RANSAC algorithm is used to segment the ground plane, and its distance threshold increases linearly with the point cloud height to adapt to the terrain undulation; For non-ground point clouds, the normal vector horizontal deflection constraint and region growing algorithm are combined to accurately extract walls and vertical structures; Optimize the ground point cloud and filter residual abnormal points through quadratic polynomial surface fitting to solve non-ideal plane interference; Use Euclidean clustering to extract dynamic objects, and use 3D morphological opening to remove suspended noise. Generate an outward expansion buffer along the segmentation boundary to ensure the edge integrity of dynamic objects and remove static background point clouds in the scene.

9. A mine trackless vehicle anti-collision system based on target recognition and structured light ranging, characterized in that, For implementing the method for preventing collision with pedestrians of a trackless mine vehicle as described in any one of claims 1-8, the system includes: an image acquisition module, an image processing module, an image display module, a voice alarm module, and a vehicle control module; The image acquisition module is configured to install structured light cameras around the trackless mine vehicle to acquire original point cloud maps in all directions of the trackless mine vehicle; The image processing module is configured to splice the original point cloud maps in all directions to obtain a panoramic point cloud map, and perform pedestrian target detection using a 3D point cloud target detection algorithm based on the panoramic point cloud map; The image display module is configured to display the panoramic point cloud map and mark the bounding box of the pedestrian target on the screen to help the driver observe the surrounding environment; The voice alarm module is configured to, when the vehicle is in a moving state, issue an emergency alarm and prompt the driver of the location of the pedestrian when a pedestrian is detected in the running direction of the vehicle and the distance value is within a first preset threshold; The vehicle control module is configured to, when the vehicle is in a moving state, automatically perform an emergency braking of the vehicle when a pedestrian is detected in the running direction of the vehicle and the distance value is within a second preset threshold.

Images