Obstacle detection method for mine and program product

Abstract

The invention discloses an obstacle detection method for a mine and a program product. The method comprises the following steps: acquiring environmental data in a mine driving environment of a transport locomotive based on a plurality of data acquisition devices deployed on the transport locomotive in the mine; obstacle identification is carried out on the environment data acquired by each data acquisition device, so that obstacle data corresponding to each data acquisition device can be obtained, according to the obstacle data corresponding to the plurality of data acquisition devices, respectively determining a detection source, a category and a moving speed corresponding to each target obstacle in the mine driving environment, wherein the detection source is used for indicating the data acquisition device which detects the target obstacle; and according to the total number of the detection sources corresponding to the target obstacle, the category of the target obstacle and the moving speed, executing an obstacle response operation corresponding to the transport locomotive so as to improve the accuracy of obstacle detection.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an obstacle detection method and program product for use in mines. Background Technology

[0002] Underground mining environments are complex and highly dangerous, with safety hazards consistently posing significant challenges to operations and the safety of personnel. Therefore, the research and widespread application of unmanned driving technology in mines is of paramount importance. This technology can directly reduce the frequency of manual underground operations, lower safety risks for personnel, and ensure more stable and efficient underground workflows, playing a crucial role in improving operational safety. While underground environments offer fewer obstacles and more varied scenarios compared to surface environments, the unique conditions still present numerous challenges: the enclosed and narrow underground space allows high concentrations of dust to adhere to equipment surfaces, and low ambient light significantly reduces the area of ​​clear observation, hindering the effective application of unmanned driving technology.

[0003] Existing obstacle detection methods in mines have significant shortcomings. When the concentration of dust underground suddenly increases, the sensing ability of some sensors rapidly declines, or they may even malfunction, failing to effectively extract key features of obstacles or making incorrect judgments due to signal interference. This drastically reduces the accuracy of obstacle detection, posing a serious threat to the operational safety of unmanned equipment. Therefore, there is an urgent need for an obstacle detection method for mines to improve the accuracy of obstacle detection. Summary of the Invention

[0004] This invention provides a method and program for obstacle detection in mines, which solves the problem that the accuracy of obstacle detection in mines decreases due to environmental factors such as high dust concentration and low ambient light.

[0005] According to one aspect of the present invention, a method for obstacle detection in a mine is provided, the method comprising:

[0006] Environmental data of the mine's operating environment is collected by a variety of data acquisition devices deployed on the transport locomotives in the mine. The data acquisition devices include lidar, millimeter-wave radar, and photosensitive cameras. The environmental data includes 3D point cloud data, millimeter-wave echo data, and environmental images.

[0007] Obstacle identification is performed on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each data acquisition device. Based on the obstacle data corresponding to the various data acquisition devices, the detection source, category, and movement speed corresponding to each target obstacle in the mine driving environment are determined. The detection source is used to indicate the data acquisition device that detected the target obstacle.

[0008] Based on the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, an obstacle response operation corresponding to the transport vehicle is performed, wherein the obstacle response operation includes braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle.

[0009] According to another aspect of the present invention, an obstacle detection device for mines is provided, the device comprising:

[0010] An environmental data acquisition module is used to collect environmental data of the mine driving environment of the transport locomotive based on a variety of data acquisition devices deployed on the transport locomotive in the mine. The data acquisition devices include lidar, millimeter-wave radar and photosensitive camera. The environmental data includes three-dimensional point cloud data, millimeter-wave echo data and environmental images.

[0011] An obstacle data determination module is used to identify obstacles in the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices. Based on the obstacle data corresponding to the various data acquisition devices, the module determines the detection source, category, and movement speed of each target obstacle in the mine driving environment. The detection source is used to indicate the data acquisition device that detected the target obstacle.

[0012] An obstacle response module is used to perform an obstacle response operation corresponding to the transport vehicle based on the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed. The obstacle response operation includes braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the obstacle detection method for mines according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the obstacle detection method for mines according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements an obstacle detection method for a mine as described in any of the embodiments of this disclosure.

[0019] The technical solution of this invention firstly involves collecting environmental data about the mine's operating environment from multiple data acquisition devices deployed on a transport locomotive in a mine. These data acquisition devices include lidar, millimeter-wave radar, and a photosensitive camera. The environmental data includes 3D point cloud data, millimeter-wave echo data, and environmental images. Acquiring multi-dimensional information from multiple data acquisition devices avoids the limitations of data collected by a single device and provides a more comprehensive reflection of the mine's operating environment. Next, obstacle identification is performed on the environmental data collected by each data acquisition device to obtain obstacle data corresponding to each device. Based on the obstacle data from the multiple data acquisition devices, the detection source, category, and movement speed of each target obstacle in the mine's operating environment are determined. The detection source is used to indicate... The data acquisition device detects the target obstacle; based on multi-data cross-validation, it reduces the error that may occur in single-data identification, improves the accuracy of obstacle identification, and obtains multi-dimensional information about the target obstacle to provide sufficient basis for subsequent obstacle response operations; finally, by executing obstacle response operations corresponding to the transport vehicle according to the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, wherein the obstacle response operations include braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle; by combining the target obstacle information to accurately take countermeasures, it effectively prevents the transport vehicle from colliding with the obstacle. The multiple data acquisition devices complement each other in the target obstacle detection process, greatly reducing the risk of collision and providing technical support for mine transportation safety.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of an obstacle detection method for mines according to Embodiment 1 of the present invention;

[0023] Figure 2a This is a flowchart of an obstacle detection method for mines according to Embodiment 2 of the present invention;

[0024] Figure 2b This is a schematic diagram of an obstacle detection method for a mine according to an embodiment of the present invention, provided by Embodiment 2 of the present invention;

[0025] Figure 2c This is a schematic diagram of obstacle data fusion detected by lidar and millimeter-wave radar in an obstacle detection method for mines according to an embodiment of the present invention;

[0026] Figure 2d This is a schematic diagram of a laser radar projected onto a photosensitive camera to capture an image, according to an embodiment of the present invention.

[0027] Figure 2e This is a schematic diagram of obstacle data fusion detected by a lidar and a photosensitive camera in an obstacle detection method for mines according to an embodiment of the present invention;

[0028] Figure 2f This is a schematic diagram of the installation of a data acquisition device for an obstacle detection method in a mine according to an embodiment of the present invention, provided by Embodiment 2 of the present invention.

[0029] Figure 3 This is a schematic diagram of the structure of an obstacle detection device for mines according to Embodiment 3 of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the obstacle detection method for mines according to embodiments of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," "third," "fourth," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0034] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0035] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0036] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0037] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0038] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0039] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0040] Example 1

[0041] Figure 1 This is a flowchart of an obstacle detection method for mines provided in Embodiment 1 of the present invention. This embodiment is applicable to the detection of obstacles in mines. The method can be executed by an obstacle detection device for mines, which can be implemented in hardware and / or software, optionally through electronic devices, such as mobile terminals, PCs, or servers. Figure 1 As shown, the method may specifically include:

[0042] S110. Environmental data of the mine driving environment of the transport locomotive is collected based on a variety of data acquisition devices deployed on the transport locomotive in the mine. The data acquisition devices include lidar, millimeter-wave radar and photosensitive camera. The environmental data includes three-dimensional point cloud data, millimeter-wave echo data and environmental images.

[0043] In this embodiment of the invention, a transport locomotive can be understood as a vehicle used inside a mine to transport materials such as coal, ore, equipment, or personnel. The various data acquisition devices deployed on the transport locomotive in the mine can be various devices installed on the transport locomotive to collect information about the environment surrounding the transport locomotive. For example, the data acquisition devices can include, but are not limited to, various types of devices such as lidar, millimeter-wave radar, and photosensitive cameras. Lidar refers to a sensor that uses laser technology to detect the position, speed, and other characteristics of target obstacles. It generates three-dimensional point cloud information of the target by emitting a laser beam and receiving the reflected laser signal. Millimeter-wave radar refers to a radar device that operates in the millimeter-wave frequency band, emitting millimeter waves and receiving the echo signal reflected from the target to obtain relevant information about target obstacles. A photosensitive camera is a device that can sense changes in light and convert them into image signals. It can capture visual information in the mine's operating environment, providing image evidence for obstacle identification. For example, photosensitive cameras can include, but are not limited to, various types of devices such as visible light cameras and infrared cameras.

[0044] The mine operating environment refers to the surrounding space of a transport vehicle when it travels within a mine. This environment may include roadways, tracks, other equipment, and potential obstacles. Environmental data can be information collected by data acquisition devices that reflects the mine operating environment. Environmental data can include, but is not limited to, various types of data such as 3D point cloud data, millimeter-wave echo data, and environmental images. 3D point cloud data, acquired by lidar, is a collection of numerous discrete points in three-dimensional space. Each point contains spatial location information, reflecting the 3D shape and position of target obstacles. Millimeter-wave echo data is data formed by the signal reflected back from a target obstacle after millimeter-wave radar emits millimeter waves. Millimeter-wave echo data can include information such as the distance, speed, and reflection intensity of the target obstacle. Environmental images refer to images captured by a photosensitive camera that visually represent the mine operating environment, allowing for direct observation of objects and obstacles within the environment.

[0045] Optionally, environmental data of the transport locomotive's operating environment in the mine can be collected using various data acquisition devices deployed on the locomotive. For example, three-dimensional point cloud data of the transport locomotive's operating environment can be collected using lidar, millimeter-wave echo data of the operating environment can be collected using millimeter-wave radar, and environmental images of the operating environment can be collected using a photosensitive camera. The lidar, millimeter-wave radar, and photosensitive camera are all installed on the transport locomotive, collecting surrounding environmental data in real time as the locomotive moves, providing comprehensive data support for subsequent obstacle detection in the mine operating environment.

[0046] S120. Obstacle identification is performed on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices. Based on the obstacle data corresponding to the various data acquisition devices, the detection source, category, and movement speed corresponding to each target obstacle in the mine driving environment are determined respectively. The detection source is used to indicate the data acquisition device that detected the target obstacle.

[0047] The obstacle data can be information related to obstacles obtained through obstacle identification. The obstacle data content may be the same or different for different data acquisition devices. The target obstacle can be a detected obstacle that may affect the safe operation of the transport vehicle. The detection source can be a data acquisition device used to indicate the detection of the target obstacle. The category can be a classification of the target obstacle according to its nature, shape, and other characteristics; for example, it can be divided into multiple categories such as personnel and equipment. The movement speed refers to how fast the target obstacle moves in the mine's operating environment.

[0048] Optionally, the environmental data collected by each data acquisition device can be analyzed and processed separately to identify whether there are obstacles in the environmental data and the relevant characteristic information of the obstacles, so as to obtain the obstacle data corresponding to each data acquisition device.

[0049] Based on the above scheme, optionally, the obstacle data corresponding to the lidar includes target clusters of target obstacles and the first velocity information of the target obstacles; the step of performing obstacle identification on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices includes: performing ground detection on the three-dimensional point cloud data collected by the lidar, and removing the detected ground points from the three-dimensional point cloud data; obtaining the ground height at each location within the lidar detection range in the mine, and filtering out the roadway top points in the three-dimensional point cloud data according to the ground height and a preset height difference threshold; clustering the three-dimensional point cloud data from which the ground points and roadway top points have been filtered out to obtain multiple obstacle clusters, selecting the target clusters of the target obstacles from the obstacle clusters, and tracking the target clusters of the target obstacles to obtain the first velocity information of the target obstacles.

[0050] The target cluster can be a set of tightly clustered point cloud data representing target obstacles, obtained by clustering the 3D point cloud data acquired by LiDAR. The target cluster reflects the approximate outline and location of the target obstacles. The first velocity information can be understood as the movement velocity information of the target obstacles obtained by tracking and analyzing the target clusters of target obstacles acquired by LiDAR.

[0051] Specifically, the three-dimensional point cloud data collected by lidar can be analyzed and processed to identify point cloud data representing the ground, and ground points that can represent the location of the mine surface can be removed from the three-dimensional point cloud data.

[0052] The detection range of a lidar system can be the spatial area within which the lidar can emit laser beams and receive reflected signals to detect obstacles. Ground height can be the vertical distance of each location within the lidar's detection range in the mine relative to a reference surface at the bottom of the mine; for example, it can be the elevation or relative height of the ground location.

[0053] Specifically, the ground elevation at each location within the detection range of the lidar in the mine can be determined. Based on a preset height difference threshold, it can be determined whether a point in the 3D point cloud data is a point at the top of the mine roadway, and points at the top of the roadway can be filtered out. Here, a point at the top of the roadway refers to a point in the 3D point cloud data collected by the lidar that represents the position of the top of the mine roadway.

[0054] Furthermore, the remaining 3D point cloud data after filtering out ground points and tunnel top points can be clustered using various clustering methods such as raster clustering or Euclidean clustering. Point cloud data that are spatially close to each other and have similar characteristics are grouped into one category, resulting in non-top and non-ground point cloud clusters and their envelopes, i.e., multiple obstacle clusters. Each obstacle cluster can be composed of a group of spatially closely adjacent point clouds to represent obstacle information.

[0055] Based on obtaining multiple obstacle clusters, these clusters can be filtered according to their attributes. For example, if the length, width, and height of an obstacle cluster are greater than a preset size threshold, it can be considered to belong to the point cloud cluster of the lane boundary wall and filtered out, thus obtaining the target cluster of the target obstacle. The target cluster of the filtered target obstacle is then detected, and information such as the position change of the target cluster is tracked in real time to calculate parameters such as the moving speed of the target obstacle.

[0056] Optionally, the target clusters of obstacles can also be tracked to obtain the three-dimensional position coordinates of the target obstacles in a three-dimensional coordinate system established by the lidar, as well as the size information of the target obstacles.

[0057] By removing ground points and tunnel top points from the 3D point cloud data, the interference of the fixed environmental structure in the mine can be effectively eliminated on obstacle recognition. Then, by clustering, filtering and tracking to obtain target clusters and first velocity information, the accuracy of obstacle data extraction is improved, and comprehensive basic data including position and motion state is provided for subsequent fusion. This avoids the impact of invalid point clouds on the detection results and can be adapted to the structural characteristics of closed underground scenes.

[0058] Based on the above scheme, optionally, the obstacle data corresponding to the millimeter-wave radar includes the two-dimensional position coordinates, second velocity information, and category of the target obstacle; the step of performing obstacle identification on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices includes: filtering out millimeter-wave echo data with echo intensity lower than a preset intensity threshold from the millimeter-wave echo data collected by the millimeter-wave radar to obtain the two-dimensional position coordinates and second velocity information of the target obstacle; classifying the target obstacle according to the second velocity information of the target obstacle and the echo intensity in the millimeter-wave echo data corresponding to the target obstacle to obtain the category of the target obstacle.

[0059] The two-dimensional position coordinates refer to the coordinate values ​​used to represent the position of the target obstacle in a two-dimensional coordinate system established based on millimeter-wave radar. The second velocity information can be the movement velocity information of the target obstacle obtained by analyzing and processing the millimeter-wave echo data collected by the millimeter-wave radar.

[0060] Echo intensity can be understood as the strength of the echo signal reflected back after millimeter waves emitted by a millimeter-wave radar encounter a target obstacle. The magnitude of the echo intensity is related to factors such as the material, surface shape, and distance of the target obstacle. Optionally, the millimeter-wave echo data collected by the millimeter-wave radar can be filtered according to a preset intensity threshold. Millimeter-wave echoes with intensity below the preset threshold can be considered false detections and filtered out. Then, the two-dimensional position coordinates and second velocity information of the target obstacle can be calculated based on the echo intensity.

[0061] Furthermore, the target obstacle can be classified based on its second velocity information and the echo intensity in the corresponding millimeter-wave echo data, thus obtaining the obstacle's category. The category can be flexibly set according to the actual business scenario requirements; for example, target obstacles with higher second velocity information or higher echo intensity can be classified as motor vehicles, while those with lower second velocity information or lower echo intensity can be classified as pedestrians.

[0062] By filtering out millimeter-wave echo data with echo intensity below a threshold, false data caused by interference factors such as dust and noise can be reduced, ensuring the accuracy of the two-dimensional position coordinates and second velocity information of the target obstacle. Combining velocity information and echo intensity to classify the target allows the obstacle category judgment to better reflect the actual situation downhole, improving the relevance of millimeter-wave radar data.

[0063] Based on the above scheme, optionally, the obstacle data corresponding to the photosensitive camera includes the region bounding box of the target obstacle in the environmental image, the category of the target obstacle, and the confidence level of the category; the step of performing obstacle identification on the environmental data collected by each of the data acquisition devices to obtain the obstacle data corresponding to each of the data acquisition devices includes: performing target obstacle detection on the environmental image collected by the photosensitive camera according to the image detection model to obtain the region bounding box of the target obstacle in the environmental image, the category of the target obstacle, and the confidence level of the category.

[0064] The bounding box can be a frame used to select the area containing a target obstacle in an environmental image captured by a photosensitive camera. The bounding box can be used to label the location and size range of the target obstacle in the environmental image. The image detection model can be a model that analyzes and processes the input environmental image and labels the specific location and category of the target obstacle. The image detection model can be trained based on sample environmental images and preset labeling information, which may include the bounding boxes of the target obstacles in the sample environmental images and the category of the target obstacle. Model confidence is set to ensure the model's recognition accuracy. For example, the image detection model can be a YOLOv5 model.

[0065] Specifically, target obstacles can be detected in environmental images captured by a photosensitive camera based on an image detection model, generating bounding boxes of the target obstacles in the environmental images and the category of the target obstacles. At the same time, the labeling model determines the confidence level of the category to which the detected target obstacles belong.

[0066] For example, if the photosensitive camera includes a visible light camera and an infrared camera, the environmental image it acquires may include a visible light image and an infrared light image. Then, based on the image detection model, target obstacle detection can be performed, and corresponding target obstacle data for the visible light camera and target obstacle data for the infrared light camera can be generated.

[0067] By using an image detection model to detect environmental images, the bounding boxes of target obstacles can be quickly located. At the same time, the category and confidence score are output, which simplifies the processing flow of photosensitive camera data. The confidence score can also provide a reliability reference for subsequent data fusion, making obstacle recognition of image data more operable in complex environments such as low light and dust, and adapting to the needs of underground visual perception.

[0068] Optionally, the two-dimensional position coordinates of the target obstacle in the photosensitive camera image coordinate system constructed based on the photosensitive camera can also be determined based on the region annotation box of the detected target obstacle in the environmental image.

[0069] Optionally, based on the target obstacle categories determined by multiple data acquisition devices, and considering factors such as the detection accuracy and data reliability of different data acquisition devices in the current scenario, corresponding weighting coefficients can be assigned to each data acquisition device. Weighted fusion can then be performed to determine the final target obstacle category, thereby improving the accuracy of category determination. Furthermore, if the photosensitive camera includes a visible light camera, and the obstacle data from multiple data acquisition devices includes obstacle data from the visible light camera, under conditions of good underground lighting and clear visible light image details, the target obstacle category determined by the visible light camera can be directly determined as the target obstacle category. Alternatively, the weight of the visible light camera in determining the target obstacle category can be adjusted accordingly to better adapt to the complex and ever-changing driving environment in mines, ensuring the rationality and reliability of obstacle category determination.

[0070] Among them, lidar can detect the lateral velocity of the target obstacle, and millimeter-wave radar can detect the longitudinal velocity of the target obstacle. This allows for the calculation of the target obstacle's moving speed and its relative speed along the transport vehicle's direction of movement. If only lidar or millimeter-wave radar detects the target obstacle's speed, the speed information in the other direction can be set to 0 or a preset speed.

[0071] S130. Based on the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, perform an obstacle response operation corresponding to the transport vehicle, wherein the obstacle response operation includes braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle.

[0072] Obstacle response operations can be measures taken to ensure the safety of transport vehicles based on relevant information about the target obstacle. Obstacle response operations may include, but are not limited to, braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle. Braking refers to actions taken to slow down or stop the transport vehicle to avoid a collision with the target obstacle. Alarm information can be generated based on the detected target obstacle and is used to alert relevant personnel to the potential obstacle risk.

[0073] Specifically, a comprehensive analysis can be performed based on the total amount of data from the detection sources corresponding to the target obstacle. Cross-validation across multiple detection sources can determine whether the target obstacle actually exists, avoiding misjudgments caused by errors in single detection source equipment or environmental interference. For example, if at least two detection sources simultaneously detect the target obstacle, its existence can be confirmed. Furthermore, for target obstacles detected by at least two detection sources, braking of the transport vehicle can be applied and / or alarm information corresponding to the target obstacle can be generated and displayed; for other target obstacles, only alarm information corresponding to the target obstacle can be generated and displayed.

[0074] Optionally, after confirming the actual existence of the target obstacle, the size of the target obstacle can be predicted based on its category, and then the obstacle response operation corresponding to the transport vehicle can be determined based on the size of the target obstacle.

[0075] Optionally, the moving speed of the target obstacle can be determined, and the moving speed of the transport vehicle can be obtained. The real-time distance between the target obstacle and the transport vehicle and the required braking distance of the transport vehicle can be calculated based on their relative moving speeds. The real-time distance and the braking distance can be compared, and the corresponding obstacle response operation can be performed, such as braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle through sound, light, images, etc.

[0076] Optionally, obstacle response operations corresponding to the transport vehicle can also be performed based on the total number of detection sources corresponding to the target obstacle, the type of the target obstacle, and the moving speed.

[0077] The technical solution of this invention firstly involves collecting environmental data about the mine's operating environment from multiple data acquisition devices deployed on a transport locomotive in a mine. These data acquisition devices include lidar, millimeter-wave radar, and a photosensitive camera. The environmental data includes 3D point cloud data, millimeter-wave echo data, and environmental images. Acquiring multi-dimensional information from multiple data acquisition devices avoids the limitations of data collected by a single device and provides a more comprehensive reflection of the mine's operating environment. Next, obstacle identification is performed on the environmental data collected by each data acquisition device to obtain obstacle data corresponding to each device. Based on the obstacle data from the multiple data acquisition devices, the detection source, category, and movement speed of each target obstacle in the mine's operating environment are determined. The detection source is used to indicate... The data acquisition device detects the target obstacle; based on multi-data cross-validation, it reduces the error that may occur in single-data identification, improves the accuracy of obstacle identification, and obtains multi-dimensional information about the target obstacle to provide sufficient basis for subsequent obstacle response operations; finally, by executing obstacle response operations corresponding to the transport vehicle according to the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, wherein the obstacle response operations include braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle; by combining the target obstacle information to accurately take countermeasures, it effectively prevents the transport vehicle from colliding with the obstacle. The multiple data acquisition devices complement each other in the target obstacle detection process, greatly reducing the risk of collision and providing technical support for mine transportation safety.

[0078] Example 2

[0079] Figure 2 is a flowchart of an obstacle detection method for mines provided in Embodiment 2 of the present invention, further describing the implementation process of determining the detection source corresponding to each target obstacle in the mine driving environment based on obstacle data corresponding to various data acquisition devices. Specific implementation details can be found in the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. As shown in Figure 2, the method may specifically include:

[0080] S210. Environmental data of the mine driving environment of the transport locomotive is collected based on a variety of data acquisition devices deployed on the transport locomotive in the mine. The data acquisition devices include lidar, millimeter-wave radar and photosensitive camera. The environmental data includes three-dimensional point cloud data, millimeter-wave echo data and environmental images.

[0081] S220. Obstacle identification is performed on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices.

[0082] S230. The obstacle data corresponding to the lidar and the obstacle data corresponding to the millimeter-wave radar are fused to obtain the first detection source data of the target obstacle in the mine driving environment. The first detection source data includes single-laser obstacle detection data, single-millimeter-wave obstacle detection data and dual-radar obstacle detection data.

[0083] The first detection source data can be a dataset representing target obstacle detection source information in the mine driving environment, obtained by fusing obstacle data corresponding to lidar and obstacle data corresponding to millimeter-wave radar. The first detection source data includes single-laser detection obstacle data, single-millimeter-wave detection obstacle data, and dual-radar detection obstacle data. Single-laser detection obstacle data refers to target obstacle data that is detected only by lidar in the first detection source data but not by millimeter-wave radar. Single-millimeter-wave detection obstacle data refers to target obstacle data that is detected only by millimeter-wave radar in the first detection source data but not by lidar. Dual-radar detection obstacle data refers to target obstacle data that is detected by both lidar and millimeter-wave radar in the first detection source data.

[0084] Optionally, obstacle data corresponding to lidar and obstacle data corresponding to millimeter-wave radar can be comprehensively analyzed and integrated to obtain the first detection source data of the target obstacle.

[0085] Based on the above scheme, optionally, the obstacle data corresponding to the lidar includes three-dimensional point cloud data, and the obstacle data corresponding to the millimeter-wave radar includes two-dimensional position coordinates; the step of fusing the obstacle data corresponding to the lidar and the obstacle data corresponding to the millimeter-wave radar to obtain the first detection source data of the target obstacle in the mine driving environment includes: projecting both the three-dimensional point cloud data corresponding to the lidar and the two-dimensional position coordinates corresponding to the millimeter-wave radar onto a bird's-eye view in the lidar coordinate system to obtain three-dimensional point cloud projection data and two-dimensional position projection data respectively; constructing a circular area with a preset value as the center of the position point indicated by the two-dimensional position projection data to obtain the radar wave obstacle area; fusing the three-dimensional point cloud projection data and the radar wave obstacle area to obtain the first detection source data of the target obstacle in the mine driving environment.

[0086] The 3D point cloud data may include, but is not limited to, the 3D position coordinates of the center point of the target obstacle detected by the lidar, the size of the target obstacle, and the orientation angle, among other data.

[0087] Optionally, the three-dimensional point cloud data of the target obstacle detected by the lidar can be projected onto the bird's-eye view under the three-dimensional lidar coordinate system established by the lidar to obtain three-dimensional point cloud projection data that represents the position distribution of the three-dimensional point cloud on the two-dimensional plane, and the two-dimensional position coordinates of the target obstacle detected by the millimeter-wave radar can be projected onto the two-dimensional plane under the bird's-eye view to obtain two-dimensional position projection data that represents the position distribution of the two-dimensional position coordinates on the two-dimensional plane.

[0088] Based on 3D point cloud projection data and 2D position projection data, since the 2D position coordinates are point information of the target obstacle, the resulting 2D position projection data is presented in the form of discrete position points, thus obtaining the position points of the 2D position projection data. A circular region with a preset radius is constructed centered on the position points of all target obstacles detected by the millimeter-wave radar, yielding the radar obstacle region, which represents the possible distribution range of target obstacles detected by the millimeter-wave radar on a 2D plane from a bird's-eye view. The preset value can be flexibly set according to the actual business scenario requirements to meet the accuracy of source determination.

[0089] The 3D point cloud projection data can be presented in the form of a size bounding box of the target obstacle. Optionally, cluster analysis can be performed on the projected 3D point cloud projection data to extract all point cloud feature points corresponding to a single target obstacle. Then, based on the distribution range of these feature points on the two-dimensional plane from a bird's-eye view, a minimum size bounding box that can encompass all projection points of the obstacle can be constructed. This size bounding box can reflect the position and size outline of the target obstacle on the two-dimensional plane.

[0090] Optionally, the 3D point cloud projection data and radar wave obstacle regions can be fused. Specifically, if 3D point cloud projection data of a lidar is detected within the radar wave obstacle region corresponding to a target obstacle, it can be matched and its corresponding first detection source data recorded as dual-radar obstacle detection data. If the same 3D point cloud projection data is detected within multiple radar wave obstacle regions corresponding to multiple target obstacles, these regions are merged and mapped to the 3D point cloud projection data, and their corresponding first detection source data is recorded as dual-radar obstacle detection data. If a radar wave obstacle region corresponding to a target obstacle matches multiple 3D point cloud projection data, it is assigned to the nearest 3D point cloud projection data, and its corresponding first detection source data is recorded as dual-radar obstacle detection data. If no 3D point cloud projection data is detected within a radar wave obstacle region corresponding to a target obstacle, its corresponding first detection source data is recorded as single-millimeter-wave obstacle detection data. If no 3D point cloud projection data is detected within a radar wave obstacle region corresponding to a target obstacle, its corresponding first detection source data is recorded as single-laser obstacle detection data.

[0091] By projecting the 3D point cloud data of LiDAR and the 2D position coordinates of millimeter-wave radar into the bird's-eye view of the LiDAR coordinate system, the fusion error caused by the difference in coordinate systems of different sensors is eliminated. Then, by constructing a circular region to achieve data association, the same obstacle detected by the two radars is accurately matched, which improves the accuracy of dual radar data fusion and lays a precise coordinate foundation for subsequent multi-source fusion.

[0092] S240. The obstacle detection data of the dual radar corresponding to the millimeter-wave radar, the obstacle detection data of the single millimeter-wave radar, and the obstacle data corresponding to the photosensitive camera are fused to obtain the second detection source data of the target obstacle in the mine driving environment. The second detection source data includes the obstacle detection data of the single millimeter-wave radar, the obstacle detection data of the single camera, and the obstacle detection data of the light wave.

[0093] The second detection source data can be a dataset representing the target obstacle detection sources in the mine driving environment, obtained by fusing obstacle detection data from dual-radar systems (millimeter-wave radar), single-millimeter-wave radar, and obstacle data from a photosensitive camera. The second detection source data includes single-millimeter-wave obstacle detection data, single-camera obstacle detection data, and optical wave obstacle detection data. Single-camera obstacle detection data can be target obstacle data detected only by the photosensitive camera in the second detection source data, but not by the millimeter-wave radar. Optical wave obstacle detection data can be target obstacle data detected by both the millimeter-wave radar and the photosensitive camera in the second detection source data.

[0094] Optionally, obstacle detection data from dual-radar detection and single-millimeter-wave detection can be fused with obstacle data from a photosensitive camera to obtain a second detection source data.

[0095] Based on the above scheme, optionally, the step of fusing the obstacle detection data of the dual radar corresponding to the millimeter-wave radar, the obstacle detection data of the single millimeter-wave radar, and the obstacle data corresponding to the photosensitive camera to obtain the second detection source data of the target obstacle in the mine driving environment includes: determining the target obstacle data corresponding to the obstacle detection data of the dual radar from the obstacle data corresponding to the millimeter-wave radar; fusing the target obstacle data and the obstacle data corresponding to the photosensitive camera to obtain the first local fusion data; constructing multiple virtual three-dimensional position coordinates of the target obstacle based on the single millimeter-wave obstacle detection data and multiple preset height values; determining the size information of the target obstacle based on the category of the target obstacle; constructing an obstacle region based on the three-dimensional position coordinates and the size information; fusing the obstacle region and the obstacle data corresponding to the photosensitive camera to obtain the second local fusion data; and determining the second detection source data of the target obstacle in the mine driving environment based on the first local fusion data and the second local fusion data.

[0096] The target obstacle data can be extracted from the obstacle data corresponding to the millimeter-wave radar, and is the target obstacle information corresponding to the obstacle data detected by the dual radars. For example, the target obstacle data may include, but is not limited to, various data such as the target obstacle's position coordinates, size, and movement speed.

[0097] Considering that millimeter-wave radar can only acquire the two-dimensional position coordinates of the target obstacle, optionally, the target obstacle data corresponding to the obstacle data detected by the dual radar can be determined from the obstacle data corresponding to the millimeter-wave radar. The three-dimensional point cloud data detected by the lidar provides the coordinate information in the z-direction and the size information of the target obstacle detected by each millimeter-wave radar. Then, the target obstacle data corresponding to the obstacle data detected by the dual radar and the obstacle data corresponding to the photosensitive camera are fused to obtain the first local fused data.

[0098] Based on this, for single-millimeter-wave obstacle detection data that has not been successfully matched with lidar, multiple virtual three-dimensional position coordinates of the target obstacle can be constructed according to the single-millimeter-wave obstacle detection data and multiple preset height values, supplementing the single-millimeter-wave detection with height information; wherein, the multiple preset height values ​​can cover the common height range of target obstacles existing in the mine, and can be set according to multiple different heights from the ground to the top of the roadway. For example, multiple z-values ​​can be created for the target obstacle. Generate multiple virtual 3D position coordinates of the target obstacle. Additionally, the orientation angle of the target obstacle can be set to 0 by default.

[0099] Furthermore, the size information of the target obstacle can be determined based on a preset category-size mapping relationship according to the category of the target obstacle detected by millimeter waves. For example, if the category is a motor vehicle, its size can be set to 4×2.5×2 according to the category-size mapping relationship; if the category is a pedestrian, its size can be set to 0.5×0.5×1.8, etc. The preset category-size mapping relationship can be pre-trained and used only as a reference during the mapping process. In actual mapping, the size information of the target obstacle can be flexibly adjusted to meet the detection accuracy requirements of the target obstacle.

[0100] Optionally, based on multiple virtual three-dimensional position coordinates of the constructed target obstacle and the size information of the target obstacle, a spatial region that can roughly cover the target obstacle can be divided in three-dimensional space as an obstacle region. The obstacle region can be used to represent the approximate range of the target obstacle detected by a single millimeter wave in three-dimensional space.

[0101] Optionally, obstacle regions determined based on single millimeter waves and obstacle data corresponding to photosensitive cameras can be fused to obtain second local fused data. This fusion method is the same as the fusion method between 3D point cloud projection data and radar wave obstacle regions. Finally, the first and second local fused data can be combined to determine the second detection source data of target obstacles in the mine driving environment.

[0102] By fusing dual radar detection data with camera data, efficient correlation of highly reliable data is ensured. For single millimeter-wave data, virtual three-dimensional coordinates and obstacle regions are constructed to compensate for the dimensionality deficiency of two-dimensional millimeter-wave data, enabling it to be effectively matched with camera image data. Combining the two local fusion methods not only ensures fusion efficiency but also solves the problem of single-source millimeter-wave data being difficult to fuse with images, thus improving the compatibility of multi-source data.

[0103] S250. The obstacle data corresponding to the lidar and the obstacle data corresponding to the photosensitive camera are fused to obtain the third detection source data of the target obstacle in the mine driving environment. The third detection source data includes single-laser detection obstacle data, single-camera detection obstacle data and light wave joint detection obstacle data.

[0104] The third detection source data can be a set of data representing the target obstacle detection sources in the mine driving environment, obtained by fusing obstacle data corresponding to LiDAR and obstacle data corresponding to photosensitive cameras. The third detection source data includes single-LiDAR obstacle detection data, single-camera obstacle detection data, and combined optical wave obstacle detection data. Combined optical wave obstacle detection data refers to target obstacle data detected by both LiDAR and photosensitive cameras in the third detection source data.

[0105] Optionally, obstacle data corresponding to the lidar and obstacle data corresponding to the photosensitive camera can be fused to obtain third detection source data.

[0106] Based on the above scheme, optionally, the obstacle data corresponding to the lidar includes three-dimensional point cloud data of the target obstacle, and the obstacle data corresponding to the photosensitive camera includes the region bounding box of the target obstacle in the environmental image; the step of fusing the obstacle data corresponding to the lidar and the obstacle data corresponding to the photosensitive camera to obtain the third detection source data of the target obstacle in the mine driving environment includes: projecting multiple feature points in the three-dimensional point cloud data of the target obstacle corresponding to the lidar onto the environmental image according to preset calibration parameters to obtain multiple projection points of the target obstacle; constructing the projection region of the target obstacle in the environmental image according to the minimum bounding box of the multiple projection points; and determining the third detection source data of the target obstacle in the mine driving environment according to the intersection-union ratio of the region bounding box of the target obstacle in the environmental image and the projection region.

[0107] The preset calibration parameters can be a set of parameters representing the transformation relationship between the lidar coordinate system and the image coordinate system of the photosensitive camera. These preset calibration parameters can be determined in advance through calibration experiments before fusing lidar-type obstacle data and photosensitive camera obstacle data.

[0108] Specifically, multiple feature points in the 3D point cloud data of the target obstacle corresponding to the LiDAR can be projected onto the environmental image captured by the photosensitive camera using preset calibration parameters to obtain multiple projection points of the target obstacle. A minimum bounding box that can include multiple projection points can then be drawn as the projection area of ​​the target obstacle. For example, if the target obstacle in the LiDAR is presented as 3D point cloud data, feature points that can reflect the key shape features of the target obstacle can be selected and converted to the photosensitive camera image coordinate system using preset calibration parameters to obtain projection points. Each feature point corresponds to one projection point.

[0109] The minimum bounding box can be the smallest rectangle that can encompass all multiple projected points. The minimum bounding box can be used to determine the overall distribution range of multiple projected points on the environmental image. The projection region can be a rectangular area constructed in the environmental image based on the minimum bounding box, and the projection region can represent the approximate projection range of the target obstacle detected by the LiDAR on the environmental image.

[0110] Optionally, the ratio of the intersection area and the union area of ​​the region bounding box and the projected area of ​​the target obstacle in the environmental image can be calculated to determine the degree of overlap between the region bounding box and the projected area, thereby determining whether the target obstacle detected by the lidar and the photosensitive camera is the same obstacle, and determining the third detection source data of the target obstacle.

[0111] Specifically, a preset intersection-union ratio (IUGR) threshold can be set. If the IUGR of the bounding box and the projected area of ​​a target obstacle in the environmental image is greater than the preset IUGR threshold, it can be considered that the LiDAR and the photosensitive camera detected the same target obstacle, and its third detection source data is recorded as light wave joint detection obstacle data. If the bounding boxes of multiple target obstacles in the environmental image match the same projected area, their third detection source data can be determined as light wave joint detection obstacle data based on their IUGR. If multiple projected areas match the bounding box of a single target obstacle in the environmental image, the distance between the multiple projected areas is calculated. If the distance is less than a preset distance threshold, the multiple projected areas can be considered as joint detection obstacle data. If an area belongs to the same target obstacle, its third detection source data is recorded as light wave joint obstacle detection data. If the distance is greater than a preset distance threshold, only multiple projection areas whose center point is close to the center point of the area label box are merged and matched with the area label box, and their third detection source data is recorded as light wave joint obstacle detection data. If the projection area does not match the area label box of the target obstacle in the environmental image, that is, the intersection-union ratio is less than or equal to the threshold, its third detection source data is recorded as single laser obstacle detection data. If the area label box of the target obstacle in the environmental image does not match the projection area, its third detection source data is recorded as single camera obstacle detection data.

[0112] Optionally, if there are multiple types of photosensitive cameras, and the region bounding boxes of multiple photosensitive cameras all match the same lidar projection area, then the photosensitive camera type corresponding to the region bounding box of the target obstacle in the environmental image corresponding to the largest cross-union ratio can be determined as the detection source information in the light wave joint detection obstacle data.

[0113] By projecting LiDAR feature points onto the environmental image according to preset calibration parameters, constructing the projection area, and calculating the intersection-union ratio with the camera's bounding box to achieve data association, the advantages of LiDAR's three-dimensional spatial positioning and the image region recognition advantage of the photosensitive camera are fully utilized. This can accurately determine whether the two have detected the same obstacle, avoid mismatch caused by differences in perspective when fusing LiDAR and photosensitive camera data, and improve the accuracy of data fusion.

[0114] S260. Based on the first detection source data, the second detection source data, and the third detection source data, determine the detection source corresponding to each target obstacle in the mine driving environment. The detection source is used to indicate the data acquisition device that detects the target obstacle.

[0115] Optionally, the detection source corresponding to each target obstacle can be determined by combining the data from the first detection source, the second detection source, and the third detection source.

[0116] Based on the above scheme, optionally, the photosensitive camera includes a visible light camera and an infrared camera, and the environmental image includes a visible light image and an infrared image; determining the detection source corresponding to each target obstacle in the mine driving environment based on the first detection source data, the second detection source data, and the third detection source data includes: fusing the visible light image and the infrared image to obtain fourth detection source data of the target obstacles in the mine driving environment, the fourth detection source data including infrared detection obstacle data, visible light detection obstacle data, and dual-light detection obstacle data; determining the detection source corresponding to each target obstacle in the mine driving environment based on the first detection source data, the second detection source data, the third detection source data, and the fourth detection source data.

[0117] Visible light cameras are cameras that can sense visible light and convert it into color or black-and-white image signals. Images captured by visible light cameras closely resemble scenes observed by the human eye. Infrared cameras are cameras that can sense infrared light and convert it into infrared image signals. Infrared cameras can capture images of obstacles in low-light or complete darkness. Visible light images are images captured by visible light cameras that show the visual characteristics of objects in a mine's operating environment under visible light. Infrared images are images captured by infrared cameras that show the thermal radiation characteristics of objects in a mine's operating environment under infrared light. Objects at different temperatures appear in different colors or brightness in infrared images.

[0118] Specifically, visible light images and infrared images can be fused to obtain fourth detection source data. The fusion method is the same as the fusion method for obstacle data corresponding to lidar and obstacle data corresponding to a photosensitive camera. The fourth detection source data refers to the data set representing the target obstacle detection source situation in the mine driving environment, obtained by fusing visible light and infrared images. The fourth detection source data includes infrared detection obstacle data, visible light detection obstacle data, and dual-light detection obstacle data. Infrared detection obstacle data refers to target obstacle data detected only by infrared images and not by visible light images in the fourth detection source data. Visible light detection obstacle data refers to target obstacle data detected only by visible light images and not by infrared images in the fourth detection source data. Dual-light detection obstacle data refers to target obstacle data detected by both visible light and infrared images in the fourth detection source data.

[0119] By refining the photosensitive camera into a visible light camera and an infrared camera, and fusing environmental images, this method addresses the complex lighting conditions in mines, ranging from low light to extreme darkness. It leverages the advantages of infrared cameras (unaffected by visible light intensity and capable of capturing thermal radiation characteristics) and visible light cameras (capable of revealing detailed textures), effectively compensating for the detection limitations of a single photosensitive camera under extreme lighting conditions through dual-light data complementarity. Furthermore, by fusing data to obtain a fourth detection source—including infrared, visible light, and dual-light detection—the detection dimensions are further enriched. This allows for coverage of detailed recognition scenarios under sufficient lighting, as well as scenarios where visible light is ineffective due to darkness or dust obstruction. This significantly improves the adaptability and comprehensiveness of obstacle detection in complex underground lighting environments, providing more robust data support for accurately identifying the detection source of each target obstacle.

[0120] Based on this, the detection source corresponding to each target obstacle in the mine driving environment can be determined according to the first detection source data, the second detection source data, the third detection source data, and the fourth detection source data. Each target obstacle has corresponding location information and detection source information. For example, assuming that the detection source corresponding to the target obstacle is four flag bits, the first bit represents the lidar, the second bit represents the millimeter-wave radar, the third bit represents the visible light camera, and the fourth bit represents the infrared camera, then 1001 can indicate that the target obstacle is detected simultaneously by the lidar and the visible light camera, and so on for the detection source information of other target obstacles.

[0121] By combining data from the first, second, third, and fourth detection sources, and associating the location information of each target obstacle with the detection source information, the detection source is determined. This achieves comprehensive coverage of multi-dimensional data, encompassing the spatial positioning and motion perception advantages of radar, and the visual recognition and illumination adaptation advantages of photosensitive cameras. Through multiple fusions, diverse characteristics of target obstacles in different environments can be captured, avoiding the problem of missing key information from single or limited detection source data. This significantly improves the accuracy and reliability of detection source determination. Cross-validation of multiple sets of detection source data effectively eliminates misjudgments from a single detection source, ensuring a high degree of consistency between the detection source information and the actual situation of the obstacle. It enhances adaptability to the complex driving environment in mines. Regardless of whether the underground environment is characterized by a sudden increase in dust concentration, drastic changes in lighting, or equipment obstruction, there is always effective data among the four types of detection source data that supports the determination of the detection source. Furthermore, the detection source information of each target obstacle can intuitively reflect its identification process, providing a clear basis for subsequent optimization of obstacle tracking and risk warning strategies for different detection source types, further ensuring the safe operation of unmanned underground vehicles.

[0122] Optionally, multiple data acquisition devices can be centrally installed at the same location on the transport locomotive to ensure a large overlap in their shared field of view and to guarantee effective data fusion. Simultaneously, to ensure the sensing range of the data acquisition devices, their installation position should not be too high or too low to avoid creating large blind spots. For example, multiple data acquisition devices can be installed under the windshield of the transport locomotive. Figure 2f As shown, 1 can be a lidar, 2 is a visible light camera, 3 is an infrared camera, and 4 is a millimeter-wave radar. This is just an example, and there are no specific restrictions on the type of data acquisition device or its installation location.

[0123] S270. Determine the category and movement speed of each target obstacle in the mine driving environment based on the obstacle data corresponding to the various data acquisition devices.

[0124] S280. Based on the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, perform an obstacle response operation corresponding to the transport vehicle, wherein the obstacle response operation includes braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle.

[0125] The technical solution of this invention firstly involves fusing the obstacle data corresponding to the lidar and the obstacle data corresponding to the millimeter-wave radar to obtain first detection source data of target obstacles in the mine driving environment. This first detection source data includes single-laser obstacle detection data, single-millimeter-wave obstacle detection data, and dual-radar obstacle detection data. The lidar acquires three-dimensional spatial details, while the millimeter-wave radar captures motion features. The fusion of these two technologies avoids the limitations of a single radar, ensuring that the first detection source data encompasses both accurate spatial information and reliable motion information, laying the foundation for subsequent multi-dimensional fusion and reducing the complexity of subsequent data processing. Next, the second detection source data of target obstacles in the mine driving environment is obtained by fusing the dual-radar obstacle detection data, the single-millimeter-wave obstacle detection data, and the obstacle data corresponding to the photosensitive camera. This second detection source data includes single-millimeter-wave obstacle detection data, single-camera obstacle detection data, and light wave obstacle detection data. Visual data from the photosensitive camera is introduced on top of the radar data to compensate for the limitations of the radar. The shortcomings in target category recognition are further verified to validate the visual matching of dual-radar target detection, supplementing the visual information of single millimeter-wave target detection and reducing the false judgment rate of millimeter-wave detection alone. Next, by fusing the obstacle data corresponding to the lidar and the obstacle data corresponding to the photosensitive camera, a third detection source data of target obstacles in the mine driving environment is obtained. The third detection source data includes single-laser detection obstacle data, single-camera detection obstacle data, and light wave joint detection obstacle data. The three-dimensional point cloud of the lidar accurately locates the spatial position of the target, and the image of the photosensitive camera provides the target's appearance details, realizing dual verification of spatial positioning and visual confirmation, improving the accuracy of category judgment, and providing a reference for subsequent determination of the final detection source. Finally, the detection source corresponding to each target obstacle in the mine driving environment is determined by the first detection source data, the second detection source data, and the third detection source data. By integrating multiple detection source data, the omission of detection source information due to a single fusion dimension is avoided, ensuring that the detection source of each target obstacle can be accurately identified, providing a reliable basis for subsequent safety response.

[0126] Example 3

[0127] Figure 3 This is a schematic diagram of a mine obstacle detection device according to Embodiment 3 of the present invention. This device is used to execute the mine obstacle detection method provided in any of the above embodiments. This device and the mine obstacle detection methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the mine obstacle detection device can be found in the embodiments of the mine obstacle detection method described above. Figure 3As shown, the device includes: an environmental data acquisition module 310, an obstacle data determination module 320, and an obstacle response module 330.

[0128] The environmental data acquisition module 310 is used to collect environmental data of the mine driving environment of the transport locomotive based on multiple data acquisition devices deployed on the transport locomotive in the mine. The data acquisition devices include lidar, millimeter-wave radar, and photosensitive cameras. The environmental data includes three-dimensional point cloud data, millimeter-wave echo data, and environmental images. The obstacle data determination module 320 is used to identify obstacles in the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each data acquisition device. Based on the obstacle data corresponding to the multiple data acquisition devices, the detection source, category, and moving speed of each target obstacle in the mine driving environment are determined. The detection source is used to indicate the data acquisition device that detected the target obstacle. The obstacle response module 330 is used to perform obstacle response operations corresponding to the transport locomotive based on the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed. The obstacle response operations include braking the transport locomotive and / or generating and displaying alarm information corresponding to the target obstacle.

[0129] The technical solution of this invention firstly involves an environmental data acquisition module 310 collecting environmental data from the mine's operating environment using various data acquisition devices deployed on a transport locomotive in the mine. These data acquisition devices include lidar, millimeter-wave radar, and a photosensitive camera. The environmental data includes 3D point cloud data, millimeter-wave echo data, and environmental images. Acquiring multi-dimensional information from multiple data acquisition devices avoids the limitations of data collected by a single device and provides a more comprehensive reflection of the mine's operating environment. Next, an obstacle data determination module 320 performs obstacle identification on the environmental data collected by each data acquisition device to obtain obstacle data corresponding to each data acquisition device. Based on the obstacle data from the various data acquisition devices, the detection source, category, and moving speed of each target obstacle in the mine's operating environment are determined. The detection source is used to indicate the data acquisition device that has detected the target obstacle; based on multi-data cross-validation, the error that may occur in single data identification is reduced, the accuracy of obstacle identification is improved, and multi-dimensional information of the target obstacle is obtained to provide sufficient basis for subsequent obstacle response operations; finally, the obstacle response module 330 executes the obstacle response operation corresponding to the transport vehicle according to the total number of the detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, wherein the obstacle response operation includes braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle; combined with the target obstacle information, precise countermeasures are taken to effectively prevent the transport vehicle from colliding with the obstacle. Multiple data acquisition devices complement each other in the target obstacle detection process, greatly reducing the risk of collision and providing technical support for mine transportation safety.

[0130] Based on the above scheme, optionally, the obstacle data corresponding to the lidar includes target clusters of target obstacles and the first velocity information of the target obstacles; the obstacle data determination module 320 includes: a ground point removal submodule, a roadway top point filtering submodule, and a target obstacle cluster data filtering submodule. The ground point removal submodule is used to perform ground detection on the three-dimensional point cloud data collected by the lidar and remove the detected ground points from the three-dimensional point cloud data; the roadway top point filtering submodule is used to obtain the ground height at each location within the lidar detection range in the mine, and filter out roadway top points in the three-dimensional point cloud data according to the ground height and a preset height difference threshold; the target obstacle cluster data filtering submodule is used to cluster the three-dimensional point cloud data from which the ground points and roadway top points have been filtered out to obtain multiple obstacle clusters, filter out the target clusters of the target obstacles from the obstacle clusters, and track the target clusters of the target obstacles to obtain the first velocity information of the target obstacles.

[0131] Based on the above scheme, optionally, the obstacle data corresponding to the millimeter-wave radar includes the two-dimensional position coordinates, second velocity information, and category of the target obstacle; the obstacle data determination module 320 includes: a millimeter-wave echo data filtering submodule and a target obstacle classification submodule. The millimeter-wave echo data filtering submodule is used to filter out millimeter-wave echo data with echo intensity lower than a preset intensity threshold from the millimeter-wave echo data collected by the millimeter-wave radar, thereby obtaining the two-dimensional position coordinates and second velocity information of the target obstacle; the target obstacle classification submodule is used to classify the target obstacle according to the second velocity information of the target obstacle and the echo intensity in the millimeter-wave echo data corresponding to the target obstacle, thereby obtaining the category of the target obstacle.

[0132] Based on the above scheme, optionally, the obstacle data corresponding to the photosensitive camera includes the region bounding box of the target obstacle in the environmental image, the category of the target obstacle, and the confidence level of the category; the obstacle data determination module 320 includes: a target obstacle detection submodule. The target obstacle detection submodule is used to perform target obstacle detection on the environmental image acquired by the photosensitive camera according to an image detection model, to obtain the region bounding box of the target obstacle in the environmental image, the category of the target obstacle, and the confidence level of the category.

[0133] Based on the above scheme, optionally, the obstacle data determination module 320 includes: a first detection source data determination submodule, a second detection source data determination submodule, a third detection source data determination submodule, and a target obstacle detection source determination submodule. The first detection source data determination submodule is used to fuse the obstacle data corresponding to the lidar and the obstacle data corresponding to the millimeter-wave radar to obtain first detection source data of the target obstacle in the mine driving environment. The first detection source data includes single-laser detection obstacle data, single-millimeter-wave detection obstacle data, and dual-radar detection obstacle data. The second detection source data determination submodule is used to fuse the dual-radar detection obstacle data corresponding to the millimeter-wave radar, the single-millimeter-wave detection obstacle data, and the obstacle data corresponding to the photosensitive camera to obtain second detection source data of the target obstacle in the mine driving environment. The source data includes single millimeter-wave obstacle detection data, single-camera obstacle detection data, and light wave obstacle detection data; the third detection source data determination submodule is used to fuse the obstacle data corresponding to the lidar and the obstacle data corresponding to the photosensitive camera to obtain the third detection source data of the target obstacles in the mine driving environment, the third detection source data including single-laser obstacle detection data, single-camera obstacle detection data, and light wave joint detection obstacle data; the target obstacle detection source determination submodule is used to determine the detection source corresponding to each target obstacle in the mine driving environment based on the first detection source data, the second detection source data, and the third detection source data.

[0134] Based on the above scheme, optionally, the photosensitive camera includes a visible light camera and an infrared camera, and the environmental image includes a visible light image and an infrared image; the target obstacle detection source determination submodule includes: a fourth detection source data determination unit and a target obstacle detection source determination unit. The fourth detection source data determination unit is used to fuse the visible light image and the infrared image to obtain fourth detection source data of the target obstacles in the mine driving environment, the fourth detection source data including infrared detection obstacle data, visible light detection obstacle data, and dual-light detection obstacle data; the target obstacle detection source determination unit is used to determine the detection source corresponding to each target obstacle in the mine driving environment based on the first detection source data, the second detection source data, the third detection source data, and the fourth detection source data.

[0135] Based on the above scheme, optionally, the obstacle data corresponding to the lidar includes three-dimensional point cloud data, and the obstacle data corresponding to the millimeter-wave radar includes two-dimensional position coordinates; the first detection source data determination submodule includes: a projection data determination unit and a first detection source data determination unit. The projection data determination unit is used to project both the three-dimensional point cloud data corresponding to the lidar and the two-dimensional position coordinates corresponding to the millimeter-wave radar into a bird's-eye view in the lidar coordinate system, respectively obtaining three-dimensional point cloud projection data and two-dimensional position projection data; the first detection source data determination unit is used to construct a circular region with a preset radius centered on the position point indicated by the two-dimensional position projection data to obtain a radar wave obstacle region, and to fuse the three-dimensional point cloud projection data and the radar wave obstacle region to obtain the first detection source data of the target obstacle in the mine driving environment.

[0136] Based on the above scheme, optionally, the second detection source data determination submodule includes: a first local fusion data determination unit, a target obstacle size determination unit, a second local fusion data determination unit, and a second detection source data determination unit. The first local fusion data determination unit is used to determine the target obstacle data corresponding to the dual-radar detection obstacle data from the obstacle data corresponding to the millimeter-wave radar, and fuse the target obstacle data with the obstacle data corresponding to the photosensitive camera to obtain first local fusion data; the target obstacle size determination unit is used to construct multiple virtual three-dimensional position coordinates of the target obstacle based on the single millimeter-wave detection obstacle data and multiple preset height values, and determine the size information of the target obstacle according to the category of the target obstacle; the second local fusion data determination unit is used to construct an obstacle region based on the three-dimensional position coordinates and the size information, and fuse the obstacle region with the obstacle data corresponding to the photosensitive camera to obtain second local fusion data; the second detection source data determination unit is used to determine the second detection source data of the target obstacle in the mine driving environment based on the first local fusion data and the second local fusion data.

[0137] Based on the above scheme, optionally, the obstacle data corresponding to the lidar includes three-dimensional point cloud data of the target obstacle, and the obstacle data corresponding to the photosensitive camera includes the region bounding box of the target obstacle in the environmental image; the third detection source data determination submodule includes: a projection point determination unit and a third detection source data determination unit. The projection point determination unit is used to project multiple feature points from the three-dimensional point cloud data of the target obstacle corresponding to the lidar onto the environmental image according to preset calibration parameters, obtaining multiple projection points of the target obstacle; the third detection source data determination unit is used to construct the projection region of the target obstacle in the environmental image based on the minimum bounding box of the multiple projection points, and determine the third detection source data of the target obstacle in the mine driving environment based on the intersection-union ratio of the region bounding box of the target obstacle in the environmental image and the projection region.

[0138] The obstacle detection device for mines provided in the embodiments of the present invention can execute the obstacle detection method for mines provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0139] Example 4

[0140] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0141] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0142] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0143] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as obstacle detection methods for mines.

[0144] In some embodiments, the obstacle detection method for a mine can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the obstacle detection method for a mine described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the obstacle detection method for a mine by any other suitable means (e.g., by means of firmware).

[0145] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0150] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0151] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0152] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for obstacle detection in mines, characterized in that, include: Environmental data of the mine's operating environment is collected by a variety of data acquisition devices deployed on the transport locomotives in the mine. The data acquisition devices include lidar, millimeter-wave radar, and photosensitive cameras. The environmental data includes 3D point cloud data, millimeter-wave echo data, and environmental images. Obstacle identification is performed on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each data acquisition device. Based on the obstacle data corresponding to the various data acquisition devices, the detection source, category, and movement speed corresponding to each target obstacle in the mine driving environment are determined. The detection source is used to indicate the data acquisition device that detected the target obstacle. Based on the total number of detection sources corresponding to the target obstacle, the category of the target obstacle, and the moving speed, an obstacle response operation corresponding to the transport vehicle is performed, wherein the obstacle response operation includes braking the transport vehicle and / or generating and displaying alarm information corresponding to the target obstacle.

2. The obstacle detection method for mines according to claim 1, characterized in that, The obstacle data corresponding to the lidar includes target clusters of obstacles and the first velocity information of the target obstacles; the step of performing obstacle identification on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices includes: Ground detection is performed on the three-dimensional point cloud data collected by the lidar, and the detected ground points are removed from the three-dimensional point cloud data; Obtain the ground height at each location within the detection range of the lidar in the mine, and filter out the top points of the roadway in the three-dimensional point cloud data based on the ground height and a preset height difference threshold; Clustering is performed on the three-dimensional point cloud data after filtering out the ground points and the top points of the alleyway to obtain multiple obstacle clusters. Target clusters of target obstacles are selected from the obstacle clusters, and the target clusters of target obstacles are tracked to obtain the first velocity information of the target obstacles.

3. The obstacle detection method for mines according to claim 1, characterized in that, The obstacle data corresponding to the millimeter-wave radar includes the two-dimensional position coordinates, second velocity information, and category of the target obstacle; The step of performing obstacle identification on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each of the data acquisition devices includes: The millimeter-wave echo data collected by the millimeter-wave radar with echo intensity lower than a preset intensity threshold is filtered out to obtain the two-dimensional position coordinates and second velocity information of the target obstacle. The target obstacle is classified according to the second velocity information of the target obstacle and the echo intensity in the millimeter-wave echo data corresponding to the target obstacle, so as to obtain the category of the target obstacle.

4. The obstacle detection method for mines according to claim 1, characterized in that, The obstacle data corresponding to the photosensitive camera includes the bounding box of the target obstacle in the environmental image, the category of the target obstacle, and the confidence level of the category; the step of performing obstacle identification on the environmental data collected by each of the data acquisition devices to obtain obstacle data corresponding to each data acquisition device includes: The image detection model is used to detect target obstacles in the environmental image captured by the photosensitive camera, so as to obtain the region bounding box of the target obstacle in the environmental image, the category of the target obstacle, and the confidence level of the category.

5. The obstacle detection method for mines according to claim 1, characterized in that, The step of determining the detection source corresponding to each target obstacle in the mine driving environment based on obstacle data corresponding to multiple data acquisition devices includes: The obstacle data corresponding to the lidar and the obstacle data corresponding to the millimeter-wave radar are fused to obtain the first detection source data of the target obstacle in the mine driving environment. The first detection source data includes single lidar detection obstacle data, single millimeter-wave detection obstacle data and dual radar detection obstacle data. The obstacle detection data of the dual radar corresponding to the millimeter-wave radar, the obstacle detection data of the single millimeter-wave radar, and the obstacle data corresponding to the photosensitive camera are fused to obtain the second detection source data of the target obstacle in the mine driving environment. The second detection source data includes single millimeter-wave obstacle detection data, single camera obstacle detection data, and light wave obstacle detection data. The obstacle data corresponding to the lidar and the obstacle data corresponding to the photosensitive camera are fused to obtain the third detection source data of the target obstacle in the mine driving environment. The third detection source data includes obstacle data detected by a single laser, obstacle data detected by a single camera, and obstacle data detected by a combination of light waves. The detection source corresponding to each target obstacle in the mine driving environment is determined based on the first detection source data, the second detection source data, and the third detection source data.

6. The obstacle detection method for mines according to claim 5, characterized in that, The photosensitive camera includes a visible light camera and an infrared camera, and the environmental image includes a visible light image and an infrared image; determining the detection source corresponding to each target obstacle in the mine driving environment based on the first detection source data, the second detection source data, and the third detection source data includes: The visible light image and the infrared image are fused to obtain the fourth detection source data of the target obstacles in the mine driving environment. The fourth detection source data includes infrared detection obstacle data, visible light detection obstacle data and dual-light detection obstacle data. The detection source corresponding to each target obstacle in the mine driving environment is determined based on the first detection source data, the second detection source data, the third detection source data, and the fourth detection source data.

7. The obstacle detection method for mines according to claim 5, characterized in that, The obstacle data corresponding to the lidar includes three-dimensional point cloud data, and the obstacle data corresponding to the millimeter-wave radar includes two-dimensional position coordinates; fusing the obstacle data corresponding to the lidar and the obstacle data corresponding to the millimeter-wave radar to obtain the first detection source data of the target obstacles in the mine driving environment includes: The three-dimensional point cloud data corresponding to the lidar and the two-dimensional position coordinates corresponding to the millimeter-wave radar are both projected into the lidar coordinate system onto the bird's-eye view to obtain three-dimensional point cloud projection data and two-dimensional position projection data, respectively. A circular region with a preset radius is constructed centered on the location point indicated by the two-dimensional position projection data to obtain the radar wave obstacle region. The three-dimensional point cloud projection data and the radar wave obstacle region are then fused to obtain the first detection source data of the target obstacle in the mine driving environment.

8. The obstacle detection method for mines according to claim 5, characterized in that, The process of fusing the obstacle detection data from the dual-radar system corresponding to the millimeter-wave radar, the obstacle detection data from the single millimeter-wave radar, and the obstacle data from the photosensitive camera to obtain the second detection source data of the target obstacles in the mine driving environment includes: The target obstacle data corresponding to the obstacle data detected by the dual radar is determined from the obstacle data corresponding to the millimeter-wave radar, and the target obstacle data and the obstacle data corresponding to the photosensitive camera are fused to obtain the first local fused data; Based on the single millimeter wave obstacle detection data and multiple preset height values, multiple virtual three-dimensional position coordinates of the target obstacle are constructed, and the size information of the target obstacle is determined according to the category of the target obstacle; An obstacle region is constructed based on the three-dimensional position coordinates and the size information. The obstacle region and the obstacle data corresponding to the photosensitive camera are fused to obtain the second local fused data. The second detection source data of the target obstacle in the mine driving environment is determined based on the first local fusion data and the second local fusion data.

9. The obstacle detection method for mines according to claim 5, characterized in that, The obstacle data corresponding to the lidar includes three-dimensional point cloud data of the target obstacle, and the obstacle data corresponding to the photosensitive camera includes the region annotation box of the target obstacle in the environmental image; The process of fusing the obstacle data corresponding to the lidar and the obstacle data corresponding to the photosensitive camera to obtain third detection source data of target obstacles in the mine driving environment includes: According to preset calibration parameters, multiple feature points in the three-dimensional point cloud data of the target obstacle corresponding to the lidar are projected onto the environmental image to obtain multiple projection points of the target obstacle; The projection region of the target obstacle is constructed in the environmental image based on the minimum bounding box of multiple projection points, and the third detection source data of the target obstacle in the mine driving environment is determined based on the intersection-union ratio of the region label box of the target obstacle in the environmental image and the projection region.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the obstacle detection method for mines as described in any one of claims 1-9.

Images