Underground metal mine environment sensing method and system

Abstract

The application provides an underground metal mine environment perception method and system, comprising: acquiring position information of a mobile device in an underground mine; converting collected image or point cloud data to a coordinate system of the mobile device according to the position information to fuse the image and the point cloud data, and obtaining a three-dimensional fused image; inputting the three-dimensional fused image into a preset recognition model to obtain position information of a target object in the three-dimensional fused image, wherein the target object comprises an obstacle and an underground mine mining face; and outputting a position relationship between the target object and the mobile device to guide the mobile device to perform mining operations.

Description

Technical Field

[0001] This invention relates to the field of intelligent mining applications, and in particular to a method and system for sensing the environment of underground metal mines. Background Technology

[0002] Existing metal mining operations largely rely on manual labor. Due to the relatively harsh mining environment, underground operations pose significant safety hazards. Furthermore, the underground working environment in metal mines is relatively complex, and the lack of effective environmental sensing methods results in low operating efficiency of the equipment. Enhancing the perception of the mining environment and providing stable and reliable sensing data for equipment operation has become an urgent problem to be solved. Summary of the Invention

[0003] In view of the problems existing in the prior art, the present invention proposes an environmental sensing method and system for underground metal mines, which mainly solves the problems of the lack of effective environmental sensing means in existing metal mining, low operation efficiency and safety hazards.

[0004] To achieve the above and other objectives, the technical solution adopted by the present invention is as follows.

[0005] This application provides a method for sensing the environment of underground metal mines, including:

[0006] Acquire the location information of mobile devices in underground mines;

[0007] Based on the location information, the acquired image or point cloud data is converted to the coordinate system of the mobile device to fuse the image and point cloud data to obtain a three-dimensional fused image;

[0008] The three-dimensional fused image is input into a preset recognition model to obtain the location information of the target object in the three-dimensional fused image, wherein the target object includes obstacles and underground mining faces;

[0009] Output the positional relationship between the target object and the mobile device to guide the mobile device to carry out mining operations.

[0010] In one embodiment of this application, obtaining the location information of a mobile device in an underground mine includes:

[0011] The corresponding image and point cloud data are acquired by the image acquisition module and point cloud acquisition module pre-set on the mobile device;

[0012] The heading angle of the mobile device is obtained according to the inertial navigation unit pre-set on the mobile device;

[0013] The pose of the mobile device is estimated based on the image, point cloud data, and heading angle to obtain the position information of the mobile device.

[0014] In one embodiment of this application, converting the acquired image or point cloud data to the coordinate system of the mobile device based on the location information to fuse the image and point cloud data includes:

[0015] The first relative position between the image acquisition module and the mobile device is determined according to the preset position parameters of the image acquisition module;

[0016] The second relative position between the point cloud acquisition module and the mobile device is determined according to the preset position parameters of the point cloud acquisition module.

[0017] Based on the first relative position and the preset first intrinsic parameter matrix of the image acquisition module, the image acquired by the image acquisition module is transformed into the coordinate system of the mobile device to obtain the first image;

[0018] Based on the second relative position of the search and the preset second intrinsic parameter matrix of the point cloud acquisition module, the point cloud data acquired by the point cloud acquisition module is converted into the coordinate system of the mobile device to obtain the first point cloud;

[0019] The first image and the first point cloud are fused to obtain a three-dimensional fused image.

[0020] In one embodiment of this application, after obtaining the three-dimensional fused image, the method further includes:

[0021] The underground mine surface data is filtered out from the 3D fused image, and the remaining 3D fused image is used for target object recognition.

[0022] In one embodiment of this application, after inputting the 3D fused image into a preset recognition model to obtain the location information of the target object in the 3D fused image, the method further includes:

[0023] The underground mining face image is extracted from the three-dimensional fused image based on the identified location information of the underground mining face.

[0024] By performing surface fitting on the underground mining face image, the surface information of the underground mining face is obtained to guide subsequent mining operations.

[0025] This application also provides an underground metal mine environmental sensing system, including:

[0026] The mobile device itself;

[0027] A data acquisition module is installed on the mobile device body and is used to collect environmental images and point cloud data during the movement of the mobile device.

[0028] A positioning module, disposed on the mobile device body, is used to determine the location information of the mobile device based on the environmental image and point cloud data;

[0029] A data fusion module is installed on the mobile device body and is used to convert the collected image or point cloud data to the coordinate system of the mobile device according to the location information so as to fuse the image and point cloud data to obtain a three-dimensional fused image.

[0030] The recognition module is installed on the mobile device body and is used to input the three-dimensional fused image into a preset recognition model to obtain the location information of the target object in the three-dimensional fused image, wherein the target object includes obstacles and underground mining faces;

[0031] A data output module, installed on the mobile device body, is used to output the positional relationship between the target object and the mobile device to guide the mobile device to carry out mining operations.

[0032] In one embodiment of this application, the data acquisition module includes: a lidar, an optical camera, an infrared camera, and an inertial navigation unit.

[0033] As described above, this application provides a method and system for sensing underground metal mines, which has the following beneficial effects.

[0034] This application identifies obstacle information and underground mining face information by merging images and point cloud data of the underground mining environment during the movement of the mobile device, which is beneficial for the subsequent movement and obstacle avoidance of the mobile device and the smooth and efficient completion of mining operations. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an underground metal ore sensing method in one embodiment of this application.

[0036] Figure 2 This is a block diagram of an underground metal mine sensing system according to one embodiment of this application. Detailed Implementation

[0037] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0038] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0039] Please see Figure 1 This application provides a method for sensing underground metal mines, which includes the following steps.

[0040] Step S01: Obtain the location information of the mobile device in the underground mine.

[0041] In one embodiment, obtaining the location information of the mobile device in an underground mine includes:

[0042] The corresponding image and point cloud data are acquired by the image acquisition module and point cloud acquisition module pre-set on the mobile device;

[0043] The heading angle of the mobile device is obtained according to the inertial navigation unit pre-set on the mobile device;

[0044] The pose of the mobile device is estimated based on the image, point cloud data, and heading angle to obtain the position information of the mobile device.

[0045] Specifically, LiDAR, infrared cameras, optical cameras, and inertial navigation units can be installed on mobile devices such as mining vehicles. LiDAR collects point cloud data of the environment surrounding the mobile device, infrared cameras collect infrared images of the environment, optical cameras collect RGB images of the environment, and the inertial navigation unit obtains the current heading angle and attitude data of the mobile device. By fusing the point cloud data, RGB images, infrared images, and attitude data, SLAM (Single-Like Algorithm) is used for qualitative positioning of the mobile device to obtain its location information, while simultaneously mapping the underground metal mining environment.

[0046] In another embodiment, data acquisition devices such as lidar, infrared cameras, and optical cameras can also be installed in fixed locations underground in metal mines.

[0047] Step S02: Based on the location information, the acquired image or point cloud data is converted to the coordinate system of the mobile device to fuse the image and point cloud data to obtain a three-dimensional fused image.

[0048] In one embodiment, converting the acquired image or point cloud data to the coordinate system of the mobile device based on the location information to fuse the image and point cloud data includes:

[0049] The first relative position between the image acquisition module and the mobile device is determined according to the preset position parameters of the image acquisition module;

[0050] The second relative position between the point cloud acquisition module and the mobile device is determined according to the preset position parameters of the point cloud acquisition module.

[0051] Based on the first relative position and the preset first intrinsic parameter matrix of the image acquisition module, the image acquired by the image acquisition module is transformed into the coordinate system of the mobile device to obtain the first image;

[0052] Based on the second relative position of the search and the preset second intrinsic parameter matrix of the point cloud acquisition module, the point cloud data acquired by the point cloud acquisition module is converted into the coordinate system of the mobile device to obtain the first point cloud;

[0053] The first image and the first point cloud are fused to obtain a three-dimensional fused image.

[0054] Specifically, current-time sensing data is acquired using acquisition devices such as lidar, infrared cameras, and optical cameras, including lidar point cloud data, infrared image data, and optical image data. The relative position of the mobile device calculated in the aforementioned steps, along with the position data of each acquisition device, is used to calculate the conversion relationship between the lidar point cloud, infrared image, and optical image, and the multiple sensing data are fused into one data set to obtain a three-dimensional fused image.

[0055] In one embodiment, after obtaining the three-dimensional fused image, the method further includes: filtering out the underground mine surface data in the three-dimensional fused image, and using the remaining three-dimensional fused image for target object recognition.

[0056] Step S03: Input the three-dimensional fused image into a preset recognition model to obtain the location information of the target object in the three-dimensional fused image, wherein the target object includes obstacles and underground mining faces.

[0057] In one embodiment, for multi-dimensional sensing data that integrates point clouds, images, and heat sources, a 3D deep neural network is used for identification to distinguish information such as people, vehicles, machines, and underground mining faces (location, size, etc.).

[0058] Step S04: Output the positional relationship between the target object and the mobile device to guide the mobile device to carry out mining operations.

[0059] In one embodiment, after inputting the 3D fused image into a preset recognition model to obtain the location information of the target object in the 3D fused image, the method further includes:

[0060] The underground mining face image is extracted from the three-dimensional fused image based on the identified location information of the underground mining face.

[0061] By performing surface fitting on the underground mining face image, the surface information of the underground mining face is obtained to guide subsequent mining operations.

[0062] By comparing information on people, vehicles, and machines with mobile device points, obstacle familiarity information is determined; surface fitting is performed on the underground mining face to obtain real-time underground mine working face information, which serves as the basis for subsequent intelligent control.

[0063] Please see Figure 2 This embodiment provides an underground metal mine environment sensing system for executing the underground metal mine environment sensing method described in the foregoing method embodiments. Since the technical principles of the system embodiment are similar to those of the foregoing method embodiments, the same technical details will not be repeated.

[0064] In one embodiment, an underground metal mine environmental sensing system includes: a mobile device body 10; a data acquisition module 11, disposed on the mobile device body, for acquiring environmental images and point cloud data during the movement of the mobile device; a positioning module 12, disposed on the mobile device body, for determining the position information of the mobile device based on the environmental images and point cloud data; a data fusion module 13, disposed on the mobile device body, for converting the acquired images or point cloud data to the coordinate system of the mobile device based on the position information to fuse the images and point cloud data to obtain a three-dimensional fused image; a recognition module 14, disposed on the mobile device body, for inputting the three-dimensional fused image into a preset recognition model to obtain the position information of target objects in the three-dimensional fused image, wherein the target objects include obstacles and underground mining faces; and a data output module 15, disposed on the mobile device body, for outputting the positional relationship between the target objects and the mobile device to guide the mobile device in mining operations.

[0065] In one embodiment, the data acquisition module includes: a lidar, an optical camera, an infrared camera, and an inertial navigation unit.

[0066] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. An underground metal mine environment sensing method, characterized by, The method comprises the following steps: acquiring position information of a mobile device in an underground mine, including: acquiring corresponding image and point cloud data through an image acquisition module and a point cloud acquisition module pre-installed on the mobile device; acquiring a heading angle of the mobile device according to an inertial navigation unit pre-installed on the mobile device; performing pose estimation on the mobile device according to the image, the point cloud data and the heading angle to obtain position information of the mobile device; converting the collected image or point cloud data to the coordinate system of the mobile device according to the position information to fuse the image and the point cloud data, and obtaining a three-dimensional fused image, including: determining a first relative position between the image acquisition module and the mobile device according to pre-set position parameters of the image acquisition module; determining a second relative position between the point cloud acquisition module and the mobile device according to pre-set position parameters of the point cloud acquisition module; converting the image collected by the image acquisition module into the coordinate system of the mobile device according to the first relative position and a pre-set first intrinsic parameter matrix of the image acquisition module to obtain a first image; converting the point cloud data collected by the point cloud acquisition module into the coordinate system of the mobile device according to the second relative position and a pre-set second intrinsic parameter matrix of the point cloud acquisition module to obtain a first point cloud; fusing the first image and the first point cloud to obtain a three-dimensional fused image; inputting the three-dimensional fused image into a pre-set recognition model to obtain position information of a target object in the three-dimensional fused image, wherein the target object includes an obstacle and an underground mining face; outputting the position relationship between the target object and the mobile device to guide the mobile device to perform mining operations.

2. The underground metal mine environment perception method according to claim 1, characterized in that, After obtaining the three-dimensional fused image, the method further comprises: filtering out underground mine ground data in the three-dimensional fused image, and using the remaining three-dimensional fused image for target object recognition.

3. The underground metal mine environment sensing method according to claim 1, characterized in that, After inputting the three-dimensional fused image into a pre-set recognition model to obtain position information of a target object in the three-dimensional fused image, the method further comprises: extracting an underground mine mining face image in the three-dimensional fused image according to the recognized underground mine mining face position information; performing curved surface fitting according to the underground mine mining face image to obtain curved surface information of the underground mine mining face to guide subsequent mining operations.

4. An underground metal mine environment sensing system using the underground metal mine environment sensing method according to any one of claims 1 to 3, characterized by, The method comprises the following steps: a mobile device body; a data acquisition module arranged on the mobile device body, used for acquiring environmental image and point cloud data during movement of the mobile device; a positioning module arranged on the mobile device body, used for determining position information of the mobile device according to the environmental image and the point cloud data; a data fusion module arranged on the mobile device body, used for converting the collected image or point cloud data to the coordinate system of the mobile device according to the position information to fuse the image and the point cloud data, and obtaining a three-dimensional fused image; an identification module arranged on the mobile device body, used for inputting the three-dimensional fused image into a pre-set recognition model to obtain position information of a target object in the three-dimensional fused image, wherein the target object includes an obstacle and an underground mining face; A data output module is arranged on the mobile device body and is configured to output the positional relationship between the target object and the mobile device to guide the mobile device to perform the mining operation.

5. The underground metal mine environment perception system of claim 4, wherein, The data collection module includes a laser radar, an optical camera, an infrared camera, and an inertial navigation unit.

Images