Intelligent geological perception method and system for deep mines based on image spectroscopy technology

Abstract

The present invention discloses a deep mine intelligent geological perception method and system based on image spectrum technology, which belongs to the field of intelligent perception. It extracts features from pre-processed depth images and hyperspectral images to obtain key feature points; uses an improved feature point matching algorithm to pair the key feature points of the depth image and the hyperspectral image, performs image registration based on the pairing results, and obtains a depth image and a hyperspectral image with aligned key feature points; based on the external parameters of the camera, maps the data of the depth image and the hyperspectral image with aligned key feature points to a unified three-dimensional space coordinate system to construct a three-dimensional model; extracts the spectral information and spatial information of the three-dimensional model, combines them and inputs them into a spectral-space fusion model to obtain a coal-rock interface; inverts temperature and gas based on spectral information and maps them to the three-dimensional model. It realizes large-scale dynamic monitoring, and also realizes automated and intelligent geological perception to ensure the safety of the mine excavation process.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent perception of deep mines, and in particular to an intelligent geological perception method and system for deep mines based on image spectroscopy technology. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With the rise of advanced technologies such as big data, cloud computing, and artificial intelligence, my country's mines are gradually moving from the past mechanized and digital development stage to the intelligent stage, and smart mines have emerged. With the gradual implementation of the concept of smart mines, intelligent and unmanned operations are gradually being implemented in mining areas, involving all aspects of mine geology, surveying, mining, mineral processing, safety, etc.

[0004] Mining production changes quickly and is time-sensitive. Spatial location information is constantly changing. The underground tunnel environment of coal mines is harsh and cluttered. The current mine informatization construction faces the challenge of the lack of standardized spatial data and business data service systems. Traditional geological perception methods have become a key factor restricting the rapid development of intelligent mines due to defects such as large workload, low efficiency, long cycle, and single-point testing. Moreover, the mine information obtained by traditional geological perception methods is single and cannot provide multi-faceted data support, which affects all aspects of mine mining and makes mine mining inefficient. The coal-rock interface of mines perceived based on traditional methods lacks a three-dimensional sense and is prone to deviations in the actual mining process, which not only affects the mining efficiency of coal, but also poses a threat to the safety of mines. Therefore, how to develop new mine geological perception technologies, improve mining efficiency, ensure safe mining, and promote intelligent management to meet the needs of mining development in the context of the new era is an urgent problem to be solved and the key to the construction of intelligent mines. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides an intelligent geological perception method and system for deep mines based on image spectroscopy technology, which constructs a three-dimensional model with spectral information based on a depth camera and a hyperspectral camera, and can use the spectral information of the three-dimensional model to invert temperature and methane gas, which not only realizes large-scale dynamic monitoring, but also realizes automated and intelligent geological perception, improves work efficiency, and ensures the safety of the mine excavation process.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] In a first aspect, the present invention provides an intelligent geological perception method for deep mines based on image spectroscopy technology, comprising:

[0008] Acquire a depth image and a hyperspectral image of the mine and perform preprocessing to obtain a preprocessed depth image and a hyperspectral image;

[0009] Using a feature extraction algorithm to extract features from the preprocessed depth image and hyperspectral image to obtain corresponding key feature points;

[0010] Using an improved feature point matching algorithm to pair the key feature points of the depth image and the hyperspectral image, and performing image registration based on the pairing results to obtain a depth image and a hyperspectral image with aligned key feature points;

[0011] Based on the camera's extrinsic parameters, the data of the depth image and hyperspectral image with key feature points aligned are mapped into a unified three-dimensional space coordinate system to construct a three-dimensional model;

[0012] The spectral information and spatial information of the three-dimensional model are extracted, combined and input into a spectral-spatial fusion model to obtain a coal-rock interface; the temperature and gas are inverted based on the spectral information and mapped onto the three-dimensional model.

[0013] A further technical solution is to use a depth camera and a hyperspectral camera to collect depth images and hyperspectral images of the mine, and to determine the intrinsic and extrinsic parameters of the camera by calibrating the camera.

[0014] A further technical solution is to use an improved contrast stretching enhancement method to highlight key features when preprocessing the depth image, specifically:

[0015] Use global contrast enhancement to find the maximum and minimum values ​​of all pixels in the depth image and map each pixel. The formula is expressed as:

[0016]

[0017] in, is the global value after mapping, The current pixel The depth value of is the maximum value of all pixel depth values, is the minimum depth value of all pixels;

[0018] Then, the depth image is enhanced by adaptive local contrast stretching. The depth image is divided into multiple sub-regions, and adaptive linear contrast stretching is applied to each sub-region. The formula is expressed as:

[0019]

[0020] in, To compare the depth value after stretching, is the original depth value in the current sub-area, and are the minimum and maximum depth values ​​in the sub-region, a and b are the upper and lower limits of the target grayscale range of contrast stretching;

[0021] Finally, the results of global and local contrast stretching are balanced, and a fusion weight parameter is introduced to obtain an image enhanced by contrast stretching.

[0022] A further technical solution is to pair the key feature points of the depth image and the hyperspectral image, and perform image registration based on the pairing results, specifically:

[0023] Performing brute force matching on key feature points of the depth image and the hyperspectral image to establish a point-to-point mapping relationship;

[0024] Based on the point-to-point mapping relationship, one of the images is aligned to the other image through affine transformation to complete the preliminary registration;

[0025] A secondary precise registration is performed based on the image similarity measurement to obtain the depth image and hyperspectral image with aligned key feature points.

[0026] A further technical solution is that the secondary precise registration includes precise optimization based on spectral similarity and local optimization based on grid division, and the specific steps are as follows:

[0027] For the matching point pairs after preliminary registration, the precise optimization based on spectral similarity is first used to select the point pairs with high spectral similarity;

[0028] On the filtered high similarity points, grid-based local refinement is used to divide the image into networks, and then window smoothing is applied within each grid.

[0029] A further technical solution for constructing a three-dimensional model is as follows:

[0030] The depth information in the depth image is converted into three-dimensional coordinate data, and the position of each pixel in the three-dimensional space is calculated through the intrinsic and extrinsic parameters of the camera to generate point cloud data;

[0031] The spectral information in the hyperspectral image is mapped to the corresponding point cloud data to generate a three-dimensional model that contains both spectral and geometric information.

[0032] A further technical solution is that the spectral-spatial fusion model is a 3D-CNN model, which is composed of an input layer, a 3D convolution layer, a maximum pooling layer, a fully connected layer and an output layer connected in sequence. The three-dimensional tensor spectral information and spatial information of the extracted three-dimensional model are fused using the channel dimension and combined into a new four-dimensional tensor, and the coal-rock interface is obtained in the input model.

[0033] In a second aspect, the present invention provides an intelligent geological perception system for deep mines based on image spectroscopy technology, comprising:

[0034] A data acquisition module is configured to: acquire a depth image and a hyperspectral image of the mine and perform preprocessing to obtain a preprocessed depth image and a hyperspectral image;

[0035] A feature extraction module is configured to: use a feature extraction algorithm to extract features from the preprocessed depth image and the hyperspectral image to obtain corresponding key feature points;

[0036] An image registration module is configured to: pair key feature points of the depth image and the hyperspectral image using an improved feature point matching algorithm, and perform image registration based on the pairing result to obtain a depth image and a hyperspectral image with aligned key feature points;

[0037] A three-dimensional model building module is configured to: map the data of the depth image and the hyperspectral image with key feature points aligned to a unified three-dimensional space coordinate system based on the extrinsic parameters of the camera to build a three-dimensional model;

[0038] The three-dimensional model perception module is configured to: extract the spectral information and spatial information of the three-dimensional model, combine them and input them into the spectral-spatial fusion model to obtain the coal-rock interface; invert the temperature and gas based on the spectral information and map them to the three-dimensional model.

[0039] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the intelligent geological perception method for deep mines based on image spectroscopy technology as described in the first aspect.

[0040] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the intelligent geological perception method for deep mines based on image spectroscopy technology as described in the first aspect are implemented.

[0041] One or more of the above technical solutions have the following beneficial effects:

[0042] The present invention combines a depth camera and a hyperspectral camera, which not only provides three-dimensional spatial information but also provides atlas dimension information. The depth camera can obtain three-dimensional depth information of geological structures, which is helpful to understand the three-dimensional morphology of geological structures. The hyperspectral camera can obtain high-spectral resolution images of geological materials. Combining hyperspectral information and depth information can simultaneously obtain the spectral characteristics (such as mineral composition, gas concentration, temperature distribution) and spatial position information of the mine.

[0043] The present invention constructs a three-dimensional model of a mine based on depth images and hyperspectral images. During the three-dimensional model construction process, an improved contrast enhancement method is used to process the depth image to highlight key features in the mine, so that these key features are clearer and more accurate in the three-dimensional model, which helps to better understand the geological structure and resource distribution and improves the accuracy of the three-dimensional model. An improved feature matching method is also used to achieve precise alignment of feature points of the depth image and the hyperspectral image, reducing model distortion or errors caused by inaccurate image alignment, helping to build a more accurate three-dimensional model and improving the application value of the three-dimensional model in geological exploration, resource assessment and other aspects.

[0044] The improved feature matching method adopted by the present invention is realized through preliminary registration and secondary precise registration. After the preliminary registration, a secondary registration including precise optimization based on spectral similarity and local refinement registration based on grid division is introduced. First: each pixel point of the hyperspectral image contains rich spectral information, not just the spatial position. Spectral similarity optimization can better utilize spectral features, ensure the spectral consistency of the registration points, and improve the physical significance of the registration. Next: independent optimization in each local area can avoid the accumulation of global errors. This is particularly important for overall image alignment, especially in scenes where the image contains complex details or local deformations.

[0045] The present invention extracts spectral information and spatial information in a three-dimensional model to construct a spectral-spatial fusion model, utilizes the difference in spectral reflectance characteristics of coal and rock in different bands, and uses the fact that the reflectivity of coal is significantly lower than that of rock in the short-wave infrared band. Combined with the spatial information of pixel points and surrounding areas, the coal and rock are partitioned to obtain the coal-rock interface, thereby improving the accuracy of coal-rock identification. The present invention can also generate a three-dimensional image of the coal-rock interface, and can accurately mark the position and thickness of the coal seam in the constructed three-dimensional model; at the same time, a three-dimensional spatial distribution map of temperature and methane gas can be obtained, which helps to better monitor the position and diffusion direction of temperature and gas leakage.

[0046] The present invention breaks the limitation of traditional detection methods that they are only single-point tests, realizes large-area geological perception, and more intuitively perceives spatial distribution and dynamic deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0048] Figure 1 It is a flow chart of the intelligent geological perception method for deep mines according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0051] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0052] Embodiment 1

[0053] like Figure 1 As shown, this embodiment discloses a deep mine intelligent geological perception method based on image spectroscopy technology, which includes the following steps:

[0054] S1: Acquire the depth image and hyperspectral image of the mine and perform preprocessing to obtain the preprocessed depth image and hyperspectral image;

[0055] In this embodiment, a depth camera and a hyperspectral camera are used to collect depth images and hyperspectral images of a mine. During the collection, the depth camera and the hyperspectral camera are ensured to be in the same position, and the image data are collected simultaneously to ensure the temporal consistency of the image data. The depth camera captures the depth information of the scene and records the distance and shape information of the target object within the field of view. The hyperspectral camera captures the spectral information of the scene, covering the light reflection characteristics at different wavelengths of visible-near infrared, short-wave infrared, and thermal infrared.

[0056] First, the acquired depth images and hyperspectral images are preprocessed, including image denoising and enhancement, to solve interference factors such as uneven lighting, dust and water mist, and narrow and long space.

[0057] For hyperspectral image data, convolution filters are used to smooth each band, and low-pass filters are used to filter out high-frequency noise or wavelet transform is used to decompose image signal threshold processing noise to remove interference from sensor noise, ambient light changes, dust, etc.

[0058] Furthermore, the hyperspectral image data also uses histogram equalization to equalize the illumination and adjusts the distortion of the illumination reflectance through color correction technology. The histogram equalization expression is:

[0059]

[0060] in, Expressed as pixel intensity value, is the pixel intensity The frequency, is the total number of pixels, It is the normalized cumulative histogram, which is used to adjust the image brightness and contrast.

[0061] For the depth image data, a bilateral filter is used to smooth the image or a median filter is used to remove noise for smoothing to remove outliers in the depth image caused by the device precision.

[0062] Among them, image denoising based on bilateral filter is expressed as:

[0063]

[0064] in, , Represented as pixels, is the image value at pixel p after filtering, is the normalization factor, represents the window range (the set of neighborhood pixels to which the filter is applied), Represented as the value of the original image at pixel q, Represented as the value of the original image at pixel p, Parameters that control the spatial distance weight decay, The parameter that controls the weight decay of pixel value similarity. exp represents the function of natural exponential.

[0065] Furthermore, the depth image also uses an improved contrast stretching enhancement method to highlight the key features in the mine. Specifically, a method combining global contrast enhancement and adaptive local contrast enhancement is used to highlight the key features and determine the target grayscale range [a, b] of contrast stretching;

[0066] First, perform global contrast stretching: 1) Find the maximum and minimum values ​​of all pixels in the depth image and map each pixel. The formula is:

[0067]

[0068] in, is the global value after mapping, The current pixel The depth value of is the maximum value of all pixel depth values, It is the minimum depth value of all pixels.

[0069] Then, the depth image is adaptively stretched and enhanced with local contrast: 1) The depth image is divided into sub-regions of appropriate size (overlapping small blocks, 16 16-pixel window), divided into blocks by sliding window;

[0070] 2) Calculate the minimum and maximum values ​​of the pixel values ​​in each sub-region, and apply adaptive linear contrast stretching to each sub-region. This operation will re-stretch the pixel value range to the target range based on the minimum and maximum pixel values ​​of each region. The formula is:

[0071]

[0072] in, To compare the depth value after stretching, is the original depth value in the current sub-area, and are the minimum and maximum depth values ​​within the sub-region, and a and b are the upper and lower limits of the target grayscale range for contrast stretching.

[0073] 3) Apply smoothing filtering to smooth the overlapping parts of adjacent areas.

[0074] Finally, the results of global and local contrast stretching are balanced and the fusion weight parameter is introduced , and get the image after contrast stretching enhancement , the formula is:

[0075]

[0076] in, To compare the image after stretching enhancement, is the fusion weight parameter, is the global value after mapping, is the current pixel, It is the depth value after comparison and stretching.

[0077] Then, calibrate the depth camera and hyperspectral camera to determine the intrinsic parameters of the camera, including focal length, principal point, distortion coefficient, and the extrinsic parameters of the camera, including posture and position. Use a calibration plate or a three-dimensional object to calibrate the depth camera and hyperspectral camera. By taking multiple pictures of the calibration plate, use the camera calibration algorithm to solve the camera's intrinsic parameters and distortion coefficient, and calculate the camera's posture information.

[0078] Image preprocessing also includes: after camera calibration, geometric transformations such as perspective transformation, translation and rotation are performed on the corresponding images according to the internal and external parameter information of the depth camera and hyperspectral camera, the original image is converted into an orthogonal projection under a standard viewing angle, and the image distortion caused by camera lens distortion is corrected.

[0079] S2: extracting features from the preprocessed depth image and hyperspectral image using a feature extraction algorithm to obtain corresponding key feature points;

[0080] In this embodiment, a feature extraction algorithm is used to find key feature points in the depth image and the hyperspectral image. The key feature points are usually local features with similarities in the two images, such as corners, edges, or areas with sudden brightness changes.

[0081] The Hessian matrix of the feature points in the SURF feature extraction algorithm is defined as:

[0082]

[0083] in, , , The image is at scale The second-order derivative of is used to detect corners and edges of images.

[0084] S3: using a feature point matching algorithm to pair key feature points of the depth image and the hyperspectral image, and performing image registration based on the pairing result to obtain a depth image and a hyperspectral image with aligned key feature points;

[0085] In this embodiment, a feature point matching algorithm is used to pair feature points of two images, namely, the depth image and the hyperspectral image, to establish a point-to-point mapping relationship.

[0086] Brute force matching is as follows: by traversing all feature point descriptors, calculating the distance between feature points in the two images, and finding the closest matching point. The distance metric uses Euclidean distance, which is expressed as:

[0087]

[0088] in, Expressed as the Euclidean distance between two feature point descriptors, is the dimension of the feature point descriptor, and They are the depth image and the hyperspectral image. and The feature point descriptor is The value of the dimension.

[0089] Through the results of key feature point matching (i.e., point-to-point mapping relationship), the transformation matrix between the two images is calculated using algorithms such as RANSAC, and one of the images is aligned to the other through affine transformation to complete the preliminary alignment.

[0090] The RANSAC algorithm calculates the transformation matrix by randomly selecting a set of matching points. , expressed as:

[0091]

[0092] in, and is the matching point between the depth image and the hyperspectral image, is the tolerance, used to find the optimal change matrix.

[0093] After the initial registration is completed, a secondary precise registration is performed based on the image similarity metric, which includes precise optimization based on spectral similarity and local optimization based on grid division to eliminate minor misalignments and ensure that the feature points of the depth image and the hyperspectral image are accurately aligned.

[0094] The specific steps of secondary precise registration are:

[0095] 1) Among them, for the matching point pairs after preliminary registration, firstly, the precise optimization based on spectral similarity is used to screen the point pairs with high spectral similarity, so as to eliminate the mismatched point pairs with inconsistent spectra, thereby optimizing the quality of the matching points. The formula is:

[0096] For the given two matching point spectral vectors A (original hyperspectral image) and B (image after preliminary registration), calculate the spectral angle mapping angle:

[0097]

[0098] Based on the spectral similarity metric, a threshold Q is set. If the metric SAM Q, then keep this matching point; if the metric value SAM>Q, then filter out this matching point.

[0099] 2) At the filtered high similarity points, local refinement based on grid partitioning is used to divide the image into networks, and then window smoothing is applied within each grid to reduce local errors and ensure high-quality configuration:

[0100] Specifically, the coordinate information of each feature point in each grid is averaged to obtain a smoothed point set to reduce local noise and errors;

[0101] The local transformation (affine transformation) is re-estimated within each grid based on the smoothed feature points, so that the hyperspectral image and the registered image in the grid area are more accurately aligned.

[0102] After the preliminary registration, the present invention introduces precise optimization based on spectral similarity and local refinement registration based on grid division. First: each pixel point of the hyperspectral image contains rich spectral information, not just the spatial position. Spectral similarity optimization can better utilize spectral features, ensure the spectral consistency of the registration points, and improve the physical significance of the registration. Next: independent optimization in each local area can avoid the accumulation of global errors. This is particularly important for overall image alignment, especially in scenes where the image contains complex details or local deformations.

[0103] S4: Based on the extrinsic parameters of the camera, the data of the depth image and the hyperspectral image with key feature points aligned are mapped into a unified three-dimensional space coordinate system to construct a three-dimensional model;

[0104] In this embodiment, the depth image data and the spectral image data are mapped to a unified three-dimensional space coordinate system according to the position information of the camera. The external parameters include position and posture. The position and posture of the camera determine the actual position of each pixel in the three-dimensional space. These pixel data are mapped to a unified space coordinate system to construct a complete three-dimensional model.

[0105] The depth information in the depth image is converted into three-dimensional coordinate data. The depth value of each pixel represents its distance from the depth camera. The position of each pixel in the three-dimensional space can be calculated through the intrinsic and extrinsic parameters of the camera. Point cloud data is thus generated, and each point in the point cloud has a three-dimensional coordinate.

[0106] Furthermore, for the depth value captured by the depth camera (represents the distance from the camera to the surface of the object), combined with the camera intrinsic parameter matrix, the coordinates of the pixel point in three-dimensional space can be calculated, expressed as:

[0107]

[0108] in, , Expressed as the optical center (principal point coordinates), , Expressed as the focal length of the camera (in pixel units along the x and y directions).

[0109] Convert the 3D point from the camera coordinate system to the world coordinate system using the camera extrinsic parameter matrix , calculate the world coordinate system, expressed as:

[0110]

[0111] in, , , is the coordinate of the pixel point in the world coordinate system, , , is the coordinate of the pixel point in the camera coordinate system, , are the camera extrinsic parameters.

[0112] The spectral information in the hyperspectral image is mapped to the corresponding three-dimensional space point cloud data. In addition to the three-dimensional coordinates ( ), it also contains its corresponding hyperspectral information ( ), generating a 3D model containing both spectral and geometric information (position) ( ).

[0113] When collecting data from multiple image frames, ensure that each image is registered with the previous and next image frames. Local registration algorithms can be used to align each image in the image sequence into a unified 3D model to ensure spatial continuity and consistency between images.

[0114] Among them, adjacent images can be accurately aligned in overlapping areas through a small-scale registration method to ensure the continuity between images in dynamic scenes. The optical flow method is used for local registration of adjacent images. The optical flow method assumes that the brightness of image pixels is constant, and its formula is expressed as:

[0115]

[0116] in, is the brightness value of the image, x and y are spatial coordinates, t is time, u and v are the optical flow components in the horizontal and vertical directions respectively, are the gradients of the image in the spatial and temporal directions, respectively.

[0117] In order to ensure the continuous transition of image features between adjacent images, the discontinuity caused by the acquisition angle or time difference between image frames can be eliminated through smoothing or image stitching methods.

[0118] S5: extracting the spectral information and spatial information of the three-dimensional model, combining them and inputting them into a spectral-spatial fusion model to obtain a coal-rock interface; inverting temperature and gas based on the spectral information, and mapping them onto the three-dimensional model.

[0119] Specifically, the three-dimensional tensor spectral information of the three-dimensional model is extracted ( ) and spatial information ( ), using channel dimension fusion, combined into a new four-dimensional tensor with dimensions , (where H is the height, W is the width, and C is the number of spectral channels, Representing the depth information of each pixel) and inputting it into the spectral-spatial fusion model to obtain the coal-rock interface; inverting the temperature and gas based on the spectral information and mapping them to the three-dimensional model.

[0120] In this embodiment, the difference in spectral reflectance characteristics of coal and rock in different bands is utilized. The reflectivity of coal is significantly lower than that of rock in the short-wave infrared band. The spatial information of the pixel points and the surrounding areas is combined to partition the coal and rock, and divide the distribution positions.

[0121] Extract spectral information and spatial information of the area around the pixel (i.e., spatial information of the hyperspectral image) from the three-dimensional model, combine the spectral information and spatial information of the area around the pixel (depth value and spatial relationship of pixels in the neighborhood), and construct a spectral-spatial fusion model, i.e., a 3D-CNN model. The 3D-CNN model consists of an input layer, a 3D convolution layer, a maximum pooling layer, a fully connected layer, and an output layer connected in sequence.

[0122] A three-dimensional convolution kernel is used to perform convolution operations in both spatial and spectral dimensions, and then the maximum pooling is used to pool the convolution output, thereby reducing the amount of data while retaining key features.

[0123] Among them, the input layer receives a four-dimensional tensor with the shape of ,in is the batch size, processing the four-dimensional tensor through the 3D convolution layer, using the size of The convolution kernel is used for feature extraction (D is the size of the convolution kernel in the spectral dimension, K is the size of the convolution kernel in the spatial dimension), and the pooling layer downsamples the feature map to reduce the dimension and extract the main features.

[0124] The model is trained by supervised learning to divide the training labels of different regions in the hyperspectral image, and the features are classified. The feature classification is to classify coal and rock.

[0125] In this embodiment, the spectrum information collected by each pixel is converted into temperature, and the temperature conditions at different positions are generated.

[0126] Spectral information (radiation intensity) is extracted from each pixel in the three-dimensional model. The relationship between spectral information and temperature is expressed as:

[0127]

[0128] in, is the emissivity of the coal seam in the mine at a certain wavelength, Indicated in wavelength The temperature is The radiation intensity at is Planck's constant, is the speed of light, is the Boltzmann constant.

[0129] Inversely solve the temperature and use the nonlinear least squares method to solve the above formula Temperature solution: By setting the temperature threshold, abnormal temperature areas are marked on the temperature heat map. These areas represent possible coal spontaneous combustion or equipment overheating in the mining area.

[0130] In this embodiment, by utilizing the characteristic of a specific obvious absorption peak of a gas (such as methane) in the mid-infrared band, the spectrum of each pixel is analyzed to generate a spatial distribution map of methane concentration and identify areas with high gas content.

[0131] Select obvious absorption peaks in the mid-infrared band and search for the absorption coefficients of these bands according to the infrared database. (about ) wavelength range, the absorption intensity of methane is relatively high, and its absorption coefficient is usually The infrared database provides a table of methane absorption coefficients under specific conditions.

[0132] Furthermore, there is a linear relationship between the concentration of a gas and the intensity of the infrared radiation it absorbs, as shown in the formula:

[0133]

[0134] in, is the absorption (difference in spectral intensity), is the radiation intensity undisturbed by methane, is the intensity of radiation absorbed by methane, is a known absorption coefficient, is the optical path length, is the methane gas concentration.

[0135] The methane concentration is inverted based on the above formula and the known absorption coefficient The concentration data for each pixel is plotted into a map or image to show the spatial distribution of methane concentration, with high concentration areas highlighted by color mapping.

[0136] Embodiment 2

[0137] This embodiment provides a deep mine intelligent geological perception system based on image spectroscopy technology, including:

[0138] A data acquisition module is configured to: acquire a depth image and a hyperspectral image of the mine and perform preprocessing to obtain a preprocessed depth image and a hyperspectral image;

[0139] A feature extraction module is configured to: use a feature extraction algorithm to extract features from the preprocessed depth image and the hyperspectral image to obtain corresponding key feature points;

[0140] An image registration module is configured to: pair key feature points of the depth image and the hyperspectral image using an improved feature point matching algorithm, and perform image registration based on the pairing result to obtain a depth image and a hyperspectral image with aligned key feature points;

[0141] A three-dimensional model building module is configured to: map the data of the depth image and the hyperspectral image with key feature points aligned to a unified three-dimensional space coordinate system based on the extrinsic parameters of the camera to build a three-dimensional model;

[0142] The three-dimensional model perception module is configured to: extract the spectral information and spatial information of the three-dimensional model, combine them and input them into the spectral-spatial fusion model to obtain the coal-rock interface; invert the temperature and gas based on the spectral information and map them to the three-dimensional model.

[0143] The data acquisition module includes two core modules: the equipment module and the synchronous acquisition module, which work together to ensure efficient and accurate collection of the required data. The equipment module includes a depth camera and a full-band hyperspectral camera; the synchronous acquisition module ensures that the depth camera and the hyperspectral camera collect data according to the predetermined timing to ensure accurate alignment of the data.

[0144] Furthermore, the timing control model is designed according to the required spatial resolution, spectral resolution and temporal resolution, including the trigger timing of the camera, exposure time, data acquisition rate, data transmission or storage strategy, etc., to achieve precise synchronization between the depth camera and the hyperspectral camera. Precision motors and drivers combined with encoder feedback devices are used to control the movement of the camera platform.

[0145] The deep mine intelligent geological perception system also includes a data transmission module, which is used to transmit the detected coal-rock interface, the inverted temperature, methane concentration and distribution to the next system. It also includes a result display module, which is used to display the coal-rock interface, the distribution of temperature and the methane concentration.

[0146] Embodiment 3

[0147] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0148] Embodiment 3

[0149] The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of embodiment 1 are performed.

[0150] The steps involved in the apparatus of the above embodiments 3 and 4 correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0151] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0152] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0153] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An intelligent geological perception method for deep mines based on image spectroscopy technology, characterized in that: include: Acquire a depth image and a hyperspectral image of the mine and perform preprocessing to obtain a preprocessed depth image and a hyperspectral image; Using a feature extraction algorithm to extract features from the preprocessed depth image and hyperspectral image to obtain corresponding key feature points; An improved feature point matching algorithm is used to pair the key feature points of the depth image and the hyperspectral image, and image registration is performed based on the pairing results to obtain a depth image and a hyperspectral image with aligned key feature points, specifically: Performing brute force matching on key feature points of the depth image and the hyperspectral image to establish a point-to-point mapping relationship; Based on the point-to-point mapping relationship, one of the images is aligned to the other image through affine transformation to complete the preliminary registration; Based on the image similarity measurement, a second precise registration is performed to obtain the depth image and hyperspectral image with aligned key feature points; The secondary precise registration includes precise optimization based on spectral similarity and local optimization based on grid division, and the specific steps are as follows: For the matching point pairs after preliminary registration, the precise optimization based on spectral similarity is first used to select the point pairs with high spectral similarity; On the filtered high similarity points, local refinement based on grid division is used to divide the image into networks, and then window smoothing is applied within each grid; Based on the camera's extrinsic parameters, the data of the depth image and hyperspectral image with key feature points aligned are mapped into a unified three-dimensional space coordinate system to construct a three-dimensional model; Extracting spectral information and spatial information of the three-dimensional model, combining and inputting into a spectral-spatial fusion model to obtain a coal-rock interface; The temperature and gas are inverted based on the spectral information and mapped onto a three-dimensional model.

2. The deep mine intelligent geological perception method based on image spectroscopy technology according to claim 1 is characterized in that: Depth cameras and hyperspectral cameras are used to collect depth images and hyperspectral images of the mine. The intrinsic and extrinsic parameters of the cameras are determined by calibrating the cameras.

3. The deep mine intelligent geological perception method based on image spectroscopy technology according to claim 1 is characterized in that: When preprocessing the depth image, an improved contrast stretching enhancement method is used to highlight key features, specifically: Use global contrast enhancement to find the maximum and minimum values ​​of all pixels in the depth image and map each pixel. The formula is expressed as: in, is the global value after mapping, The current pixel The depth value of is the maximum value of all pixel depth values, is the minimum depth value of all pixels; Then, the depth image is enhanced by adaptive local contrast stretching. The depth image is divided into multiple sub-regions, and adaptive linear contrast stretching is applied to each sub-region. The formula is expressed as: in, To compare the depth value after stretching, is the original depth value in the current sub-area, and are the minimum and maximum depth values ​​in the sub-region, a and b are the upper and lower limits of the target grayscale range of contrast stretching; Finally, the results of global and local contrast stretching are balanced, and a fusion weight parameter is introduced to obtain an image enhanced by contrast stretching.

4. The deep mine intelligent geological perception method based on image spectroscopy technology according to claim 1 is characterized in that: The specific steps of building a three-dimensional model are as follows: The depth information in the depth image is converted into three-dimensional coordinate data, and the position of each pixel in the three-dimensional space is calculated through the intrinsic and extrinsic parameters of the camera to generate point cloud data; The spectral information in the hyperspectral image is mapped to the corresponding point cloud data to generate a three-dimensional model that contains both spectral and geometric information.

5. The deep mine intelligent geological perception method based on image spectroscopy technology according to claim 1 is characterized in that: The spectral-spatial fusion model is a 3D-CNN model, which is composed of an input layer, a 3D convolutional layer, a maximum pooling layer, a fully connected layer and an output layer connected in sequence. The three-dimensional tensor spectral information and spatial information of the extracted three-dimensional model are fused using the channel dimension to form a new four-dimensional tensor, and the coal-rock interface is obtained in the input model.

6. The deep mine intelligent geological perception system based on image spectroscopy technology is characterized by: include: A data acquisition module is configured to: acquire a depth image and a hyperspectral image of the mine and perform preprocessing to obtain a preprocessed depth image and a hyperspectral image; A feature extraction module is configured to: use a feature extraction algorithm to extract features from the preprocessed depth image and the hyperspectral image to obtain corresponding key feature points; The image registration module is configured to: pair the key feature points of the depth image and the hyperspectral image using an improved feature point matching algorithm, and perform image registration based on the pairing results to obtain the depth image and the hyperspectral image with aligned key feature points, specifically: Performing brute force matching on key feature points of the depth image and the hyperspectral image to establish a point-to-point mapping relationship; Based on the point-to-point mapping relationship, one of the images is aligned to the other image through affine transformation to complete the preliminary registration; Based on the image similarity measurement, a second precise registration is performed to obtain the depth image and hyperspectral image with aligned key feature points; The secondary precise registration includes precise optimization based on spectral similarity and local optimization based on grid division, and the specific steps are as follows: For the matching point pairs after preliminary registration, the precise optimization based on spectral similarity is first used to select the point pairs with high spectral similarity; On the filtered high similarity points, local refinement based on grid division is used to divide the image into networks, and then window smoothing is applied within each grid; A three-dimensional model building module is configured to: map the data of the depth image and the hyperspectral image with key feature points aligned to a unified three-dimensional space coordinate system based on the extrinsic parameters of the camera to build a three-dimensional model; A three-dimensional model perception module is configured to: extract spectral information and spatial information of the three-dimensional model, combine and input them into a spectral-spatial fusion model to obtain a coal-rock interface; The temperature and gas are inverted based on the spectral information and mapped onto a three-dimensional model.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the deep mine intelligent geological perception method based on image spectroscopy technology as described in any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the deep mine intelligent geological perception method based on image spectroscopy technology as described in any one of claims 1 to 5 are implemented.

Images