A method for extracting ground fissures in mining areas from UAV images based on prior location knowledge

Abstract

The present invention discloses a method for extracting ground fissures in mining areas from drone images based on prior position knowledge, which belongs to the field of remote sensing image processing. The present invention uses the prior knowledge extracted by the prior position extractor to gradually constrain the feature space of the model, thereby reducing the difference between the source domain and the target domain data to improve the effect of migration training. At the same time, the Gaussian detail enhancement module is used to gradually enhance the details of the ground fissure image, weaken the interference of complex landforms, and finally achieve accurate extraction of ground fissures in complex mining areas in high-resolution drone images. Therefore, the present invention solves the problems of poor migration training effect and difficulty in extracting small cracks when extracting ground fissures in drone images by existing deep learning methods, and plays an important role in ensuring safe production in mining areas and ecological restoration in mining areas.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method for extracting ground fissures in mining areas using unmanned aerial vehicle images based on priori position knowledge. Background Art

[0002] The accurate investigation and extraction of ground fissures in mining areas is crucial to ensure safe production and ecological restoration in mining areas. Traditionally, ground fissures in mining areas mainly rely on manual on-site observations, using total stations, global positioning systems (GPS) and other means to measure the location of ground fissures. Field investigations are time-consuming and laborious. Since ground fissures are usually sub-meter in size, the current resolution of optical satellite images is difficult to clearly display ground fissures. At present, drone photogrammetry and remote sensing technology are developing rapidly and have been widely used in many fields. Drone photogrammetry technology is flexible and low-cost, and can provide images with a resolution in the centimeter range, providing a suitable data source for monitoring ground fissures in mining areas.

[0003] As an important branch of machine learning technology, deep learning networks have shown significant advantages in the field of image processing, and deep learning has also been widely used in image crack extraction. In the task of road crack extraction, a large amount of sample label data and the uniformity of the road environment provide significant advantages for deep learning methods. However, in the task of extracting ground fissures in mining areas, there are challenges of limited sample data and complex geological conditions. In response to these problems, transfer training and attention mechanism technologies can solve the above problems to a certain extent. However, existing transfer learning methods often ignore the specific attributes of the data. When the data-rich source domain data and the data-scarce target domain data environment are very different, the transfer learning effect is often poor. At the same time, in the task of extracting objects under complex geological conditions, relying solely on technologies such as multi-scale feature fusion and attention mechanism, lacking the integration of domain expertise, it is impossible to accurately locate the area of ​​concern, and there are certain difficulties in realizing the refined extraction of objects under the interference of complex objects. Summary of the invention

[0004] The present invention provides a method and device for extracting ground fissures in mining areas from drone images based on prior position knowledge, so as to solve the problems that the existing deep learning methods have poor migration training effect and difficulty in extracting small cracks when extracting ground fissures in drone images.

[0005] The first aspect of the present invention provides a method for extracting ground fissures in mining areas from drone images based on prior position knowledge, comprising the following steps:

[0006] Obtaining drone images of the mining area, and preprocessing the drone images to obtain original images of ground fissures;

[0007] Extracting prior position knowledge of ground fissures by using a prior position knowledge extractor, combining the original crack map with the prior position knowledge of ground fissures to generate a prior map of ground fissures;

[0008] Downsampling the original ground fissure image, and using prior position knowledge to constrain the downsampled feature space to obtain a downsampled first feature map; downsampling the ground fissure prior image, and using multiple scale Gaussian convolution kernels to enhance the details of the downsampled feature map to obtain a downsampled second feature map;

[0009] The first feature map and the second feature map are superimposed to obtain a target feature map, and the target feature map is upsampled to obtain ground fissure information in the drone image.

[0010] Optionally, in one embodiment of the present invention, the drone image is preprocessed, including one or more of radiation correction, geometric correction, registration, stitching, and cropping.

[0011] Optionally, in one embodiment of the present invention, before extracting the priori position knowledge of the ground fissure using the priori position knowledge extractor, the method further includes:

[0012] The first-order Gaussian difference function is used as the characteristic response function to construct a priori position knowledge extractor. The ground fissure gradient amplitude is weighted by the first-order Gaussian difference function to enhance the ground fissure edge. The expression of the priori position knowledge extractor F is:

[0013]

[0014] Among them, x is the variable perpendicular to the direction of the ground fissure, y is the variable parallel to the direction of the ground fissure, L is the length of the defined template, σ is the standard deviation of the Gaussian distribution, and m is the mean of the Gaussian distribution.

[0015] Optionally, in one embodiment of the present invention, the prior position knowledge extractor uses a Sobel filter to calculate the gradients in the x and y directions in the ground fissure image, and calculates the gradient amplitude and direction, constructs a gradient direction histogram, divides the gradient direction into multi-directional bins, counts the number of gradients in each bin, and uses the constructed feature response function to weight the gradient amplitude, enhance the edge of the ground fissure, mark the part where the feature response exceeds the threshold as an edge, use a sliding window to perform density statistics on the pixels of the ground fissure edge map, and map the density statistics into prior position knowledge.

[0016] Optionally, in one embodiment of the present invention, the ground fissure prior map is downsampled, and multiple scale Gaussian convolution kernels are used to enhance the details of the downsampled feature map to obtain a downsampled second feature map, including:

[0017] Use multiple Gaussian convolution kernels of different scales to perform convolution operations on the ground fissure prior map, and obtain three fuzzy feature maps T1, T2, and T3 of different scales:

[0018] T1=G1*I, T2=G2*I, T3=G3*I

[0019] Among them, G1, G2, and G3 are Gaussian kernels with standard deviations σ1=1, σ2=2, and σ3=4, respectively, and I is the prior map of ground fissures;

[0020] Using three fuzzy feature maps of different scales, we extract detail feature maps D1, D2, and D3. The extraction method is as follows:

[0021] D1=I-T1, D2=T1-T2, D3=T2-T3

[0022] The sigmoid function is used to map the detail feature maps D1, D2, and D3 of three different scales into the probability space, and they are multiplied with the feature map of the ground fissure prior map respectively. The three different results of multiplication with the feature map of the ground fissure prior map are superimposed and then downsampled to obtain the final detail-enhanced second feature map.

[0023] The second aspect of the present invention provides a device for extracting ground fissures in mining areas using drone images based on prior position knowledge, comprising:

[0024] An acquisition module is used to acquire drone images of the mining area and pre-process the drone images to obtain an original image of the ground fissures;

[0025] A priori processing module, used for extracting the priori position knowledge of the ground fissures by using the priori position knowledge extractor, combining the original crack map with the priori position knowledge of the ground fissures, and generating a priori map of the ground fissures;

[0026] A constraint and enhancement module is used to downsample the original ground fissure image, and use prior position knowledge to constrain the downsampled feature space to obtain a downsampled first feature map, downsample the ground fissure prior image, and use multiple scale Gaussian convolution kernels to enhance the details of the downsampled feature map to obtain a downsampled second feature map;

[0027] The extraction module is used to superimpose the first feature map and the second feature map to obtain a target feature map, and upsample the target feature map to obtain ground fissure information in the drone image.

[0028] Optionally, in one embodiment of the present invention, the drone image is preprocessed, including one or more of radiation correction, geometric correction, registration, stitching, and cropping.

[0029] Optionally, in one embodiment of the present invention, the device further comprises:

[0030] A construction module is used to construct a priori position knowledge extractor using a first-order Gaussian difference function as a characteristic response function, weighting the ground fissure gradient amplitude through the first-order Gaussian difference function to enhance the ground fissure edge. The expression of the priori position knowledge extractor F is:

[0031]

[0032] Among them, x is the variable perpendicular to the direction of the ground fissure, y is the variable parallel to the direction of the ground fissure, L is the length of the defined template, σ is the standard deviation of the Gaussian distribution, and m is the mean of the Gaussian distribution.

[0033] Optionally, in one embodiment of the present invention, the prior position knowledge extractor uses a Sobel filter to calculate the gradients in the x and y directions in the ground fissure image, and calculates the gradient amplitude and direction, constructs a gradient direction histogram, divides the gradient direction into multi-directional bins, counts the number of gradients in each bin, and uses the constructed feature response function to weight the gradient amplitude, enhance the edge of the ground fissure, mark the part where the feature response exceeds the threshold as an edge, use a sliding window to perform density statistics on the pixels of the ground fissure edge map, and map the density statistics into prior position knowledge.

[0034] Optionally, in one embodiment of the present invention, the ground fissure prior map is downsampled, and multiple scale Gaussian convolution kernels are used to enhance the details of the downsampled feature map to obtain a downsampled second feature map, including:

[0035] Use multiple Gaussian convolution kernels of different scales to perform convolution operations on the ground fissure prior map, and obtain three fuzzy feature maps T1, T2, and T3 of different scales:

[0036] T1=G1*I, T2=G2*I, T3=G3*I

[0037] Among them, G1, G2, and G3 are Gaussian kernels with standard deviations σ1=1, σ2=2, and σ3=4, respectively, and I is the prior map of ground fissures;

[0038] Using three fuzzy feature maps of different scales, we extract detail feature maps D1, D2, and D3. The extraction method is as follows:

[0039] D1=I-T1, D2=T1-T2, D3=T2-T3

[0040] The sigmoid function is used to map the detail feature maps D1, D2, and D3 of three different scales into the probability space, and they are multiplied with the feature map of the ground fissure prior map respectively. The three different results of multiplication with the feature map of the ground fissure prior map are superimposed and then downsampled to obtain the final detail-enhanced second feature map.

[0041] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for extracting ground fissures in mining areas using drone images based on prior position knowledge as described in the above embodiment.

[0042] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned method for extracting ground fissures in mining areas using drone images based on prior position knowledge.

[0043] A fourth aspect of the present invention provides a computer program product, including a computer program, which is executed to implement the method for extracting ground fissures in mining areas using drone images based on prior position knowledge as described in the above embodiments.

[0044] The method and device for extracting ground fissures in mining areas from drone images based on prior position knowledge according to the embodiment of the present invention have the following beneficial effects:

[0045] (1) The present invention uses the first-order Gaussian difference function as the main body to construct a priori position knowledge extractor to extract the priori position knowledge of ground fissures, and gradually constrains the downsampled feature space with the priori position map, thereby narrowing the gap between the source domain data and the target data during the migration training process and improving the effect of migration training;

[0046] (2) The present invention improves the network's perception of small ground cracks during the sampling process of the ground crack prior map through the Gaussian detail enhancement module. At the same time, it weakens the interference of the complex mining environment. Compared with other methods, the ability of the present invention to extract small ground cracks is greatly improved.

[0047] (3) Compared with other deep learning methods, the accuracy of the present invention has been greatly improved and it can adapt to mining areas with more complex geological conditions.

[0048] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0050] Figure 1 A flowchart of a method for extracting ground fissures in mining areas from drone images based on prior position knowledge according to an embodiment of the present invention;

[0051] Figure 2 A network architecture diagram of a method for extracting ground fissures in mining areas using drone images based on prior location knowledge according to an embodiment of the present invention;

[0052] Figure 3 It is a structural diagram of a priori position knowledge extractor according to an embodiment of the present invention;

[0053] Figure 4 It is a flow chart of prior position knowledge constraint transfer training according to an embodiment of the present invention;

[0054] Figure 5 A structural diagram of a Gaussian detail enhancement module according to an embodiment of the present invention;

[0055] Figure 6 A map of the Qinghai Maduo earthquake research area according to an embodiment of the present invention;

[0056] Figure 7 A study area map of the Lainuoer mining area in Inner Mongolia according to an embodiment of the present invention;

[0057] Figure 8 A comparison diagram of ground fissure extraction results using different methods in an Inner Mongolia open-pit mine according to an embodiment of the present invention;

[0058] Fig. 9 The present invention is a schematic diagram of the structure of a device for extracting ground fissures in mining areas using drone images based on prior position knowledge according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0060] The following describes the method and device for extracting ground fissures in mining areas from drone images based on prior position knowledge according to an embodiment of the present invention with reference to the accompanying drawings. In view of the problems mentioned in the background technology center above that the existing deep learning methods have poor transfer training effects and difficulty in extracting small cracks when extracting ground fissures in drone images, the present invention provides a method for extracting ground fissures in mining areas from drone images based on prior position knowledge. In this method, by designing a prior position knowledge extractor, the extraction of prior position knowledge of ground fissures is realized with a first-order Gaussian difference function as the response function. Based on the U-Net3+ model, a double downsampling route is introduced, and in the branch of the downsampling of the original image of the ground fissures, a prior position knowledge constraint is introduced to gradually constrain the feature space of the downsampling. In the downsampling branch of the prior image of the ground fissures, a Gaussian detail enhancement module is added to enhance the details of the downsampling feature map of the branch and improve the network's perception of small cracks. By integrating the prior position knowledge of the ground fissures into the U-Net3+ network, the present invention effectively improves the accuracy of the network in extracting ground fissures, and at the same time enhances the ability to extract small cracks, which can play an important role in ensuring safe production in mining areas and ecological restoration in mining areas.

[0061] like Figure 1 and Figure 2 As shown in FIG. 1 , the method for extracting ground fissures in mining areas from drone images based on prior location knowledge includes the following steps:

[0062] In step S101, a drone image of the mining area is obtained, and the drone image is preprocessed to obtain an original image of the ground fissures.

[0063] In an embodiment of the present invention, after obtaining the drone image, the drone image is preprocessed by means of radiation correction, geometric correction, registration, splicing, cropping, etc. to improve the subsequent extraction accuracy of the image. The preprocessed drone image can be used to construct a deep learning data set for training the model constructed in the embodiment of the present invention.

[0064] In step S102, a priori position knowledge extractor is used to extract the priori position knowledge of the ground fissures, and the original fracture map is combined with the priori position knowledge of the ground fissures to generate a priori map of the ground fissures.

[0065] It can be understood that according to the gray value change of the ground fissures being similar to an inverted Gaussian function, a priori position knowledge detector is constructed using the first-order Gaussian difference function as the response function to extract the priori position knowledge of the ground fissures.

[0066] Specifically, the crack gray value change curve is similar to the inverse Gaussian function, and the image gray value shows a trend of first decreasing and then increasing. The first-order Gaussian difference function is used as the characteristic response function to construct the prior position knowledge extractor. The specific function expression is shown in formula (1). The first-order Gaussian difference function is used to weight the ground crack gradient amplitude and enhance the ground crack edge. The expression of the prior position knowledge extractor F is:

[0067]

[0068] Among them, x and y are variables perpendicular to and parallel to the ground fissure direction, respectively. For Gaussian distribution, the probability that a variable falls within the range of [-3σ, 3σ] is as high as 99.7%, so it is used as the value range of variable x; L is defined as the length of the template; σ and m are the standard deviation and mean of Gaussian distribution, respectively.

[0069] Optionally, in one embodiment of the present invention, Figure 3 As shown in Figure 1, the prior position knowledge extractor mainly consists of four parts:

[0070] 1) Use the Sobel filter to calculate the gradients in the x and y directions of the ground fissure image, and calculate the gradient amplitude and direction;

[0071] 2) Construct a gradient direction histogram, divide the gradient direction into multi-directional bins, and count the number of gradients in each bin;

[0072] 3) Using the constructed characteristic response function, the gradient amplitude is weighted to enhance the edge of the ground fissure, and the part where the characteristic response exceeds the threshold is marked as the edge;

[0073] 4) Use a sliding window to perform density statistics on the pixels of the ground fissure edge map and map the density statistics into prior position knowledge.

[0074] In step S103, the original ground fissure image is downsampled, and the downsampled feature space is constrained by using prior position knowledge to obtain a downsampled first feature map. The ground fissure prior image is downsampled, and multiple scale Gaussian convolution kernels are used to enhance the details of the downsampled feature map to obtain a downsampled second feature map.

[0075] Optionally, in one embodiment of the present invention, Figure 4 and Figure 5 As shown in the figure, a Gaussian detail enhancement module is constructed using multiple scale Gaussian convolution kernels to obtain detail information of different scales of the image, and these detail information are integrated into the original image to achieve multi-scale detail enhancement. Based on the U-Net3+ network, a double downsampling route is designed. In the branch of downsampling the original ground fissure image, the prior position knowledge constraint is introduced, and the Gaussian detail enhancement module is added to the downsampling route of the prior ground fissure image.

[0076] Specifically, multiple Gaussian convolution kernels of different scales are used to perform convolution operations on the ground fissure prior map to obtain three fuzzy feature maps T1, T2, and T3 of different scales, as shown in formula (2):

[0077] T1=G1*I, T2=G2*I, T3=G3*I (2)

[0078] Among them, G1, G2, and G3 are Gaussian kernels with standard deviations σ1=1, σ2=2, and σ3=4, respectively, and I is the prior map of ground fissures;

[0079] Using three fuzzy feature maps of different scales, we extract detail feature maps D1, D2, and D3. The extraction method is as follows:

[0080] D1=I-T1, D2=T1-T2, D3=T2-T3 (3)

[0081] Using the sigmoid function, as shown in formula (4), the detail feature maps D1, D2, and D3 of three different scales are mapped to the probability space and multiplied with the feature map of the ground crack prior map respectively. The three different results of multiplication with the feature map of the ground crack prior map are superimposed and then downsampled to obtain the final detail-enhanced second feature map:

[0082]

[0083] Based on the Unet3+ network, the encoding part is improved. Two independent downsampling branches are designed to introduce the obtained prior location map into the branch X of the ground fissure original image downsampling. En In the four downsampling, the prior position map is introduced to narrow the gap between the source domain and the target domain and improve the effect of transfer training. The design method is shown in formula (5):

[0084] (θ, λ)=J(θ)+λ*g(θ) (5)

[0085] Among them, J(θ) is the target domain and g(θ) is the constraint condition.

[0086] Add the Gaussian detail enhancement module to the branch Y of the ground fissure prior map downsampling En In the above example, we obtain the feature map Y after detail enhancement. f .

[0087] In step S104, the first feature map and the second feature map are superimposed to obtain a target feature map, and the target feature map is upsampled to obtain ground fissure information in the drone image.

[0088] The downsampling branch X En The feature map Xf With the downsampling branch Y En The feature map Y f The layers are superimposed, and the number of channels is restored by downsampling to obtain the feature maps of the network coding part, namely feat1, feat2, feat3, feat4, and feat5. The decoding network is used for upsampling to restore the feature map of the coding part and obtain the ground fissure information in the drone image.

[0089] The method for extracting ground fissures in mining areas using drone images based on prior position knowledge of the present invention is described in detail below through specific embodiments.

[0090] like Figure 6 and Figure 7 As shown, case study area 1 is the 2021 Qinghai Maduo M w Drone orthophotos released after the 7.4-magnitude earthquake. Study area 2 is located in the Lainuoer mining area, Zamanzhouli City, Inner Mongolia Autonomous Region, China. The data source is the data collected by the DJI Phantom 4RTK drone on May 16, 2021. The camera resolution is 4864×3648 and the field of view is 84°. Then, the drone data processing software PIX4D mapper 4.5.2 version was used to stitch the collected multiple single visible light images to obtain a visible light orthophoto with a spatial resolution of 2.9 cm for the experimental area 2. Compared with the Inner Mongolia open-pit mine data, the Qinghai Maduo earthquake ground fissure data has the advantages of a single geographical environment and a large amount of data. Therefore, the Qinghai Maduo data is used as the source data for network transfer learning, and the Inner Mongolia open-pit mine data is used as the target data.

[0091] The method for extracting ground fissures in mining areas from drone images based on prior position knowledge in an embodiment of the present invention specifically includes:

[0092] Step 1: UAV image acquisition and preprocessing.

[0093] Step 2: Dataset creation. Dataset 1 and Dataset 2 were created using the Qinghai Maduo earthquake ground fissure data and Inner Mongolia open-pit mine data.

[0094] Step 3: Experimental environment configuration. PyTorch 1.11.7 framework was used to build the PF-Unet3+ network. All tasks were performed on a workstation equipped with a CPU (13th Gen Intel(R) Core(TM) i7-13700KF) and a GPU (NVIDIAGeForce RTX 4090). The number of training epochs was set to 100, the size of each small batch of data was 10, the initial learning rate was set to 7e-3, the loss function was Cross Entorpy Loss, and when optimizing network parameters, the stochastic gradient descent (SGD) optimizer was selected, and the momentum parameter was set to 0.8. At the same time, weight decay was applied with a decay rate of 1e-5.

[0095] Step 4: Ground fissure extraction. In this experiment, DeepLabV3+, U-Net, U-Net3+, and PF-Unet were used for comparative experiments. First, pre-training was performed using dataset 1, and the trained weights were transferred. Based on the pre-training, training was performed using dataset 2. The experimental comparison results are shown in Figure 2. Figure 8 As shown. Through comparative experiments, it can be seen that Deeplabv3+ has many ground cracks missing in the extraction results; U-Net network has a good extraction effect on ground cracks, but there is a phenomenon of missing extraction for small cracks, and there is a phenomenon of discontinuous extraction of ground cracks. The U-Net3+ network has a certain improvement in the extraction ability of small ground cracks, and there are fewer ground cracks missing in extraction; compared with other experiments, PF-Unet3+ has greatly reduced the phenomenon of missing extraction of ground cracks, and the extraction results of small ground cracks are significantly improved, and the extraction results are more continuous. The Precision (Pr), Recall (Re), F1-score (F1) and Mean Intersection over Union (MIoU) indicators are used to fairly evaluate the crack extraction results. Table 1 shows the evaluation indicators of PF-Unet3+ on Dataset 2. It is only lower than the U-Net model in MIoU, but the precision rate is 94.24%, the recall rate is 96.15%, and the F1-Score is 88.24%, which is much better than other methods.

[0096] Table 1

[0097] Method Precision(%) Recall(%) F1-Score(%) MIoU(%) Deeplabv3+ 80.10 86.01 76.24 78.36 U-Net 81.56 85.83 77.15 79.48 U-Net3+ 92.95 94.14 87.87 93.86 PF-Unet3+ 94.24 96.15 88.24 93.26

[0098] The method for extracting ground fissures in mining areas from drone images based on prior position knowledge in an embodiment of the present invention obtains the prior position knowledge of ground fissures, and at the same time, combines the original image of ground fissures with the prior position to generate a prior map of ground fissures. Then, on the basis of the U-Net3+ network, an independent downsampling branch is added. In the branch of downsampling the original image of ground fissures, the prior position knowledge constraint is introduced to gradually constrain the feature space of downsampling. In the downsampling branch of the ground fissure prior map, a Gaussian detail enhancement module is added to enhance the details of the downsampled feature map. The feature maps of the two downsamples are superimposed to obtain the final feature map. Finally, the obtained feature map is jump-connected and upsampled to obtain the final ground fissure extraction map. The present invention improves the problems of poor migration training effect and weak extraction ability of small ground fissures in current deep learning methods based on the extracted prior position knowledge of ground fissures and the designed detail enhancement module.

[0099] Next, a device for extracting ground fissures in mining areas using drone images based on prior position knowledge proposed in an embodiment of the present invention will be described with reference to the accompanying drawings.

[0100] Fig. 9 The present invention is a schematic diagram of the structure of a device for extracting ground fissures in mining areas using drone images based on prior position knowledge according to an embodiment of the present invention.

[0101] like Fig. 9 As shown, the mining area ground fissure extraction device 10 based on prior position knowledge using drone images includes: an acquisition module 100, a priori processing module 200, a constraint and enhancement module 300 and an extraction module 400.

[0102] The acquisition module 100 is used to acquire drone images of the mining area and pre-process the drone images to obtain the original images of the ground fissures.

[0103] The priori processing module 200 is used to extract the priori position knowledge of the ground fissures by using the priori position knowledge extractor, combine the original crack map with the priori position knowledge of the ground fissures, and generate a priori map of the ground fissures.

[0104] The constraint and enhancement module 300 is used to downsample the original ground fissure image, and use the prior position knowledge to constrain the downsampled feature space to obtain a downsampled first feature map, downsample the ground fissure prior image, and use multiple scale Gaussian convolution kernels to enhance the details of the downsampled feature map to obtain a downsampled second feature map.

[0105] The extraction module 400 is used to superimpose the first feature map and the second feature map to obtain a target feature map, and upsample the target feature map to obtain ground fissure information in the drone image.

[0106] Optionally, in one embodiment of the present invention, the drone image is preprocessed, including one or more of radiation correction, geometric correction, registration, stitching, and cropping.

[0107] Optionally, in one embodiment of the present invention, the mining area ground fissure extraction device 10 based on drone imagery and prior position knowledge further includes:

[0108] The construction module is used to construct a priori position knowledge extractor by taking the first-order Gaussian difference function as the characteristic response function, weighting the ground fissure gradient amplitude by the first-order Gaussian difference function, and enhancing the ground fissure edge. The expression of the priori position knowledge extractor F is:

[0109]

[0110] Among them, x is the variable perpendicular to the direction of the ground fissure, y is the variable parallel to the direction of the ground fissure, L is the length of the defined template, σ is the standard deviation of the Gaussian distribution, and m is the mean of the Gaussian distribution.

[0111] Optionally, in one embodiment of the present invention, the prior position knowledge extractor uses a Sobel filter to calculate the gradients in the x and y directions in the ground fissure image, and calculates the gradient amplitude and direction, constructs a gradient direction histogram, divides the gradient direction into multi-directional bins, counts the number of gradients in each bin, and uses the constructed feature response function to weight the gradient amplitude, enhance the edge of the ground fissure, mark the part where the feature response exceeds the threshold as an edge, use a sliding window to perform density statistics on the pixels of the ground fissure edge map, and map the density statistics into prior position knowledge.

[0112] Optionally, in one embodiment of the present invention, the ground fissure prior map is downsampled, and multiple scale Gaussian convolution kernels are used to enhance the details of the downsampled feature map to obtain a downsampled second feature map, including:

[0113] Use multiple Gaussian convolution kernels of different scales to perform convolution operations on the ground fissure prior map, and obtain three fuzzy feature maps T1, T2, and T3 of different scales:

[0114] T1=G1*I, T2=G2*I, T3=G3*I

[0115] Among them, G1, G2, and G3 are Gaussian kernels with standard deviations σ1=1, σ2=2, and σ3=4, respectively, and I is the prior map of ground fissures;

[0116] Using three fuzzy feature maps of different scales, we extract detail feature maps D1, D2, and D3. The extraction method is as follows:

[0117] D1=I-T1, D2=T1-T2, D3=T2-T3

[0118] The sigmoid function is used to map the detail feature maps D1, D2, and D3 of three different scales into the probability space, and they are multiplied with the feature map of the ground fissure prior map respectively. The three different results of multiplication with the feature map of the ground fissure prior map are superimposed and then downsampled to obtain the final detail-enhanced second feature map.

[0119] It should be noted that the aforementioned explanation of the embodiment of the method for extracting ground fissures in mining areas using drone images based on prior position knowledge is also applicable to the device for extracting ground fissures in mining areas using drone images based on prior position knowledge in this embodiment, and will not be repeated here.

[0120] According to the device for extracting ground fissures in mining areas from drone images based on prior position knowledge proposed in an embodiment of the present invention, the feature space of the model is gradually constrained using the prior knowledge extracted by the prior position extractor, thereby reducing the difference between the source domain and the target domain data to improve the effect of migration training. At the same time, the details of the ground fissure images are gradually enhanced using the Gaussian detail enhancement module, weakening the interference of complex landforms, and ultimately achieving accurate extraction of ground fissures in complex mining areas in high-resolution drone images, playing an important role in ensuring safe production in mining areas and ecological restoration in mining areas.

[0121] Furthermore, an embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for extracting ground fissures in mining areas using drone images based on prior location knowledge as described in the above embodiment.

[0122] Furthermore, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, it implements the above-mentioned method for extracting ground fissures in mining areas using drone images based on prior position knowledge.

[0123] In addition, an embodiment of the present invention also provides a computer program product, including a computer program, which is executed to implement the method for extracting ground fissures in mining areas using drone images based on prior position knowledge in the above embodiment.

[0124] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0125] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0126] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

Claims

1. A method for extracting ground fissures in mining areas from drone images based on prior location knowledge, characterized in that: The following steps are involved: Obtaining drone images of the mining area, and preprocessing the drone images to obtain original images of ground fissures; Extracting prior position knowledge of ground fissures by using a prior position knowledge extractor, combining the original crack map with the prior position knowledge of ground fissures to generate a prior map of ground fissures; Downsampling the original ground fissure image, and using prior position knowledge to constrain the downsampled feature space to obtain a downsampled first feature map; downsampling the ground fissure prior image, and using multiple scale Gaussian convolution kernels to enhance the details of the downsampled feature map to obtain a downsampled second feature map; The first feature map and the second feature map are superimposed to obtain a target feature map, and the target feature map is upsampled to obtain ground fissure information in the drone image; Before extracting the priori position knowledge of the ground fissures using the priori position knowledge extractor, the method further includes: The first-order Gaussian difference function is used as the characteristic response function to construct a priori position knowledge extractor. The ground fissure gradient amplitude is weighted by the first-order Gaussian difference function to enhance the ground fissure edge. The expression of the priori position knowledge extractor F is: Among them, x is the variable perpendicular to the direction of the ground fissure, y is the variable parallel to the direction of the ground fissure, L is the length of the definition template, σ is the standard deviation of the Gaussian distribution, and m is the mean of the Gaussian distribution; The prior position knowledge extractor uses a Sobel filter to calculate the gradients in the x and y directions in the ground fissure image, and calculates the gradient amplitude and direction, constructs a gradient direction histogram, divides the gradient direction into multi-directional bins, counts the number of gradients in each bin, and uses the constructed feature response function to weight the gradient amplitude, enhance the edge of the ground fissure, mark the part where the feature response exceeds the threshold as the edge, and use a sliding window to perform density statistics on the pixels of the ground fissure edge map, and map the density statistics value into prior position knowledge; The ground fissure prior map is downsampled, and multiple scale Gaussian convolution kernels are used to enhance the details of the downsampled feature map to obtain the downsampled second feature map, including: Use multiple Gaussian convolution kernels of different scales to perform convolution operations on the ground fissure prior map, and obtain three fuzzy feature maps T1, T2, and T3 of different scales: T1=G1*I, T2=G2*I, T3=G3*I Among them, G1, G2, and G3 are Gaussian kernels with standard deviations σ1=1, σ2=2, and σ3=4, respectively, and I is a priori map of ground fissures. Using three fuzzy feature maps of different scales, the detail feature maps D1, D2, and D3 are extracted. The extraction method is as follows: D1=I-T1, D2=T1-T2, D3=T2-T3 The sigmoid function is used to map the detail feature maps D1, D2, and D3 of three different scales into the probability space, and they are multiplied with the feature map of the ground fissure prior map respectively. The three different results of multiplication with the feature map of the ground fissure prior map are superimposed and then downsampled to obtain the final detail-enhanced second feature map.

2. The method according to claim 1, characterized in that The drone image is preprocessed, including one or more of radiation correction, geometric correction, registration, stitching, and cropping.

3. A device for extracting ground fissures in mining areas from drone images based on prior position knowledge, characterized in that: include: An acquisition module is used to acquire drone images of the mining area and pre-process the drone images to obtain an original image of the ground fissures; A priori processing module, used for extracting the priori position knowledge of the ground fissures by using the priori position knowledge extractor, combining the original crack map with the priori position knowledge of the ground fissures, and generating a priori map of the ground fissures; A constraint and enhancement module is used to downsample the original ground fissure image, and use prior position knowledge to constrain the downsampled feature space to obtain a downsampled first feature map, downsample the ground fissure prior image, and use multiple scale Gaussian convolution kernels to enhance the details of the downsampled feature map to obtain a downsampled second feature map; An extraction module, used for superimposing the first feature map and the second feature map to obtain a target feature map, and upsampling the target feature map to obtain ground fissure information in the drone image; A construction module is used to construct a priori position knowledge extractor using a first-order Gaussian difference function as a characteristic response function, weighting the ground fissure gradient amplitude through the first-order Gaussian difference function to enhance the ground fissure edge. The expression of the priori position knowledge extractor F is: Among them, x is the variable perpendicular to the direction of the ground fissure, y is the variable parallel to the direction of the ground fissure, L is the length of the definition template, σ is the standard deviation of the Gaussian distribution, and m is the mean of the Gaussian distribution; The prior position knowledge extractor uses a Sobel filter to calculate the gradients in the x and y directions in the ground fissure image, and calculates the gradient amplitude and direction, constructs a gradient direction histogram, divides the gradient direction into multi-directional bins, counts the number of gradients in each bin, and uses the constructed feature response function to weight the gradient amplitude, enhance the edge of the ground fissure, mark the part where the feature response exceeds the threshold as the edge, and use a sliding window to perform density statistics on the pixels of the ground fissure edge map, and map the density statistics value into prior position knowledge; The ground fissure prior map is downsampled, and multiple scale Gaussian convolution kernels are used to enhance the details of the downsampled feature map to obtain the downsampled second feature map, including: Use multiple Gaussian convolution kernels of different scales to perform convolution operations on the ground fissure prior map, and obtain three fuzzy feature maps T1, T2, and T3 of different scales: T1=G1*I, T2=G2*I, T3=G3*I Among them, G1, G2, and G3 are Gaussian kernels with standard deviations σ1=1, σ2=2, and σ3=4, respectively, and I is a priori map of ground fissures. Using three fuzzy feature maps of different scales, the detail feature maps D1, D2, and D3 are extracted. The extraction method is as follows: D1=I-T1, D2=T1-T2, D3=T2-T3 The sigmoid function is used to map the detail feature maps D1, D2, and D3 of three different scales into the probability space, and they are multiplied with the feature map of the ground fissure prior map respectively. The three different results of multiplication with the feature map of the ground fissure prior map are superimposed and then downsampled to obtain the final detail-enhanced second feature map.

4. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for extracting ground fissures in mining areas using drone images based on prior position knowledge as described in any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method for extracting ground fissures in mining areas using drone images based on prior position knowledge as described in any one of claims 1-2.

6. A computer program product comprising a computer program, characterized in that The computer program is executed to implement the method for extracting ground fissures in mining areas using drone images based on prior location knowledge as described in any one of claims 1-2.

Images