Mine geological disaster detection method, device, equipment, medium and program product

Abstract

The invention provides a mine geological disaster detection method, device and equipment, a medium and a program product, and is applied to the technical field of geological detection and the technical field of computers. The method comprises steps that a first remote sensing image and a second remote sensing image of a to-be-detected target area are acquired in real time, and the first remote sensing image and the second remote sensing image are images of the same place at different time; performing feature extraction on the first remote sensing image by using a first network structure of a deep learning model to obtain a first image feature; performing feature extraction on the second remote sensing image by using a second network structure of the deep learning model to obtain a second image feature; performing feature fusion on the first image feature and the second image feature by using a third network structure of a deep learning model to obtain a third image feature; and performing geological disaster detection on the third image feature by using a fourth network structure of the deep learning model to obtain a geological disaster detection result of the to-be-detected target area.

Description

Technical Field

[0001] The present invention relates to the fields of geological detection technology and computer technology, and in particular to a method, device, equipment, medium and program product for detecting geological disasters in mines. Background Art

[0002] With the accelerated development of industrialization, the development and utilization of mineral resources has gradually become increasingly important. However, mining activities will inevitably have a serious impact on the geological environment. These environmental problems include but are not limited to geological disasters such as landslides, collapses, ground subsidence, and ground fissures, which will seriously threaten the ecological security of mining areas and surrounding areas.

[0003] Traditionally, the detection of geohazards in mining activities usually uses remote sensing technology to obtain remote sensing images for data processing, and the identification and interpretation of geohazards are usually achieved through human-computer collaboration.

[0004] In the process of realizing the concept of the present invention, the inventors found that there are at least the following problems in the related technology: in the process of human-computer collaborative interpretation, the identification of geological disasters depends on manual experience, and the accuracy of the interpretation of geological disasters is low. Summary of the invention

[0005] In view of this, the present invention provides a method, device, equipment, medium and program product for detecting geological disasters in mines.

[0006] One aspect of the present invention provides a method for detecting geological disasters in mines, comprising: acquiring a first remote sensing image and a second remote sensing image of a target area to be detected in real time, wherein the first remote sensing image and the second remote sensing image are images of the same place at different times; using a first network structure of a deep learning model to perform feature extraction on the first remote sensing image to obtain a first image feature, wherein the first image feature represents the geological features of the target area to be detected of the first remote sensing image; using a second network structure of the deep learning model to perform feature extraction on the second remote sensing image to obtain a second image feature, wherein the second image feature represents the geological features of the target area to be detected of the second remote sensing image; using a third network structure of the deep learning model to perform feature fusion on the first image feature and the second image feature to obtain a third image feature, wherein the third image feature represents the different image change features between the first image feature and the second image feature; using a fourth network structure of the deep learning model to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, wherein the geological disaster detection result includes whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the change process of the geological disaster that occurs.

[0007] Optionally, the third network structure of the deep learning model is used to perform feature fusion on the first image feature and the second image feature to obtain a third image feature, including: performing feature splicing on the first image feature and the second image feature to obtain an image splicing feature; performing convolution processing on the first image feature and the second image feature to obtain a fourth image feature; and performing a product operation on the image splicing feature and the fourth image feature to obtain the third image feature.

[0008] Optionally, performing convolution processing on the first image feature and the second image feature to obtain a fourth image feature includes: performing a difference operation on the first image feature and the second image to obtain a second intermediate image feature; and performing convolution processing on the second intermediate image feature to obtain the fourth image feature.

[0009] Optionally, the first network structure of the deep learning model is used to extract features of the first remote sensing image to obtain first image features, including: using an encoder module of the first network structure to perform image encoding on the first remote sensing image to obtain first intermediate image features; and using a decoder module of the first network structure to decode the first intermediate image features to obtain first image features.

[0010] Optionally, the fourth network structure of the deep learning model is used to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, including: performing a convolution operation on the third image feature to obtain a third intermediate image feature; using a fully connected layer to perform geological disaster detection on the third intermediate image feature to obtain an address disaster detection result of the target area to be detected.

[0011] Optionally, the above method also includes: performing orthorectification processing on the first remote sensing image to obtain a corrected first remote sensing image; performing registration and fusion processing on the corrected first remote sensing image to obtain a fused first remote sensing image; performing image enhancement processing on the fused first remote sensing image to obtain an enhanced first remote sensing image; performing image cropping processing on the enhanced first remote sensing image to obtain a cropped first remote sensing image.

[0012] Another aspect of the present invention provides a mining geological disaster detection device, including: an image acquisition module, used to acquire a first remote sensing image and a second remote sensing image of a target area to be detected in real time, wherein the first remote sensing image and the second remote sensing image are images of the same place at different times; a first image feature extraction module, used to extract features of the first remote sensing image using a first network structure of a deep learning model to obtain first image features, wherein the first image features represent geological features of the target area to be detected in the first remote sensing image; a second image feature extraction module, used to extract features of the second remote sensing image using a second network structure of the deep learning model to obtain second image features, wherein the second image features The image features represent the geological features of the target area to be detected in the second remote sensing image; the third image feature fusion module is used to use the third network structure of the deep learning model to fuse the first image features and the second image features to obtain the third image features, wherein the third image features represent the different image change characteristics between the first image features and the second image features; the detection module is used to use the fourth network structure of the deep learning model to perform geological disaster detection on the third image features to obtain the geological disaster detection results of the target area to be detected, wherein the geological disaster detection results include whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the changing process of the geological disaster that occurs.

[0013] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0014] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0015] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when being executed.

[0016] According to an embodiment of the present invention, a first remote sensing image and a second remote sensing image of the same place at different times are acquired in real time through remote sensing technology, and a first network structure of a deep learning model is used to extract features of the first remote sensing image to obtain first image features, and a second network structure is used to extract features of the second remote sensing image to obtain second image features; and a third network structure is used to fuse the first image features and the second image features to obtain third image features; and a fourth network structure is used to detect geological disasters on the third image features to obtain geological disaster detection results for the target area to be detected. Due to the use of a technical means of combining remote sensing technology and a deep learning model to identify and interpret geological disasters in remote sensing images, the technical problems of low interpretation accuracy and untimely monitoring of geological disasters in mining activities caused by human-machine collaboration in the prior art are at least partially overcome, thereby achieving automatic identification of geological disaster categories and development processes, and improving the accuracy of geological disaster identification and interpretation, providing an important reference for monitoring geological disasters in mining activities. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0018] Figure 1 An exemplary system architecture to which a method and device for detecting mining geological disasters according to an embodiment of the present invention can be applied is shown;

[0019] Figure 2 A flow chart of a method for detecting mining geological disasters according to an embodiment of the present invention is shown;

[0020] Figure 3 A flow chart of a method for obtaining a third image feature according to an embodiment of the present invention is shown;

[0021] Figure 4 A schematic diagram of a mine geological disaster detection method according to an embodiment of the present invention is shown;

[0022] Figure 5 A block diagram showing a mining geological disaster detection device according to an embodiment of the present invention; and

[0023] Figure 6 A block diagram of an electronic device suitable for implementing a method for detecting mining geological disasters according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0029] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0030] An embodiment of the present invention provides a method for detecting geological disasters in mines, comprising: acquiring a first remote sensing image and a second remote sensing image of a target area to be detected in real time, wherein the first remote sensing image and the second remote sensing image are images of the same place at different times; using a first network structure of a deep learning model to perform feature extraction on the first remote sensing image to obtain a first image feature, wherein the first image feature represents the geological features of the target area to be detected of the first remote sensing image; using a second network structure of the deep learning model to perform feature extraction on the second remote sensing image to obtain a second image feature, wherein the second image feature represents the geological features of the target area to be detected of the second remote sensing image; using a third network structure of the deep learning model to perform feature fusion on the first image feature and the second image feature to obtain a third image feature, wherein the third image feature represents the different image change features between the first image feature and the second image feature; using a fourth network structure of the deep learning model to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, wherein the geological disaster detection result includes whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the change process of the geological disaster that occurs.

[0031] Figure 1 The exemplary system architecture of the mine geological disaster detection method and device according to the embodiment of the present invention is shown. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0032] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0033] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0035] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0036] It should be noted that the mine geological disaster detection method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the mine geological disaster detection device provided in the embodiment of the present invention can generally be set in the server 105. The mine geological disaster detection method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the mine geological disaster detection device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the mine geological disaster detection method provided in the embodiment of the present invention can also be performed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or by other terminal devices that are different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the mining geological disaster detection device provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0037] For example, the first remote sensing image and the second remote sensing image may be originally stored in any one of the first terminal device 101, the second terminal device 102 or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored in an external storage device and may be imported into the first terminal device 101. Then, the first terminal device 101 may locally execute the mining geological disaster detection method provided by the embodiment of the present invention, or send the first remote sensing image and the second remote sensing image to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the first remote sensing image and the second remote sensing image may execute the mining geological disaster detection method provided by the embodiment of the present invention.

[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0039] Figure 2 A flow chart of a mine geological disaster detection method according to an embodiment of the present invention is shown.

[0040] like Figure 2 As shown, the method 200 includes operations S210 to S250.

[0041] In operation S210, a first remote sensing image and a second remote sensing image of a target area to be detected are acquired in real time, wherein the first remote sensing image and the second remote sensing image are images of the same location at different times.

[0042] According to an embodiment of the present invention, the target area to be detected may be a preset mining area in a mine to be detected. A remote sensing image may be generated in real time for the geological environment of the target area to be detected by a remote sensing device.

[0043] According to an embodiment of the present invention, after a preset time period of a first remote sensing image of a target area to be detected is acquired, a second remote sensing image can be acquired again in the same target area to be detected. The first remote sensing image and the second remote sensing image are acquired at different times at the same location. The preset time period can be one month, three months, half a year, etc. The preset time period is set according to the actual geological disaster detection needs, and the present invention does not make specific limitations.

[0044] According to an embodiment of the present invention, the second remote sensing image may reflect the surface texture information and ground object information of the target area to be detected when the second remote sensing image is acquired after a preset time period.

[0045] In operation S220, a first network structure of a deep learning model is used to perform feature extraction on the first remote sensing image to obtain a first image feature, wherein the first image feature represents a geological feature of a target area to be detected in the first remote sensing image.

[0046] According to an embodiment of the present invention, the first remote sensing image can reflect the surface texture information and object information of the target area to be detected when the first remote sensing image is obtained. The surface texture information can identify the area, objects, etc. in the first remote sensing image, and reflect the landform type, soil type, object type, landform shape, etc. of the target area to be detected. The object information can feedback the object information in the target area to be detected, such as buildings, trees, mountains, etc.

[0047] According to an embodiment of the present invention, the deep learning model may be a model for detecting geological disasters in a target area to be detected and built based on a dual-stream network structure. The first network structure may be a branch structure of the dual-stream network structure.

[0048] According to the embodiment of the present invention, the first remote sensing image can be feature extracted through the first network structure to obtain geological feature information in the first remote sensing image.

[0049] In operation S230, a second network structure of the deep learning model is used to perform feature extraction on the second remote sensing image to obtain second image features, wherein the second image features represent geological features of the target area to be detected in the second remote sensing image.

[0050] According to an embodiment of the present invention, the second remote sensing image may reflect the surface texture information and ground object information of the target area to be detected when the second remote sensing image is acquired after a preset time period.

[0051] According to an embodiment of the present invention, the second network structure may be another branch structure of the dual-stream network structure. The second network structure and the first network structure share network parameters. The second remote sensing image may be feature extracted by the second network structure to obtain geological features of the target area to be detected in the second remote sensing image.

[0052] In operation S240, the first image feature and the second image feature are feature-fused using a third network structure of the deep learning model to obtain a third image feature, wherein the third image feature represents different image change features between the first image feature and the second image feature.

[0053] According to an embodiment of the present invention, the third network structure may be a feature fusion network structure, which performs feature fusion on the first image feature and the second image feature to obtain an image feature that is different from the first image feature and the second image feature, that is, a third image feature.

[0054] According to an embodiment of the present invention, the different image features among the first image feature and the second image feature represent image change features of geological changes between the first remote sensing image and the second remote sensing image within a preset time period in the target area to be detected.

[0055] In operation S250, the fourth network structure of the deep learning model is used to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, wherein the geological disaster detection result includes whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the changing process of the geological disaster.

[0056] According to an embodiment of the present invention, the fourth network structure can be a discriminator network, which is used to perform geological disaster detection on the third image features obtained through feature fusion, so as to determine whether a geological disaster has occurred in the area where geological changes have occurred in the target area to be detected, the type of geological disaster that has occurred, and the changing process of the geological disaster.

[0057] According to an embodiment of the present invention, the types of geological disasters may include but are not limited to landslides, collapses, ground subsidence, and ground fissures. The change process of the occurrence of a geological disaster may be a process from the geological features of the first remote sensing image to the geological features of the second remote sensing image, from the appearance of the geological disaster to its disappearance (i.e., repair), or from the repair of the geological disaster to its appearance, etc.

[0058] According to an embodiment of the present invention, a first remote sensing image and a second remote sensing image of the same place at different times are acquired in real time through remote sensing technology, and a first network structure of a deep learning model is used to extract features of the first remote sensing image to obtain first image features, and a second network structure is used to extract features of the second remote sensing image to obtain second image features; and a third network structure is used to fuse the first image features and the second image features to obtain third image features; and a fourth network structure is used to detect geological disasters on the third image features to obtain geological disaster detection results for the target area to be detected. Due to the use of a technical means of combining remote sensing technology and a deep learning model to identify and interpret geological disasters in remote sensing images, the technical problems of low interpretation accuracy and untimely monitoring of geological disasters in mining activities caused by human-machine collaboration in the prior art are at least partially overcome, thereby achieving automatic identification of geological disaster categories and development processes, and improving the accuracy of geological disaster identification and interpretation, providing an important reference for monitoring geological disasters in mining activities.

[0059] According to an embodiment of the present invention, the method may further include: performing orthorectification processing on the first remote sensing image to obtain a corrected first remote sensing image; performing registration and fusion processing on the corrected first remote sensing image to obtain a fused first remote sensing image; performing image enhancement processing on the fused first remote sensing image to obtain an enhanced first remote sensing image; performing image cropping processing on the enhanced first remote sensing image to obtain a cropped first remote sensing image.

[0060] According to an embodiment of the present invention, the orthorectification processing is performed on the first remote sensing image in order to correct the geometric distortion in the first remote sensing image caused by the tilt angle of the sensor of the remote sensing device or the terrain undulations, so that the position of each pixel in the first remote sensing image accurately corresponds to the actual position on the ground, and the error generated in the projection process is corrected, thereby obtaining the corrected first remote sensing image.

[0061] According to an embodiment of the present invention, the spectral information of the multi-spectral whole scene data and the high spatial resolution data of the panchromatic whole scene data are accurately aligned and fused for the corrected first remote sensing image to generate a remote sensing image that contains both rich spectral information and high spatial resolution, thereby obtaining the fused first remote sensing image.

[0062] According to an embodiment of the present invention, the fused first remote sensing image is subjected to image enhancement and cropping, so that the first remote sensing image obtained after cropping has a high-quality image base map to ensure the accuracy of the remote sensing image data in subsequent analysis and application.

[0063] According to the embodiment of the present invention, the operation flow of preprocessing the second remote sensing image is the same as the operation of preprocessing the first remote sensing image, which will not be repeated here.

[0064] According to an embodiment of the present invention, a first network structure of a deep learning model is used to perform feature extraction on a first remote sensing image to obtain a first image feature, including: using an encoder module of the first network structure to perform image encoding on the first remote sensing image to obtain a first intermediate image feature; and using a decoder module of the first network structure to decode the first intermediate image feature to obtain the first image feature.

[0065] According to an embodiment of the present invention, the first network structure of the deep learning model may include an encoder module and a decoder module. The first remote sensing image may be input into the encoder module of the first network structure for image encoding to obtain a first intermediate image feature.

[0066] According to an embodiment of the present invention, the first remote sensing image is subjected to image encoding, and this process may be a process of increasing the dimension, so that the geological texture information in the target area to be detected in the first remote sensing image can be better extracted.

[0067] According to an embodiment of the present invention, the obtained first intermediate image feature can be input into the decoder module of the first network structure for decoding processing to obtain the first image feature. This process can be a process of reducing the dimension, so that the key features of the first remote sensing image can be extracted. For example, the key features can be geological morphological features, geological shape features, etc. of the target area to be detected in the first remote sensing image.

[0068] According to an embodiment of the present invention, the second network structure of the deep learning model may also include an encoder module and a decoder module. The encoder module in the first network structure and the encoder module in the second network structure may share shared weights; the decoder module in the first network structure and the decoder module in the second network structure may share shared weights.

[0069] According to an embodiment of the present invention, the method of extracting features from the second remote sensing image using the second network structure to obtain second image features is the same as the method of extracting features from the first remote sensing image using the first network structure to obtain first image features, which will not be repeated here.

[0070] Figure 3 A flow chart of a method for obtaining a third image feature according to an embodiment of the present invention is shown.

[0071] like Figure 3 As shown, the method 300 may include operations S310 to S330.

[0072] In operation S310, feature stitching is performed on the first image feature and the second image feature to obtain an image stitching feature.

[0073] In operation S320, convolution processing is performed on the first image feature and the second image feature to obtain a fourth image feature.

[0074] In operation S330, a product operation is performed on the image stitching feature and the fourth image feature to obtain a third image feature.

[0075] According to an embodiment of the present invention, the first image feature can be transformed into and second image features Stitching is performed to obtain image stitching features, thereby obtaining smooth and seamless panoramic image features reconstructed by stitching .

[0076] According to an embodiment of the present invention, the feature stitching method does not change the depth information of the first image feature and the second image feature, but increases the expression capability of the depth model by expanding the width of the image feature.

[0077] According to an embodiment of the present invention, a convolution operation can be performed on the first image feature and the second image feature to obtain a fourth image feature, including: performing a difference operation on the first image feature and the second image feature to obtain a second intermediate image feature; and performing a convolution operation on the second intermediate image feature to obtain a fourth image feature.

[0078] According to an embodiment of the present invention, by and second image feature Perform feature subtraction and calculate the absolute value. The result represents the different feature parts between the first image feature and the second image feature, that is, the second intermediate image feature. , which can be expressed as .

[0079] According to an embodiment of the present invention, the second intermediate image feature is then convolved to obtain a two-dimensional score matrix , where H is the height of the remote sensing image feature map, W is the width of the remote sensing image feature map, and R is a real number matrix. The two-dimensional fractional matrix S can represent the part where geological changes occur between the first image feature and the second image feature, that is, the fourth image feature.

[0080] According to an embodiment of the present invention, the image stitching feature can be The third image feature is obtained by multiplying the two-dimensional score matrix S representing the fourth image feature That is, the characteristic parts where geological changes have occurred are extracted from the image stitching features.

[0081] According to an embodiment of the present invention, the fourth network structure of the deep learning model is used to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, including: performing a convolution operation on the third image feature to obtain a third intermediate image feature; and using a fully connected layer to perform geological disaster detection on the third intermediate image feature to obtain an address disaster detection result of the target area to be detected.

[0082] According to an embodiment of the present invention, the third image feature can be input into the fourth network structure for convolution operation to restore the third image feature to the same image size as the first image feature and the second image feature, thereby obtaining a third intermediate image feature.

[0083] According to an embodiment of the present invention, the fully connected layer in the fourth network structure can be used to perform a (2+4)-dimensional prediction on each pixel point of the third intermediate image feature, wherein the first two dimensions represent whether the geological disaster appears in the first remote sensing image or the second remote sensing image to characterize the changing process of the geological disaster, and the following four dimensions represent the type of geological disaster.

[0084] According to an embodiment of the present invention, for example, one-hot encoding can be used to represent the change process of geological disasters (such as Table 1) and the categories of geological disasters (such as Table 2), that is:

[0085] Table 1

[0086]

[0087] Table 2

[0088]

[0089] According to an embodiment of the present invention, for example, a pixel point of the third intermediate image feature is predicted to obtain a (2+4)-dimensional prediction of the pixel point. For example, the (2+4)-dimensional prediction of the pixel point is (0, 1, 1, 0, 0, 0,). This indicates that the first two dimensions (0, 1) of the pixel point indicate that the geological disaster changes from the first remote sensing image to the second remote sensing image from non-existence to appearance. That is, no geological disaster occurred at the pixel point in the first remote sensing image, but a geological disaster occurred at the pixel point in the second remote sensing image. Similarly, if the prediction of the first two dimensions is (1, 0), it indicates that a geological disaster occurred at the pixel point in the first remote sensing image, and after repair, displacement, etc., the geological disaster at the pixel point in the second remote sensing image disappears. The last four dimensions (1, 0, 0, 0) of the pixel point indicate that the type of geological disaster that occurred at the pixel point is a landslide. If the (2+4)-dimensional prediction obtained for the pixel is (0,0,0,0,0,0), it means that there is no change process of geological disasters at the pixel point from the first remote sensing image to the second remote sensing image, that is, no geological disasters have occurred. By combining the above pixel points that obtain the same prediction results, the prediction results for the third intermediate image features can be obtained. The prediction results represent the geological disaster results of the target area to be detected, that is, whether a geological disaster has occurred in the target area to be detected, the type of geological disasters that have occurred, and the change process.

[0090] According to an embodiment of the present invention, the network structure of the deep learning model can be used to detect geological disasters in the first remote sensing image and the second remote sensing image, and the geological disaster detection results of the target area to be detected can be automatically identified and interpreted, thereby improving the accuracy of detection and providing an important reference for the impact of geological environment in ore mining activities.

[0091] According to an embodiment of the present invention, before the deep learning model is used to process the first remote sensing image and the second remote sensing image, it is necessary to train the deep learning model to obtain a trained deep learning model for detecting ore geological disasters. The trained deep learning model can be obtained through the following training process.

[0092] According to an embodiment of the present invention, a first remote sensing image sample and a second remote sensing image sample of a target area to be detected can be acquired in real time, and the first remote sensing image sample and the second remote sensing image sample are image samples of the same location at different times.

[0093] According to an embodiment of the present invention, the first remote sensing image sample and the second remote sensing image sample may be preprocessed by orthorectification, registration and fusion, image enhancement and cropping, respectively, to obtain the preprocessed first remote sensing image sample and the second remote sensing image sample.

[0094] According to the embodiments of the present invention, based on the knowledge of geological experts and previous geological survey data, landslides usually occur on slopes, manifested as soil or rock mass sliding down along a weak surface, and image features include a dustpan-shaped landslide perimeter and a steeper horseshoe-shaped sliding body. Collapse occurs on steep slopes, manifested as a sudden fall of rock or soil, appearing in an arc or triangle shape on the image, with lighter tones. Ground collapse is mostly caused by mining activities, appearing in a ring or ellipse shape on the image, with varying shades of light and dark. Ground fissures are caused by underground resource exploitation or excessive groundwater development, and appear as dark lines on the image.

[0095] According to an embodiment of the present invention, remote sensing interpretation and annotation can be performed on a first real sample of a remote sensing image and a second real sample of a remote sensing image of a target area to be detected, wherein the first real sample of a remote sensing image and the second real sample of a remote sensing image are real samples of images of the same location at different times.

[0096] Specifically, the first remote sensing image real sample and the second remote sensing image real sample both cover the mining area and its surrounding areas. Based on the knowledge of geological experts and the previous geological survey data, the performance of landslides, collapses, ground subsidence and ground fissures in the image is accurately marked for different types of geological disasters in the first remote sensing image real sample and the second remote sensing image real sample. Each type of disaster will be assigned a specific label. In the annotation, the polygon tool is used to draw along the disaster boundary to ensure that each labeled area clearly defines the scope and form of the disaster. At the same time, labels are also marked to indicate the disappearance and appearance of geological disasters, that is, one of the disappearance and appearance is only present in the first remote sensing image real sample, and the other is only present in the second remote sensing image real sample. For example, if a geological disaster appears in the first remote sensing image real sample, the label that has appeared is marked. Conversely, if the geological disaster disappears in the second remote sensing image real sample, the disappeared label is marked, indicating that the geological disaster from the first remote sensing image real sample to the second remote sensing image real sample has undergone a change from appearance to disappearance.

[0097] According to an embodiment of the present invention, the preprocessed first remote sensing image sample, the preprocessed second remote sensing image sample, the real sample of the first remote sensing image and the real sample of the second remote sensing image can be input into the deep learning model to obtain the prediction results of the geological disasters in the target area to be detected, the predicted detection results of the geological disasters in the target area to be detected and the real detection results of the geological disasters are calculated to obtain the loss value, and based on the loss value, the parameters of the deep learning model are adjusted until the loss value reaches a preset threshold, and the training is stopped to obtain a trained deep learning model.

[0098] Figure 4 A schematic diagram of a mine geological disaster detection method according to an embodiment of the present invention is shown.

[0099] like Figure 4 As shown, in the schematic diagram 400, the first remote sensing image and the second remote sensing image of the target area to be detected are preprocessed in real time to obtain the preprocessed first remote sensing image 401 and the second remote sensing image 402. The preprocessed first remote sensing image 401 is input into the first network structure 403 of the deep learning model, and the first network structure 403 includes an encoder module 403-1 and a decoder module 403-2 to obtain the first image feature 404. The preprocessed second remote sensing image 402 is input into the second network structure 405 of the deep learning model, and the second network structure 405 includes an encoder module 405-1 and a decoder module 405-2 to obtain the second image feature 406. Using the third network structure 407 of the deep learning network, the first image feature 404 and the second image feature 406 are subtracted and convolved to obtain the fourth image feature 408, and the first image feature 404 and the second image feature 406 are feature spliced ​​to obtain the image splicing feature 409. The fourth image feature 408 and the image stitching feature 409 are multiplied to obtain a third image feature 410. The third image feature 410 is input into the fourth network structure 411 of the deep learning model for geological disaster detection to obtain a geological disaster detection result 412 of the target area to be detected. The geological disaster detection result 412 includes whether a geological disaster occurs in the target area to be detected 412-1, the type of geological disaster 412-2, and the change process 412-3 of the geological disaster.

[0100] Figure 5 A block diagram of a mining geological disaster detection device according to an embodiment of the present invention is shown.

[0101] like Figure 5 As shown, the device 500 includes an image acquisition module 510 , a first image feature extraction module 520 , a second image feature extraction module 530 , a third image feature fusion module 540 and a detection module 550 .

[0102] The image acquisition module 510 is used to acquire a first remote sensing image and a second remote sensing image of a target area to be detected in real time, wherein the first remote sensing image and the second remote sensing image are images of the same location at different times.

[0103] The first image feature extraction module 520 is used to extract features from the first remote sensing image using the first network structure of the deep learning model to obtain first image features, wherein the first image features represent geological features of the target area to be detected in the first remote sensing image.

[0104] The second image feature extraction module 530 is used to extract features from the second remote sensing image using the second network structure of the deep learning model to obtain second image features, wherein the second image features represent geological features of the target area to be detected in the second remote sensing image.

[0105] The third image feature fusion module 540 is used to use the third network structure of the deep learning model to perform feature fusion on the first image feature and the second image feature to obtain a third image feature, wherein the third image feature represents the different image change characteristics between the first image feature and the second image feature.

[0106] The detection module 550 is used to use the fourth network structure of the deep learning model to perform geological disaster detection on the third image feature to obtain the geological disaster detection results of the target area to be detected, wherein the geological disaster detection results include whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the changing process of the geological disaster.

[0107] According to an embodiment of the present invention, the third image feature fusion module 540 includes: an image stitching feature obtaining submodule, a fourth image feature obtaining submodule and a third image feature obtaining submodule.

[0108] The image stitching feature acquisition submodule is used to stitch the first image feature and the second image feature to obtain an image stitching feature.

[0109] The fourth image feature acquisition submodule is used to perform convolution processing on the first image feature and the second image feature to obtain a fourth image feature.

[0110] The third image feature acquisition submodule is used to perform a product operation on the image stitching feature and the fourth image feature to obtain the third image feature.

[0111] According to an embodiment of the present invention, the fourth image feature obtaining submodule includes: a second intermediate image feature obtaining unit and a fourth image feature obtaining unit.

[0112] The second intermediate image feature obtaining unit is used to perform a difference operation on the first image feature and the second image feature to obtain the second intermediate image feature.

[0113] The fourth image feature obtaining unit is used to perform convolution processing on the second intermediate image feature to obtain the fourth image feature.

[0114] According to an embodiment of the present invention, the first image feature extraction module 520 includes: a first intermediate image feature acquisition submodule and a first image feature acquisition submodule.

[0115] The first intermediate image feature acquisition submodule is used to use the encoder module of the first network structure to perform image encoding on the first remote sensing image to obtain the first intermediate image feature.

[0116] The first image feature acquisition submodule is used to decode the first intermediate image feature using the decoder module of the first network structure to obtain the first image feature.

[0117] According to an embodiment of the present invention, the detection module 550 includes: a third intermediate image feature obtaining submodule and a detection result obtaining submodule.

[0118] The third intermediate image feature acquisition submodule is used to perform a convolution operation on the third image feature to obtain the third intermediate image feature.

[0119] The detection result acquisition submodule is used to use the fully connected layer to perform geological disaster detection on the third intermediate image features to obtain the address disaster detection results of the target area to be detected.

[0120] According to an embodiment of the present invention, the device 500 further includes: a correction processing module, a fusion processing module, an image enhancement processing module and a cropping processing module.

[0121] The correction processing module is used to perform orthorectification processing on the first remote sensing image to obtain a corrected first remote sensing image.

[0122] The fusion processing module is used to perform registration and fusion processing on the corrected first remote sensing image to obtain a fused first remote sensing image.

[0123] The image enhancement processing module is used to perform image enhancement processing on the fused first remote sensing image to obtain an enhanced first remote sensing image.

[0124] The cropping processing module is used to perform image cropping processing on the enhanced first remote sensing image to obtain a cropped first remote sensing image.

[0125] According to an embodiment of the present invention, any multiple of the image acquisition module 510, the first image feature extraction module 520, the second image feature extraction module 530, the third image feature fusion module 540 and the detection module 550 can be combined in one module / submodule / unit for implementation, or any one of the modules / submodules / units can be split into multiple modules / submodules / units. Alternatively, at least part of the functions of one or more modules / units / subunits in these modules / submodules / units can be combined with at least part of the functions of other modules / submodules / units and implemented in one module / submodule / unit. According to an embodiment of the present invention, at least one of the image acquisition module 510, the first image feature extraction module 520, the second image feature extraction module 530, the third image feature fusion module 540 and the detection module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the image acquisition module 510, the first image feature extraction module 520, the second image feature extraction module 530, the third image feature fusion module 540 and the detection module 550 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0126] It should be noted that the mine geological disaster detection device part in the embodiment of the present invention corresponds to the mine geological disaster detection method part in the embodiment of the present invention. The description of the mine geological disaster detection device part specifically refers to the mine geological disaster detection method part, which will not be repeated here.

[0127] Figure 6 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is shown. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0128] like Figure 6As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0129] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0130] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0131] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0132] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0133] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0134] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the mine geological disaster detection method provided by the embodiment of the present invention.

[0135] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0136] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0137] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0139] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The present invention does not depart from the scope of the present invention, and those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for detecting geological disasters in mines, characterized in that: The method comprises: Acquire a first remote sensing image and a second remote sensing image of a target area to be detected in real time, wherein the first remote sensing image and the second remote sensing image are images of the same location at different times; Using a first network structure of a deep learning model to perform feature extraction on the first remote sensing image to obtain a first image feature, wherein the first image feature represents a geological feature of the target area to be detected in the first remote sensing image; Using the second network structure of the deep learning model to perform feature extraction on the second remote sensing image to obtain second image features, wherein the second image features represent geological features of the target area to be detected in the second remote sensing image; Using the third network structure of the deep learning model to perform feature fusion on the first image feature and the second image feature to obtain a third image feature, wherein the third image feature represents different image change features between the first image feature and the second image feature; The fourth network structure of the deep learning model is used to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, wherein the geological disaster detection result includes whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the changing process of the geological disaster.

2. The method according to claim 1, characterized in that: The step of using the third network structure of the deep learning model to perform feature fusion on the first image feature and the second image feature to obtain a third image feature includes: Performing feature stitching on the first image feature and the second image feature to obtain an image stitching feature; Performing convolution processing on the first image feature and the second image feature to obtain a fourth image feature; A product operation is performed on the image stitching feature and the fourth image feature to obtain the third image feature.

3. The method according to claim 2, characterized in that The performing convolution processing on the first image feature and the second image feature to obtain a fourth image feature includes: performing a difference operation on the first image feature and the second image feature to obtain a second intermediate image feature; The second intermediate image feature is subjected to convolution processing to obtain the fourth image feature.

4. The method according to claim 1, characterized in that: The step of extracting features from the first remote sensing image using the first network structure of the deep learning model to obtain first image features includes: Using the encoder module of the first network structure to perform image encoding on the first remote sensing image to obtain a first intermediate image feature; The first intermediate image feature is decoded by using the decoder module of the first network structure to obtain the first image feature.

5. The method according to claim 1, characterized in that The method of using the fourth network structure of the deep learning model to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected includes: Performing a convolution operation on the third image feature to obtain a third intermediate image feature; The fully connected layer is used to perform geological disaster detection on the third intermediate image feature to obtain the address disaster detection result of the target area to be detected.

6. The method according to claim 1, characterized in that The method further comprises: Performing orthorectification processing on the first remote sensing image to obtain a corrected first remote sensing image; Performing registration and fusion processing on the corrected first remote sensing image to obtain a fused first remote sensing image; Performing image enhancement processing on the fused first remote sensing image to obtain an enhanced first remote sensing image; The enhanced first remote sensing image is subjected to image cropping processing to obtain a cropped first remote sensing image.

7. A mining geological disaster detection device, characterized in that: The device comprises: An image acquisition module, used for acquiring in real time a first remote sensing image and a second remote sensing image of a target area to be detected, wherein the first remote sensing image and the second remote sensing image are images of the same location at different times; A first image feature extraction module, configured to extract features from the first remote sensing image using a first network structure of a deep learning model to obtain first image features, wherein the first image features represent geological features of the target area to be detected in the first remote sensing image; A second image feature extraction module, configured to extract features from the second remote sensing image using the second network structure of the deep learning model to obtain second image features, wherein the second image features represent geological features of the target area to be detected in the second remote sensing image; A third image feature fusion module, configured to perform feature fusion on the first image feature and the second image feature using the third network structure of the deep learning model to obtain a third image feature, wherein the third image feature represents different image change features between the first image feature and the second image feature; A detection module is used to use the fourth network structure of the deep learning model to perform geological disaster detection on the third image feature to obtain a geological disaster detection result of the target area to be detected, wherein the geological disaster detection result includes whether a geological disaster occurs in the target area to be detected, the type of geological disaster that occurs, and the changing process of the geological disaster.

8. An electronic device comprising: one or more processors; a memory for storing one or more programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor implements the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images