Mining area ground fracture monitoring method and device, electronic equipment and storage medium

Abstract

The invention discloses a mining area ground fracture monitoring method and device, electronic equipment and a storage medium, and relates to the technical field of image recognition, and the method comprises the steps: obtaining an original image of a mining area ground fracture, segmenting the original image based on a preset semantic segmentation model, and obtaining a ground fracture extraction result; a ground fracture skeleton is extracted from the ground fracture extraction result, and a ground fracture structure line graph is generated according to key points of the ground fracture skeleton; and connecting associated structure lines in the ground fracture structure line graph to obtain a ground fracture strike line. According to the method, the deep learning semantic segmentation model and the selective connection between the structural lines are adopted, so that the monitoring efficiency of the ground fissures in the mining area is improved, the precision and reliability of ground fissure monitoring are greatly improved, and effective technical support is provided for mine geological disaster investigation and ecological environment management.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, electronic equipment and storage medium for monitoring ground fissures in mining areas. Background Art

[0002] Mineral resources are an important material basis for social and economic development, and the development of mineral resources provides a basic guarantee for economic development and people's lives. At the same time, mining has also had a huge impact on the ecological environment, including various types of mining geological disasters directly or indirectly caused by mining activities. After underground minerals are mined, large-scale subsidence will occur on the surface. Ground fissures are an important sign for judging subsidence areas. Timely acquisition of ground fissure distribution information is of great significance to the investigation of mining geological disasters and ecological environment governance.

[0003] At present, the main method for extracting mine ground fissures from drone aerial images is manual visual interpretation, which is time-consuming and labor-intensive. There are very few applicable automated ground fissure monitoring methods. When used for ground fissure extraction in mining areas, they are easily affected by complex factors such as the geographical environment of the mining area. There is a problem of low accuracy in ground fissure extraction. Summary of the invention

[0004] The main purpose of the present application is to provide a method, device, electronic equipment and storage medium for monitoring ground fissures in mining areas, aiming to solve the technical problem of low accuracy in extracting ground fissures by automated ground fissure monitoring methods.

[0005] To achieve the above purpose, the present application proposes a method for monitoring ground fissures in a mining area, the method comprising:

[0006] Acquire an original image of ground fissures in a mining area, and segment the original image based on a preset semantic segmentation model to obtain a ground fissure extraction result;

[0007] Extracting a ground fissure skeleton from the ground fissure extraction result, and generating a ground fissure structure line diagram according to key points of the ground fissure skeleton;

[0008] Connect the related structural lines in the ground fissure structural line diagram to obtain the ground fissure strike line.

[0009] In one embodiment, the ground fissure extraction result includes a ground fissure binary map, and the step of extracting a ground fissure skeleton from the ground fissure extraction result and generating a ground fissure structure line map according to key points of the ground fissure skeleton includes:

[0010] Obtaining a connected domain of ground fissures in the ground fissure binary map, and performing denoising and connectivity completion on the connected domain to obtain a processed ground fissure binary map;

[0011] Extracting the ground fissure skeleton from the processed ground fissure binary image based on a preset image thinning algorithm;

[0012] The key points are extracted from the ground fissure skeleton based on a preset iterative adaptive point algorithm, and the key points are connected to obtain the ground fissure structure line diagram.

[0013] In one embodiment, the step of connecting the associated structural lines in the ground fissure structural line map to obtain the ground fissure strike line comprises:

[0014] Obtaining the endpoints of each structure line in the ground fissure structure line diagram;

[0015] For any one of the structural lines, determine the direction angle of the structural line according to the corresponding endpoint of the structural line, and retrieve the remaining endpoints within a preset distance range of the corresponding endpoint of the structural line, wherein the remaining endpoints are the endpoints of the remaining structural lines;

[0016] In the case where the remaining endpoints exist within the preset distance range, the remaining endpoints are screened based on the strike angle to obtain target remaining endpoints, and a connection relationship between the target remaining endpoints and the corresponding endpoints of the structure line is determined;

[0017] After traversing each endpoint, each connection relationship is deduplicated, and the structure line endpoints in the ground fissure structure line diagram are connected based on the deduplicated connection relationships to obtain the ground fissure strike line.

[0018] In one embodiment, the step of screening the remaining endpoints based on the direction angle to obtain the remaining target endpoints includes:

[0019] Filtering the remaining endpoints based on the connection angles between the remaining endpoints and the endpoint to obtain candidate remaining endpoints, wherein the candidate remaining endpoints are the remaining endpoints whose difference between the connection angle and the direction angle is less than a preset angle;

[0020] For any candidate remaining endpoint among the candidate remaining endpoints, determining the similarity between a first image around the candidate remaining endpoint and a second image around a corresponding endpoint of the structure line, and testing the similarity to obtain a test result;

[0021] After traversing each candidate remaining endpoint, the candidate remaining endpoint corresponding to the qualified search result in each inspection result is used as the target remaining endpoint.

[0022] In one embodiment, the preset semantic segmentation model is trained based on annotated images captured and visually interpreted of ground fissures in a mining area, the preset semantic segmentation model includes an encoder and a decoder, and the mining area ground fissure monitoring method further includes:

[0023] Constructing a model training data set based on the labeled image, and performing data enhancement in multiple preset forms on the model training data set to obtain enhanced data sets;

[0024] Inputting each of the enhanced data sets into the encoder respectively, and performing comparative learning based on the ground fissure feature information output after processing by the encoder, wherein the encoder is optimized based on a preset comparative loss function during the comparative learning process;

[0025] Inputting the ground fissure feature information into the decoder, so as to extract context information of the ground fissure feature information in the decoder through a preset isotropic convolution, and performing ground fissure feature fusion according to the context information and feature information output by each layer of the neural network in the encoder, so as to obtain a ground fissure training extraction result;

[0026] The regional loss function of the preset semantic segmentation model is calculated based on the ground fissure training extraction result, and the decoder is optimized based on the regional loss function.

[0027] In one embodiment, the encoder includes a plurality of residual networks and a plurality of dilated residual networks, and before the step of inputting the ground fissure feature information into the decoder, the step further includes:

[0028] Extracting spatial attention features and channel attention features from candidate ground fissure feature information processed by multiple residual networks and multiple dilated residual networks to obtain spatial features and channel features of the candidate ground fissure feature information;

[0029] The spatial features and the channel features are combined to obtain the ground fissure feature information.

[0030] In one embodiment, after the step of performing comparative learning based on the ground fissure feature information output after processing by each encoder, the step further includes:

[0031] Obtaining feature information output by each layer of the neural network in the encoder;

[0032] Based on each feature information, edge reinforcement learning of the ground cracks is performed to obtain edge attention mapping parameters, so that the edge attention mapping parameters are used to generate the ground crack training extraction results in the decoder, wherein, in the edge reinforcement learning process of the ground cracks, the feature information is input into an edge reinforcement branch composed of multiple residual blocks and multiple gated convolution blocks, so that the edge attention mapping parameters are calculated by the edge reinforcement branch, and the edge reinforcement branch is optimized by a preset edge loss function.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a mining area ground fissure monitoring device, the mining area ground fissure monitoring device comprising:

[0034] A segmentation module is used to obtain the original image taken of the ground fissures in the mining area, and segment the original image based on a preset semantic segmentation model to obtain a ground fissure extraction result;

[0035] A generating module, used for extracting a ground fissure skeleton from the ground fissure extraction result, and generating a ground fissure structure line diagram according to key points of the ground fissure skeleton;

[0036] The connection module is used to connect the related structure lines in the ground fissure structure line diagram to obtain the ground fissure trend line.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the mining area ground fissure monitoring method as described above.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the mining area ground fissure monitoring method as described above are implemented.

[0039] One or more technical solutions proposed in this application have at least the following technical effects:

[0040] The present application first obtains original images taken of ground fissures in a mining area, and segments the original images based on a preset semantic segmentation model to obtain ground fissure extraction results, so as to process the original images using the preset semantic segmentation model, thereby realizing rapid and accurate extraction of ground fissure information from a complex mining environment, effectively reducing the workload of manual visual interpretation, and improving the efficiency and accuracy of ground fissure identification; extracting ground fissure skeletons from the ground fissure extraction results, and generating ground fissure structure line diagrams based on key points of the ground fissure skeletons, thereby realizing accurate depiction of ground fissure morphology and generation of structure line diagrams, achieving the effect of accurately expressing the spatial distribution characteristics of ground fissures, and facilitating further analysis of the direction of ground fissures; connecting the associated structure lines in the ground fissure structure line diagram to obtain ground fissure strike lines, thereby realizing accurate and complete drawing of ground fissure strike lines by analyzing the relationship between structure lines, and facilitating providing a more intuitive and effective basis for mine geological disaster investigation and ecological environmental governance.

[0041] In summary, this application avoids the time-consuming and labor-intensive traditional manual visual interpretation and the incomplete and low-precision extraction of ground fissures by automated monitoring methods by adopting a deep learning semantic segmentation model and selective connection between structural lines. It achieves the effect of quickly and accurately extracting ground fissure information in mining areas, accurately depicting the skeleton and structural line diagram of ground fissures, and accurately drawing complete ground fissure trend lines. It not only improves the monitoring efficiency of ground fissures in mining areas, but also greatly improves the accuracy and reliability of ground fissure monitoring, providing effective technical support for mine geological disaster investigation and ecological environmental governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the method for monitoring ground fissures in a mining area of ​​the present application;

[0045] Figure 2 A schematic diagram of ground fissure extraction results of the mining area ground fissure monitoring method provided in Example 1 of the present application;

[0046] Figure 3 A schematic diagram of key points and structural lines of a mining area ground fissure monitoring method provided in Example 1 of the present application;

[0047] Figure 4 A schematic diagram of a process flow provided for Embodiment 2 of the method for monitoring ground fissures in mining areas of the present application;

[0048] Figure 5 A schematic diagram of the isotropic convolution of the method for monitoring ground fissures in mining areas provided in Example 2 of the present application;

[0049] Figure 6 A schematic diagram of double attention for a method for monitoring ground fissures in a mining area provided in Example 2 of the present application;

[0050] Figure 7 A schematic diagram of the edge reinforcement branch of the mining area ground fissure monitoring method provided in Example 2 of the present application;

[0051] Figure 8 A schematic diagram of a process flow provided for Embodiment 3 of the method for monitoring ground fissures in a mining area of ​​the present application;

[0052] Fig. 9 A schematic diagram of a brief process of a method for monitoring ground fissures in a mining area provided in Example 3 of the present application;

[0053] Fig.10 A schematic diagram of the semantic segmentation model architecture of the mining area ground fissure monitoring method provided in Example 3 of the present application;

[0054] Fig.11 This is a schematic diagram of the module structure of a ground fissure monitoring device in a mining area according to an embodiment of the present application;

[0055] Fig.12 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the mining area ground fissure monitoring method in the embodiment of the present application.

[0056] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0058] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0059] The main solution of the embodiment of the present application is: obtain the original image taken of the ground fissures in the mining area, and segment the original image based on a preset semantic segmentation model to obtain a ground fissure extraction result; extract the ground fissure skeleton from the ground fissure extraction result, and generate a ground fissure structure line map according to the key points of the ground fissure skeleton; connect the related structural lines in the ground fissure structure line map to obtain the ground fissure strike line.

[0060] The current method of extracting mine ground fissures from drone aerial images is mainly manual visual interpretation, which is time-consuming and labor-intensive. There are very few applicable automated ground fissure monitoring methods, and when used for ground fissure extraction in mining areas, they are easily affected by complex factors such as the geographical environment of the mining area. There is a problem of low accuracy in ground fissure extraction.

[0061] The present application provides a solution, which avoids the time-consuming and labor-intensive traditional manual visual interpretation and the incomplete and low-precision ground fissure extraction of automated monitoring methods by adopting a deep learning semantic segmentation model and selective connection between structural lines. It achieves the effect of quickly and accurately extracting ground fissure information in mining areas, accurately depicting the skeleton and structural line diagram of ground fissures, and accurately drawing complete ground fissure trend lines. It not only improves the monitoring efficiency of ground fissures in mining areas, but also greatly improves the accuracy and reliability of ground fissure monitoring, providing effective technical support for mining geological disaster investigation and ecological environmental governance.

[0062] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions. The following takes an electronic device as an example to illustrate this embodiment and the following embodiments.

[0063] Based on this, the present application embodiment provides a method for monitoring ground fissures in a mining area, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for monitoring ground fissures in mining areas of the present application.

[0064] In this embodiment, the mining area ground fissure monitoring method includes steps S10 to S30:

[0065] Step S10, obtaining an original image of the ground fissures in the mining area, and segmenting the original image based on a preset semantic segmentation model to obtain a ground fissure extraction result;

[0066] It should be noted that the original image refers to the surface photos or videos taken in the mining area by drones or other photographic equipment, which record the actual distribution of ground fissures on the ground in the mining area; the semantic segmentation model refers to a deep learning model that can classify each pixel in the image into a predetermined category (such as ground fissures, background, etc.) according to the contextual information of the pixel, thereby achieving pixel-level image understanding; the ground fissure extraction result refers to the binary image obtained after processing the original image with the semantic segmentation model, that is, the ground fissure binary map, in which the ground fissure area is marked as a white connected domain, distinguished from the black background area.

[0067] It is understandable that since manual visual interpretation in the traditional ground fissure monitoring process is time-consuming and labor-intensive, and will be affected by a large number of subjective factors, performing step S10 can avoid the time-consuming and subjectivity of traditional manual visual interpretation, thereby automatically identifying and extracting ground fissure information from complex mining area surface images, and realizing fast and accurate extraction of ground fissure information from original images, thereby improving the monitoring efficiency of ground fissures.

[0068] For example, please refer to Figure 2 , obtain the original image A, and perform the segmentation processing shown in the S operation in the figure on A based on the preset semantic segmentation model to obtain the ground fissure extraction result B, where the dark area in B is the 0-value area in the ground fissure binary map, which represents the ground fissure, and the white area in B is the 1-value area in the ground fissure binary map, which represents the surrounding environment of the ground fissure.

[0069] Step S20, extracting a ground fissure skeleton from the ground fissure extraction result, and generating a ground fissure structure line map according to key points of the ground fissure skeleton;

[0070] It should be noted that the ground fissure skeleton refers to one or more thin lines extracted from the ground fissure extraction results through the refinement algorithm. These thin lines represent the center line or main direction of the ground fissure; the key points refer to specific points on the ground fissure skeleton. These points usually represent the turning points, bifurcation points or endpoints of the ground fissure, which are of great significance for understanding the structure and direction of the ground fissure; the ground fissure structure line diagram refers to a graphic generated based on the key points of the ground fissure skeleton, which shows the spatial structure and main characteristics of the ground fissure.

[0071] It can be understood that in order to accurately depict the morphology of the ground fissures from the ground fissure extraction results, step S20 is performed. By extracting the key points of the ground fissure skeleton, an effective data basis is provided for further improving the integrity of the ground fissure extraction, which can avoid inaccurate and incomplete ground fissure information, thereby providing detailed information for the morphological analysis and feature extraction of the ground fissures.

[0072] In a feasible implementation manner, the ground fissure extraction result includes a ground fissure binary map, and step S20 may include steps S21 to S23:

[0073] Step S21, obtaining a connected domain of ground fissures in the ground fissure binary map, and performing denoising and connectivity completion on the connected domain to obtain a processed ground fissure binary map;

[0074] It should be noted that the connected domain refers to an area in the ground fissure binary map, which is composed of a series of adjacent pixels with the same pixel value. These areas represent a continuous ground fissure segment; the ground fissure binary map refers to an image obtained by semantic segmentation or other image processing methods, in which the ground fissure area is represented by white pixels and the background area is represented by black pixels, thus forming an image with only two colors.

[0075] It can be understood that since the binary map of ground fissures directly segmented by the semantic segmentation model often has problems of noise and weak connectivity, step S21 is performed to remove noise and complete the connectivity of cracks on the prediction result of the semantic segmentation model, that is, the binary map of ground fissures, to obtain complete and continuous cracks, so as to facilitate the subsequent extraction of the complete ground fissure skeleton structure.

[0076] For example, the connected domain in the ground fissure binary map is obtained based on OpenCV, such as Figure 2The dark area of ​​B in the image is identified by eight connected domains, and the connected domain is identified by each identified block. The number of pixels in each identified block is calculated, and then the connected domain with a pixel number less than the threshold is deleted to complete the connected domain denoising. The KD-tree-based crack connection algorithm is then used to complete the connected domain, that is, the KD-tree algorithm is used to search for the nearest neighbor endpoints, and the crack endpoints with a distance less than the threshold are connected to obtain the processed ground crack binary map.

[0077] Step S22, extracting the ground fissure skeleton from the processed ground fissure binary image based on a preset image thinning algorithm;

[0078] It should be noted that the image thinning algorithm refers to an image processing technology that can thin the target edge in the ground fissure binary image into a single-pixel wide line to obtain the skeleton of the ground fissure. For example, the Zhang-Suen parallel thinning algorithm can continuously delete pixels on the object border without interrupting connectivity until no more pixels can be deleted.

[0079] It can be understood that in order to extract the skeleton of the ground fissure from the binary image of the ground fissure after denoising and connectivity completion, step S22 is recorded. Extracting the skeleton of the ground fissure through the image thinning algorithm can avoid the problem of inaccurate ground fissure contours, thereby achieving accurate depiction of the ground fissure morphology.

[0080] Exemplarily, the Zhang-Suen parallel thinning algorithm is used to first form an eight-domain map of the center point pixel, as shown in Table 1 below:

[0081] P9 P2 P3 P8 P1 P4 P7 P6 P5

[0082] Table 1

[0083] Then, remove the boundary points that satisfy both of the following conditions:

[0084] ①2≤N(P1)≤6; ②S(P1)=1; ③P2×P4×P6=0; ④P4×P6×P8=0

[0085] Among them, N(P1) is the number of non-zero neighboring points of P1, and S(P1) is the number of times the value of two adjacent pixels changes from 0 to 1 in the clockwise direction starting from P2. Then delete the boundary points that meet the following conditions at the same time:

[0086] ①2≤N(P1)≤6; ②S(P1)=1; ③P2×P4×P8=0; ④P2×P6×P8=0

[0087] The above two-step deletion operation constitutes one iteration until no point meets the marking condition. At this time, the remaining points form a region, which is the refined skeleton, that is, the ground fissure skeleton.

[0088] Step S23, extracting the key points from the ground fissure skeleton based on a preset iterative adaptive point algorithm, and connecting the key points to obtain the ground fissure structure line map.

[0089] It should be noted that the iterative adaptive point algorithm refers to an algorithm for extracting key points from the ground fissure skeleton, namely the Douglas–Peucker (DP) algorithm, which finds characteristic points on the skeleton, such as turning points and bifurcation points, in an iterative manner.

[0090] It can be understood that in order to extract key points from the ground fissure skeleton and generate a structural line diagram by connecting these key points, step S23 is performed to avoid misjudgment of the direction and structure of the ground fissure, thereby achieving accurate representation of the spatial distribution of the ground fissure.

[0091] For example, the DP algorithm is used to extract key points from the ground fissure skeleton, such as Figure 3 Each point in the neutron graph A, and each key point is as follows Figure 3 The connection operation shown in C in the figure gives the ground fissure structure line diagram, as shown in Figure 3 In the subgraph B in the point set, the DP algorithm first selects the first and last points in the point set, connects them into a straight line, and then for each other point in the point set, calculates the distance from the other point to the straight line, finds the point with the largest distance, and checks whether the distance is greater than a predetermined threshold. If so, keep this point and recursively apply the algorithm to all points between this point and the starting point and the end point; if not, delete all intermediate points. Finally, repeat this process until there are no more points to be eliminated.

[0092] In this embodiment, by performing connected domain analysis, denoising and connectivity completion, image refinement algorithm and iterative adaptive point algorithm, the erroneous recognition due to noise, fractures and incomplete connected domains, as well as the loss or misjudgment of ground fissure features due to inaccurate skeleton extraction and key point positioning deviation are avoided. The ground fissure information is accurately extracted from the ground fissure binary map, the ground fissure skeleton is refined, the key points are accurately identified and the structural line map is constructed, thereby providing high-precision and reliable ground fissure analysis results for the monitoring and assessment of mining geological disasters.

[0093] Step S30, connecting the related structure lines in the ground fissure structure line diagram to obtain the ground fissure strike line.

[0094] It should be noted that the ground fissure strike line refers to the line obtained by selectively connecting the key points in the ground fissure structure line diagram, which represents the main extension direction of the ground fissure.

[0095] It is understandable that, since it is usually difficult to clearly display the direction and distribution of ground fissures in traditional ground fissure monitoring methods, performing step S30 can avoid misjudgment of the actual trend of the ground fissures, ensure the integrity of the ground fissures, and thus achieve accurate analysis of the spatial distribution and direction of the ground fissures.

[0096] In a feasible implementation, step S30 may include steps S31 to S34:

[0097] Step S31, obtaining the endpoints of each structure line in the ground fissure structure line diagram;

[0098] Step S32, for any one of the structural lines, determining the direction angle of the structural line according to the corresponding endpoint of the structural line, and retrieving the remaining endpoints within a preset distance range of the corresponding endpoint of the structural line, wherein the remaining endpoints are the endpoints of the remaining structural lines;

[0099] For example, the direction of all cracks in the ground crack structure line map is calculated. Let O and P be the starting point and end point (i.e., the two end points) of a crack, and the direction of the crack is ∠OP, with the unit of °. Then, a search range is created with O as the center and d as the radius. For the point q within the preset distance range that does not belong to the crack where OP is located, i (i=1, 2, ..., N) calculate the distance Oq i and angle ∠Oq i , N is the number of retrieved points.

[0100] It should be noted that the strike angle refers to the angle between the line connecting the two end points of a certain structure line in the ground fissure structure line diagram and the horizontal line or reference direction. This angle is used to indicate the extension direction of the ground fissure at that point.

[0101] It is understandable that due to the complex geographical environment such as the presence of a large amount of vegetation distribution in the mining area, and the limited clarity of images taken by shooting equipment such as drones, actual continuous ground fissures often appear as broken states in the image due to environmental factors or equipment limitations. Therefore, performing step S32 can avoid errors in judging the direction of ground fissures, thereby providing an effective data basis for ensuring the accuracy of ground fissure network connections.

[0102] Step S33, when the remaining endpoints exist within the preset distance range, screening the remaining endpoints based on the strike angle to obtain target remaining endpoints, and determining the connection relationship between the target remaining endpoints and the corresponding endpoints of the structure line;

[0103] It should be noted that the remaining target endpoints refer to other structural line endpoints within a preset distance range that have the same or similar strike angle as the endpoint currently being processed, and these points are potential connection candidate points.

[0104] It can be understood that in order to select suitable points from possible connection points and establish a connection relationship, step S33 is performed to avoid erroneous or unnecessary connections, thereby ensuring the continuity and correctness of the ground fissure strike line.

[0105] Step S34, after traversing each endpoint, deduplicate each connection relationship, and connect the structure line endpoints in the ground fissure structure line diagram based on the deduplicated connection relationships to obtain the ground fissure strike line.

[0106] It can be understood that since there may be multiple different endpoints to be connected at the endpoints of the same structural line, step S34 is performed to avoid redundancy and intersection of the ground fissure direction line by selecting the most appropriate connection relationship among the connection relationships of the multiple endpoints to be connected, thereby ensuring the clarity and accuracy of the ground fissure direction line.

[0107] In this embodiment, by determining the endpoints of the structure line, calculating the strike angle, screening the target endpoints, establishing connection relationships, and removing duplicates and connecting endpoints, the problems of inaccurate positioning of ground fissure endpoints, incorrect strike angle judgment, unnecessary or incorrect endpoint connections, and redundancy and intersection of strike lines are avoided. The endpoints of the ground fissure structure line diagram are accurately identified, the ground fissure strike is accurately determined, the ground fissure network connection relationship is effectively established, and the ground fissure strike line is clearly drawn, thereby providing accurate ground fissure analysis results for mine geological disaster monitoring and ecological environmental assessment.

[0108] This embodiment provides a method for monitoring ground fissures in mining areas. By adopting a deep learning semantic segmentation model and selective connection between structural lines, it avoids the time-consuming and labor-intensive traditional manual visual interpretation and the incomplete and low-precision ground fissure extraction of automated monitoring methods. It achieves the effect of quickly and accurately extracting ground fissure information in mining areas, accurately depicting the skeleton and structural line diagram of ground fissures, and accurately drawing complete ground fissure trend lines. It not only improves the monitoring efficiency of ground fissures in mining areas, but also greatly improves the accuracy and reliability of ground fissure monitoring, providing effective technical support for mining geological disaster investigation and ecological environmental governance.

[0109] In a feasible implementation manner, the step of screening the remaining endpoints based on the direction angle in step S33 to obtain the remaining target endpoints may include steps S331 to S333:

[0110] Step S331, screening the remaining endpoints based on the connection angle between the remaining endpoints and the endpoint to obtain candidate remaining endpoints, wherein the candidate remaining endpoints are the remaining endpoints whose difference between the connection angle and the direction angle is less than a preset angle;

[0111] It should be noted that the candidate remaining endpoints refer to other structural line endpoints that have similar strike angles to the current endpoint within a preset angle range, and the endpoints are potential connection points of the current endpoint.

[0112] It can be understood that in order to filter out candidate endpoints that may be connected to the current endpoint from all other endpoints, step S331 is performed, which can avoid endpoints that are too different from the current endpoint from being incorrectly connected, thereby achieving the accuracy of endpoint connection between ground fissure structure lines.

[0113] For example, points with consistent directions are screened and points with angle differences exceeding a threshold value t are removed, that is, only |∠OP-∠Oq is retained. i |≤, and obtain the remaining candidate endpoints, where O and P are the two endpoints of a crack, the direction of the crack is ∠OP, the unit is °, ∠Oq i is the point q within the preset distance range that does not belong to the crack where OP is located i (i=1, 2, ..., N) and O, where N is the number of retrieved points.

[0114] Step S332, for any candidate remaining endpoint among the candidate remaining endpoints, determine the similarity between the first image around the candidate remaining endpoint and the second image around the corresponding endpoint of the structure line, and test the similarity to obtain a test result;

[0115] It should be noted that the first image refers to the image of the area near the endpoint of other structural lines that have similar direction angles to the current endpoint within a preset angle range; the second image refers to the image of the area near the endpoint of the current structural line; the similarity refers to the degree of similarity between two images in the image histogram, which can be quantified by comparing the distance between the two histograms.

[0116] It is understandable that structural lines with similar strike angles may not be actually connected but are ground fissures that are broken in the image. Therefore, step S332 is performed to further verify the similarity between the candidate endpoints and the images around the current endpoints, thereby avoiding structural lines with dissimilar geographical backgrounds from being incorrectly connected together, thereby ensuring the accuracy of the connection between ground fissure structural lines.

[0117] For example, the similarity test is to extract the image around the location of the endpoint, calculate the image histogram, and compare the similarity of the histogram H. If the histograms are exactly the same, the similarity is 1, otherwise it is 0. The calculation formula is as follows:

[0118]

[0119] Among them, I and J represent the pixel matrix around a pixel point in the structure line, and N is the number of histograms.

[0120] Step S333, after traversing each candidate remaining endpoint, the candidate remaining endpoint corresponding to the retrieval result that has passed the inspection in each inspection result is used as the target remaining endpoint.

[0121] It can be understood that in order to determine the remaining endpoints of the final target from the inspection results, step S333 is performed, which can avoid unqualified candidate endpoints, that is, candidate breakpoints with low similarity from being included in the final connection relationship, thereby generating erroneous ground fissure trend lines, ensuring that the final connection relationship is reliable and valid.

[0122] In this embodiment, by screening candidate endpoints based on angles and calculating the similarity between images at the endpoints and performing verification, the problem of incorrect connection and irrelevant structural lines being incorrectly associated due to excessive angle differences is avoided, and the remaining target endpoints with the same direction and similar shape as the current endpoint are accurately identified from a large number of endpoints, thereby ensuring the rationality and accuracy of the connection of the ground fissure trend line, and providing reliable data support for ground fissure analysis.

[0123] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 4 , Figure 4 This is a flow chart of the second embodiment of the method for monitoring ground fissures in mining areas of the present application.

[0124] The preset semantic segmentation model is trained based on annotated images taken and visually interpreted of ground fissures in a mining area. The preset semantic segmentation model includes an encoder and a decoder. The mining area ground fissure monitoring method also includes steps S01 to S04:

[0125] Step S01, constructing a model training data set based on the annotated image, and performing data enhancement in multiple preset forms on the model training data set to obtain enhanced data sets;

[0126] It should be noted that the labeled image refers to the image whose location and category of ground fissures have been marked by the user through visual interpretation. These marked information is used to guide the training of the model; the model training dataset refers to the data set used to train the semantic segmentation model, which consists of labeled images and their corresponding annotation information; the enhanced dataset refers to the data set generated by performing a series of data transformations (such as rotation, scaling, cropping, etc.) on the original model training dataset, which is used to improve the generalization ability of the model.

[0127] It is understandable that in order to create diverse training data, step S01 is performed, which can avoid the overfitting of the model to a specific data distribution and the problem of poor generalization ability of the model, thereby improving the robustness and generalization ability of the model in identifying ground fissures under different conditions.

[0128] Step S02, inputting each of the enhanced data sets into the encoder respectively, and performing comparative learning based on the ground fissure feature information output after processing by the encoder, wherein the encoder is optimized based on a preset comparative loss function during the comparative learning process;

[0129] It should be noted that the ground fissure feature information refers to the data representation extracted by the encoder that can represent the characteristics of the ground fissures; the contrast loss function refers to the loss function used to measure the similarity between two feature vectors. This loss function is used in contrastive learning to guide the model to learn to distinguish between positive and negative samples.

[0130] It is understandable that since the encoder's ability to extract ground fissure features is usually not strong, step S02 is performed to improve the encoder's ability to extract features through comparative learning methods, which can avoid inaccurate feature representation, shorten the model training time, and reduce the sample size requirements of the data set, thereby reducing labor costs while improving the generalization performance and segmentation accuracy of the model.

[0131] For example, in the contrastive learning process, given a set of augmented datasets {x k}, which includes a pair of positive examples x i and x j , the contrastive prediction task aims to predict the i Identifying {x k} k≠i x j . Randomly sample a mini-batch of N samples and generate an augmented view on the mini-batch samples, resulting in 2N data points. In this embodiment, no negative examples are explicitly specified, but given a positive pair, the other 2N-2 augmented examples in the mini-batch are considered negative examples. The similarity is defined as cosine similarity:

[0132]

[0133] Among them, u and v represent samples of different enhanced data sets, respectively. Then the loss function of a pair of positive pairs (i, j) is defined as:

[0134]

[0135] Among them, 1 [k≠i]is an indicator function, which is 1 when k≠i, and τ is a temperature parameter. The final loss is calculated for all positive pairs (i, j) and (j, i) in a small batch, and the weights and biases of each layer of the neural network in the encoder are adjusted by minimizing the loss function.

[0136] Step S03, inputting the ground fissure feature information into the decoder, so as to extract context information of the ground fissure feature information in the decoder through a preset isotropic convolution, and performing ground fissure feature fusion according to the context information and feature information output by each layer of the neural network in the encoder, to obtain a ground fissure training extraction result;

[0137] It should be noted that isotropic atrous convolution refers to a convolution operation that increases the receptive field by inserting "holes" in the convolution kernel without increasing the number of parameters or reducing the resolution. Based on the concepts of one-dimensional convolution and atrous convolution, an isotropic atrous convolution is proposed, which can make full use of five types of atrous convolutions: omnidirectional, horizontal, vertical, left diagonal and right diagonal, to extract contextual information of ground fissure feature information from different directions; contextual information refers to the relationship and interaction information between ground fissure features and their surrounding environment; ground fissure training extraction results refer to the ground fissure feature information processed by the decoder, which is used to generate the final semantic segmentation results.

[0138] It is understandable that the fusion quality of traditional ground fissure features is often poor, so step S03 is performed to fully utilize the contextual information of the ground fissures through isotropic void convolution, which can avoid ignoring the relationship between the ground fissures and the surrounding environment, thereby improving the fusion quality of the ground fissure features and improving the accuracy of ground fissure monitoring.

[0139] For example, please refer to Figure 5 , with X∈R H×W×C Represents the input tensor, where H, W, and C represent the width, height, and number of channels of the input tensor, respectively. In the dilated convolution module, after the 1×1 convolution, X is input into 7 parallel paths, 4 of which contain a dilated convolution of one shape each, 2 paths are 3×1 convolution (horizontal kernel) and 1×3 convolution (vertical kernel), and 1 3×3 convolution is an omnidirectional dilated convolution. Then, the 7 output feature maps are connected, and 1×1 convolution and upsampling are performed to obtain the output of the dilated convolution. Let w∈R 2k+1 is a unidirectional hole convolution of size 2k+1,

[0140] `

[0141] D=(D h , D w ) represents the direction of w, Z D ∈R H×W×C The result of the unidirectional dilated convolution can be defined as:

[0142]

[0143] Among them, X*w represents the convolution operation, D represents the direction vector of the convolution, and (0, 1), (1, 0), (1, 1) and (-1, 1) represent horizontal, vertical, left diagonal and right diagonal convolutions respectively. We set k = 3, 4, 5 so that the hole convolution in each direction has different scales, thereby expanding the receptive field. In this part, each position in the output feature map is allowed to establish a relationship with multiple positions in the five directions of the input feature map, thereby extracting the context information of the ground fissure feature information.

[0144] Step S04, calculating the regional loss function of the preset semantic segmentation model based on the ground fissure training extraction result, and optimizing the decoder based on the regional loss function.

[0145] It should be noted that the regional loss function refers to the loss function used to measure the difference between the ground fissure area predicted by the model and the actual marked area.

[0146] It can be understood that in order to optimize the decoder so that the decoder can predict the ground fissure area more accurately, step S04 is performed. By minimizing the regional loss function to optimize the weights and biases of each layer in the decoder, inaccurate semantic segmentation results can be avoided, thereby reducing the prediction error of the model and realizing the accurate prediction of the ground fissure area by the semantic segmentation model.

[0147] For example, in order to better optimize the model, the regional loss function intersection over union (IoU) is used to effectively evaluate the gap between the predicted cracks and the real labeled cracks. IoU is the ratio of the intersection and union of the real label and the predicted value. MIoU is the average of the intersection over union of each class in the data set (i.e., the ground crack training extraction result). The calculation formula is as follows:

[0148]

[0149] Among them, TP, FN, FP represent the corresponding true positive, false negative and false positive numbers. MIoU Loss is defined as:

[0150]

[0151] One problem with MIoU Loss is that the IoU weights of various samples are equal, which may cause the model prediction to be biased towards non-crack areas. However, in the crack image, i.e., the ground crack training extraction results, the positive samples (crack pixels) only account for a small part compared to the negative samples (non-crack pixels). Therefore, Focal Loss is introduced to solve the problem of extremely unbalanced positive and negative samples, which can be expressed as:

[0152]

[0153] Focal Loss focuses on difficult examples by giving a larger weight to samples that are difficult to classify. For positive samples, the loss obtained by using samples with a large prediction probability becomes smaller, while the loss of samples with a small prediction probability (difficult examples) becomes larger, thereby increasing the focus on difficult examples. However, it introduces additional parameters, which increases the difficulty of parameter adjustment. Therefore, through weighted mixing, MIoU Loss and Focal Loss are combined to obtain a hybrid regional loss function, namely the regional loss function, which is calculated as follows:

[0154]

[0155] Among them, β is a hyperparameter used to balance MIoU Loss and Focal Loss. By minimizing the loss function in this area, the weights and biases of each layer in the model are adjusted and optimized.

[0156] In this embodiment, by performing data enhancement, contrastive learning, isotropic convolution and multi-scale feature fusion, the problems of model overfitting, inaccurate feature representation, insufficient utilization of contextual information and low semantic segmentation accuracy are avoided, and the effects of improving the generalization ability of the model, enhancing the feature extraction ability, improving the quality of ground fissure feature fusion and accurately predicting the ground fissure area are achieved, so that the semantic segmentation model can more robustly and accurately identify and segment ground fissures.

[0157] In a feasible implementation manner, before step S03, steps S031 to S032 may also be included:

[0158] Step S031, extracting spatial attention features and channel attention features from the candidate ground fissure feature information processed by multiple residual networks and multiple dilated residual networks to obtain spatial features and channel features of the candidate ground fissure feature information;

[0159] It should be noted that the residual network refers to a deep learning network structure, which allows the gradient to flow directly to the deeper layers of the network by introducing jump connections, thereby effectively solving the gradient vanishing problem in deep network training; the dilated residual network is based on the residual network, and introduces dilated convolution to increase the receptive field while keeping the number of parameters unchanged, and is used to process features with a large spatial range; the candidate ground fissure feature information refers to the feature data that has been preliminarily processed by the residual network and the dilated residual network. These feature data contain possible information about the ground fissures, but have not been further refined; the spatial attention feature refers to the feature learned in the network that can highlight the importance of the spatial position of the ground fissure in the image; the channel attention feature refers to the feature learned in the network that can emphasize the importance of different feature channels (i.e., different types of features) for ground fissure identification; the spatial feature refers to the feature extracted from the candidate ground fissure feature information and related to the spatial distribution of the ground fissure in the image; the channel feature refers to the feature extracted from the candidate ground fissure feature information and related to different feature channels.

[0160] It is understandable that due to the limitation of the local receptive field, the residual network and the dilated residual network can only capture feature information at each step, which is insufficient to obtain a large amount of valuable global features and contextual information, hindering the derivation of long-term memory. Therefore, step S031 is performed to extract important features in space and channels from the network by introducing double attention, which can avoid ignoring the key information of the ground cracks in the space and channel dimensions, resulting in insufficient feature representation or loss of important information, thereby further improving the representation ability and recognition accuracy of the ground crack feature information.

[0161] Step S032, combining the spatial features and the channel features to obtain the ground fissure feature information.

[0162] It can be understood that in order to effectively combine the spatial and channel features to obtain a more comprehensive representation of ground fissure features, step S032 is performed, which can avoid problems such as performance degradation caused by insufficient feature fusion, thereby enhancing the model's comprehensive recognition ability for ground fissures and improving the accuracy and robustness of semantic segmentation.

[0163] For example, please refer to Figure 6In the spatial attention part, the input feature X, i.e., the candidate ground fissure feature information processed by multiple residual networks and multiple dilated residual networks, is first passed into two 1x1 convolutional layers, and then the feature matrix is ​​transposed, reshaped, and multiplied. The result is passed through the Softmax layer to generate the weighted spatial attention matrix S, which can represent the spatial location connection of any two pixels. Then the input feature X is transformed by 1x1 convolution again, and the spatial attention matrix S is multiplied with the transformed feature to generate the fused feature. Then the final feature map OS is created by performing element aggregation operations on the input feature x and the fused feature. OS selectively aggregates contextual information and improves the compactness and consistency of the ground fissure feature. In the channel attention part, the input feature X is matrix transposed, reshaped, and multiplied, and then the weighted channel attention matrix C is obtained through the Softmax layer. C can represent the mutual dependence between any two feature map channels. Then the feature fusion is performed by multiplying the channel matrix C and the input feature to generate the final output feature map OC. OC enhances the long-term semantic interdependence between channels, and the obtained ground fissure feature has a strong semantic response. Finally, OS and OC are connected to obtain high-level semantic features including global spatial position and channel features, namely, ground fissure feature information.

[0164] In this embodiment, by introducing dual attention including spatial attention and channel attention, global context information is obtained from the features extracted from the network, thereby avoiding the problem of important information being ignored or weakened in the feature extraction process and the problem of insufficient feature representation, and ensuring that the model can simultaneously focus on the key information of the ground fissures in the spatial position and feature channels, thereby improving the accuracy of ground fissure detection and the model's adaptability to complex scenarios.

[0165] In a feasible implementation manner, step S02 may further include steps S021 to S022:

[0166] Step S021, obtaining feature information output by each layer of the neural network in the encoder;

[0167] It should be noted that the feature information refers to the data output from each layer of the neural network of the encoder, which contains feature representations of ground fissures at multiple scales.

[0168] Step S022, performing edge reinforcement learning of ground cracks based on each feature information to obtain edge attention mapping parameters, so as to use the edge attention mapping parameters to generate the ground crack training extraction results in the decoder, wherein, in the edge reinforcement learning process of the ground cracks, the feature information is input into an edge reinforcement branch composed of multiple residual blocks and multiple gated convolution blocks, so that the edge attention mapping parameters are calculated by the edge reinforcement branch, and the edge reinforcement branch is optimized by a preset edge loss function.

[0169] It should be noted that edge reinforcement learning is a learning process that focuses on enhancing the feature representation of the edge of the ground fissure. Through learning, the model can more accurately identify and locate the edge of the ground fissure; the edge attention mapping parameter refers to the parameter used to emphasize the edge features of the ground fissure, which is used in the decoder to generate more accurate ground fissure training extraction results; the residual block refers to a network structure that can transfer gradients through jump connections, effectively solve the gradient vanishing problem in deep network training, and deepen the network depth at the same time; the gated convolution block is a special convolution block that controls the flow of information through a gating mechanism, enhances the network's selectivity for features, and improves the accuracy of feature representation; the edge loss function is a loss function used to optimize the edge reinforcement branch. This function is designed to measure the accuracy of the model's prediction of the edge of the ground fissure and optimize the network parameters through back propagation.

[0170] It is understandable that since the existing model often has low recognition accuracy for the edges of ground cracks, step S022 is performed to enhance the edge features of ground cracks by calculating edge attention mapping parameters and edge loss functions, thereby improving the accuracy of the model's detection of ground cracks and the model's ability to process ground crack details.

[0171] For example, please refer to Figure 7 , the edge enhancement branch consists of multiple residual blocks (squares in the figure) interleaved with gated convolution blocks (rectangles in the figure), where the gated convolution blocks make the branch process only information related to the boundary. N represents the number of positions, t∈0,1,2,...,N is the running index, where r t and e t represents the intermediate representation of the corresponding regular and edge streams processed using gated convolution. During the operation of the gated convolution block, first, by connecting e t and r t , followed by a 1×1 convolutional layer with normalization, followed by a sigmoid function σ, to obtain the edge attention map α t : In addition, the edge loss function is to deeply supervise the edge enhancement branch by measuring the difference between the predicted edge map and the binary edge map generated by the GT map, which is supervised by the cross entropy loss function (BCE). The edge loss function is defined as:

[0172]

[0173] Where (x, y) is the edge ground truth map G e and the predicted edge graph S e The coordinates of each pixel in G e is to use the ground truth map G s where w and h represent the width and height of the corresponding map, respectively.

[0174] In this embodiment, by extracting and strengthening the output feature information of each layer of the neural network in the encoder, edge attention mapping parameters are calculated, and these parameters are used in the decoder to generate more accurate ground crack training extraction results, so that the model can more effectively focus on and locate the edges of the ground cracks, and optimize the weights and biases of each layer in the edge enhancement branch through the edge loss function, thereby avoiding the problems of inaccurate ground crack edge recognition and insufficient feature representation, improving the accuracy of ground crack edge detection and enhancing the model's ability to learn ground crack features.

[0175] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those of the above-mentioned first and / or second embodiments can be referred to the above introduction, and will not be described in detail later. Figure 8 , Figure 8 This is a flow chart of the third embodiment of the method for monitoring ground fissures in mining areas of the present application. The method for monitoring ground fissures in mining areas may further include steps S100 to S200 before step S10:

[0176] Step S100, obtaining original mining data of the mining area, and determining the image acquisition range and image control point information according to the original mining data;

[0177] It should be noted that the original mining data includes mining area DEM (Digital Elevation Model) data and working face mining data.

[0178] Step S200: collecting an original image to be processed based on the image collection range and the image control point information, and performing orthophoto processing on the original image to be processed to obtain the original image.

[0179] It should be noted that orthophoto processing at least includes image stitching, aerial triangulation, mosaicking and other processing processes to obtain an orthophoto (i.e., the original image).

[0180] It is understandable that since images taken directly by equipment such as drones often cannot be directly used for segmentation processing by semantic segmentation models, or the original images processed by semantic segmentation models are usually difficult to directly reflect the actual environmental information of the mining area, step S200 is performed to obtain the original images that can reflect the objective and real geographical environment by orthophoto processing the collected images, thereby providing an effective and reliable data basis for subsequent analysis of ground fissures in mining areas.

[0181] In this embodiment, by collecting images based on the original mining data of the mining area and performing orthophoto processing on the collected images, it is ensured that the collected data is accurate and effective, and the overall efficiency of the ground fissure analysis process is improved.

[0182] In a feasible implementation manner, after the step of extracting the ground fissure skeleton from the ground fissure extraction result in step S20, steps S201 to S202 may also be included:

[0183] Step S201, determining the direction angle of the ground fissure skeleton, and generating a normal line of the ground fissure skeleton according to the direction angle;

[0184] Step S202, determining the ground fissure width based on the distance between the normal line and the crack boundary of the ground fissure skeleton, and saving the ground fissure width, wherein the ground fissure width is used to be associated with the ground fissure strike line for output.

[0185] In a feasible implementation manner, step S30 may further include step S301 after step S30:

[0186] Step S301, calculating the ground fissure length and ground fissure width direction based on the ground fissure strike line, wherein the ground fissure length and the ground fissure direction are used to be associated with the ground fissure strike line and output.

[0187] For example, in order to help understand the implementation process of the mining area ground fissure monitoring method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Fig. 9 , Fig. 9 A brief schematic diagram of a method for monitoring ground fissures in a mining area is provided, specifically:

[0188] First, the mining area data, i.e., the original mining data of the mining area, is obtained, and the monitoring area, i.e., the image acquisition range and the image control point information, is delineated according to the mining area data, and the UAV flight plan is formulated. After the ground image control points are arranged and the UAV is taken in flight, the original image to be processed is obtained. After the original image to be processed is processed by image stitching and aerial triangulation, the original image is obtained. The original image is manually visually interpreted, i.e., the ground fissures are annotated to obtain an annotated image, and the annotated image is divided according to a preset ratio to obtain a model training data set. The constructed ground fissure model, i.e., the preset semantic segmentation model, is trained by the model training data set, and the original image is segmented by the trained semantic segmentation model to obtain the ground fissure extraction result. At this time, the maximum width (i.e., the ground fissure width) and the average width of the ground fissure can be calculated from the ground fissure extraction result. The ground fissure skeleton and the key points of the ground fissure skeleton are extracted from the ground fissure extraction result, so as to generate a ground fissure structure line map based on the key points, and then the endpoints of the ground fissure structure line map are selectively connected to obtain the ground fissure strike line. At this time, the ground fissure length and ground fissure direction can be calculated from the ground fissure strike line. Finally, the crack vector is generated and attached, that is, the ground crack strike line, as well as the ground crack width, length, direction and other data are associated and output, all of which are Shapefile format data, and the attribute information is stored in the form of attribute table DBF, so as to meet the application needs of various scenarios in various forms of crack products: crack line vector (including length, width, strike information), crack surface vector (including length, width, strike, area information), crack skeleton grid, and crack surface grid.

[0189] For further information, please refer to Fig.10 , Fig.10 This is a diagram of the semantic segmentation model architecture. The input data in the figure is the original image. The encoder part contains two layers of residual networks, namely Layer2 and Layer3, and two layers of dilated residual networks, namely Layer4 and Layer5. The projection head in the figure is obtained using a layer with 2 Dense and an activation layer, where the activation function is a ReLU nonlinear function. The contrast function defined at the output of the projection head can achieve excellent evaluation performance. The dual attention part includes the spatial attention part SAM and the channel attention part CAM, such as Figure 6 As shown in FIG. 1 , the edge attention map calculated by the edge enhancement branch can be fused into C, the aggregated part after double attention processing, and the ground fissure feature information.

[0190] This application also provides a mining area ground fissure monitoring device, please refer to Fig.11 , the mining area ground fissure monitoring device comprises:

[0191] The segmentation module 10 is used to obtain the original image taken of the ground fissures in the mining area, and segment the original image based on a preset semantic segmentation model to obtain the ground fissure extraction result;

[0192] A generating module 20, for extracting a ground fissure skeleton from the ground fissure extraction result, and generating a ground fissure structure line diagram according to key points of the ground fissure skeleton;

[0193] The connection module 30 is used to connect the related structure lines in the ground fissure structure line diagram to obtain the ground fissure strike line.

[0194] The mining area ground fissure monitoring device provided by the present application adopts the mining area ground fissure monitoring method in the above embodiment, which can solve the technical problem of low accuracy of ground fissure extraction by the automated ground fissure monitoring method. Compared with the prior art, the beneficial effects of the mining area ground fissure monitoring device provided by the present application are the same as the beneficial effects of the mining area ground fissure monitoring method provided by the above embodiment, and the other technical features of the mining area ground fissure monitoring device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0195] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the mining area ground fissure monitoring method in the above-mentioned embodiment one.

[0196] Reference below Fig.12, the electronic device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. In particular, according to the embodiments disclosed in the present application, the process described with reference to the flowchart above can be implemented as a computer software program. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0197] The electronic device provided by the present application adopts the mining area ground fissure monitoring method in the above embodiment, which can solve the technical problem of low accuracy of ground fissure extraction by the automated ground fissure monitoring method. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the mining area ground fissure monitoring method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0198] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the mining area ground fissure monitoring method in the above-mentioned embodiment.

[0199] In this embodiment, the computer readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted using any appropriate medium.

[0200] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains an original image of ground fissures in a mining area, and segments the original image based on a preset semantic segmentation model to obtain ground fissure extraction results; extracts ground fissure skeletons from the ground fissure extraction results, and generates a ground fissure structure line map based on key points of the ground fissure skeletons; and connects associated structure lines in the ground fissure structure line map to obtain ground fissure strike lines.

[0201] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned mining area ground fissure monitoring method, and can solve the technical problem that the automatic ground fissure monitoring method has low accuracy in extracting ground fissures. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the mining area ground fissure monitoring method provided by the above-mentioned embodiment, and will not be repeated here.

Claims

1. A method for monitoring ground fissures in a mining area, characterized in that: The mining area ground fissure monitoring method comprises: Acquire the original image of the ground fissures in the mining area, and segment the original image based on a preset semantic segmentation model to obtain a ground fissure extraction result; Extracting a ground fissure skeleton from the ground fissure extraction result, and generating a ground fissure structure line diagram according to key points of the ground fissure skeleton; The related structural lines in the ground fissure structural line diagram are connected to obtain the ground fissure strike line.

2. The method for monitoring ground fissures in a mining area according to claim 1, characterized in that: The ground fissure extraction result includes a ground fissure binary map, and the steps of extracting a ground fissure skeleton from the ground fissure extraction result and generating a ground fissure structure line map according to key points of the ground fissure skeleton include: Obtaining a connected domain of ground fissures in the ground fissure binary map, and performing denoising and connectivity completion on the connected domain to obtain a processed ground fissure binary map; Extracting the ground fissure skeleton from the processed ground fissure binary image based on a preset image thinning algorithm; The key points are extracted from the ground fissure skeleton based on a preset iterative adaptive point algorithm, and the key points are connected to obtain the ground fissure structure line diagram.

3. The method for monitoring ground fissures in mining areas according to claim 1, characterized in that: The step of connecting the associated structural lines in the ground fissure structural line diagram to obtain the ground fissure strike line comprises: Obtaining the endpoints of each structure line in the ground fissure structure line diagram; For any one of the structural lines, determine the direction angle of the structural line according to the corresponding endpoint of the structural line, and retrieve the remaining endpoints within a preset distance range of the corresponding endpoint of the structural line, wherein the remaining endpoints are the endpoints of the remaining structural lines; In the case where the remaining endpoints exist within the preset distance range, the remaining endpoints are screened based on the strike angle to obtain target remaining endpoints, and a connection relationship between the target remaining endpoints and the corresponding endpoints of the structure line is determined; After traversing each endpoint, each connection relationship is deduplicated, and the structure line endpoints in the ground fissure structure line diagram are connected based on the deduplicated connection relationships to obtain the ground fissure strike line.

4. The method for monitoring ground fissures in a mining area according to claim 3, characterized in that: The step of screening the remaining endpoints based on the direction angle to obtain the remaining target endpoints includes: Filtering the remaining endpoints based on the connection angles between the remaining endpoints and the endpoint to obtain candidate remaining endpoints, wherein the candidate remaining endpoints are the remaining endpoints whose difference between the connection angle and the direction angle is less than a preset angle; For any candidate remaining endpoint among the candidate remaining endpoints, determining the similarity between a first image around the candidate remaining endpoint and a second image around a corresponding endpoint of the structure line, and testing the similarity to obtain a test result; After traversing each candidate remaining endpoint, the candidate remaining endpoint corresponding to the qualified search result in each inspection result is used as the target remaining endpoint.

5. The method for monitoring ground fissures in mining areas according to claim 1, characterized in that: The preset semantic segmentation model is trained based on annotated images captured and visually interpreted from ground fissures in a mining area, the preset semantic segmentation model includes an encoder and a decoder, and the mining area ground fissure monitoring method further includes: Constructing a model training data set based on the labeled image, and performing data enhancement in multiple preset forms on the model training data set to obtain enhanced data sets; Inputting each of the enhanced data sets into the encoder respectively, and performing comparative learning based on the ground fissure feature information output after processing by the encoder, wherein the encoder is optimized based on a preset comparative loss function during the comparative learning process; The ground fissure feature information is input into the decoder, so as to extract context information of the ground fissure feature information in the decoder through a preset isotropic convolution, and perform ground fissure feature fusion according to the context information and feature information output by each layer of the neural network in the encoder to obtain a ground fissure training extraction result; The regional loss function of the preset semantic segmentation model is calculated based on the ground fissure training extraction result, and the decoder is optimized based on the regional loss function.

6. The method for monitoring ground fissures in a mining area according to claim 5, characterized in that: The encoder includes a plurality of residual networks and a plurality of dilated residual networks, and before the step of inputting the ground fissure feature information into the decoder, the step further includes: Extracting spatial attention features and channel attention features from candidate ground fissure feature information processed by multiple residual networks and multiple dilated residual networks to obtain spatial features and channel features of the candidate ground fissure feature information; The spatial features and the channel features are combined to obtain the ground fissure feature information.

7. The method for monitoring ground fissures in a mining area according to claim 5, characterized in that: After the step of performing comparative learning based on the ground fissure feature information output after processing by each encoder, the step further includes: Obtaining feature information output by each layer of the neural network in the encoder; Based on each feature information, edge reinforcement learning of the ground cracks is performed to obtain edge attention mapping parameters, so that the edge attention mapping parameters are used to generate the ground crack training extraction results in the decoder, wherein, in the edge reinforcement learning process of the ground cracks, the feature information is input into an edge reinforcement branch composed of multiple residual blocks and multiple gated convolution blocks, so that the edge attention mapping parameters are calculated by the edge reinforcement branch, and the edge reinforcement branch is optimized by a preset edge loss function.

8. A ground fissure monitoring device in a mining area, characterized in that: The mining area ground fissure monitoring device comprises: A segmentation module is used to obtain the original image taken of the ground fissures in the mining area, and segment the original image based on a preset semantic segmentation model to obtain a ground fissure extraction result; A generating module, used for extracting a ground fissure skeleton from the ground fissure extraction result, and generating a ground fissure structure line diagram according to key points of the ground fissure skeleton; The connection module is used to connect the related structure lines in the ground fissure structure line diagram to obtain the ground fissure trend line.

9. An electronic device, characterized in that: The device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for monitoring ground fissures in a mining area as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the mining area ground fissure monitoring method as described in any one of claims 1 to 7 are implemented.

Images