Slope crack monitoring method, device, medium and equipment

Abstract

The invention discloses a slope crack monitoring method and device, a medium and equipment. The method comprises the following steps: acquiring slope crack image data; and processing the slope crack image data by using the slope crack recognition model with the optimal parameters to obtain a slope crack recognition image of which the confidence exceeds a preset value. A slope crack identification model is determined based on a backbone network added with an SE attention mechanism module, a Neck feature enhancement network added with a feature pyramid network and a path aggregation network, and a YOLOv5 model of a head network provided with three 1 * 1 convolutional layers, so that the method has more excellent robustness and precision; slope crack monitoring information is obtained based on the slope crack recognition image and the pre-obtained three-dimensional model of the slope crack area, formation and expansion of cracks can be found as early as possible, geological disasters such as landslide and collapse of the slope can be early warned in advance, corresponding safety measures are taken, and the safety of the slope is improved. The technical problems that the features of the slope crack image shot by the unmanned aerial vehicle are not obvious, and the slope crack cannot be accurately recognized and monitored are solved.

Description

Technical Field

[0001] The present application relates to the technical field of coal mine safety engineering, and in particular to a slope crack monitoring method, device, medium and equipment. Background Art

[0002] During open-pit mining, cracks will appear on the slopes due to factors such as geological structure, rock mechanical properties, and mining activities. If they are not discovered in time and appropriate remedial measures are not taken, they will cause serious hazards to construction safety such as landslides and collapses. Therefore, the significance of slope crack monitoring is very important.

[0003] Slope crack monitoring generally adopts a combination of on-site observation and measurement. Through manual observation and instrument measurement, such as laser instruments, total stations and other tools, the cracks, displacements and other deformations of the slope are recorded and analyzed, so as to timely discover and grasp the deformation of the slope. However, this method is greatly affected by human factors, and the observation results have certain errors. In addition, there are high and steep slopes in open-pit mines, and engineering personnel cannot directly reach them, and thus cannot observe cracks.

[0004] When using drone technology for non-contact aerial photography to obtain images of slope cracks, some open-pit mines have complex high and steep slopes with scattered trees and power towers. In order to ensure flight safety, the drones fly at a high distance, and the features of the slope crack images taken are not obvious, making it impossible to accurately identify and monitor slope cracks. Summary of the invention

[0005] The main purpose of this application is to provide a slope crack monitoring method, device, medium and equipment, aiming to solve the technical problem that the features of slope crack images taken by drones are not obvious and cannot accurately identify and monitor slope cracks.

[0006] To achieve the above-mentioned purpose, the present application provides a slope crack monitoring method, device, medium and equipment, including: obtaining slope crack image data; processing the slope crack image data using a slope crack recognition model with optimal parameters to obtain a slope crack recognition image with a confidence level exceeding a preset value, wherein the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module, a Neck feature enhancement network with a feature pyramid network and a path aggregation network, and a YOLOv5 model with a head network having three 1x1 convolutional layers; and obtaining slope crack monitoring information based on the slope crack recognition image and a pre-acquired three-dimensional model of the slope crack area.

[0007] Optionally, before obtaining the slope crack image data, the slope crack monitoring method further includes: obtaining slope crack image acquisition data; processing the slope crack image acquisition data based on a target detection algorithm to obtain slope crack image data; wherein the target detection algorithm is obtained by replacing the Mosaic image enhancement algorithm in the YOLOv5 model with a multi-scale Retinex image processing algorithm.

[0008] Optionally, before processing the slope crack image data using the slope crack recognition model with optimal parameters, the slope crack monitoring method also includes: obtaining a training set, a validation set and a test set of the annotated slope crack image data; iteratively training the slope crack recognition model based on the training set, monitoring the performance of the slope crack recognition model based on the validation set, supervising the training of the slope crack recognition model based on a center loss function combined with a focal-loss loss function, testing the recognition accuracy of the monitored slope crack recognition model based on the test set, and obtaining a monitored slope crack recognition model with optimal parameters.

[0009] Optionally, the slope crack recognition model is determined based on the YOLOv5 model of a backbone network with an SE attention mechanism module added, a Neck feature enhancement network with a feature pyramid network and a path aggregation network added, and a head network with three 1x1 convolutional layers, including: constructing a backbone network according to a first convolutional module, a second convolutional module, a first composite convolutional structure module, a third convolutional module, a second composite convolutional structure module, a fourth convolutional module, a third composite convolutional structure module, a fifth convolutional module, an SE attention mechanism module, and an SPPF module connected in sequence; and determining the slope crack recognition model according to the backbone network, a Neck feature enhancement network with a feature pyramid network and a path aggregation network added, and a head network with three 1x1 convolutional layers.

[0010] Optionally, the obtaining of slope crack monitoring information based on the slope crack identification image and a pre-acquired three-dimensional model of the slope crack area includes: determining the slope crack area according to the acquired three-dimensional model of the slope crack area; acquiring a slope crack identification image having a confidence level exceeding a preset value; determining a three-dimensionally reconstructed slope crack area based on the slope crack identification image and the slope crack area exceeding the preset value; and determining the slope crack monitoring information based on the three-dimensionally reconstructed slope crack area.

[0011] Optionally, after determining the slope crack monitoring information based on the three-dimensionally reconstructed slope crack area, the slope crack monitoring method further includes: outputting an alarm record based on the slope crack monitoring information.

[0012] Optionally, the multi-scale Retinex image processing algorithm includes: decomposing the slope crack image acquisition data into the product of the illumination component of the ambient light and the reflection component of the target object carrying the image detail information to obtain the original image after multi-scale decomposition; performing Gaussian blur processing on the original image after multi-scale decomposition to obtain a blurred image; and merging the components of each scale to obtain an enhanced image.

[0013] In order to solve the above technical problems, the present invention also provides a slope crack monitoring device, including: an acquisition module, used to acquire slope crack image data; a processing module, used to process the slope crack image data using a slope crack recognition model with optimal parameters to obtain a slope crack recognition image with a confidence level exceeding a preset value, wherein the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module added, a Neck feature enhancement network with a feature pyramid network and a path aggregation network added, and a YOLOv5 model with a head network having three 1x1 convolutional layers; an output module, used to obtain slope crack monitoring information based on the slope crack recognition image and a pre-acquired three-dimensional model of the slope crack area.

[0014] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, which includes instructions, and when the instructions are run on a computer, the computer executes any one of the slope crack monitoring methods.

[0015] In order to solve the above technical problems, the present invention also provides a monitoring device, which includes: at least one processor, a memory and an input and output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the slope crack monitoring method.

[0016] The embodiments of the present application propose a slope crack monitoring method, device, medium and equipment, which obtains slope crack image data; uses a slope crack recognition model with optimal parameters to process the slope crack image data to obtain a slope crack recognition image with a confidence level exceeding a preset value; determines the slope crack recognition model based on a backbone network with an SE attention mechanism module, a Neck feature enhancement network with a feature pyramid network and a path aggregation network, and a YOLOv5 model with a head network having three 1x1 convolutional layers; and combines the deep learning network model for recognition and analysis to accurately and effectively grasp the changes in the slope cracks; obtains slope crack monitoring information based on the slope crack recognition image and a three-dimensional model of the pre-acquired slope crack area, and can detect the formation and expansion of cracks early, and warn in advance that geological disasters such as landslides and collapses may occur on the slope, so as to take corresponding safety measures to protect the safety of personnel and property, thereby solving the technical problem that the features of the slope crack images taken by drones are not obvious and the slope cracks cannot be accurately identified and monitored. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a slope crack monitoring method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the structure of the YOLOv5 backbone network of the slope crack monitoring method provided in the embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of the structure of a slope crack monitoring device provided in an embodiment of the present application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] With the rise of drone technology, drones can be used for patrol, surveillance, aerial photography and other tasks in air traffic, which reduces the burden of manpower, improves the safety and efficiency of traffic flow, and expands the scope of monitoring and inspection. In September 2018, the 62nd meeting of the World Customs Organization Harmonized System Committee (HSC) decided to classify drones as "flying cameras". Drones are equipped with high-definition cameras to capture actual scene images in a non-contact manner and provide real-time feedback of flight images. In the face of special dangerous environments, they can not only safely collect image materials and record actual flight images, but also ensure the safety of personnel, making aerial monitoring safer and more accurate.

[0023] Deep learning is the study of the inherent laws and representation levels of sample data. It can accurately identify and classify various types of images and sounds through the construction of large amounts of data and deep neural networks, and better characterize the rich inherent information of the data. With the development of deep learning, mainstream deep models have also emerged. YOLOv5 is currently a common and commonly used deep learning method in the field of target detection. YOLOv5 uses a lightweight network structure to reduce the number of parameters and calculations of the model, making the model lighter while maintaining high performance. Among them, the image data enhancement method of YOLOv5 uses the Mosaic algorithm, which splices four different images into one image, which can increase the generalization and richness of the data set, but the image data may lose some detailed information due to the complexity of the image, and the image visual effect is poor. The multi-scale Retinex image processing algorithm can better and effectively avoid the problems of poor image contrast, brightness, color, and poor visual effect. The algorithm linearly combines the enhancement results at different scales while retaining the image detail information, fully taking into account the local information and the overall information, and then adjusting the contrast and brightness of the image, which can produce a single output image with good dynamic range compression, color stability and good hue restoration. In general, the multi-scale Retinex image processing algorithm can better handle problems such as image illumination changes and color restoration by combining information at different scales, thereby improving the visual effect of the image.

[0024] During the open-pit mining process, cracks will appear on the slopes due to factors such as geological structure, rock mechanical properties, and mining activities. If not discovered in time and appropriate remedial measures are taken, it will cause serious harm to construction safety such as landslides and collapses. Therefore, the significance of slope crack monitoring is very important. Slope crack monitoring generally adopts a method combining on-site observation and measurement. Through manual observation and instrument measurement, such as laser instruments, total stations and other tools, the deformation conditions such as cracks and displacements of the slopes are recorded and analyzed, so as to timely discover and grasp the deformation of the slopes. However, this method is greatly affected by human factors, the observation results have certain errors, and there are high and steep slopes in the open-pit mines, which cannot be reached directly by engineering personnel, and thus cracks cannot be observed. Therefore, the present invention uses drone technology for non-contact aerial photography to obtain images of slope cracks, but it should also be noted that the high and steep slopes of some open-pit mines have complex landforms, trees are scattered, and power towers are also set up. In order to ensure flight safety, the drones fly at a high distance, and the features of the slope crack images taken back are not obvious. To this end, multi-scale Retinex is introduced into YOLOv5 for identification, so as to accurately identify and monitor slope cracks. Finally, early warning analysis and statistics of crack identification results are carried out to realize slope crack early warning.

[0025] Figure 1For the flow chart of the slope crack monitoring method provided in the embodiment of the present application, refer to Figure 1 The slope crack monitoring method may be executed by a processor, and the slope crack monitoring method may include:

[0026] S10, obtaining slope crack image data.

[0027] Specifically, there are high and steep slopes in the open-pit mine, which cannot be reached directly by engineering personnel. The present invention uses a drone equipped with a P1 camera to obtain image data. During the specific implementation, the parameters of the drone equipment used are shown in Table 1.

[0028] Table 1 UAV performance table

[0029] parameter Specific value weight 3.6kg without battery, 6.3kg with dual batteries Maximum take-off weight 9kg Maximum flight altitude 5000m Maximum wind speed 15m / s Maximum flight time 55min Operating temperature -20° to 50° Maximum horizontal flight speed S mode 7m / s, P mode 17m / s Maximum ascent speed S mode 6m / s, P mode 5m / s Maximum descent speed (vertical) S mode 5m / s, P mode 4m / s Maximum flight altitude 7000m

[0030] The power is the kinetic energy of the equipment and the basis of safe flight. Before flying, check whether the power of each device is sufficient. The number of battery packs required by the drone is calculated according to the flight time. Normally, the flight time of a set of batteries is about half an hour. When the route is automatically planned, the flight time will be automatically generated. The drone will calculate how many sets of batteries are needed to complete the specified flight mission based on the flight time. In addition, if allowed, it is best to have a charging place close to the flight area to ensure the energy supply of the flight battery.

[0031] The flight area file is drawn in advance in the relevant surveying and mapping software or mapping software. The flight area file format is .kml. Save the drawn kml file to a USB flash drive and then import it into the remote controller. After importing the kml file into the remote controller, the drone will automatically plan the flight route according to the flight area.

[0032] The monitor should check whether the camera memory card is inserted before flight and check the remaining capacity of the memory card. The monitor should delete the pictures in the memory card after transferring the flight data to the computer after each flight. Otherwise, the flight mission will be interrupted when the memory card is full.

[0033] The drone is equipped with a P1 camera to obtain two-dimensional and three-dimensional images, with a flight altitude of 150m and an intelligent posing mode. The heading overlap rate is set to 80%, and the lateral overlap rate is set to 60%. If the overlap rate is not enough, it is easy to have stripes, holes, and black holes. The acquired pictures are saved in the camera memory card, in a folder named with the flight number and time, and the picture is saved in JPG format. Finally, the picture data is transmitted to the server.

[0034] In the embodiment of the application, before step S10, the slope crack monitoring method may further include:

[0035] S40. Processing the slope crack image acquisition data based on a target detection algorithm to obtain slope crack image data; wherein the target detection algorithm is obtained by replacing the Mosaic image enhancement algorithm in the YOLOv5 model with a multi-scale Retinex image processing algorithm.

[0036] Specifically, the multi-scale Retinex image processing algorithm may include the following specific implementation steps:

[0037] S41, decomposing the slope crack image acquisition data into the product of the illumination component of the ambient light and the reflection component of the target object carrying the image detail information, to obtain the original image after multi-scale decomposition;

[0038] S42, performing Gaussian blur processing on the original image after multi-scale decomposition to obtain a blurred image;

[0039] S43, merging the scale components to obtain an enhanced image.

[0040] For example, in the special geomorphic environment of high and steep slopes, the slope crack targets are small. If encountering special weather or foggy environment, the slope crack data obtained by the drone is fuzzy and the image visual effect is poor. To address this problem, the present invention replaces the traditional Mosaic image enhancement algorithm of YOLOv5 with the multi-scale Retinex image processing algorithm, mainly by decomposing the image into components of different scales, and then adjusting the contrast of each scale component respectively, so as to more effectively deal with the problem of illumination changes and better capture the detailed information of the image, and finally merge the scale components to obtain the enhanced image.

[0041] The formula of the multi-scale Retinex image processing algorithm is as follows:

[0042] F(x,y)=L(x,y)×R(x,y)

[0043] LogR(x,y)=LogF(x,y)-LogL(x,y)

[0044] Among them, (x, y) is the coordinate of the corresponding pixel point, F(x, y) represents the original image collected by the device, R(x, y) is the reflection component of the target object carrying the image detail information, and L(x, y) is the illumination component of the ambient light. Then, Gaussian blur is performed on the image at each scale to obtain the blurred image:

[0045] Log(R(x,y))=Log(R(x,y))+Weight(i)×(Log(F i (x,y))-Log(L i (x,y)))

[0046] Among them, i represents the scale reading, Weight(i) represents the weight of each scale, and the sum is 1. Finally, the image output is expressed as:

[0047]

[0048] Among them, G i (x,y) is the scale parameter δ i Gaussian function, N is the number of scales, generally N = 3 represents large, medium and small scales, λ i It is a scale variable and needs to be adjusted according to the specific situation. i is a Gaussian surround scaling function with a value between 80 and 100. The enhanced image data is obtained through the above steps and used in the later model training.

[0049] S20. Process the slope crack image data using the slope crack recognition model with optimal parameters to obtain a slope crack recognition image with a confidence level exceeding a preset value, wherein the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module added, a Neck feature enhancement network with a feature pyramid network and a path aggregation network added, and a YOLOv5 model with a head network having three 1x1 convolutional layers.

[0050] Figure 2 The backbone network structure diagram of the YOLOv5 model provided in the embodiment of the present application is shown in FIG. Figure 2 In an embodiment of the present application, the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module, a Neck feature enhancement network with a feature pyramid network and a path aggregation network, and a YOLOv5 model with a head network having three 1x1 convolutional layers, and may include:

[0051] Constructing a backbone network according to the first convolution module, the second convolution module, the first composite convolution structure module, the third convolution module, the second composite convolution structure module, the fourth convolution module, the third composite convolution structure module, the fifth convolution module, the SE attention mechanism module, and the SPPF module connected in sequence;

[0052] The slope crack recognition model is determined according to the backbone network, the Neck feature enhancement network with the feature pyramid network and the path aggregation network added, and the YOLOv5 model of the head network with three 1x1 convolutional layers.

[0053] In an embodiment of the present invention, in order to improve the model recognition accuracy, the present invention adds an SE attention mechanism module to the backbone network structure of YOLOv5 based on the YOLOv5 model structure. Among them, the SE attention mechanism module mainly improves the model performance by compressing the feature map after local transformation of the convolution through Squeeze and excitation. In the compression process, the input feature map (H×W×C) is compressed into a vector through global average pooling, and the entire spatial feature on a channel is encoded as a global feature, and then mapped to a 1×1×C vector through a fully connected layer to achieve the compression function. In order to learn the relationship between each channel, the backbone network of YOLOv5 uses a sigmoid activation function to compress each element in this vector to between 0 and 1 during the excitation process, and multiplies it with the original input feature map to obtain a weighted feature map.

[0054]

[0055] s=F excitation (F squeeze (u c ),W)=σ(g(F squeeze (u c ),W))=σ(W2RELU(W1F squeeze (u c )))

[0056] F(u c ,s c )=u c ×s c

[0057] Among them, F squeeze (u c ) is the feature output after compression, H is the image height, W is the image width, C is the number of channels, u c is the original feature input. s is the feature output after excitation, σ() is the sigmoid activation function, W1 and W2 are the input shared weights, respectively. is a sigmoid gating mechanism, F(u c ,s c ) is the final feature output.

[0058] After the model structure is determined, the processor is put into model training according to the reasonable learning rate, batch processing size, number of iterations, activation function and loss function. The present invention combines the center loss function with the Focal-loss loss function to jointly supervise the network training, and jointly supervise the model training to find the optimal parameters. The smaller the loss function, the better the model performance. The model effect of using a loss function alone is poor, so the algorithm compares the effects and effects of multiple loss functions, selects multiple suitable loss functions, assigns weights to them, and jointly measures the quality of the model. Focal Loss is a loss function used to solve the problem of class imbalance. It adjusts the weights of positive and negative samples so that samples that are difficult to classify receive greater attention during the training process. The center point loss is used to predict the center point position of the target box, helping the model to more accurately regress the position of the target box, thereby improving the accuracy of target detection. The present invention combines the center loss function with the Focal-loss loss function to jointly supervise the network training, and jointly supervise the model training to find the optimal parameters.

[0059] The following is the definition of the center loss function:

[0060]

[0061] Among them, L b is the center loss, x i is the category feature, b yi is the feature center of the i-th category, and k represents the batch processing size. The Focal Loss function is defined as:

[0062]

[0063]

[0064]

[0065] Among them, L pt is the Focal Loss loss function, y is the label value, the positive sample label value is 1, and the negative sample label value is 0. To balance the weights of positive and negative samples, the value range is [0, 1]; P t It represents the probability (i.e. confidence) of a certain category predicted by the model.

[0066] The loss function of the entire network is:

[0067] L=L b +δL pt

[0068] Among them, δ is the balance scalar of the two loss functions. The scalar value is continuously optimized during the retraining process to minimize L.

[0069] The processor calculates the confidence of the new feature map through the slope crack recognition model; the lower the confidence, the lower the overlap with the features in the trained model, and there is no abnormality; the higher the confidence, the higher the overlap with the features in the trained model; the processor sets the confidence to be greater than 0.7 to determine that the slope crack exists, and when it is less than 0.7, it determines that there is no abnormality and there is no slope crack. Through multiple slope crack recognition result pictures, the processor can identify the location and changes of the slope cracks.

[0070] The processor calculates the confidence of the new feature map through the slope crack recognition model; the lower the confidence, the lower the overlap with the features in the trained model, and there is no abnormality; the higher the confidence, the higher the overlap with the features in the trained model; the processor determines that when the confidence is greater than 0.7, there is a slope crack, and when it is less than 0.7, it is considered that there is no abnormality and there is no slope crack. The processor can identify the location and changes of the cracks through multiple slope crack recognition result pictures.

[0071] Specifically, before implementing step S20, the slope crack monitoring method may further include the following specific implementation steps:

[0072] S51, obtaining a training set, a validation set, and a test set of the annotated slope crack image data;

[0073] S52, iteratively train the slope crack recognition model based on the training set, monitor the performance of the slope crack recognition model based on the validation set, supervise the training of the slope crack recognition model based on the center loss function and the focal-loss loss function, test the recognition accuracy of the monitored slope crack recognition model based on the test set, and obtain the monitored slope crack recognition model with optimal parameters.

[0074] For example, the model user annotates the slope crack data using the labeling tool labelme to prevent the model from learning redundant and irrelevant features that affect the recognition results. The model user configures the number of positive and negative samples to 10:1, sets the labels of these samples without targets to empty, and saves the output format in txt format.

[0075] S30, obtaining slope crack monitoring information based on the slope crack recognition image and a pre-acquired three-dimensional model of the slope crack area.

[0076] Specifically, step S30 may include the following specific implementation steps:

[0077] S31, determining the slope crack area according to the acquired three-dimensional model of the slope crack area;

[0078] S32, obtaining a slope crack recognition image with a confidence level exceeding a preset value;

[0079] S33, determining the slope crack area after three-dimensional reconstruction based on the slope crack recognition image and the slope crack area exceeding the preset value;

[0080] S34. Determine the slope crack monitoring information based on the three-dimensionally reconstructed slope crack area.

[0081] When the confidence level is greater than 0.7, the processor determines that the recognition result is valid. The processor uses the 3D modeling software to build a 3D model of the slope crack area to determine the slope crack area. Then, based on the image data obtained by the imported drone, the processor selects the aerial triangulation calculation parameters and the local coordinate system to complete the 3D reconstruction. After the 3D reconstruction is completed, the processor obtains the point coordinate information (longitude and latitude of the slope crack) and the length and width of the slope crack according to the software function.

[0082] After implementing step S34, the slope crack monitoring method may further include the following specific implementation steps:

[0083] S35. Outputting an alarm record based on the slope crack monitoring information.

[0084] Specifically, first, the processor names the slope, then divides the slope into multiple areas, and determines the area where the crack is located based on the recognition result image. There may be multiple cracks in an area. The processor numbers the cracks. When the processor enters the alarm information, it enters the alarm information according to the slope-area-crack number, forming a one-to-one correspondence, that is, the alarm record is associated with the slope name, area and crack number.

[0085] When the recognition result is valid, the processor will transmit the data back to the background to form an alarm record. The alarm record mainly includes: aerial survey time, alarm time, target longitude and latitude coordinates, target area, whether there is an abnormality, crack length, crack width, warning level, processing and approval opinions, and recognition result pictures. Among them, the processor determines the specific longitude and latitude information of the slope crack according to the target longitude and latitude, which is convenient for the later staff to locate the slope crack; the processor determines whether there is an abnormality, that is, there are two output results, 0 represents normal, that is, the confidence is greater than 0.7, and there is no slope crack; 1 represents abnormality, that is, the confidence is less than 0.7, and there is a slope crack; the processor determines the alarm time as the corresponding alarm time when the model detects a slope crack; the warning level is defined according to the length and width of the slope crack, and the warning level is divided into three types: low, medium, and high. The order of the level is proportional to the length and width of the slope crack. The longer and wider the slope crack, the higher the warning level; the recognition result picture is convenient for the staff to intuitively view the slope crack. After the alarm record is generated, an alarm prompt pop-up window will be formed, and the early warning system has a built-in voice broadcast function. When an alarm record is generated, a voice prompt will be given to inform the alarm event so that the staff can take corresponding solutions in time.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] The present invention utilizes drone aerial photography technology to perform non-contact photography, and data collection is convenient, safe, and efficient.

[0088] Compared with the traditional method, the slope crack recognition method trained by UAV aerial photography technology combined with the improved YOLOv5 model can better handle the problems of image illumination change and color restoration by combining information of different scales through the multi-scale Retinex image processing algorithm, thereby improving the visual effect of the image. The training parameters and network architecture of the slope crack recognition model are debugged based on the optimized image data, so that the slope crack recognition model has better robustness and accuracy. The Focal-loss loss function and the center function are combined in the training process. Compared with the only loss function, the results are more accurate and uploading the recognition results to the early warning system is more reliable.

[0089] By monitoring the cracks in the slopes with drones, the formation and expansion of cracks can be detected early, and geological disasters such as landslides and collapses can be warned in advance, so that corresponding safety measures can be taken to protect the safety of personnel and property. By monitoring the changes in the cracks in the slopes with drones, the stability of the slopes can be evaluated, the deformation and displacement of the slopes can be understood, and engineers and geological experts can be helped to judge the stability of the slopes and take corresponding improvement and repair measures. Slope instability may lead to soil erosion, soil erosion and environmental damage. By monitoring the cracks in the slopes, drones combined with the slope crack identification model can detect slope problems in a timely manner, take corresponding repair and protection measures, help protect the ecological environment, and promote the ecological recovery of the slopes. Therefore, the social benefits obtained by the open-pit mine slope crack monitoring system and method are outstanding and worthy of promotion.

[0090] Based on the above embodiments, an embodiment of the present invention further provides a slope crack monitoring device 100. Figure 3 This is a schematic diagram of the structure of the slope crack monitoring device provided in the embodiment of the present application, refer to Figure 3 The slope crack monitoring device 100 may include an acquisition module 101, a processing module 102 and an output module 103, wherein the acquisition module 101 is used to acquire slope crack image data; the processing module 102 is used to process the slope crack image data using a slope crack recognition model with optimal parameters to obtain a slope crack recognition image with a confidence level exceeding a preset value, wherein the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module added, a Neck feature enhancement network with a feature pyramid network and a path aggregation network added, and a YOLOv5 model with a head network having three 1x1 convolutional layers; the output module 103 is used to obtain slope crack monitoring information based on the slope crack recognition image and a pre-acquired three-dimensional model of the slope crack area.

[0091] Based on the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium including instructions, which, when executed on a computer, enables the computer to execute a slope crack monitoring method.

[0092] Based on the above embodiment, an embodiment of the present invention further provides a monitoring device, the monitoring device comprising:

[0093] at least one processor, memory, and input-output unit;

[0094] The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the slope crack monitoring method.

[0095] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A slope crack monitoring method, characterized in that: include: Acquire slope crack image data; Processing the slope crack image data using a slope crack recognition model with optimal parameters to obtain a slope crack recognition image with a confidence level exceeding a preset value, wherein the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module, a Neck feature enhancement network with a feature pyramid network and a path aggregation network, and a YOLOv5 model with a head network having three 1x1 convolutional layers; Slope crack monitoring information is obtained based on the slope crack identification image and a pre-acquired three-dimensional model of the slope crack area.

2. The slope crack monitoring method according to claim 1, characterized in that: Before acquiring the slope crack image data, the slope crack monitoring method further includes: Obtain slope crack image acquisition data Processing the slope crack image acquisition data based on a target detection algorithm to obtain slope crack image data; The target detection algorithm is obtained by replacing the Mosaic image enhancement algorithm in the YOLOv5 model according to the multi-scale Retinex image processing algorithm.

3. The slope crack monitoring method according to claim 1, characterized in that: Before processing the slope crack image data using the slope crack recognition model with optimal parameters, the slope crack monitoring method further includes: Obtain training sets, validation sets and test sets of annotated slope crack image data; The slope crack recognition model is iteratively trained based on the training set, the performance of the slope crack recognition model is monitored based on the validation set, the slope crack recognition model is supervised and trained based on the center loss function and the focal-loss loss function, and the recognition accuracy of the monitored slope crack recognition model is tested based on the test set to obtain a monitored slope crack recognition model with optimal parameters.

4. The slope crack monitoring method according to claim 1, characterized in that: The slope crack recognition model is determined based on the backbone network with SE attention mechanism module, the Neck feature enhancement network with feature pyramid network and path aggregation network, and the YOLOv5 model with head network with 3 1x1 convolution layers, including: Constructing a backbone network according to the first convolution module, the second convolution module, the first composite convolution structure module, the third convolution module, the second composite convolution structure module, the fourth convolution module, the third composite convolution structure module, the fifth convolution module, the SE attention mechanism module, and the SPPF module connected in sequence; The slope crack recognition model is determined according to the backbone network, the Neck feature enhancement network with the feature pyramid network and the path aggregation network added, and the YOLOv5 model of the head network with three 1x1 convolutional layers.

5. The slope crack monitoring method according to claim 1, characterized in that: The step of obtaining slope crack monitoring information based on the slope crack identification image and a pre-acquired three-dimensional model of the slope crack area includes: Determine the slope crack area according to the acquired three-dimensional model of the slope crack area; Acquire a slope crack identification image with a confidence level exceeding a preset value; Determine the slope crack area after three-dimensional reconstruction based on the slope crack identification image and the slope crack area exceeding the preset value; The slope crack monitoring information is determined based on the three-dimensionally reconstructed slope crack area.

6. The slope crack monitoring method according to claim 5, characterized in that: After determining the slope crack monitoring information based on the three-dimensionally reconstructed slope crack area, the slope crack monitoring method further includes: Output an alarm record based on the slope crack monitoring information.

7. The slope crack monitoring method according to claim 2, characterized in that: The multi-scale Retinex image processing algorithm includes: Decompose the slope crack image acquisition data into the product of the illumination component of the ambient light and the reflection component of the target object carrying the image detail information, and obtain the original image after multi-scale decomposition; Performing Gaussian blur processing on the original image after multi-scale decomposition to obtain a blurred image; The components at each scale are combined to obtain the enhanced image.

8. A slope crack monitoring device, characterized in that: include: An acquisition module, used for acquiring slope crack image data; A processing module is used to process the slope crack image data using a slope crack recognition model with optimal parameters to obtain a slope crack recognition image with a confidence level exceeding a preset value, wherein the slope crack recognition model is determined based on a backbone network with an SE attention mechanism module, a Neck feature enhancement network with a feature pyramid network and a path aggregation network, and a YOLOv5 model with a head network having three 1x1 convolutional layers; The output module is used to obtain slope crack monitoring information based on the slope crack identification image and the pre-acquired three-dimensional model of the slope crack area.

9. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the slope crack monitoring method according to any one of claims 1 to 7.

10. A monitoring device, characterized in that: The monitoring equipment includes: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the slope crack monitoring method according to any one of claims 1 to 7.

Images