Dynamic mountain dangerous rock detection method and device based on machine vision, and electronic equipment

Abstract

The embodiments of the present application provide a method and device for dynamically detecting dangerous mountain rocks based on machine vision, and an electronic device. The method includes: determining a mountain detection area, and obtaining an image of the surface of the dangerous mountain rocks by photographing the mountain detection area; using an unmanned aerial vehicle to strike the mountain detection area; photographing the strike image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks that appear in the mountain detection area and the craters formed by the strike in the strike image; by comparing the surface image of the dangerous mountain rocks and the strike image, determining whether there is a risk of detachment of the dangerous mountain rocks in the mountain detection area. Based on this, the embodiments of the present application replace manual detection with an unmanned aerial vehicle, improve the detection efficiency of dangerous mountain rocks, reduce the risk coefficient during manual detection, and can improve the investigation efficiency of dangerous mountain rocks, reduce the danger of operators, and improve the accuracy of dangerous rock detection by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the dangerous mountain rocks.

Description

Technical Field

[0001] The present application relates to the technical field of mountain exploration, and particularly to a dynamic mountain dangerous rock detection method and device based on machine vision, and an electronic device. Background Art

[0002] The falling of mountain dangerous rocks will bring unexpected dangers in real life, especially having important impacts on aspects such as the operation and construction of mountain roads, mineral development, and mountain areas of scenic spots. The time of falling of mountain dangerous rocks is random and uncertain, and at the same time, the volume of the fallen mountain rocks cannot be determined. The detection of mountain dangerous rocks is an important part of safety operation detection. At present, most of the existing mountain dangerous rock detection technologies detect dangerous rocks manually, and remote sensing technology is also used to detect large-scale mountain landslides and mountain changes. However, remote sensing technology can only detect large-scale mountain movements, and can only obtain the transformation trend of the mountain, and cannot detect whether there is a risk of falling of mountain dangerous rocks. Manually detecting mountain dangerous rocks requires manually knocking on the mountain to achieve the purpose of making the mountain dangerous rocks fall off in advance. Manually detecting mountains requires a large amount of manpower and material resources. At the same time, the fallen dangerous rocks during manual detection of the mountain will pose a physical danger to the detector. If the fallen mountain dangerous rocks are large in volume, it will be life-threatening to the detection personnel. Moreover, in the face of extreme weather, it is impossible to detect the mountain manually. At the same time, as the operation time increases, the efficiency of manual detection will also decrease. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a dynamic mountain dangerous rock detection method and device based on machine vision, and an electronic device, which can improve the efficiency of mountain dangerous rock investigation, reduce the danger of operators, and improve the accuracy of dangerous rock detection.

[0004] To achieve the above object, the first aspect of the embodiments of the present application proposes a dynamic mountain dangerous rock detection method based on machine vision, and the method includes:

[0005] Determine the mountain detection area, and obtain the surface image of the mountain dangerous rock by photographing the mountain detection area;

[0006] Use an unmanned aerial vehicle to strike the mountain detection area;

[0007] Take the strike image after the unmanned aerial vehicle strikes the mountain, and mark the cracks and craters formed by the strike that appear in the mountain detection area in the strike image;

[0008] By comparing the surface image of the mountain dangerous rock and the strike image, determine whether there is a risk of falling of the mountain dangerous rock in the mountain detection area.

[0009] In some embodiments, after determining the mountain body detection area and obtaining the surface image of the mountain body dangerous rock by photographing the mountain body detection area, the following steps are included:

[0010] Preprocess the surface image of the mountain body dangerous rock, and the preprocessing includes denoising the surface image of the mountain body dangerous rock by using a Gaussian filtering algorithm.

[0011] In some embodiments, the use of the unmanned aerial vehicle to strike the mountain body detection area includes:

[0012] Use the unmanned aerial vehicle to strike the mountain body detection area multiple times by using the pentagon strike method to obtain a strike result;

[0013] Determine the detection features of the mountain body dangerous rock in the mountain body detection area based on the strike result.

[0014] In some embodiments, the use of the unmanned aerial vehicle to strike the mountain body detection area multiple times by using the pentagon strike method includes:

[0015] Take off the unmanned aerial vehicle, perform regional segmentation on the mountain body detection area, and segment it into multiple unit areas with equal area sizes, where the shape of the unit area is a rectangle;

[0016] Mark the four vertices and the center point of the unit area to determine the projectile positions where the unmanned aerial vehicle needs to launch projectiles at the unit area;

[0017] Perform projectile shooting on the marked projectile positions respectively, take a photograph each time after a strike, until all the marked unit areas are struck.

[0018] In some embodiments, the detection features of the mountain body dangerous rock include at least one of the following:

[0019] The size of the bullet marks left in the mountain body detection area after the unmanned aerial vehicle strikes the mountain body;

[0020] The length and width of the cracks that appear in the mountain body detection area after the unmanned aerial vehicle strikes the mountain body;

[0021] The size of the area where mountain body rocks fall off and the amount of mountain body rocks that fall off in the mountain body detection area after the unmanned aerial vehicle strikes the mountain body.

[0022] In some embodiments, the determination of whether there is a risk of falling off of the mountain body dangerous rock in the mountain body detection area by comparing the surface image of the mountain body dangerous rock and the strike image includes:

[0023] Divide the surface image of the mountain body dangerous rock and the strike image into two sets;

[0024] Detect the image changes before and after the impact of each of the mountain detection areas according to two sets;

[0025] Determine the detection features of mountain dangerous rocks according to the image changes;

[0026] Input the detection features of mountain dangerous rocks into a pre-trained ResNet network for dangerous rock detection to determine whether there is a risk of detachment of mountain dangerous rocks in the mountain detection area.

[0027] In some embodiments, the training method of the ResNet network includes:

[0028] Create a pre-training data set, which consists of feature pictures in the attention mechanism. The feature pictures include bullet mark pictures, crack pictures after impact, and rock detachment area pictures;

[0029] Annotate the feature images to determine the range values between normal and abnormal conditions of dangerous rock detachment;

[0030] Train the pre-constructed ResNet network according to the annotated feature images and the range values.

[0031] To achieve the above object, a second aspect of the embodiments of the present application proposes a dynamic mountain dangerous rock detection device based on machine vision. The device includes:

[0032] An acquisition module for determining a mountain detection area and acquiring a surface image of a mountain dangerous rock by photographing the mountain detection area;

[0033] An impact module for using an unmanned aerial vehicle to impact the mountain detection area;

[0034] A marking module for photographing the impact image after the unmanned aerial vehicle impacts the mountain to mark the cracks and impact craters that appear in the mountain detection area in the impact image;

[0035] A judgment module for judging whether there is a risk of detachment of mountain dangerous rocks in the mountain detection area by comparing the surface image of the mountain dangerous rock and the impact image.

[0036] To achieve the above object, a third aspect of the embodiments of the present application proposes an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect above is implemented.

[0037] To achieve the above object, a fourth aspect of the embodiments of the present application proposes a computer-readable storage medium storing a computer program, which when executed by a processor implements the method described in the first aspect above.

[0038] The method, device and electronic device for dynamic mountain dangerous rock detection based on machine vision proposed in the present application determine a mountain detection area, obtain a surface image of the mountain dangerous rock by photographing the mountain detection area; use an unmanned aerial vehicle to strike the mountain detection area; photograph the strike image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks appearing in the mountain detection area and the craters formed by the strike in the strike image; by comparing the surface image of the mountain dangerous rock and the strike image, it is judged whether there is a risk of detachment of the mountain dangerous rock in the mountain detection area. Based on this, compared with the existing manual detection method, the embodiment of the present application replaces manual detection with an unmanned aerial vehicle, improves the detection efficiency of mountain dangerous rocks, reduces the risk coefficient during manual detection, and can improve the detection efficiency of mountain dangerous rocks by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the mountain dangerous rock, reduce the danger of operators, and improve the accuracy of dangerous rock detection. By comparing the images before and after the unmanned aerial vehicle strikes the mountain dangerous rock area, the detection of mountain dangerous rocks based on machine vision is realized, so that the mountain dangerous rocks with the risk of detachment can be effectively detected and investigated. Description of the Drawings

[0039] Figure 1 is a flowchart of the method for dynamic mountain dangerous rock detection based on machine vision provided by the embodiment of the present application;

[0040] Figure 2 is Figure 1 a flowchart of step S102 in

[0041] Figure 3 is Figure 2 a flowchart of step S201 in

[0042] Figure 4 is Figure 1 a flowchart of step S104 in

[0043] Figure 5 is a flowchart of the training method of the ResNet network provided by the embodiment of the present application;

[0044] Figure 6 is a schematic structural diagram of the device for dynamic mountain dangerous rock detection based on machine vision provided by the embodiment of the present application;

[0045] Figure 7 is a schematic hardware structure diagram of the electronic device provided by the embodiment of the present application. Detailed Embodiments

[0046] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different module division in the device or a different order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0048] Aiming at the technical problems in the prior art that manual detection of dangerous rocks has a high risk factor, long operation time and low detection efficiency, the embodiments of the present application provide a method and device for dynamically detecting dangerous rocks on mountain bodies based on machine vision, and an electronic device. The detection area of the mountain body is determined, and the surface image of the dangerous rocks on the mountain body is obtained by photographing the detection area of the mountain body; the detection area of the mountain body is struck by an unmanned aerial vehicle; the strike image after the unmanned aerial vehicle strikes the mountain body is photographed to mark the cracks and the craters formed by the strike that appear in the detection area of the mountain body in the strike image; by comparing the surface image of the dangerous rocks on the mountain body and the strike image, it is judged whether the dangerous rocks on the mountain body in the detection area have the risk of falling off. Based on this, compared with the existing manual detection method, the embodiments of the present application replace manual detection with an unmanned aerial vehicle, improve the detection efficiency of dangerous rocks on the mountain body, reduce the risk factor during manual detection, and can improve the inspection efficiency of dangerous rocks on the mountain body and reduce the danger of operators by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the dangerous rocks on the mountain body, and improve the accuracy of dangerous rock detection. By comparing the images before and after the unmanned aerial vehicle strikes the dangerous rock area on the mountain body, the detection of dangerous rocks on the mountain body based on machine vision is realized, so that the dangerous rocks on the mountain body with the risk of falling off can be effectively detected and inspected.

[0049] The method and device for dynamically detecting dangerous rocks on mountain bodies based on machine vision and the electronic device provided by the embodiments of the present application are specifically described through the following embodiments. First, the method for dynamically detecting dangerous rocks on mountain bodies based on machine vision in the embodiments of the present application is described.

[0050] Figure 1 is an optional flowchart of the method for dynamically detecting dangerous rocks on mountain bodies based on machine vision provided by the embodiments of the present application, Figure 1 and the method in may include but is not limited to steps S101 to S104.

[0051] Step S101, determine the detection area of the mountain body, and obtain the surface image of the dangerous rocks on the mountain body by photographing the detection area of the mountain body;

[0052] Step S102, use an unmanned aerial vehicle to strike the mountain detection area;

[0053] Step S103, capture the strike image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks that appear in the mountain detection area and the craters formed by the strike in the strike image;

[0054] Step S104, by comparing the surface image of the mountain dangerous rock and the strike image, determine whether there is a risk of detachment of the mountain dangerous rock in the mountain detection area.

[0055] In some embodiments, determine the mountain detection area, and obtain the surface image of the mountain dangerous rock by photographing the mountain detection area; use an unmanned aerial vehicle to strike the mountain detection area; capture the strike image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks that appear in the mountain detection area and the craters formed by the strike in the strike image; by comparing the surface image of the mountain dangerous rock and the strike image, determine whether there is a risk of detachment of the mountain dangerous rock in the mountain detection area. Based on this, compared with the existing manual detection method, the embodiment of the present application replaces manual detection with an unmanned aerial vehicle, improves the detection efficiency of mountain dangerous rocks, reduces the risk coefficient during manual detection, and can improve the detection efficiency of mountain dangerous rocks by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the mountain dangerous rock, reduce the danger of operating personnel, and improve the accuracy of dangerous rock detection.

[0056] In some embodiments, photograph the surface image of the mountain, and determine the detection range to be detected. Determine the types of mountain dangerous rocks to be detected. Different types of dangerous rocks have different hardnesses, and different types of projectiles are required at the same time. Therefore, it is very important to determine the types of mountain dangerous rocks. Take off the unmanned aerial vehicle, perform regional segmentation on the range to be detected, divide it into different unit areas, and the area size of each area is equal. Perform a pentagonal marking on the segmented areas to determine the five positions where projectiles need to be launched in each area. Shoot projectiles at the marked projectile positions respectively, take a picture each time after a strike, until all the marked areas are struck. Compare the obtained images to see whether there is a risk of detachment of the mountain dangerous rock in this area. By comparing the images before and after the unmanned aerial vehicle strikes the mountain dangerous rock area, the detection of mountain dangerous rocks based on machine vision is realized, so as to effectively detect and investigate the mountain dangerous rocks with the risk of detachment.

[0057] In some embodiments, photograph the detection area to be detected, collect the image features of the area to be detected. After obtaining the collected image, first perform image preprocessing on the image, and use the Gaussian filtering algorithm to achieve the purpose of denoising the image.

[0058] In some embodiments, since the acquired image range is large, it is necessary to delimit the detection range. By delimiting the detection range, the actual position where dangerous rocks are likely to occur can be more accurately located.

[0059] In some embodiments, since the method adopted in the embodiments of the present application is to strike with an unmanned aerial vehicle, it is necessary to determine the hardness of the projectiles to be launched. Different types of projectiles have different hardnesses and different striking effects. In the embodiments of the present application, the hardness of the mountain rock is determined in advance to determine the required projectiles. This method can improve the striking effect of the projectiles and the detection accuracy of the detection.

[0060] In some embodiments, due to the need for the striking effects of projectiles at different positions, the area to be detected in the embodiments of the present application is divided. To ensure the accuracy of the detection effect, when dividing the area in the embodiments of the present application, it is ensured that the area of each region is equal, and the shape of the divided region is a rectangle.

[0061] In some embodiments, making pentagonal marks on the segmented detection area can ensure the effectiveness of the strike by the unmanned aerial vehicle, and at the same time can ensure that the unmanned aerial vehicle can strike the designated position. The pentagonal marking method proposed in the embodiments of the present application refers to marking the four vertices and the center point of the segmented area. Through the pentagonal marking method, it can be ensured that obvious deformation occurs to the dangerous rocks in the area after the projectiles strike the area, thereby ensuring that dangerous rocks with a risk of falling off can be detected.

[0062] Please refer to Figure 2 , in some embodiments, step S102 may include but is not limited to steps S201 to S205:

[0063] Step S201, using the unmanned aerial vehicle to strike the mountain detection area multiple times with the pentagonal strike method to obtain the strike results;

[0064] Step S202, determining the detection features of the mountain dangerous rocks in the mountain detection area based on the strike results.

[0065] In some embodiments, the embodiments of the present application use the pentagonal strike method to detect the risk of mountain dangerous rocks falling off. The pentagonal strike method can more efficiently determine whether there is a risk of dangerous rocks falling off in the area, and at the same time improves the detection efficiency of dangerous rocks. Specifically, when implementing, the unmanned aerial vehicle takes off, and the area to be detected is segmented into different unit areas, and the area of each area is equal. Pentagonal marks are made on the segmented areas to determine five positions where projectiles need to be launched in each area. Projectiles are shot at the marked projectile positions respectively, and one shot is taken each time until all the marked areas are struck. Based on this, the pentagonal strike method can detect the risk of mountain dangerous rocks falling off to the greatest extent, and effectively solves the problem of reduced detection accuracy caused by uneven strikes.

[0066] In some embodiments, making a pentagonal mark on the segmented detection area can ensure the effectiveness of the UAV strike and also ensure that the UAV can strike the designated position. The pentagonal marking method proposed in the embodiments of the present application refers to marking the four vertices and the center point of the segmented area. By using the pentagonal marking method, it can be ensured that the dangerous rocks in the area undergo obvious deformation after the projectile strikes the area, thereby ensuring that dangerous rocks with a risk of falling off can be detected.

[0067] In some embodiments, for the detection features of mountain dangerous rocks, the embodiments of the present application use a UAV for strike operations, which can increase the detection features of mountain dangerous rocks. Specifically, after the UAV strikes the mountain, bullet marks will be left, and the size of the bullet marks can become a detection feature of the mountain dangerous rocks; after the UAV strikes the mountain, cracks will appear in the strike area, and the length and width of the cracks can also become detection features of the dangerous rocks; after the UAV strikes the mountain, whether there will be detachment of mountain rocks, the size of the detachment area and the amount of detachment of mountain rocks can also become detection features of the mountain dangerous rocks. The amount of detachment of mountain rocks can be calculated based on the density and volume relationship after the UAV strike to obtain the amount of detachment of mountain rocks.

[0068] Please refer to Figure 3 , in some embodiments, step S201 may include but is not limited to steps S301 to S303:

[0069] Step S301, take off the UAV and segment the mountain detection area into multiple unit areas with equal areas, where the shape of the unit area is a rectangle;

[0070] Step S302, mark the four vertices and the center point of the unit area to determine the projectile positions where the UAV needs to launch projectiles at the unit area;

[0071] Step S303, perform projectile shooting at the marked projectile positions respectively, take a picture each time after a strike, until all the marked unit areas are struck.

[0072] In some embodiments, take off the unmanned aerial vehicle, perform regional segmentation on the mountain detection area, divide it into different unit areas with equal area sizes for each unit area, mark the segmented unit areas, and determine five positions where projectiles need to be launched in each area. Launch projectiles at the marked locations for strikes. When the unmanned aerial vehicle launches projectiles in each detected area, it needs to launch them in sequence. First, strike the central position, and then strike the four vertices of the rectangular detection from left to right in turn. When the unmanned aerial vehicle launches projectiles for strikes, it will take a picture of the detection area every time it launches a projectile. Each divided detection area is struck by projectiles five times, and at the same time, the aircraft takes pictures five times to obtain five images. By using the pentagonal strike method to detect the risk of mountain rockfall, it is possible to more accurately determine whether there is a risk of mountain rockfall in this area. The pentagonal strike method can more efficiently determine whether there is a risk of rockfall in this area, and at the same time, it also improves the detection efficiency of rockfall detection.

[0073] Please refer to Figure 4 , in some embodiments, step S104 may include but is not limited to steps S401 to S404:

[0074] Step S401, divide the mountain rockfall surface image and the strike image into two sets;

[0075] Step S402, detect the image changes before and after the strike in each mountain detection area according to the two sets;

[0076] Step S403, determine the detection features of the mountain rockfall according to the image changes;

[0077] Step S404, input the detection features of the mountain rockfall into a pre-trained ResNet network for rockfall detection to determine whether there is a risk of rockfall in the mountain rockfall in the mountain detection area.

[0078] In some embodiments, count the collected mountain images, number the images of each corner strike, and divide them into two sets before and after the strike. After collecting the data images, use an image detection algorithm to perform image detection on the collected images. Since the embodiments of the present application use a regional segmentation method to segment the captured mountain images, it is necessary to detect the image changes before and after the strike in each area.

[0079] In some embodiments, in order to better detect the risk of dangerous rock detachment on a mountain, the embodiments of the present application propose a ResNet mountain dangerous rock detachment detection algorithm integrating an attention mechanism. The core of this algorithm is to detect the risk of dangerous rock detachment by inputting the detachment characteristics of mountain dangerous rocks. In the embodiments of the present application, the characteristics of mountain dangerous rock detachment are input into the attention mechanism in the ResNet network, and the input characteristics are used to assist the ResNet network in detecting dangerous rocks.

[0080] In some embodiments, first, a ResNet network is constructed. The ResNet network is a deep neural network, which can avoid the phenomenon of training degradation caused by setting too many training network layers. The ResNet network does not directly fit the network mapping of each layer, but realizes cross-layer connection of the network through the way of skip-connections. On the basis of the above connection method, the ResNet network solves the training degradation phenomenon that occurs when building a multi-layer network in a deep neural network by fitting the data residuals. Since there are many characteristics of mountain dangerous rocks input in the embodiments of the present application, more training layers need to be set to achieve the effect of dangerous rock detection.

[0081] In some embodiments, due to the characteristics of mountain dangerous rocks, the common detection methods include detecting whether there are cracks in the mountain and whether landslides occur on the mountain. The embodiments of the present application use an unmanned aerial vehicle for striking operations, which can increase the detection characteristics of mountain dangerous rocks. The specific manifestations are as follows: bullet marks will be left after the unmanned aerial vehicle strikes the mountain, and the size of the bullet marks can become a detection characteristic of mountain dangerous rocks; after the unmanned aerial vehicle strikes the mountain, cracks will appear in the striking area, and the length and width of the cracks can also become detection characteristics of dangerous rocks; after the unmanned aerial vehicle strikes the mountain, whether there will be detachment of mountain rocks, the size of the detachment area and the amount of detached mountain rocks can also become detection characteristics of mountain dangerous rocks. The amount of detached mountain rocks can be calculated through the density and volume relationship after the unmanned aerial vehicle strikes to obtain the amount of detached mountain rocks.

[0082] In some embodiments, in order to more accurately determine whether there is a risk of detachment of dangerous rocks in this area, the embodiments of the present application use the attention mechanism to make the ResNet network pay more attention to the detachment characteristics of mountain dangerous rocks when detecting the target, thereby improving the detection of the risk of dangerous rock detachment. The attention mechanism input in the embodiments of the present application mainly includes: bullet mark size, crack length and width after striking, rock detachment area, rock detachment amount, etc.

[0083] The embodiment of this application abandons the existing manual detection scheme, replaces manual labor with an unmanned aerial vehicle, improves the detection efficiency, reduces the risk coefficient during manual detection, and at the same time combines the pentagonal strike method mentioned in the present invention to strike the area to be detected. After the strike, the detection algorithm proposed by the embodiment of this application is used to detect whether there is a risk of mountain rockfall.

[0084] Please refer to Figure 5 , in some embodiments, the training method of the ResNet network may include but is not limited to steps S501 to S503:

[0085] Step S501, create a pre-training data set, which is composed of feature pictures in the attention mechanism. The feature pictures include bullet mark pictures, crack pictures after impact, and rock detachment area pictures;

[0086] Step S502, label the feature images to determine the range values between normal and abnormal rockfall;

[0087] Step S503, train the pre-constructed ResNet network according to the labeled feature images and range values.

[0088] In some embodiments, after constructing the above ResNet network integrating the attention mechanism, before performing rockfall detection, it is necessary to create a pre-training data set. The pre-training data set of the embodiment of this application is composed of feature pictures in the attention mechanism, mainly including bullet mark pictures, crack pictures after impact, and rock detachment area pictures. Each type of feature image will be pre-labeled to determine the range values between normal and abnormal, including the bullet mark range under normal conditions, the length and width of cracks under normal conditions, and the size of the rock detachment area under normal conditions, etc.

[0089] In some embodiments, after the data is labeled, the ResNet network constructed by the embodiment of this application can be trained. After the training is completed, it is necessary to detect the accuracy of the trained ResNet network to ensure that the ResNet network integrating the attention mechanism proposed by the embodiment of this application can achieve the detection accuracy. If the detection accuracy cannot be achieved, the network needs to be retrained until the detection accuracy can be achieved. After the network is detected, the unmanned aerial vehicle can take off to detect the mountain rockfall of the mountain to be detected.

[0090] Based on this, this method is applied to the detection of mountain body status, and analyzes whether there is a risk of falling of mountain dangerous rocks by analyzing the surface map of the mountain body captured by the camera. Since the existing detection method for mountain dangerous rocks is periodic manual inspection and detection, this method is greatly affected by the work experience and subjective factors of the detection personnel, has a certain danger to the personal safety of the inspection personnel, and has a strong dependence on the weather. The labor intensity of workers increases, and the efficiency is low, the accuracy is not high, and it is dangerous, greatly increasing the detection cost. The embodiment of this application can solve the above problems through machine vision technology. The embodiment of this application also proposes a hitting detection method for the above problems. First, determine the dangerous rock area to be detected, use an unmanned aerial vehicle to launch a projectile to hit the dangerous rock, and then judge the falling situation of the dangerous rocks in this area by comparing the photos before and after the hitting. Since the hitting area of the projectile launched by the unmanned aerial vehicle is limited, the attachment state of the dangerous rocks in this area can be determined by multiple hits. The existing detection means is static detection. The embodiment of this application uses the hitting effect of the unmanned aerial vehicle combined with machine vision detection to construct a dynamic detection means, and judges the attachment state of the dangerous rocks by obtaining the difference in image features before and after the hitting. The present invention can improve the efficiency of mountain dangerous rock inspection, reduce the danger of operating personnel, and improve the accuracy of dangerous rock detection. For the above problems, the embodiment of this application proposes a mountain dangerous rock detection technology using an unmanned aerial vehicle combined with machine vision technology, which can solve the above problems and realize the function of detecting and warning the falling of mountain dangerous rocks. In the aspect of image detection, in order to highlight the different features before and after the mountain body is hit, the embodiment of this application proposes a ResNet mountain dangerous rock falling detection algorithm that integrates the attention mechanism. Integrating the attention mechanism can more accurately detect the changes after the mountain body is hit, and the ResNet network can improve the detection efficiency of the algorithm and reduce the problem of training degradation caused by too much data.

[0091] Please refer to Figure 6 , the embodiment of this application also provides a dynamic mountain dangerous rock detection device based on machine vision, which can implement the above-mentioned dynamic mountain dangerous rock detection method based on machine vision. The device includes:

[0092] An acquisition module 610, configured to determine a mountain body detection area, and acquire a surface image of mountain dangerous rocks by photographing the mountain body detection area;

[0093] A striking module 620, configured to use an unmanned aerial vehicle to strike the mountain body detection area;

[0094] A marking module 630, configured to photograph the striking image after the unmanned aerial vehicle strikes the mountain body, and mark the cracks and craters formed by the strike that appear in the mountain body detection area in the striking image;

[0095] A judgment module 640, configured to judge whether there is a risk of falling of the mountain dangerous rocks in the mountain body detection area by comparing the surface image of the mountain dangerous rocks and the striking image.

[0096] Based on this, for the dynamic mountain dangerous rock detection device based on machine vision according to the embodiments of the present application, the acquisition module 610 determines the mountain detection area, and acquires the surface image of the mountain dangerous rock by photographing the mountain detection area; the striking module 620 uses an unmanned aerial vehicle to strike the mountain detection area; the marking module 630 photographs the striking image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks and the craters formed by the strike that appear in the mountain detection area in the striking image; the judgment module 640 determines whether there is a risk of detachment of the mountain dangerous rock in the mountain detection area by comparing the surface image of the mountain dangerous rock and the striking image. In the embodiments of the present application, the mountain detection area is determined, and the surface image of the mountain dangerous rock is acquired by photographing the mountain detection area; the mountain detection area is struck by using an unmanned aerial vehicle; the striking image after the unmanned aerial vehicle strikes the mountain is photographed, so as to mark the cracks and the craters formed by the strike that appear in the mountain detection area in the striking image; whether there is a risk of detachment of the mountain dangerous rock in the mountain detection area is determined by comparing the surface image of the mountain dangerous rock and the striking image. Based on this, compared with the existing manual detection method, the embodiments of the present application replace manual detection with an unmanned aerial vehicle, improve the detection efficiency of mountain dangerous rocks, reduce the risk coefficient during manual detection, and can improve the detection efficiency of mountain dangerous rocks and reduce the danger of operators by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the mountain dangerous rock, and improve the accuracy of dangerous rock detection. By comparing the images before and after the unmanned aerial vehicle strikes the mountain dangerous rock area, the dynamic detection of mountain dangerous rocks based on machine vision is realized, so that the mountain dangerous rocks with a risk of detachment can be effectively detected and investigated.

[0097] The specific implementation manner of the dynamic mountain dangerous rock detection device based on machine vision is basically the same as the specific embodiments of the above-mentioned dynamic mountain dangerous rock detection method based on machine vision, and will not be elaborated here.

[0098] The embodiments of the present application further provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the above-mentioned dynamic mountain dangerous rock detection method based on machine vision is implemented. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0099] Please refer to Figure 7 , Figure 7 which schematically shows the hardware structure of an electronic device according to another embodiment. The electronic device includes:

[0100] The processor 701 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0101] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 702 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 702 and are called by the processor 701 to execute the machine vision-based dynamic mountain rockfall detection method in the embodiments of the present application, that is, by determining a mountain detection area and obtaining a surface image of the mountain rockfall by photographing the mountain detection area; using an unmanned aerial vehicle to strike the mountain detection area; photographing the strike image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks and craters formed by the strike that appear in the mountain detection area in the strike image; by comparing the surface image of the mountain rockfall and the strike image, judging whether there is a risk of detachment of the mountain rockfall in the mountain detection area. Based on this, compared with the existing manual detection method in the embodiments of the present application, the unmanned aerial vehicle is used to replace manual detection, which improves the detection efficiency of mountain rockfalls, reduces the risk coefficient during manual detection, and can improve the investigation efficiency of mountain rockfalls by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the mountain rockfall, reducing the danger to operators and improving the accuracy of rockfall detection. By comparing the images before and after the unmanned aerial vehicle strikes the mountain rockfall area, the machine vision-based mountain rockfall detection is realized, so as to effectively detect and investigate the mountain rockfalls with a risk of detachment.

[0102] The input / output interface 703 is used to implement information input and output.

[0103] The communication interface 704 is used to implement communication interaction between this device and other devices, and can implement communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0104] The bus transmits information between the various components of the device (such as the processor 701, the memory 702, the input / output interface 703, and the communication interface 704).

[0105] Among them, the processor 701, the memory 702, the input / output interface 703, and the communication interface 704 are communicatively connected to each other inside the device through a bus.

[0106] An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for dynamically detecting mountain rockfall based on machine vision is implemented.

[0107] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0108] The method for dynamically detecting mountain rockfall based on machine vision, the device for dynamically detecting mountain rockfall based on machine vision, the electronic device, and the storage medium provided by the embodiments of the present application determine a mountain detection area, and obtain an image of the surface of the mountain rockfall by photographing the mountain detection area; use an unmanned aerial vehicle to strike the mountain detection area; photograph the strike image after the unmanned aerial vehicle strikes the mountain, so as to mark the cracks and the craters formed by the strike that appear in the mountain detection area in the strike image; by comparing the image of the surface of the mountain rockfall and the strike image, determine whether there is a risk of detachment of the mountain rockfall in the mountain detection area. Based on this, compared with the existing manual detection method, the embodiments of the present application replace manual detection with an unmanned aerial vehicle, improve the detection efficiency of mountain rockfall, reduce the risk coefficient during manual detection, and can improve the efficiency of checking mountain rockfall by obtaining the difference in image features before and after the unmanned aerial vehicle strikes the mountain rockfall, reduce the danger of operating personnel, and improve the accuracy of rockfall detection. By comparing the images before and after the unmanned aerial vehicle strikes the mountain rockfall area, the detection of mountain rockfall based on machine vision is realized, so that the mountain rockfall with a risk of detachment can be effectively detected and checked.

[0109] Those of ordinary skill in the art can understand that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or can be implemented as hardware, or can be implemented as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable programs, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically includes computer-readable programs, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0110] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0111] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown, or combine some steps, or different steps.

[0112] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations.

[0114] In the description of the present application and the above-mentioned accompanying drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0115] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (one) of the following" or its similar expression refers to any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0116] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0117] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, in each embodiment of the present application, each functional unit can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media that can store programs such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0120] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings, and thus do not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A dynamic mountain dangerous rock detection method based on machine vision, characterized in that, The method includes: Determine the mountain body detection area, and obtain the surface image of the mountain dangerous rock by photographing the mountain body detection area; Use an unmanned aerial vehicle to strike the mountain body detection area; Take the strike image after the unmanned aerial vehicle strikes the mountain body, and mark the cracks that appear in the mountain body detection area and the craters formed by the strike in the strike image; By comparing the surface image of the mountain dangerous rock and the strike image, determine whether there is a risk of detachment of the mountain dangerous rock in the mountain body detection area.

2. The method according to claim 1, characterized in that After determining the mountain body detection area and obtaining the surface image of the mountain dangerous rock by photographing the mountain body detection area, it includes: Preprocess the surface image of the mountain dangerous rock, and the preprocessing includes denoising the surface image of the mountain dangerous rock by using a Gaussian filtering algorithm.

3. The method according to claim 2, wherein The using the unmanned aerial vehicle to strike the mountain body detection area includes: Use the unmanned aerial vehicle to strike the mountain body detection area multiple times by using the pentagon strike method to obtain a strike result; Based on the strike result, determine the detection features of the mountain dangerous rock in the mountain body detection area.

4. The method according to claim 3, wherein The using the unmanned aerial vehicle to strike the mountain body detection area multiple times by using the pentagon strike method includes: Take off the unmanned aerial vehicle, perform regional segmentation on the mountain body detection area, and segment it into multiple unit areas with equal area sizes, where the shape of the unit area is a rectangle; Mark the four vertices and the center point of the unit area to determine the projectile positions where the unmanned aerial vehicle needs to launch projectiles for the unit area; Perform projectile shooting on the marked projectile positions respectively, take a picture each time after a strike, until all the marked unit areas are struck.

5. The method according to claim 3, wherein The detection features of the mountain dangerous rock include at least one of the following: The size of the bullet marks left in the mountain body detection area after the unmanned aerial vehicle strikes the mountain body; The length and width of the cracks that appear in the mountain body detection area after the unmanned aerial vehicle strikes the mountain body; The size of the area where mountain rocks fall off and the amount of mountain rock detachment in the mountain body detection area after the unmanned aerial vehicle strikes the mountain body.

6. The method according to claim 1, wherein The by comparing the surface image of the mountain dangerous rock and the strike image to determine whether there is a risk of detachment of the mountain dangerous rock in the mountain body detection area includes: Divide the surface image of the mountain dangerous rock and the strike image into two sets; Detect the image changes before and after the strike of each mountain body detection area according to the two sets; Determine the detection features of the mountain dangerous rock according to the image changes; Input the detection features of the mountain dangerous rock into a pre-trained ResNet network for dangerous rock detection to determine whether there is a risk of detachment of the mountain dangerous rock in the mountain body detection area.

7. The method according to claim 6, wherein The training method of the ResNet network includes: Create a pre-training data set, which consists of feature pictures in the attention mechanism, and the feature pictures include bullet mark pictures, crack pictures after strike, and mountain rock detachment area pictures; Annotate the feature images to determine the range values between normal and abnormal situations of dangerous rock detachment; Train the pre-constructed ResNet network according to the annotated feature images and the range values.

8. A dynamic mountain dangerous rock detection device based on machine vision, characterized in that, The device includes: An acquisition module, configured to determine a mountain body detection area, and acquire an image of the surface of mountain body dangerous rocks by photographing the mountain body detection area; A striking module, configured to use an unmanned aerial vehicle to strike the mountain body detection area; A marking module, configured to photograph a striking image after the unmanned aerial vehicle strikes the mountain body, so as to mark cracks that appear in the mountain body detection area and craters formed by the strike in the striking image; A judgment module, configured to judge whether there is a risk of detachment of the mountain body dangerous rocks in the mountain body detection area by comparing the image of the surface of the mountain body dangerous rocks and the striking image.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the method for dynamically detecting mountain body dangerous rocks based on machine vision according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for dynamically detecting mountain body dangerous rocks based on machine vision according to any one of claims 1 to 7.

Images