Tailing pond unmanned aerial vehicle intelligent inspection method based on deep learning

Abstract

The invention discloses a tailings pond unmanned aerial vehicle intelligent inspection method based on deep learning. The method comprises the steps that a current situation CAD graph of a to-be-inspected area is acquired; constructing a python path planning model based on an ant algorithm; based on the current situation CAD graph and the python path planning model, planning an aerial photography path of a to-be-inspected area, and determining a target path of tailing pond unmanned aerial vehicle intelligent inspection corresponding to the to-be-inspected area; performing aerial photography on a to-be-inspected area according to the target path based on the tailing pond unmanned aerial vehicle, and determining a to-be-recognized image; inputting the to-be-recognized image into a pre-trained tailings pond typical hidden danger detection model for recognition to obtain a recognition result; giving out early warning when determining that the hidden danger exists in the identification result; according to the method, the inspection path of the unmanned aerial vehicle can be efficiently planned, hidden danger detection and recognition are carried out by using deep learning and a pre-trained model, early warning is given out once hidden danger conditions are found, related personnel can quickly take countermeasures, and the safety of a tailing pond is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of tailings pond unmanned aerial vehicle (UAV) intelligent inspection, and in particular to a tailings pond unmanned aerial vehicle (UAV) intelligent inspection method based on deep learning. Background Art

[0002] There are more than 6,000 tailings ponds in my country, including more than 900 "overhead ponds" and more than 900 third-class tailings ponds. Tailings ponds are man-made debris flow hazards with high potential energy. Once a dam breaks, it will cause serious casualties, property losses and adverse social impacts.

[0003] Tailings ponds are mostly built in remote mountainous areas. It takes a long time to go to the tailings pond site. In addition, the road conditions in some areas are poor and the road surface is uneven. When inspecting the surrounding mountains, there is a risk of injury to workers inspecting the pond area. According to on-site manual monitoring, (1) in order to understand the conditions of various systems in the tailings pond, such as flood diversion ditches and dam bodies, it is necessary to inspect around the tailings pond. The inspection route can only be planned based on the road conditions, and it is impossible to conduct a comprehensive and reasonable inspection of the inspection locations in the tailings pond. (2) Manual inspections are time-consuming and cannot understand the overall construction situation. Only part of the system can be inspected. Most of the monitoring points in the online monitoring system also monitor the relevant conditions of the point, such as the displacement of the dam body. What is monitored is the displacement of the dam body at that location, which cannot reflect the displacement of the dam body of the entire tailings pond; (3) Manual inspections cannot detect hidden dangers and issue warnings in a timely manner, leading to tailings pond accidents.

[0004] Therefore, there is an urgent need for a tailings pond UAV intelligent inspection method based on deep learning to solve the above problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems in the above-mentioned technologies to a certain extent. To this end, the purpose of the present invention is to propose a tailings pond UAV intelligent inspection method based on deep learning, which can efficiently plan the inspection path of the UAV, use deep learning and pre-trained models to detect and identify hidden dangers, and issue early warnings in time once hidden dangers are found, which helps relevant personnel to quickly take countermeasures and ensure the safety of the tailings pond.

[0006] To achieve the above objectives, the embodiment of the present invention proposes a tailings pond drone intelligent inspection method based on deep learning, comprising:

[0007] Obtain the current CAD drawing of the area to be inspected;

[0008] Build a python path planning model based on the ant algorithm;

[0009] Based on the current CAD map and the python path planning model, the aerial photography path of the area to be inspected is planned, and the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected is determined;

[0010] Based on the tailings pond drone, aerial photography of the area to be inspected is performed according to the target path to determine the image to be identified;

[0011] The image to be identified is input into a pre-trained typical hidden danger detection model for tailings ponds for identification, and the identification result is obtained;

[0012] An early warning is issued when it is determined that there are hidden dangers in the identification results.

[0013] Preferably, the aerial photography path of the area to be inspected is planned based on the current CAD map and the python path planning model, and the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected is determined, including:

[0014] Analyze the current CAD map of the area to be inspected to determine the area information in the current CAD map of the area to be inspected; and set nodes in the current CAD map based on the area information to obtain a plurality of nodes;

[0015] Performing two-dimensional coordinate conversion on the current CAD image to obtain two-dimensional coordinates corresponding to a number of nodes;

[0016] The two-dimensional coordinates corresponding to each node are input into a pre-built python model for iteration to determine the target path of the tailings pond drone intelligent inspection corresponding to the area to be inspected.

[0017] Preferably, the method for constructing a typical hidden danger detection model for tailings ponds includes:

[0018] Obtain a typical hidden danger image dataset;

[0019] The typical hidden danger image data set is input into the YOLOX model for training to obtain an initial tailings pond typical hidden danger detection model;

[0020] The I oU values ​​were set respectively to screen the initial tailings pond typical hidden danger detection model, and the model with the highest mAP value was selected as the final tailings pond typical hidden danger detection model.

[0021] Preferably, obtaining a typical hidden danger image data set includes:

[0022] Using drones to take aerial photos of tailings ponds in different regions, the sub-dams, drainage systems and surrounding mountain areas of the tailings ponds are photographed from different angles and heights to obtain a first image data set;

[0023] Acquire monitoring data from a tailings pond video monitoring system, perform image extraction on the monitoring data, and obtain a second image data set;

[0024] Obtaining the daily inspection records of the inspection personnel to obtain a third image data set; the daily inspection records include the mobile phone shooting records of the inspection personnel during the inspection process;

[0025] Merging the first image data set, the second image data set and the third image data set to obtain an initial image data set;

[0026] Based on the image data annotation tool Labelimg, the hidden dangers in the initial image dataset are annotated to obtain a typical hidden danger image dataset.

[0027] Preferably, the hidden dangers in the initial image dataset are annotated based on the image data annotation tool Labelimg to obtain a typical hidden danger image dataset, including:

[0028] Set labels for five typical hidden dangers in Labelimg;

[0029] Take any hidden danger image to be labeled and get the target hidden danger image;

[0030] Create a block at the location containing a typical hidden danger in the target hidden danger image and select a label corresponding to the typical hidden danger for annotation;

[0031] Save the labeled target hidden danger image in Pascal VOC XML format;

[0032] Traverse all the images to be annotated and obtain the Pascal VOC XML format files corresponding to all the images to be annotated;

[0033] All the annotated images to be annotated and their corresponding XML files are taken as typical hidden danger image datasets.

[0034] Preferably, before inputting the image to be identified into a pre-trained typical hidden danger detection model for tailings ponds for identification, the method further includes:

[0035] Take any image to be recognized;

[0036] Grayscale the image to be identified to obtain a grayscale image;

[0037] Take any pixel in the grayscale image as the target pixel; determine the area to be processed with a size of P×Q with the target pixel as the center;

[0038] The Euclidean distances between the target pixel and other pixel points in the area to be processed except the target pixel are calculated respectively to obtain a plurality of Euclidean distances;

[0039] The maximum value among several Euclidean distances is taken as the texture intensity value corresponding to the target pixel;

[0040] Get the gray value corresponding to each pixel in the area to be processed;

[0041] Calculate the mean of the grayscale values ​​of each pixel in the area to be processed to obtain the target mean;

[0042] Calculate the absolute value of the difference between the grayscale value of the target pixel and the target mean as the grayscale difference value corresponding to the target pixel;

[0043] Determine the noise evaluation value corresponding to the target pixel point based on the preset algorithm according to the texture intensity value and the grayscale difference value corresponding to the target pixel point;

[0044] Comparing the noise evaluation value with a preset noise evaluation threshold, and when it is determined that the noise evaluation value is greater than or equal to the preset noise evaluation threshold, taking the target pixel point as a target noise point;

[0045] Traverse the pixel points in the grayscale image to obtain several target noise points;

[0046] Filter and reduce noise on several target noise points to obtain an initial denoised image;

[0047] Performing image quality evaluation on the initial denoised image to obtain an evaluation value corresponding to the initial denoised image;

[0048] Comparing the evaluation value with a preset evaluation value threshold;

[0049] When it is determined that the evaluation value is greater than or equal to the preset evaluation value threshold, the initial denoised image is used as the denoised image to be identified.

[0050] Preferably, the preset algorithm includes:

[0051]

[0052] Among them, X i Represents the noise evaluation value of the i-th target pixel in the grayscale image; n i N represents the number of different gray values ​​contained in the area to be processed corresponding to the i-th target pixel in the gray image; i N represents the total number of pixels in the processing area corresponding to the i-th target pixel in the grayscale image; ij Indicates the number of grayscale values ​​in the processing area corresponding to the i-th target pixel in the grayscale image that are equal to the j-th grayscale value; T i Represents the grayscale difference value corresponding to the i-th target pixel in the grayscale image; K iRepresents the texture intensity value corresponding to the i-th target pixel in the grayscale image; e represents a natural constant.

[0053] Preferably, performing image quality evaluation on the initial denoised image to obtain an evaluation value corresponding to the initial denoised image includes:

[0054] Inputting the initial denoised image into a Cartesian rectangular coordinate system to obtain the coordinates corresponding to each pixel point;

[0055] Determine the image evaluation value corresponding to the initial denoised image according to the coordinates corresponding to each pixel point and a preset evaluation algorithm;

[0056]

[0057] Where S represents the image evaluation value corresponding to the initial denoised image; x represents the maximum value of the horizontal coordinate of the pixel point in the Cartesian rectangular coordinate system corresponding to the initial denoised image; y represents the maximum value of the vertical coordinate of the pixel point in the Cartesian rectangular coordinate system corresponding to the initial denoised image; f(a,b) represents the grayscale value of the pixel point with coordinates (a,b) in the Cartesian rectangular coordinate system corresponding to the initial denoised image; M represents the length of the initial denoised image; N represents the width of the initial denoised image; f max Indicates the maximum grayscale value of the pixel in the initial denoised image; f min Indicates the minimum grayscale value of the pixels in the initial denoised image.

[0058] Preferably, typical hidden dangers of the tailings pond include excessively high water level, insufficient dry beach length, blocked drainage wells, blocked drainage chutes and damage to surrounding mountains.

[0059] Preferably, after issuing the warning, the method further includes:

[0060] Generate warning information and send it to the drone controller in the form of a mobile phone text message for manual review;

[0061] After manual verification, the warning information will be reported.

[0062] The present invention discloses a tailings pond UAV intelligent inspection method based on deep learning. Through CAD drawings and path planning models, the inspection path of the UAV can be efficiently planned, and unnecessary flight time and repetition can be reduced. Hidden danger detection and identification are performed using deep learning and pre-trained models, which improves the accuracy and reliability of detection and reduces the interference of human factors. Once hidden dangers are found, early warnings are issued in time, which helps relevant personnel to quickly take countermeasures to ensure the safety of the tailings pond. Based on accurate identification results and early warning information, powerful data support can be provided for subsequent maintenance and management, which facilitates the formulation of more reasonable decisions. Compared with traditional manual inspections, UAV inspections can reduce labor costs and operational risks, and at the same time can cover the inspection area more comprehensively, and can perform comprehensive and reasonable inspections on the inspection positions in the tailings pond, shorten the inspection time, and timely discover hidden dangers and issue early warnings.

[0063] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 This is a flow chart of a tailings pond drone intelligent inspection method based on deep learning according to an embodiment of the present invention;

[0067] Figure 2 is a flow chart of determining a target path according to an embodiment of the present invention;

[0068] Figure 3 It is a flowchart of a method for constructing a pre-trained typical hidden danger detection model for tailings ponds according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0070] Example 1

[0071] like Figure 1 As shown, a tailings pond drone intelligent inspection method based on deep learning includes steps S1-S6:

[0072] S1: Obtain the current CAD map of the area to be inspected;

[0073] S2: Build a python path planning model based on the ant algorithm;

[0074] S3: planning the aerial photography path of the area to be inspected based on the current CAD map and the python path planning model, and determining the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected;

[0075] S4: The tailings pond drone takes aerial photos of the area to be inspected according to the target path to determine an image to be identified;

[0076] S5: input the image to be identified into a pre-trained typical hidden danger detection model for tailings ponds for identification, and obtain an identification result;

[0077] S6: Issue an early warning when it is determined that there are hidden dangers in the identification results.

[0078] In this embodiment, the current CAD drawing refers to a drawing drawn by computer-aided design (CAD) software that reflects the current actual situation of a certain area; it usually contains various geometric shapes, sizes, locations and other detailed information of the area, such as the layout of buildings, topography, distribution of infrastructure, etc. These drawings can provide accurate basic data and reference basis for various projects, planning, design, etc.

[0079] In this embodiment, a python path planning model is constructed based on the ant algorithm, including: defining maps and nodes: creating classes that represent nodes on the map and the connection relationships between them. Initializing pheromones: initializing pheromone values ​​for edges on the map. Ant construction: creating an ant class, with ants having attributes such as current position and paths traveled; ants selecting the next node: calculating the probability of selecting the next node based on pheromones and heuristic information, and making a selection. Ant movement: ants move according to the selected nodes and update the path; pheromone update: after all ants complete a traversal, update the pheromone according to the path traveled by the ants; iterative process: repeating the movement of ants and pheromone updates until a certain number of iterations is reached or other termination conditions are met; obtaining the optimal path: finding the optimal path from the paths traveled by all ants.

[0080] In this embodiment, the tilt angle of the drone camera gimbal can be set to 45 degrees, the flight altitude is always maintained at a height of 15 meters from vertical obstacles, the automatic shooting time interval is 3 seconds, and the flight speed is 5m / s.

[0081] In this embodiment, the recognition result is judged for similarity with the preset tailings pond typical hidden danger database, that is, the image in the recognition result is calculated for similarity with the image in the preset tailings pond typical hidden danger database, and the similarity calculation method includes but is not limited to Euclidean distance similarity calculation and cosine similarity calculation.

[0082] In this embodiment, the pre-trained typical hidden danger detection model for tailings ponds is a model that uses YOLOX as the target detection algorithm, uses PyTorch as the deep learning framework to establish the model, and uses a typical hidden danger image dataset in the PascalVOC XML dataset format to train the model to obtain a typical hidden danger detection model for tailings ponds.

[0083] In this embodiment, the target path is the optimal path calculated by the Python path planning model.

[0084] The beneficial effects of the above technical solution are: through CAD drawings and path planning models, the inspection path of the drone can be efficiently planned, reducing unnecessary flight time and repetition; deep learning and pre-trained models are used for hidden danger detection and identification, which improves the accuracy and reliability of detection and reduces the interference of human factors; once hidden dangers are found, early warnings are issued in time, which helps relevant personnel to take countermeasures quickly to ensure the safety of the tailings pond; based on accurate identification results and early warning information, it can provide strong data support for subsequent maintenance and management, and facilitate the formulation of more reasonable decisions; compared with traditional manual inspections, drone inspections can reduce labor costs and operational risks, and at the same time can cover the inspection area more comprehensively, and can perform comprehensive and reasonable inspections of the inspection locations in the tailings pond, shorten the inspection time, and promptly discover hidden dangers and issue early warnings.

[0085] Example 2

[0086] like Figure 2 As shown, based on the current CAD map and the python path planning model, the aerial photography path of the area to be inspected is planned to determine the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected, including S31-S33:

[0087] S31: Analyze the current CAD map of the area to be inspected to determine the area information in the current CAD map of the area to be inspected; and perform node setting on the current CAD map based on the area information to obtain a plurality of nodes;

[0088] S32: performing two-dimensional coordinate conversion on the current CAD image to obtain two-dimensional coordinates corresponding to a plurality of nodes;

[0089] S33: Input the two-dimensional coordinates corresponding to each node into a pre-built python model for iteration to determine the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected.

[0090] In this embodiment, the regional information includes drainage wells, drainage chutes, surrounding mountains, sub-dams, dry beaches and water level gauge information.

[0091] In this embodiment, firstly, a distance matrix and a pheromone matrix are constructed, and the pheromone is inversely proportional to the distance when selecting a path; each coordinate point in the model is traversed one by one, the first point is the initial point, and the last point is the end point. If there is a coordinate point in the middle that cannot be passed, it will automatically transition to the next point, and the total length of the path is calculated after traversing all the coordinate points; then the total length of the path is recalculated through roulette selection, move operation and flip operation to determine the final target path; then the pheromone and distance are initialized, a new starting point is selected, each drone is traversed, the latest path is compared with the current optimal solution, and the optimal solution is updated; in this way, it is gradually iterated to obtain the final optimal target path.

[0092] The beneficial effects of the above technical solution are: through the analysis of the current CAD diagram and node setting, the specific characteristics and key positions of the area to be inspected can be accurately obtained, providing a basis for subsequent path planning; the two-dimensional coordinate transformation makes the data more convenient for model processing and calculation, and can be better combined with the Python model; using the pre-built Python model for iteration, the optimal tailings pond drone intelligent inspection target path can be efficiently calculated, improving the scientificity and accuracy of path planning; it is helpful to realize the automation and intelligence of tailings pond inspection, improve the inspection efficiency and quality, and also reduce the difficulty and risk of manual operation; it can carry out targeted path planning according to the actual area information, adapt to the special needs of different areas to be inspected, and has strong flexibility and adaptability.

[0093] Example 3

[0094] like Figure 3 As shown in the figure, the construction method of the typical hidden danger detection model of the tailings pond includes S51-S53:

[0095] S51: Acquire a typical hidden danger image dataset;

[0096] S52: inputting the typical hidden danger image data set into the YOLOX model for training to obtain an initial tailings pond typical hidden danger detection model;

[0097] S53: The I oU values ​​are set respectively to screen the initial tailings pond typical hidden danger detection model, and the model with the highest mAP value is selected as the final tailings pond typical hidden danger detection model.

[0098] In this embodiment, the IoU values ​​are set to 0.5 and 0.75 as conditions for the model output results, and the model with the highest mAP value is selected as the final tailings pond typical hidden danger detection model.

[0099] The beneficial effects of the above technical scheme are: by obtaining a typical hidden danger image data set, a rich and targeted data basis is provided for model training, which helps to improve the model's ability to identify typical hidden dangers in tailings ponds; using the YOLOX model for training, its advanced technical architecture and performance advantages can be used to quickly and effectively learn the hidden danger characteristics; using different IoU values ​​for screening can optimize the performance and accuracy of the model and find the model configuration that is most suitable for detecting typical hidden dangers in tailings ponds; finally, the model with the highest mAP value is selected as the final model, which ensures the superiority of the model in overall detection performance and improves the accuracy and reliability of detection; the model constructed by this method helps to realize the automated and intelligent detection of typical hidden dangers in tailings ponds, can discover hidden dangers in a timely manner, and provide strong support for the safety management of tailings ponds.

[0100] Example 4

[0101] Obtain typical hidden danger image datasets, including:

[0102] Using drones to take aerial photos of tailings ponds in different regions, the sub-dams, drainage systems and surrounding mountain areas of the tailings ponds are photographed from different angles and heights to obtain a first image data set;

[0103] Acquire monitoring data from a tailings pond video monitoring system, perform image extraction on the monitoring data, and obtain a second image data set;

[0104] Obtaining the daily inspection records of the inspection personnel to obtain a third image data set; the daily inspection records include the mobile phone shooting records of the inspection personnel during the inspection process;

[0105] Merging the first image data set, the second image data set and the third image data set to obtain an initial image data set;

[0106] Based on the image data annotation tool Labelimg, the hidden dangers in the initial image dataset are annotated to obtain a typical hidden danger image dataset.

[0107] In this embodiment, Labelimg is a commonly used image data annotation tool that can efficiently add accurate annotation information to image data and provide a high-quality data foundation for subsequent machine learning and deep learning tasks.

[0108] The beneficial effects of the above technical scheme are: image data sets are obtained through multiple channels, including drone aerial photography, video monitoring system extraction and patrol personnel records, covering different angles, heights and scenes, making the data more comprehensive and rich, and better reflecting the actual situation of the tailings pond; combined with the video monitoring system and daily inspection data, real-time hidden dangers can be captured in time, reflecting a certain dynamic monitoring capability; the mobile phone shooting records of patrol personnel are closer to the actual work scene, which is helpful to discover some details that are easy to overlook; merging image data sets from multiple sources is conducive to comprehensive analysis and utilization, and improves the utilization value of data; using image data annotation tools to annotate hidden dangers provides accurate guidance and goals for subsequent model training, and improves the accuracy of model learning and detection; overall, it helps to more accurately detect and identify typical hidden dangers of tailings ponds, thereby improving the safety management level of tailings ponds and reducing potential risks.

[0109] Example 5

[0110] Based on the image data annotation tool Labelimg, the hidden dangers in the initial image dataset are annotated to obtain a typical hidden danger image dataset, including:

[0111] Set labels for five typical hidden dangers in Labelimg;

[0112] Take any hidden danger image to be labeled and get the target hidden danger image;

[0113] Create a block at the location containing a typical hidden danger in the target hidden danger image and select a label corresponding to the typical hidden danger for annotation;

[0114] Save the labeled target hidden danger image in Pascal VOC XML format;

[0115] Traverse all the images to be annotated and obtain the Pascal VOC XML format files corresponding to all the images to be annotated;

[0116] All the annotated images to be annotated and their corresponding XML files are taken as typical hidden danger image datasets.

[0117] In this embodiment, five typical hidden dangers include too high reservoir water level, insufficient dry beach length, blocked drainage wells, blocked drainage chutes and damage to surrounding mountains.

[0118] In this embodiment, Pascal VOC XML is a file format for describing target object information in an image. It usually contains relevant attributes of the image (such as file name, size, etc.) and detailed annotation information of the target object in the image, such as the object category, bounding box coordinates, etc. This format is widely used in the field of computer vision, especially in tasks such as target detection. It provides a standardized way for data annotation and management, which facilitates different algorithms and tools to read, process and use annotated data.

[0119] The beneficial effects of the above technical scheme are as follows: by setting labels for five typical hidden dangers, the labeling work has a clear classification standard, which is convenient for subsequent data processing and analysis; blocks are created in the image for the hidden danger locations and the corresponding labels are accurately selected for labeling, which improves the accuracy and reliability of labeling; the labeling results are saved in Pascal VOC XML format, which is convenient for data storage, exchange and sharing, and is conducive to compatibility with other systems or algorithms; all images to be labeled are traversed to ensure that each image in the data set is accurately labeled, thereby increasing the integrity of the data set; the obtained typical hidden danger image data set can be better used for tasks such as model training, which helps to improve the ability to identify and analyze hidden dangers; standardized processes and tools make labeling work more efficient and repeatable, providing a stable foundation for subsequent research and application.

[0120] Example 6

[0121] Before the image to be identified is input into the pre-trained typical hidden danger detection model of tailings pond for identification, it also includes:

[0122] Take any image to be recognized;

[0123] Grayscale the image to be identified to obtain a grayscale image;

[0124] Take any pixel in the grayscale image as the target pixel; determine the area to be processed with a size of P×Q with the target pixel as the center;

[0125] The Euclidean distances between the target pixel and other pixel points in the area to be processed except the target pixel are calculated respectively to obtain a plurality of Euclidean distances;

[0126] The maximum value among several Euclidean distances is taken as the texture intensity value corresponding to the target pixel;

[0127] Get the gray value corresponding to each pixel in the area to be processed;

[0128] Calculate the mean of the grayscale values ​​of each pixel in the area to be processed to obtain the target mean;

[0129] Calculate the absolute value of the difference between the grayscale value of the target pixel and the target mean as the grayscale difference value corresponding to the target pixel;

[0130] Determine the noise evaluation value corresponding to the target pixel point based on the preset algorithm according to the texture intensity value and the grayscale difference value corresponding to the target pixel point;

[0131] Comparing the noise evaluation value with a preset noise evaluation threshold, and when it is determined that the noise evaluation value is greater than or equal to the preset noise evaluation threshold, taking the target pixel point as a target noise point;

[0132] Traverse the pixel points in the grayscale image to obtain several target noise points;

[0133] Filter and reduce noise on several target noise points to obtain an initial denoised image;

[0134] Performing image quality evaluation on the initial denoised image to obtain an evaluation value corresponding to the initial denoised image;

[0135] Comparing the evaluation value with a preset evaluation value threshold;

[0136] When it is determined that the evaluation value is greater than or equal to the preset evaluation value threshold, the initial denoised image is used as the denoised image to be identified.

[0137] In this embodiment, all the images to be identified are subjected to denoising based on the above solution to obtain all the images to be identified after denoising.

[0138] In this embodiment, filtering noise reduction includes but is not limited to median filtering noise reduction.

[0139] The beneficial effects of the above technical solution are: by monitoring the noise of the pixels in the image to be identified, evaluating the target pixels based on the grayscale difference value and texture intensity value of the pixels, completing the screening of noise pixels and improving the screening accuracy; by evaluating the overall image quality of the image to be identified after denoising, the accuracy of subsequent recognition is guaranteed.

[0140] Example 7

[0141] The preset algorithm includes:

[0142]

[0143] Among them, X i Represents the noise evaluation value of the i-th target pixel in the grayscale image; n i N represents the number of different gray values ​​contained in the area to be processed corresponding to the i-th target pixel in the gray image; i N represents the total number of pixels in the processing area corresponding to the i-th target pixel in the grayscale image; ijIndicates the number of grayscale values ​​in the processing area corresponding to the i-th target pixel in the grayscale image that are equal to the j-th grayscale value; T i Represents the grayscale difference value corresponding to the i-th target pixel in the grayscale image; K i Represents the texture intensity value corresponding to the i-th target pixel in the grayscale image; e represents a natural constant.

[0144] The beneficial effects of the above technical solution are: the noise evaluation value is calculated based on multiple factors such as the distribution of different grayscale values, grayscale difference values, and texture intensity values, making the evaluation more comprehensive and accurate; the noise level of each target pixel can be quantified more accurately, providing an accurate basis for subsequent processing; it can be flexibly adjusted and calculated according to the specific characteristics of different areas in the image (such as grayscale changes and texture conditions); it helps to identify and process noise points more specifically, improving the overall processing effect and image quality.

[0145] Example 8

[0146] The image quality of the initial denoised image is evaluated to obtain an evaluation value corresponding to the initial denoised image, including:

[0147] Inputting the initial denoised image into a Cartesian rectangular coordinate system to obtain the coordinates corresponding to each pixel point;

[0148] Determine the image evaluation value corresponding to the initial denoised image according to the coordinates corresponding to each pixel point and a preset evaluation algorithm;

[0149]

[0150] Where S represents the image evaluation value corresponding to the initial denoised image; x represents the maximum value of the horizontal coordinate of the pixel point in the Cartesian rectangular coordinate system corresponding to the initial denoised image; y represents the maximum value of the vertical coordinate of the pixel point in the Cartesian rectangular coordinate system corresponding to the initial denoised image; f(a,b) represents the grayscale value of the pixel point with coordinates (a,b) in the Cartesian rectangular coordinate system corresponding to the initial denoised image; M represents the length of the initial denoised image; N represents the width of the initial denoised image; f max Indicates the maximum grayscale value of the pixel in the initial denoised image; f min Indicates the minimum grayscale value of the pixels in the initial denoised image.

[0151] The beneficial effects of the above technical solution are: taking into account the position and grayscale value of each pixel in the image, comprehensively evaluating the image quality, and being able to more accurately reflect the overall situation; providing an objective evaluation standard through clear formulas and calculations to reduce the influence of subjective factors; analyzing the grayscale value difference of each coordinate point, which helps to capture the detail changes and quality differences in the image; and being applicable to initial denoised images of different sizes and contents, with good versatility.

[0152] Example 9

[0153] Typical hidden dangers of the tailings pond include excessively high water level, insufficient dry beach length, clogged drainage wells, clogged drainage chutes and damage to surrounding mountains.

[0154] Example 10

[0155] After the warning is issued, it also includes:

[0156] Generate warning information and send it to the drone controller in the form of a mobile phone text message for manual review;

[0157] After manual verification, the warning information will be reported.

[0158] The beneficial effects of the above technical solution are: sending early warning information in the form of mobile phone text messages can ensure that the control end personnel receive it in time, thereby improving the response speed; manual review increases accuracy and reliability, and reduces the risk of false alarms; the process of reviewing before reporting ensures the quality of information, avoids unnecessary confusion and wrong decisions; and promotes effective interaction and collaboration between the control end and the early warning system.

[0159] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A tailings pond drone intelligent inspection method based on deep learning, characterized in that: include: Obtain the current CAD drawing of the area to be inspected; Build a python path planning model based on the ant algorithm; Based on the current CAD map and the python path planning model, the aerial photography path of the area to be inspected is planned, and the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected is determined; Based on the tailings pond drone, aerial photography of the area to be inspected is performed according to the target path to determine the image to be identified; The image to be identified is input into a pre-trained typical hidden danger detection model for tailings ponds for identification, and the identification result is obtained; An early warning is issued when it is determined that there are hidden dangers in the identification results.

2. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 1 is characterized in that: Based on the current CAD map and the python path planning model, the aerial photography path of the area to be inspected is planned, and the target path of the tailings pond UAV intelligent inspection corresponding to the area to be inspected is determined, including: Analyze the current CAD map of the area to be inspected to determine the area information in the current CAD map of the area to be inspected; and set nodes in the current CAD map based on the area information to obtain a plurality of nodes; Performing two-dimensional coordinate conversion on the current CAD image to obtain two-dimensional coordinates corresponding to a number of nodes; The two-dimensional coordinates corresponding to each node are input into a pre-built python model for iteration to determine the target path of the tailings pond drone intelligent inspection corresponding to the area to be inspected.

3. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 1 is characterized in that: The construction method of the typical hidden danger detection model of tailings pond includes: Obtain a typical hidden danger image dataset; The typical hidden danger image data set is input into the YOLOX model for training to obtain an initial tailings pond typical hidden danger detection model; The IoU values ​​were set respectively to screen the initial tailings pond typical hidden danger detection model, and the model with the highest mAP value was selected as the final tailings pond typical hidden danger detection model.

4. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 3 is characterized in that: Obtain typical hidden danger image datasets, including: Using drones to take aerial photos of tailings ponds in different regions, the sub-dams, drainage systems and surrounding mountain areas of the tailings ponds are photographed from different angles and heights to obtain a first image data set; Acquire monitoring data from a tailings pond video monitoring system, perform image extraction on the monitoring data, and obtain a second image data set; Obtaining the daily inspection records of the inspection personnel to obtain a third image data set; the daily inspection records include the mobile phone shooting records of the inspection personnel during the inspection process; Merging the first image data set, the second image data set and the third image data set to obtain an initial image data set; Based on the image data annotation tool Label img, the hidden dangers in the initial image dataset are annotated to obtain a typical hidden danger image dataset.

5. The tailings pond drone intelligent inspection method based on deep learning as described in claim 4 is characterized in that: Based on the image data annotation tool Labelimg, the hidden dangers in the initial image dataset are annotated to obtain a typical hidden danger image dataset, including: Set labels for five typical hidden dangers in Labelimg; Take any hidden danger image to be labeled and obtain the target hidden danger image; Create a block at the location containing a typical hidden danger in the target hidden danger image and select a label corresponding to the typical hidden danger for annotation; Save the labeled target hidden danger image in Pascal VOC XML format; Traverse all the images to be annotated and obtain the Pascal VOC XML format files corresponding to all the images to be annotated; All the annotated images to be annotated and their corresponding XML files are taken as typical hidden danger image datasets.

6. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 1 is characterized in that: Before the image to be identified is input into the pre-trained typical hidden danger detection model of tailings pond for identification, it also includes: Take any image to be recognized; Grayscale the image to be identified to obtain a grayscale image; Take any pixel in the grayscale image as the target pixel; determine the area to be processed with a size of P×Q with the target pixel as the center; The Euclidean distances between the target pixel and other pixel points in the area to be processed except the target pixel are calculated respectively to obtain a plurality of Euclidean distances; The maximum value among several Euclidean distances is taken as the texture intensity value corresponding to the target pixel; Obtain the grayscale value corresponding to each pixel in the area to be processed; Calculate the mean of the grayscale values ​​of each pixel in the area to be processed to obtain the target mean; Calculate the absolute value of the difference between the grayscale value of the target pixel and the target mean as the grayscale difference value corresponding to the target pixel; Determine the noise evaluation value corresponding to the target pixel point based on the preset algorithm according to the texture intensity value and the grayscale difference value corresponding to the target pixel point; Comparing the noise evaluation value with a preset noise evaluation threshold, and when it is determined that the noise evaluation value is greater than or equal to the preset noise evaluation threshold, taking the target pixel point as a target noise point; Traverse the pixel points in the grayscale image to obtain several target noise points; Filter and reduce noise on several target noise points to obtain an initial denoised image; Performing image quality evaluation on the initial denoised image to obtain an evaluation value corresponding to the initial denoised image; Comparing the evaluation value with a preset evaluation value threshold; When it is determined that the evaluation value is greater than or equal to the preset evaluation value threshold, the initial denoised image is used as the denoised image to be identified.

7. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 6 is characterized in that: The preset algorithm includes: Among them, X i Represents the noise evaluation value of the i-th target pixel in the grayscale image; n i N represents the number of different gray values ​​contained in the area to be processed corresponding to the i-th target pixel in the gray image; i N represents the total number of pixels in the processing area corresponding to the i-th target pixel in the grayscale image; ij Indicates the number of grayscale values ​​in the processing area corresponding to the i-th target pixel in the grayscale image that are equal to the j-th grayscale value; T i Represents the grayscale difference value corresponding to the i-th target pixel in the grayscale image; K i Represents the texture intensity value corresponding to the i-th target pixel in the grayscale image; e represents a natural constant.

8. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 6 is characterized in that: The image quality of the initial denoised image is evaluated to obtain an evaluation value corresponding to the initial denoised image, including: Inputting the initial denoised image into a Cartesian rectangular coordinate system to obtain the coordinates corresponding to each pixel point; Determine the image evaluation value corresponding to the initial denoised image according to the coordinates corresponding to each pixel point and a preset evaluation algorithm; Where S represents the image evaluation value corresponding to the initial denoised image; x represents the maximum value of the horizontal coordinate of the pixel point in the Cartesian rectangular coordinate system corresponding to the initial denoised image; y represents the maximum value of the vertical coordinate of the pixel point in the Cartesian rectangular coordinate system corresponding to the initial denoised image; f(a,b) represents the grayscale value of the pixel point with coordinates (a,b) in the Cartesian rectangular coordinate system corresponding to the initial denoised image; M represents the length of the initial denoised image; N represents the width of the initial denoised image; f max Indicates the maximum grayscale value of the pixel in the initial denoised image; f min Indicates the minimum grayscale value of the pixels in the initial denoised image.

9. The tailings pond drone intelligent inspection method based on deep learning as claimed in claim 1, characterized in that: Typical hidden dangers of the tailings pond include excessively high water level, insufficient dry beach length, clogged drainage wells, clogged drainage chutes and damage to surrounding mountains.

10. The tailings pond drone intelligent inspection method based on deep learning according to claim 1, characterized in that: After the warning is issued, it also includes: Generate warning information and send it to the drone controller in the form of a mobile phone text message for manual review; After manual verification, the warning information will be reported.

Images