A reservoir safety intelligent inspection method and system based on drone

Abstract

The embodiment of the present invention provides a reservoir safety intelligent inspection method and system based on a drone, which belongs to the field of intelligent inspection technology. The method includes: determining a target reservoir and using a target drone to collect data on the target reservoir to obtain the initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target drone; performing target recognition on the initial image data to obtain the target abnormal object of the target reservoir; performing edge recognition based on the initial image data to obtain the target edge information of the target reservoir; obtaining the initial area information of the target reservoir at the initial position based on the target edge information; determining the first water level information of the target reservoir based on the initial area information; determining the second water level information of the target reservoir based on the initial ranging data and the initial position; determining the target water level information of the target reservoir based on the first water level information and the second water level information; determining the target inspection result of the target reservoir based on the target abnormal object and the target water level information.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection technology, and in particular to a reservoir safety intelligent inspection method and system based on unmanned aerial vehicle (UAV). Background Art

[0002] In modern water conservancy project management, regular inspection of reservoirs is a key measure to ensure their long-term safety, efficient operation and environmental protection. Reservoirs not only provide important support for agricultural irrigation, urban water supply, power generation, flood control and drought relief, but also play an irreplaceable role in maintaining ecological balance and ensuring public safety. Therefore, a comprehensive and efficient inspection system is essential for reservoir management. However, despite the continuous advancement of technology, existing inspection methods still face significant challenges in fully covering all areas of reservoirs. Traditional reservoir inspections usually rely on manual monitoring, which has many limitations, such as limited vision, high labor costs, low inspection efficiency, and difficulty in real-time monitoring of reservoir dynamic changes. Especially in reservoirs with complex terrain or large scale, manual inspections cannot go deep into every corner and are prone to miss potential safety hazards. Although modern technologies such as drones can cover a large area of ​​the reservoir surface, there is a problem that the inspection is not accurate and timely, which affects the accuracy and response speed of reservoir management. Summary of the invention

[0003] The main purpose of the embodiments of the present invention is to provide a method and system for intelligent reservoir safety inspection based on drones, aiming to solve the problem in related technologies that reservoir inspections are not accurate and timely enough, thus affecting the accuracy and response speed of reservoir management.

[0004] In a first aspect, an embodiment of the present invention provides a reservoir safety intelligent inspection method based on a drone, comprising:

[0005] Determine a target reservoir, and use a target UAV to collect data on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target UAV;

[0006] Performing target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir;

[0007] Perform edge recognition according to the initial image data to obtain target edge information corresponding to the target reservoir;

[0008] Obtaining initial area information corresponding to the target reservoir at the initial position according to the target edge information;

[0009] Determine first water level information corresponding to the target reservoir according to the initial area information;

[0010] Determine second water level information corresponding to the target reservoir according to the initial distance measurement data and the initial position;

[0011] Determine target water level information corresponding to the target reservoir according to the first water level information and the second water level information;

[0012] A target inspection result corresponding to the target reservoir is determined according to the target abnormal object and the target water level information.

[0013] In a second aspect, an embodiment of the present invention provides a reservoir safety intelligent inspection system based on a drone, comprising:

[0014] A data acquisition module is used to determine a target reservoir, and use a target UAV to perform data acquisition on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target UAV;

[0015] A target recognition module, used for performing target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir;

[0016] An edge recognition module, used for performing edge recognition according to the initial image data to obtain target edge information corresponding to the target reservoir;

[0017] An area calculation module, used for obtaining the initial area information corresponding to the target reservoir at the initial position according to the target edge information;

[0018] A first water level calculation module, used to determine first water level information corresponding to the target reservoir according to the initial area information;

[0019] A second water level calculation module, used to determine the second water level information corresponding to the target reservoir according to the initial distance measurement data and the initial position;

[0020] a water level determination module, configured to determine target water level information corresponding to the target reservoir according to the first water level information and the second water level information;

[0021] A result determination module is used to determine the target inspection result corresponding to the target reservoir according to the target abnormal object and the target water level information.

[0022] The embodiment of the present invention provides a reservoir safety intelligent inspection method and system based on unmanned aerial vehicles, the method comprising: determining a target reservoir, and using a target unmanned aerial vehicle to collect data on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target unmanned aerial vehicle, so that the use of unmanned aerial vehicles for data collection can greatly improve the efficiency and safety of inspections. Unmanned aerial vehicles can quickly reach a designated location and collect data in a complex or dangerous environment, reducing the dependence on manpower and material resources, and also reducing the risk in the inspection process. Then, target recognition is performed on the initial image data to obtain target abnormal objects corresponding to the target reservoir, so that abnormal objects in the reservoir, such as floating objects, damaged ships, illegal constructions, etc., can be automatically detected. This automated recognition method not only improves the accuracy of inspections, but also greatly shortens the time to discover problems, which helps to respond to and deal with potential safety hazards in a timely manner. Then, edge recognition is performed based on the initial image data to obtain target edge information corresponding to the target reservoir, so that the outline of the reservoir can be clearly outlined, providing accurate basic data for subsequent water level calculations. Thus, the initial area information corresponding to the target reservoir at the initial position is obtained according to the target edge information. The accurate acquisition of the initial area information helps to grasp the water level state of the reservoir more accurately, and then the first water level information of the target reservoir is calculated according to the initial area information. And the second water level information corresponding to the target reservoir is determined according to the initial ranging data and the initial position, so as to determine the target water level information corresponding to the target reservoir according to the first water level information and the second water level information, and then obtain more accurate and comprehensive water level data. This double verification method improves the reliability and stability of water level monitoring. Finally, the target inspection result corresponding to the target reservoir is determined according to the target abnormal object and the target water level information, so that a more comprehensive and detailed inspection result can be obtained. This comprehensive evaluation method helps to more accurately judge the operating status and potential risks of the reservoir, and provides strong support for subsequent maintenance and management. This method solves the problem that the reservoir inspection is not accurate and timely in the relevant technology, thus affecting the accuracy and response speed of reservoir management, and also improves the efficiency and accuracy of the inspection, and significantly reduces the manual risk in the inspection process, which is suitable for safety monitoring of reservoirs of various sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of a flow chart of a reservoir safety intelligent inspection method based on a drone provided in an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of the module structure of a drone-based intelligent reservoir safety inspection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0028] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0029] The embodiment of the present invention provides a method and system for intelligent inspection of reservoir safety based on drones. The method and system for intelligent inspection of reservoir safety based on drones can be applied to a terminal device, which can be an electronic device such as a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The terminal device can be a server or a server cluster.

[0030] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] Please refer to Figure 1 , Figure 1 A schematic flow chart of a method for intelligent reservoir safety inspection based on drones provided in an embodiment of the present invention.

[0032] like Figure 1 As shown, the UAV-based intelligent reservoir safety inspection method includes steps S101 to S108.

[0033] Step S101, determine a target reservoir, and use a target UAV to collect data on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target UAV.

[0034] For example, the reservoir that needs to be inspected is determined as the target reservoir according to the inspection requirements, and the specific location of the target reservoir is determined using the global positioning system. The target drone is determined, and the target drone needs to have sufficient flight endurance, a high-resolution camera system, stable transmission capabilities, and ranging functions (such as a laser rangefinder). In this way, the target drone is controlled to reach the target reservoir, and it is ensured that the target drone can cover the area corresponding to the entire target reservoir, so as to collect data at a suitable height and angle, and then obtain the initial image data and initial ranging data corresponding to the target reservoir under the initial position of the target drone for data collection.

[0035] Step S102: performing target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir.

[0036] Exemplarily, a deep learning model for target recognition tasks is selected, such as YOLO (You Only Look Once), Faster R-CNN, SSD (Single Shot MultiBox Detector), etc., so as to use the image and label information corresponding to the target object in the target reservoir to fine-tune the deep learning model to obtain the target recognition model corresponding to the target reservoir, so as to perform target recognition on the initial image data according to the target recognition model to obtain the target abnormal object corresponding to the target reservoir. The target abnormal object is an object that did not exist or should not exist in the target library before.

[0037] In some embodiments, the target recognition of the initial image data to obtain the target abnormal object corresponding to the target reservoir includes: performing feature extraction on the initial image data to obtain initial feature information corresponding to the initial target in the initial image data; performing feature fusion on the initial feature information to obtain target feature information corresponding to the initial target; determining the target cluster center corresponding to the initial image data based on the initial feature information and the target feature information, and calculating the distance information between each sub-feature information in the initial image data and the target cluster center; performing data clustering on the initial image data based on the distance information to obtain target contour information corresponding to the initial target; obtaining corresponding target area information from the initial image data based on the target contour information, and determining corresponding spatial distribution information based on the target area information; performing target recognition based on the spatial distribution information and the target feature information to obtain the target abnormal object corresponding to the target reservoir.

[0038] Exemplarily, feature extraction such as edge detection, texture analysis, and color histogram is performed on the initial image data to obtain initial feature information corresponding to the initial target in the initial image data.

[0039] Exemplarily, the extracted initial feature information is fused using a feature fusion method such as weighted averaging, principal component analysis (PCA), or feature concatenation to generate target feature information.

[0040] Exemplarily, a position is arbitrarily selected from the target feature information to be determined as the initial cluster center, and then the initial cluster center corresponding to the minimum sum of squares of the difference between the initial feature information and the initial cluster center is determined as the target cluster center corresponding to the initial image data.

[0041] Exemplarily, the Euclidean distance calculation formula is used to calculate the distance information between each sub-feature information in the initial image data and the target cluster center, so that when the distance information is small, it is determined to be the same cluster, and then the outermost pixel information corresponding to each cluster is determined to obtain the target contour information corresponding to the initial target.

[0042] Exemplarily, according to the target contour information, the corresponding target area information is extracted from the initial image data and then the spatial distribution analysis of the target area information is performed, such as calculating the center of gravity, area, perimeter, etc. of the target area information to obtain the spatial distribution information. Finally, according to the preset recognition rules or standards, the spatial distribution information (such as center of gravity, area, perimeter, etc.) is combined with the target feature information (such as texture, color, edge, etc.), and the set recognition rules are applied to perform recognition operations on each target area. By comparing the spatial distribution information and feature information with the preset recognition standards, it is determined whether the target area meets the characteristics of the target abnormal object. If the target area information meets the recognition conditions of the target abnormal object, the area is marked as a target abnormal object.

[0043] In some embodiments, the initial feature information includes first gradient information, first entropy information and first contrast information, and the feature fusion of the initial feature information to obtain the target feature information corresponding to the initial target includes: performing a maximum value solution on the first gradient information to obtain second gradient information; determining an erosion operator, and processing the first entropy information according to the erosion operator to connect the initial image information to obtain second entropy information; filtering the first contrast information to obtain second contrast information; determining a first parameter and a second parameter, and determining a first weight corresponding to the second gradient information, a second weight corresponding to the second entropy information, and a third weight corresponding to the second contrast information according to the first parameter, the second parameter and the second gradient information; and fusing the second gradient information, the second entropy information and the second contrast information according to the first weight, the second weight and the third weight to determine the target feature information.

[0044] Exemplarily, an edge detection algorithm (such as Sobel, Canny, etc.) is used to extract gradient information from the initial image information to determine the first gradient information. An entropy value calculation algorithm is used to calculate the entropy information of the initial image information to determine the first entropy information, and a contrast analysis tool is used to calculate the contrast information of the initial image information to determine the first contrast information.

[0045] Exemplarily, the maximum value of the first gradient information is solved, that is, the maximum gradient value is found in the gradient information to form the second gradient information. The erosion operator is determined. The erosion operator is a morphological operation used to eliminate small noise points or isolated points in the image. Thus, the first entropy information is eroded and weighted using the erosion operator, and then the initial image information is regionally connected to obtain the second entropy information.

[0046] Exemplarily, a filtering algorithm such as Gaussian filtering, median filtering, etc. is used to filter the first contrast information to smooth the contrast information and reduce noise, thereby obtaining the second contrast information.

[0047] Exemplarily, the first parameter and the second parameter are set according to actual needs and experience. The first parameter and the second parameter are used to adjust the weights of the gradient, entropy, and contrast information. For example, the first parameter and the second gradient information are multiplied and then added to the second parameter to obtain the first weight corresponding to the second gradient information, and then the second gradient information is scaled according to the first parameter to obtain the third weight corresponding to the second contrast information, and then the second weight corresponding to the second entropy information is obtained by subtracting the first weight and the third weight from 1.

[0048] Exemplarily, according to the calculated first weight, second weight and third weight, the second gradient information, the second entropy information and the second contrast information are weighted fused, so that the result of the weighted fusion is determined as the target feature information, which integrates the characteristics of gradient, entropy and contrast and can more comprehensively describe the initial target.

[0049] In some embodiments, the determining the target cluster center corresponding to the initial image data according to the initial feature information and the target feature information, and calculating the distance information between each sub-feature information in the initial image data and the target cluster center, includes: determining the feature center from the target feature information, and obtaining the initial cluster center according to the feature center; updating the initial cluster center according to the first difference information between the initial feature information and the initial cluster center, and obtaining the target cluster center; determining the edge parameter, calculating the second difference information between the sub-feature information and the corresponding associated cluster center in the target cluster center, and calculating the third difference information between the sub-feature information and other cluster centers in the target cluster center except the associated cluster center; determining the distance information between each sub-feature information in the initial image data and the target cluster center according to the second difference information, the third difference information and the edge parameter; wherein the distance information is obtained according to the following formula:

[0050]

[0051] in, represents the distance information between the ith sub-feature information and the target cluster center, n represents the number of targets corresponding to the target cluster center, lb represents a logarithmic function with base 2, exp represents a natural exponential function, represents the i-th sub-feature information and the corresponding initial feature information, represents the tth target cluster center, represents the kth target cluster center, represents the second difference information, represents the third difference information, represents the edge parameter.

[0052] Exemplarily, a feature information is randomly selected from the target feature information to be determined as the feature center, and then the feature center is determined as the initial cluster center, so as to calculate the first difference information between the initial feature information and each initial cluster center, and then the square of the first difference information is summed, so that when the square sum result meets the preset threshold, the initial cluster center is determined as the target cluster center. If the square sum result does not meet the preset threshold, the first difference information is summed, and the sum result is added to the initial cluster center to achieve the purpose of updating to obtain the updated initial cluster center, and then the first difference information is recalculated until the square sum result of the recalculated first difference information meets the preset threshold, then the initial cluster center is determined as the target cluster center.

[0053] Exemplarily, an edge parameter is set to adjust the sensitivity between target cluster centers. The edge parameter can help distinguish similar and different cluster centers. For each sub-feature information, the difference between it and the corresponding associated cluster center in the target cluster center, i.e., the second difference information, is calculated. And the difference between each sub-feature information and other cluster centers in the target cluster center except the associated cluster center, i.e., the third difference information, is calculated. Thus, according to the second difference information, the third difference information and the edge parameter, combined with the following formula, the distance information between each sub-feature information and the target cluster center is calculated:

[0054]

[0055] in, represents the distance information between the ith sub-feature information and the target cluster center, n represents the number of targets corresponding to the target cluster center, lb represents a logarithmic function with base 2, exp represents a natural exponential function, represents the i-th sub-feature information and the corresponding initial feature information, represents the tth target cluster center, represents the kth target cluster center, represents the second difference information, represents the third difference information, represents the edge parameter.

[0056] Exemplarily, by combining the second difference information, the third difference information and the edge parameter to calculate the distance information, the relationship between the sub-feature information and the target cluster center can be considered more comprehensively. By calculating the second difference information and the third difference information between the sub-feature information and different cluster centers, similar features can be distinguished more finely. This helps to more accurately identify and classify targets in complex image data. The introduction of edge parameters can flexibly adjust the sensitivity between target cluster centers, so that by calculating the distance information between the sub-feature information and the target cluster center, it can better adapt to different data distributions and feature changes, thereby providing good support for subsequent data clustering to obtain the target contour information corresponding to the initial target.

[0057] Step S103: performing edge recognition according to the initial image data to obtain target edge information corresponding to the target reservoir.

[0058] Exemplarily, edge detection algorithms such as Sobel, Canny, Laplacian, etc. are used, and relevant parameters (such as threshold, filter size, etc.) are adjusted to perform edge recognition on the initial image data, thereby obtaining target edge information corresponding to the target reservoir. The target edge information is used to characterize the edge information corresponding to the water stored in the target reservoir.

[0059] In some embodiments, the edge recognition based on the initial image data to obtain the target edge information corresponding to the target reservoir includes: determining a rotation factor and an adjustment factor, and determining a multi-directional convolution function based on the rotation factor and the adjustment factor in combination with a Gaussian kernel function; performing convolution processing on the initial image data according to the multi-directional convolution function to obtain a corresponding initial processing result; performing threshold segmentation processing on the initial processing result to obtain a target processing result, and analyzing the target processing result to obtain target pixel information in the target processing result that is not connected with other areas; calculating the pixel distance between the target pixel information and the center position of the connected area in the target processing result; dividing the target pixel information according to the pixel distance to obtain the target area corresponding to the target pixel information in the connected area of ​​the target processing result; and determining the target edge information corresponding to the target reservoir according to the target area and the target processing result.

[0060] Exemplarily, a suitable rotation factor and adjustment factor are selected according to actual needs. The rotation factor is used to control the range of change of the convolution direction. The adjustment factor is used to adjust the sensitivity and effect of the convolution kernel. The adjustment factor can be adjusted according to the contrast and noise level of the image. Thus, the Gaussian kernel function is combined with the rotation factor and the adjustment factor to define a multi-directional convolution function, which is used to ensure that the initial image information can adapt to the convolution requirements in multiple directions.

[0061] Exemplarily, a multi-directional convolution function is applied to the initial image information to obtain an initial processing result of the initial image data. The initial processing result may be a convolved image or a multi-dimensional data array. Thus, a suitable threshold is selected according to the characteristics of the initial processing result. The threshold is used to segment the image into a target area and a background area. Thus, the initial processing result is segmented into a target processing result.

[0062] For example, in the target processing result, target pixel information that is not connected to other regions is identified, that is, isolated or discrete pixel information in the target processing result is identified as the target pixel information.

[0063] Exemplarily, the color difference and position difference between the target pixel information and the center position of the connected area are calculated, and the pixel coordinates covered when the target pixel information and the center position of the connected area are connected by a straight line are calculated, so as to obtain the edge change information between the target pixel information and the center position of the connected area according to the pixel coordinates, so as to fuse the color difference, position difference and edge change information to obtain

[0064] The pixel distance between the target pixel information and the center position of the connected area.

[0065] Exemplarily, the target pixel information is divided according to the calculated pixel distance, and its target area in the connected area of ​​the target processing result is determined, so that the target edge information of the target reservoir is obtained according to the divided target area and the target processing result.

[0066] Step S104: obtaining initial area information corresponding to the target reservoir at the initial position according to the target edge information.

[0067] Exemplarily, a target fitting curve corresponding to the target reservoir is obtained by performing curve fitting according to the target edge information, and then the initial area information corresponding to the target reservoir at the initial position is obtained by performing area calculation on the target fitting curve using an integral function.

[0068] Step S105: determining first water level information corresponding to the target reservoir according to the initial area information.

[0069] Exemplarily, different water levels of a target reservoir and their corresponding area data are collected to establish a mapping table, so that the initial area information is queried in the mapping table to obtain the first water level information corresponding to the target reservoir.

[0070] In some embodiments, determining the first water level information corresponding to the target reservoir based on the initial area information includes: determining a target reference position, and determining an area conversion relationship between the initial position and the target reference position; converting the initial area information to the target area information corresponding to the target reservoir at the target reference position according to the area conversion relationship; and using a target mapping table to perform a water level query based on the target area information to obtain the first water level information corresponding to the target reservoir.

[0071] Exemplarily, the target reference position can be a standard water level, a datum surface, or a reference point for a specific measurement. The difference and conversion relationship between the initial position and the target reference position are analyzed. This may involve calculations of elevation changes, projection transformations, or other geometric relationships. Based on the analysis results, an area conversion relationship from the initial position to the target reference position is established. This can be a simple proportional relationship, a linear transformation, or other mathematical models.

[0072] Exemplarily, the initial area information is converted from the initial position to the target reference position using the established area conversion relationship, so that the initial area information is adjusted according to the area conversion relationship to obtain the target area information corresponding to the target reservoir at the target reference position.

[0073] Exemplarily, different water levels of the target reservoir at the target reference position and their corresponding area data are collected to create a target mapping table, and then the water level value matching the converted target area information is searched in the target mapping table. This water level value is the first water level information corresponding to the target reservoir at the target reference position.

[0074] Step S106: Determine second water level information corresponding to the target reservoir according to the initial distance measurement data and the initial position.

[0075] Exemplarily, the initial distance measurement data includes the vertical distance from the initial position to the target reservoir water surface. The initial distance measurement data is used in combination with the altitude of the initial position to determine the absolute height of the reservoir water surface, that is, the second water level information, through simple mathematical operations.

[0076] For example, if the ranging data shows that the distance from the measuring point to the water surface is 10 meters, and the altitude of the measuring point is 100 meters, then the second water level information of the reservoir may be 90 meters.

[0077] In some embodiments, determining the second water level information corresponding to the target reservoir based on the initial ranging data and the initial position includes: calculating the vertical distance between the initial position and the target reference position; adjusting the initial ranging data based on the vertical distance to obtain target ranging data; and obtaining the second water level information corresponding to the target reservoir based on the difference calculation between the target reference position and the target ranging data.

[0078] For example, the target reference position can be a standard reference point, such as a datum plane or a reference point of a specific measurement. The vertical distance between the initial position and the target reference position is calculated. This can be obtained by a simple subtraction operation: the vertical distance is equal to the altitude of the target reference position minus the altitude of the initial position.

[0079] Exemplarily, the initial distance measurement data is adjusted according to the calculated vertical distance. If the altitude of the target reference position is higher than the initial position, the vertical distance is added to the initial distance measurement data; if the altitude of the target reference position is lower than the initial position, the vertical distance is subtracted from the initial distance measurement data, and then the target distance measurement data is obtained by adjusting the initial distance measurement data.

[0080] Exemplarily, the second water level information of the target reservoir is calculated according to the target reference position and the adjusted target ranging data. For example, the second water level information is obtained by subtracting the target ranging data from the altitude of the target reference position.

[0081] Step S107: Determine target water level information corresponding to the target reservoir according to the first water level information and the second water level information.

[0082] Exemplarily, when the difference between the first water level information and the second water level information is less than or equal to a preset value, the first water level information and the second water level information are averaged to obtain the target water level information corresponding to the target reservoir. When the difference between the first water level information and the second water level information is greater than the preset value, the target drone is used to re-collect the initial image data and initial ranging data corresponding to the target reservoir until the target water level information is obtained.

[0083] Step S108: determining a target inspection result corresponding to the target reservoir according to the target abnormal object and the target water level information.

[0084] For example, the target abnormal objects include but are not limited to abnormal objects such as floating objects and algae on the surface of the target reservoir, so that the corresponding water level change of the target reservoir is determined according to the target water level information, and then the corresponding target inspection results are generated according to the preset rules based on the target abnormal objects and water level changes. So that the target user can clearly understand the current status of the target reservoir according to the target inspection results.

[0085] In some embodiments, determining the target inspection result corresponding to the target reservoir based on the target abnormal object and the target water level information includes: obtaining associated area information corresponding to the target abnormal object, and determining the target proportion corresponding to the target abnormal object based on the initial area information and the associated area information; obtaining the corresponding historical water level information within a preset time, and determining the water level change information corresponding to the target reservoir based on the historical water level information and the target water level information; identifying the abnormal state of the target reservoir based on the target proportion and the water level change information to obtain a target judgment result; determining the target inspection result corresponding to the target reservoir based on the target judgment result, the target proportion and the water level change information.

[0086] Exemplarily, associated edge information corresponding to the target abnormal object is obtained from the target edge information, and then the associated edge information is integrated to obtain associated area information corresponding to the target abnormal object, and then the associated area information and the initial area information are ratio-calculated to determine the target proportion corresponding to the target abnormal object.

[0087] Exemplarily, historical water level information within a preset time period is obtained, for example, historical water level information corresponding to the previous time interval of the current moment, so as to calculate water level change information by comparing the historical water level information with the target water level information.

[0088] Exemplarily, the target proportion and water level change information are combined to analyze the abnormal state of the target reservoir to obtain the target judgment result. For example, if the target proportion exceeds a preset threshold (such as 5% or 10%), it may indicate that there is a serious abnormal object problem. If the water level change is large and related to abnormal objects (such as the accumulation of floating objects causing the water level to rise suddenly), it may indicate that there is an abnormal state, and then determine that the target judgment result is that there is an obvious abnormality, otherwise determine that the target judgment result is that the reservoir is normal.

[0089] For example, when the target judgment result is that there is an obvious abnormality, all identified target abnormal objects and their corresponding area information and target proportion are listed, and water level change information and its impact are obtained. Thus, the identified abnormal state and its potential impact are described in detail. And according to the expert database, treatment suggestions for abnormal states and target abnormal objects are obtained, such as cleaning, repair, monitoring, etc., so as to generate corresponding target inspection results.

[0090] In some embodiments, after obtaining the target inspection result, the method further includes: sending the target inspection result to a target terminal corresponding to a target user, so that the target user receives the target inspection result corresponding to the target reservoir according to the target terminal.

[0091] Exemplarily, the target users who need to receive the target inspection results are clearly identified. The target users may be reservoir managers, engineers, government department personnel, etc., and the target terminal types commonly used by each target user are determined, such as computers, mobile devices (mobile phones, tablets, etc.), and then the access address of the target terminal of each target user is obtained, such as email addresses, login information of web applications, etc. Therefore, the transmission method is selected according to the type of target user and target terminal. Common transmission methods include email, file transfer protocol, messaging applications, etc. Therefore, the target inspection result document is sent to the target terminal corresponding to the target user according to the transmission method. After sending the target inspection result, you can confirm whether the target user has successfully received it through a message or phone call.

[0092] See also Figure 2 , Figure 2A reservoir safety intelligent inspection system 200 based on a drone is provided in an embodiment of the present application. The reservoir safety intelligent inspection system 200 based on a drone includes a data acquisition module 201, a target recognition module 202, an edge recognition module 203, an area calculation module 204, a first water level calculation module 205, a second water level calculation module 206, a water level determination module 207, and a result determination module 208. The data acquisition module 201 is used to determine a target reservoir, and use a target drone to perform data acquisition on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target drone; the target recognition module 202 is used to perform target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir; the edge recognition module 203 is used to perform target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir; and the edge recognition module 203 is used to perform target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir. , used for performing edge recognition according to the initial image data to obtain the target edge information corresponding to the target reservoir; an area calculation module 204, used for obtaining the initial area information corresponding to the target reservoir at the initial position according to the target edge information; a first water level calculation module 205, used for determining the first water level information corresponding to the target reservoir according to the initial area information; a second water level calculation module 206, used for determining the second water level information corresponding to the target reservoir according to the initial ranging data and the initial position; a water level determination module 207, used for determining the target water level information corresponding to the target reservoir according to the first water level information and the second water level information; a result determination module 208, used for determining the target inspection result corresponding to the target reservoir according to the target abnormal object and the target water level information.

[0093] In some implementations, the target recognition module 202 performs, during the process of performing target recognition on the initial image data to obtain the target abnormal object corresponding to the target reservoir:

[0094] Performing feature extraction on the initial image data to obtain initial feature information corresponding to an initial target in the initial image data;

[0095] Performing feature fusion on the initial feature information to obtain target feature information corresponding to the initial target;

[0096] Determining a target cluster center corresponding to the initial image data according to the initial feature information and the target feature information, and calculating distance information between each sub-feature information in the initial image data and the target cluster center;

[0097] Performing data clustering on the initial image data according to the distance information to obtain target contour information corresponding to the initial target;

[0098] Obtaining corresponding target area information from the initial image data according to the target contour information, and determining corresponding spatial distribution information according to the target area information;

[0099] Target recognition is performed according to the spatial distribution information and the target feature information to obtain the target abnormal object corresponding to the target reservoir.

[0100] In some implementations, the initial feature information includes first gradient information, first entropy information, and first contrast information. In the process of performing feature fusion on the initial feature information to obtain target feature information corresponding to the initial target, the target recognition module 202 performs:

[0101] Performing a maximum value calculation on the first gradient information to obtain second gradient information;

[0102] Determine a corrosion operator, process the first entropy information according to the corrosion operator to connect the initial image information in regions, and obtain second entropy information;

[0103] Performing filtering processing on the first contrast information to obtain second contrast information;

[0104] Determine a first parameter and a second parameter, and determine a first weight corresponding to the second gradient information, a second weight corresponding to the second entropy information, and a third weight corresponding to the second contrast information according to the first parameter, the second parameter, and the second gradient information;

[0105] The target feature information is determined by fusing the second gradient information, the second entropy information and the second contrast information according to the first weight, the second weight and the third weight.

[0106] In some implementations, the target recognition module 202, in the process of determining the target cluster center corresponding to the initial image data according to the initial feature information and the target feature information, and calculating the distance information between each sub-feature information in the initial image data and the target cluster center, performs:

[0107] Determining a feature center from the target feature information, and obtaining an initial clustering center according to the feature center;

[0108] According to first difference information between the initial feature information and the initial cluster center, and according to the first difference information, the initial cluster center is updated to obtain a target cluster center;

[0109] Determine edge parameters, calculate second difference information between the sub-feature information and the corresponding associated cluster center in the target cluster center, and calculate third difference information between the sub-feature information and other cluster centers in the target cluster center except the associated cluster center;

[0110] Determine the distance information between each sub-feature information in the initial image data and the target cluster center according to the second difference information, the third difference information and the edge parameter;

[0111] The distance information is obtained according to the following formula:

[0112]

[0113] in, represents the distance information between the ith sub-feature information and the target cluster center, n represents the number of targets corresponding to the target cluster center, lb represents a logarithmic function with base 2, exp represents a natural exponential function, represents the i-th sub-feature information and the corresponding initial feature information, represents the tth target cluster center, represents the kth target cluster center, represents the second difference information, represents the third difference information, represents the edge parameter.

[0114] In some implementations, the edge recognition module 203 performs, during the process of obtaining the target edge information corresponding to the target reservoir by performing edge recognition according to the initial image data:

[0115] Determine a rotation factor and an adjustment factor, and determine a multi-directional convolution function according to the rotation factor and the adjustment factor in combination with a Gaussian kernel function;

[0116] Performing convolution processing on the initial image data according to the multi-directional convolution function to obtain a corresponding initial processing result;

[0117] Performing threshold segmentation processing on the initial processing result to obtain a target processing result, and analyzing the target processing result to obtain target pixel information in the target processing result that is not connected with other areas;

[0118] Calculating the pixel distance between the target pixel information and the center position of the connected area in the target processing result;

[0119] Dividing the target pixel information according to the pixel distance to obtain a target area corresponding to the target pixel information in the connected area of ​​the target processing result;

[0120] The target edge information corresponding to the target reservoir is determined according to the target area and the target processing result.

[0121] In some implementations, the first water level calculation module 205, during the process of determining the first water level information corresponding to the target reservoir according to the initial area information, executes:

[0122] Determining a target reference position, and determining an area conversion relationship between the initial position and the target reference position;

[0123] Converting the initial area information to target area information corresponding to the target reservoir at the target reference position according to the area conversion relationship;

[0124] The target mapping table is used to query the water level according to the target area information to obtain the first water level information corresponding to the target reservoir.

[0125] In some implementations, the second water level calculation module 206, during the process of determining the second water level information corresponding to the target reservoir according to the initial ranging data and the initial position, executes:

[0126] Calculating a vertical distance between the initial position and the target reference position;

[0127] Adjusting the initial distance measurement data according to the vertical distance to obtain target distance measurement data;

[0128] The second water level information corresponding to the target reservoir is obtained by performing a difference calculation between the target reference position and the target distance measurement data.

[0129] In some implementations, the result determination module 208, during the process of determining the target inspection result corresponding to the target reservoir according to the target abnormal object and the target water level information, executes:

[0130] Obtaining associated area information corresponding to the target abnormal object, and determining a target proportion corresponding to the target abnormal object according to the initial area information and the associated area information;

[0131] Obtaining historical water level information corresponding to a preset time, and determining water level change information corresponding to the target reservoir according to the historical water level information and the target water level information;

[0132] According to the target proportion and the water level change information, the abnormal state of the target reservoir is identified to obtain a target judgment result;

[0133] The target inspection result corresponding to the target reservoir is determined according to the target judgment result, the target proportion and the water level change information.

[0134] In some implementations, after obtaining the target inspection result, the result determination module 208 further executes:

[0135] The target inspection result is sent to a target terminal corresponding to a target user, so that the target user receives the target inspection result corresponding to the target reservoir through the target terminal.

[0136] In some embodiments, the drone-based reservoir safety intelligent inspection system 200 can be applied to terminal devices.

[0137] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the drone-based reservoir safety intelligent inspection system 200 described above can refer to the corresponding process in the aforementioned drone-based reservoir safety intelligent inspection method embodiment, and will not be repeated here.

[0138] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any drone-based intelligent reservoir safety inspection method provided in the description of the embodiment of the present invention.

[0139] The storage medium may be an internal storage unit of the terminal device described in the foregoing embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc., equipped on the terminal device.

[0140] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0141] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0142] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A reservoir safety intelligent inspection method based on drones, characterized in that: The method comprises: Determine a target reservoir, and use a target UAV to collect data on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target UAV; Performing target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir; Determine a rotation factor and an adjustment factor, and determine a multi-directional convolution function based on the rotation factor and the adjustment factor combined with a Gaussian kernel function; perform convolution processing on the initial image data according to the multi-directional convolution function to obtain a corresponding initial processing result; perform threshold segmentation processing on the initial processing result to obtain a target processing result, and analyze the target processing result to obtain target pixel information that is not connected with other areas in the target processing result; calculate the pixel distance between the target pixel information and the center position of the connected area in the target processing result; divide the target pixel information according to the pixel distance to obtain the target area corresponding to the target pixel information in the connected area of ​​the target processing result; determine the target edge information corresponding to the target reservoir according to the target area and the target processing result; Obtaining initial area information corresponding to the target reservoir at the initial position according to the target edge information; Determine a target reference position, and determine an area conversion relationship between the initial position and the target reference position; convert the initial area information to target area information corresponding to the target reservoir at the target reference position according to the area conversion relationship; use a target mapping table to perform a water level query according to the target area information to obtain first water level information corresponding to the target reservoir; Calculate the vertical distance between the initial position and the target reference position; adjust the initial distance measurement data according to the vertical distance to obtain target distance measurement data; obtain the second water level information corresponding to the target reservoir according to the difference calculation between the target reference position and the target distance measurement data; Determine target water level information corresponding to the target reservoir according to the first water level information and the second water level information; A target inspection result corresponding to the target reservoir is determined according to the target abnormal object and the target water level information.

2. The method according to claim 1, characterized in that The performing target recognition on the initial image data to obtain the target abnormal object corresponding to the target reservoir includes: Performing feature extraction on the initial image data to obtain initial feature information corresponding to an initial target in the initial image data; Performing feature fusion on the initial feature information to obtain target feature information corresponding to the initial target; Determining a target cluster center corresponding to the initial image data according to the initial feature information and the target feature information, and calculating distance information between each sub-feature information in the initial image data and the target cluster center; Performing data clustering on the initial image data according to the distance information to obtain target contour information corresponding to the initial target; Obtaining corresponding target area information from the initial image data according to the target contour information, and determining corresponding spatial distribution information according to the target area information; Target recognition is performed according to the spatial distribution information and the target feature information to obtain the target abnormal object corresponding to the target reservoir.

3. The method according to claim 2, characterized in that The initial feature information includes first gradient information, first entropy information and first contrast information, and the step of performing feature fusion on the initial feature information to obtain target feature information corresponding to the initial target includes: Performing maximum value calculation on the first gradient information to obtain second gradient information; Determine a corrosion operator, process the first entropy information according to the corrosion operator to connect the initial image data in regions, and obtain second entropy information; Performing filtering processing on the first contrast information to obtain second contrast information; Determine a first parameter and a second parameter, and determine a first weight corresponding to the second gradient information, a second weight corresponding to the second entropy information, and a third weight corresponding to the second contrast information according to the first parameter, the second parameter, and the second gradient information; The target feature information is determined by fusing the second gradient information, the second entropy information and the second contrast information according to the first weight, the second weight and the third weight.

4. The method according to claim 2, characterized in that: The determining the target cluster center corresponding to the initial image data according to the initial feature information and the target feature information, and calculating the distance information between each sub-feature information in the initial image data and the target cluster center, includes: Determining a feature center from the target feature information, and obtaining an initial clustering center according to the feature center; According to first difference information between the initial feature information and the initial cluster center, and according to the first difference information, the initial cluster center is updated to obtain a target cluster center; Determine edge parameters, calculate second difference information between the sub-feature information and the corresponding associated cluster center in the target cluster center, and calculate third difference information between the sub-feature information and other cluster centers in the target cluster center except the associated cluster center; Determine the distance information between each sub-feature information in the initial image data and the target cluster center according to the second difference information, the third difference information and the edge parameter; The distance information is obtained according to the following formula: ； in, represents the distance information between the ith sub-feature information and the target cluster center, n represents the number of targets corresponding to the target cluster center, lb represents a logarithmic function with base 2, exp represents a natural exponential function, represents the i-th sub-feature information and the corresponding initial feature information, represents the tth target cluster center, represents the kth target cluster center, represents the second difference information, represents the third difference information, represents the edge parameter.

5. The method according to claim 1, characterized in that The step of determining the target inspection result corresponding to the target reservoir according to the target abnormal object and the target water level information includes: Obtaining associated area information corresponding to the target abnormal object, and determining a target proportion corresponding to the target abnormal object according to the initial area information and the associated area information; Obtaining historical water level information corresponding to a preset time, and determining water level change information corresponding to the target reservoir according to the historical water level information and the target water level information; According to the target proportion and the water level change information, the abnormal state of the target reservoir is identified to obtain a target judgment result; The target inspection result corresponding to the target reservoir is determined according to the target judgment result, the target proportion and the water level change information.

6. The method according to any one of claims 1 to 5, characterized in that After obtaining the target inspection result, the method further includes: The target inspection result is sent to a target terminal corresponding to a target user, so that the target user receives the target inspection result corresponding to the target reservoir through the target terminal.

7. A reservoir safety intelligent inspection system based on drones, characterized in that: include: A data acquisition module is used to determine a target reservoir, and use a target UAV to perform data acquisition on the target reservoir to obtain initial image data and initial ranging data corresponding to the target reservoir at the initial position of the target UAV; A target recognition module, used for performing target recognition on the initial image data to obtain a target abnormal object corresponding to the target reservoir; An edge recognition module is used to determine a rotation factor and an adjustment factor, and determine a multi-directional convolution function according to the rotation factor and the adjustment factor combined with a Gaussian kernel function; and perform convolution processing on the initial image data according to the multi-directional convolution function to obtain a corresponding initial processing result; Performing threshold segmentation processing on the initial processing result to obtain a target processing result, and analyzing the target processing result to obtain target pixel information in the target processing result that is not connected with other areas; Calculating the pixel distance between the target pixel information and the center position of the connected area in the target processing result; Divide the target pixel information according to the pixel distance to obtain the target area corresponding to the target pixel information in the connected area of ​​the target processing result; determine the target edge information corresponding to the target reservoir according to the target area and the target processing result; An area calculation module, used for obtaining the initial area information corresponding to the target reservoir at the initial position according to the target edge information; The first water level calculation module is used to determine a target reference position and determine an area conversion relationship between the initial position and the target reference position; according to the area conversion relationship, the initial area information is converted into target area information corresponding to the target reservoir at the target reference position; Using the target mapping table to query the water level according to the target area information, to obtain the first water level information corresponding to the target reservoir; A second water level calculation module is used to calculate the vertical distance between the initial position and the target reference position; adjust the initial distance measurement data according to the vertical distance to obtain target distance measurement data; and obtain the second water level information corresponding to the target reservoir according to the difference calculation between the target reference position and the target distance measurement data; a water level determination module, configured to determine target water level information corresponding to the target reservoir according to the first water level information and the second water level information; A result determination module is used to determine the target inspection result corresponding to the target reservoir according to the target abnormal object and the target water level information.

Images