Unmanned aerial vehicle inspection abnormal target detection method and device based on deep learning

Abstract

The invention discloses an unmanned aerial vehicle routing inspection abnormal target detection method and device based on deep learning, and relates to the technical field of unmanned aerial vehicle routing inspection, and the method comprises the steps: obtaining routing inspection data collected by an unmanned aerial vehicle in a photovoltaic routing inspection region according to a routing inspection route, and obtaining environment feature data and photovoltaic feature data through separation according to the routing inspection data, updating a photovoltaic environment model based on the environment characteristic data and the photovoltaic characteristic data, determining an abnormal influence factor of a current inspection task according to the photovoltaic environment model, and inputting the photovoltaic characteristic data and the abnormal influence factor into the photovoltaic inspection model to obtain an abnormal photovoltaic characteristic of a photovoltaic panel, and determining an abnormal target according to the abnormal photovoltaic characteristics, and outputting abnormal information of the abnormal target. Through the above mode, the relation between the environment change and the photovoltaic is used for judging the working condition of the photovoltaic, and the accuracy of determining the abnormal target in photovoltaic inspection by the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of drone inspection technology, and in particular to a method and device for detecting abnormal targets in drone inspection based on deep learning. Background Art

[0002] As a vital component of clean energy, the operational efficiency and stability of photovoltaic power plants directly impact the reliability of energy supply. Traditional inspections of photovoltaic power plants rely primarily on manual methods, which suffer from low efficiency, high costs, and limited coverage. With the development of drone technology, drone inspections have gradually become a crucial tool for photovoltaic power plant maintenance. However, current drone inspections rely solely on visual inspections of photovoltaic panels, providing only a partial assessment of their subjective condition. This output can lead to misjudgments, potentially misidentifying normal photovoltaic panels as abnormal. Consequently, current drone inspections have a low accuracy rate for detecting abnormal photovoltaic outputs, resulting in wasted maintenance resources.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a method and device for detecting abnormal targets in drone inspections based on deep learning, aiming to solve the technical problem of low accuracy of abnormal photovoltaic output in drone inspections in the existing technology.

[0005] To achieve the above objectives, this application provides a method for detecting abnormal targets in drone inspections based on deep learning, which includes: Obtaining inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separating the inspection data into environmental characteristic data and photovoltaic characteristic data; updating a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining an abnormal impact factor of a current inspection task according to the photovoltaic environment model; Inputting the photovoltaic characteristic data and the abnormal influencing factors into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; An abnormal target is determined according to the abnormal photovoltaic feature, and abnormal information of the abnormal target is output.

[0006] In one embodiment, the step of obtaining inspection data collected by a drone in a photovoltaic inspection area along an inspection route, and separating the inspection data to obtain environmental characteristic data and photovoltaic characteristic data includes: Obtain a three-dimensional cloud map of the photovoltaic inspection area, map the inspection route to the three-dimensional cloud map, and obtain the drone inspection route; Acquire inspection data collected by the drone along the drone inspection route, separate the inspection data based on data type, and obtain three-dimensional point cloud data and optical inspection image data; An environmental area and a photovoltaic area are determined according to the three-dimensional point cloud data, and the optical inspection image data are fused with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data.

[0007] In one embodiment, the steps of determining an environmental region and a photovoltaic region based on the three-dimensional point cloud data, and fusing the optical inspection image data with the environmental region and the photovoltaic region, respectively, to obtain environmental characteristic data and photovoltaic characteristic data include: generating a scene fence according to the photovoltaic position in the photovoltaic inspection area, and dividing the photovoltaic inspection area into an environmental scene and a photovoltaic scene by the scene fence; Overlaying the three-dimensional point cloud data with the scene fence to determine an overlapping area of ​​the scene fence in the three-dimensional point cloud data; Obtaining an environmental area and a photovoltaic area according to the scene type of the overlapping area and the three-dimensional point cloud data respectively; The photovoltaic inspection image is fused with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data.

[0008] In one embodiment, the step of fusing the photovoltaic inspection image with the environmental area and the photovoltaic area to obtain environmental characteristic data and photovoltaic characteristic data includes: Generating a three-dimensional point cloud coordinate system using the three-dimensional point cloud data, and mapping the three-dimensional point cloud data to the three-dimensional point cloud coordinate system to obtain three-dimensional point cloud coordinate values; Determine a sequence of three-dimensional point cloud coordinate values ​​corresponding to the area ranges of the environmental area and the photovoltaic area; The photovoltaic inspection image is mapped into a three-dimensional point cloud coordinate system, the pixel values ​​of the photovoltaic inspection image are aligned with the three-dimensional point cloud coordinate system, the photovoltaic inspection image is fused with the three-dimensional point cloud coordinate system, and based on the three-dimensional point cloud coordinate value sequence, the photovoltaic inspection image is divided to obtain environmental feature data and photovoltaic feature data.

[0009] In one embodiment, the step of updating the photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining the abnormal impact factor of the current inspection task according to the photovoltaic environment model includes: Determine the environmental factors and photovoltaic factors in the current photovoltaic environmental model; Updating the environmental factor to the environmental characteristic data, and updating the photovoltaic factor to the photovoltaic characteristic data, to obtain a new environmental factor and a new photovoltaic factor; Performing a two-dimensional mapping between the new environmental factor and the new photovoltaic factor to construct a two-dimensional joint probability distribution matrix, and determining an entanglement value of the new environmental factor and the new photovoltaic factor according to the marginal distribution of the new environmental factor and the new photovoltaic factor, wherein the entanglement value is the degree of influence of the new environmental factor on the new photovoltaic factor; The entanglement value is determined as the abnormal impact factor of the current inspection task.

[0010] In one embodiment, the step of inputting the photovoltaic characteristic data and the abnormal influencing factor into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of a photovoltaic panel includes: Inputting the photovoltaic characteristic data and the abnormal influencing factors into a first processing module in a photovoltaic inspection model, wherein the first processing module is a convolutional neural network; Updating the initial convolution kernel and the initial pooling factor of the convolutional neural network according to the abnormal impact factor to obtain a real-time convolution kernel and a real-time pooling factor; Performing image convolution on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data; performing pooling processing on the photovoltaic convolution data based on the real-time pooling factor to obtain photovoltaic image features; Anomaly detection is performed on the photovoltaic image features to obtain abnormal photovoltaic features.

[0011] In one embodiment, the step of performing image convolution on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data includes: Normalizing and denoising the photovoltaic characteristic data to obtain preprocessed photovoltaic characteristic data; performing convolution processing on the pre-processed photovoltaic characteristic data based on the real-time convolution kernel, multiplying a convolution window corresponding to the real-time convolution kernel by a characteristic value of the photovoltaic characteristic data to obtain a photovoltaic characteristic map; Photovoltaic convolution data is obtained according to the photovoltaic characteristic map and the activation function.

[0012] In one embodiment, the step of performing pooling processing on the photovoltaic convolution data based on the real-time pooling factor to obtain photovoltaic image features includes: Determining a pooling window size, a step size, and a pooling type for a pooling operation according to the real-time pooling factor; Performing block processing on the photovoltaic convolution data to obtain a plurality of pooled input blocks; Performing a pooling operation on the pooling input block according to the pooling window size, the step size, and the pooling type to obtain a pooling output block; The pooled output blocks are spliced ​​to obtain photovoltaic image features.

[0013] In one embodiment, the step of determining an abnormal target according to the abnormal photovoltaic feature and outputting abnormal information of the abnormal target includes: determining an abnormal type of the abnormal photovoltaic characteristic; Mapping coordinate values ​​of the position information of the abnormal photovoltaic feature in a three-dimensional point cloud coordinate system; An abnormal target is determined according to the mapped coordinate value, abnormal information is obtained according to the abnormal target and the abnormal type, and the abnormal information is output.

[0014] In addition, to achieve the above objectives, the present application also proposes a deep learning-based drone inspection abnormal target detection device, which includes: The data acquisition module is used to obtain the inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separate the environmental characteristic data and photovoltaic characteristic data according to the inspection data; A model updating module, configured to update a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determine an abnormal impact factor of a current inspection task according to the photovoltaic environment model; An inspection and detection module, configured to input the photovoltaic characteristic data and the abnormal influencing factors into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; The abnormality output module is used to determine an abnormal target according to the abnormal photovoltaic characteristics and output abnormal information of the abnormal target.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a deep learning-based drone inspection abnormal target detection device, and the deep learning-based drone inspection abnormal target detection device includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor. The computer program is configured to implement the steps of the deep learning-based drone inspection abnormal target detection method as described above.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the abnormal target detection method for drone inspection based on deep learning are implemented as described above.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the deep learning-based drone inspection abnormal target detection method as described above.

[0018] The present application provides a method for detecting abnormal targets in drone inspections based on deep learning. The method obtains inspection data collected by drones in photovoltaic inspection areas according to inspection routes, separates environmental characteristic data and photovoltaic characteristic data based on the inspection data, updates a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, determines the abnormal impact factor of the current inspection task based on the photovoltaic environment model, inputs the photovoltaic characteristic data and the abnormal impact factor into the photovoltaic inspection model, obtains abnormal photovoltaic characteristics of the photovoltaic panel, determines abnormal targets based on the abnormal photovoltaic characteristics, and outputs abnormal information of the abnormal targets. In the above manner, the connection between environmental changes and photovoltaics is used to judge the working conditions of photovoltaics, thereby improving the accuracy of drones in determining abnormal targets in photovoltaic inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flowchart of the first embodiment of the method for detecting abnormal targets during drone inspections based on deep learning in this application; Figure 2 This is a schematic diagram of the feature extraction process of an embodiment of the deep learning-based UAV inspection abnormal target detection method of this application; Figure 3 This is a schematic diagram of the module structure of the abnormal target detection device for drone inspection based on deep learning in an embodiment of the present application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the deep learning-based drone inspection abnormal target detection method in the embodiment of the present application.

[0022] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0024] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of the embodiment of the present application is: obtaining inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separating the environmental characteristic data and photovoltaic characteristic data according to the inspection data; updating the photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining the abnormal influencing factors of the current inspection task according to the photovoltaic environment model; inputting the photovoltaic characteristic data and the abnormal influencing factors into the photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; determining abnormal targets according to the abnormal photovoltaic characteristics, and outputting abnormal information of the abnormal targets.

[0026] Currently, photovoltaic power plants are a vital component of clean energy, and their operational efficiency and stability directly impact the reliability of energy supply. Traditional inspections of photovoltaic power plants rely primarily on manual methods, which are subject to issues such as low efficiency, high costs, and limited coverage. With the development of drone technology, drone inspections have gradually become a crucial tool for photovoltaic power plant operations and maintenance. However, current drone inspections only examine the appearance of photovoltaic panels, providing only a partial understanding of their subjective condition. This output can lead to misjudgment, resulting in normal photovoltaic panels being misidentified as abnormal. Consequently, current drone inspections have a low accuracy rate for detecting abnormal photovoltaic output, resulting in wasted maintenance resources.

[0027] The present application provides a solution, which obtains inspection data collected by a drone in a photovoltaic inspection area along an inspection route, separates environmental characteristic data and photovoltaic characteristic data based on the inspection data, updates a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, determines the abnormal impact factor of the current inspection task based on the photovoltaic environment model, inputs the photovoltaic characteristic data and the abnormal impact factor into the photovoltaic inspection model, obtains the abnormal photovoltaic characteristics of the photovoltaic panel, determines the abnormal target based on the abnormal photovoltaic characteristics, and outputs the abnormal information of the abnormal target. In this way, the connection between environmental changes and photovoltaics is used to judge the working condition of photovoltaics, thereby improving the accuracy of drones in determining abnormal targets in photovoltaic inspections.

[0028] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the aforementioned functions, such as a deep learning-based drone inspection abnormal target detection device, etc., and this embodiment does not specifically limit this. The following uses a deep learning-based drone inspection abnormal target detection device as an example to illustrate this embodiment and the following embodiments.

[0029] The present invention provides a method for detecting abnormal targets in drone inspections based on deep learning. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the deep learning-based abnormal target detection method for drone inspections in this application.

[0030] In this embodiment, the deep learning-based drone inspection abnormal target detection method includes steps S10 to S40: Step S10: obtaining inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separating the inspection data to obtain environmental characteristic data and photovoltaic characteristic data; It should be noted that the PV inspection area refers to the PV installation location. Since the installation location of PV panels often changes with the environment, the PV inspection area includes the projection area of ​​the PV panels on the installation area and the surrounding environmental area. The inspection route is the flight path of the drone when performing an inspection mission. When performing an inspection mission, the drone will be at a certain height above the PV panels. The cameras, radars, and other sensors deployed on the drone can obtain inspection data within the PV inspection area. The inspection data includes camera images of the PV inspection area and radar data.

[0031] It is understood that environmental characteristic data refers to environmental data captured by drones during inspections within the PV inspection area, describing environmental characteristics such as wind speed, humidity, temperature, and plant foliage distribution. Photovoltaic characteristic data refers to image information of the PV panels captured by drones during inspections, including light and shadow images of the PV surface and texture images such as cracks.

[0032] In specific implementations, when abnormal target detection is performed on photovoltaics, the objects of detection are abnormal photovoltaics in photovoltaics, including abnormal cracks on the photovoltaic surface and abnormal foreign object occlusion. Foreign object occlusion here refers to the situation where foreign objects fall on photovoltaics. When the mission center initiates an inspection task for abnormal photovoltaic detection, it can mobilize a fleet of drones to inspect the photovoltaic inspection area according to the inspection route and collect inspection data based on the inspection route. After obtaining the inspection data, the data in the inspection data can be separated according to its purpose to obtain environmental feature data and photovoltaic feature data. The environmental feature data can describe the environmental information during the current inspection task, while the photovoltaic feature information describes the surface information of the photovoltaic during the current inspection task.

[0033] In a feasible embodiment, the step of obtaining inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separating the environmental characteristic data and the photovoltaic characteristic data according to the inspection data includes: Obtain a three-dimensional cloud map of the photovoltaic inspection area, map the inspection route to the three-dimensional cloud map, and obtain the drone inspection route; Acquire inspection data collected by the drone along the drone inspection route, separate the inspection data based on data type, and obtain three-dimensional point cloud data and optical inspection image data; An environmental area and a photovoltaic area are determined according to the three-dimensional point cloud data, and the optical inspection image data are fused with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data.

[0034] It should be noted that a 3D cloud map refers to a high-precision 3D digital model generated by 3D point cloud data collected by drone-mounted sensors (such as lidar and multi-view cameras) across the PV inspection area. This data is then used to create a 3D reconstruction algorithm. 3D point cloud data is a collection of points in three dimensions, each containing both position and reflection intensity information. This data can be used to precisely delineate environmental areas, such as terrain, vegetation, and the location of PV panels.

[0035] In a specific implementation, when a drone collects inspection data along a patrol route in a photovoltaic inspection area, a 3D cloud map of the inspection area can be obtained. This 3D cloud map can be either the initial 3D cloud map or the 3D cloud map from the previous inspection mission. During the inspection mission, the inspection route can be mapped to the 3D cloud map, resulting in the drone inspection route. The drone inspection route can be a set of coordinate sequences. The inspection data collected by the drone along the patrol route is then obtained and separated according to data type, including radar data such as 3D point cloud data and image data such as optical inspection image data. The 3D point cloud data can then be used to determine environmental and photovoltaic zones. The environmental zone is defined as the area surrounding the photovoltaic power station that is not covered by photovoltaic equipment, such as terrain, vegetation, and buildings. The photovoltaic zone is defined as the area where photovoltaic panels and their mountings are located. This zone segmentation can be performed automatically based on geometric features in the 3D point cloud data, such as height and reflectivity differences, or by pre-set scene fences. This allows for differentiation between environmental factors and photovoltaic panel anomalies. It is important to note that both the environmental and photovoltaic zones are represented as 3D point cloud data. After distinguishing the environmental area and the photovoltaic area, the photovoltaic inspection image data can be fused with the environmental area and the photovoltaic area respectively, that is, the photovoltaic inspection image can be fused with the three-dimensional point cloud data, and the coordinate data of the current photovoltaic inspection image data in the three-dimensional point cloud data can be determined. At the same time, this coordinate can be matched with the corresponding pixel in the image, thereby obtaining the environmental feature data and photovoltaic feature data.

[0036] In a feasible embodiment, the steps of determining the environmental area and the photovoltaic area based on the three-dimensional point cloud data, and fusing the optical inspection image data with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data include: generating a scene fence according to the photovoltaic position in the photovoltaic inspection area, and dividing the photovoltaic inspection area into an environmental scene and a photovoltaic scene by the scene fence; Overlaying the three-dimensional point cloud data with the scene fence to determine an overlapping area of ​​the scene fence in the three-dimensional point cloud data; Obtaining an environmental area and a photovoltaic area according to the scene type of the overlapping area and the three-dimensional point cloud data respectively; The photovoltaic inspection image is fused with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data.

[0037] It should be noted that the scene fence is a virtual fence used to distinguish the environmental area from the photovoltaic area, and in this embodiment is a coordinate set.

[0038] In the specific implementation, refer to Figure 2 , Figure 2 The figure is a schematic diagram of the feature extraction process. In the photovoltaic inspection area, a virtual scene fence is generated based on the location information of the photovoltaic panels (such as coordinates or boundaries), and the area is divided into the environmental scene (non-photovoltaic area) and the photovoltaic scene (the area where the photovoltaic panels are located). Since the layout of the photovoltaic panels is usually a regular rectangular array, this embodiment uses the rectangular layout as an example for explanation. The inspection methods of other layouts such as circular layout are similar to this embodiment. The scene fence is defined based on the boundary coordinates. Let The PV panel boundary coordinates of the block are:

[0039] Then the fence of the overall photovoltaic scene is the union of all photovoltaic panel boundaries and can be expressed as:

[0040] Assume that a photovoltaic power station has three photovoltaic panels, and their coordinate ranges are: Photovoltaic panel 1:

[0041] Photovoltaic panel 2:

[0042] Photovoltaic panel 3:

[0043] Then the scene fence is defined as:

[0044] The environmental area is the remaining area.

[0045] The steps of fusing the photovoltaic inspection image with the environmental area and the photovoltaic area to obtain environmental characteristic data and photovoltaic characteristic data include: Generating a three-dimensional point cloud coordinate system using the three-dimensional point cloud data, and mapping the three-dimensional point cloud data to the three-dimensional point cloud coordinate system to obtain three-dimensional point cloud coordinate values; Determine a sequence of three-dimensional point cloud coordinate values ​​corresponding to the area ranges of the environmental area and the photovoltaic area; The photovoltaic inspection image is mapped into a three-dimensional point cloud coordinate system, the pixel values ​​of the photovoltaic inspection image are aligned with the three-dimensional point cloud coordinate system, the photovoltaic inspection image is fused with the three-dimensional point cloud coordinate system, and based on the three-dimensional point cloud coordinate value sequence, the photovoltaic inspection image is divided to obtain environmental feature data and photovoltaic feature data.

[0046] In the specific implementation, the 3D point cloud data is superimposed with the scene fence to determine the overlapping area of ​​the scene fence in the 3D point cloud data. At this time, the 3D point cloud data can be aligned with the spatial range of the scene fence to determine which points belong to the photovoltaic scene, that is, the overlapping area. , to determine whether it is within the scene fence, that is, to determine whether this area belongs to the photovoltaic scene. When , it is determined to be a photovoltaic scene, otherwise it is an environmental scene. The judgment method is:

[0047] Assume that a point in the 3D point cloud , check whether its coordinates are within the range of photovoltaic panel 1 ( ), since the conditions are met, the point belongs to the photovoltaic scene. If a point ,Since y=15 exceeds the y range of the photovoltaic panel (y≤10), it belongs to the ambient scenario.

[0048] When dividing the environmental area and photovoltaic area according to the scene type and 3D point cloud data of the overlapping area, the area classification can be further refined according to the reflectivity in the point cloud data. Specifically, it can be segmented according to the threshold. When the reflectivity is greater than the photovoltaic reflection threshold, it is determined to be a photovoltaic area point, and when the reflectivity is less than the photovoltaic reflection threshold, it is determined to be an environmental area point. Assume that the photovoltaic reflection threshold is If the reflectivity of a point is 0.85, it is classified as a photovoltaic area, and if the reflectivity is 0.3, it is classified as an ambient area.

[0049] When the photovoltaic inspection image is fused with the environmental area and photovoltaic area to obtain environmental feature data and photovoltaic feature data, the optical inspection image can be aligned with the three-dimensional point cloud coordinate system and environmental features and photovoltaic features can be extracted by region. Mapping to 3D point cloud coordinate system , the mapping formula is:

[0050] Among them, K is the camera intrinsic parameter matrix, R and t are the rotation matrix and translation vector.

[0051] After the mapping is completed, environmental characteristic data and photovoltaic characteristic data can be obtained based on the photovoltaic inspection image, the environmental area and the photovoltaic area.

[0052] Step S20, updating a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining an abnormal impact factor of a current inspection task according to the photovoltaic environment model; It should be noted that the photovoltaic environment model refers to a dynamic linear model formed for the photovoltaic area and environmental area within the photovoltaic inspection area, which can establish a linear connection between environmental data and photovoltaic data. The abnormal influencing factor quantifies the contribution of environmental factors to photovoltaic anomalies, such as insufficient light, high temperature or shading leading to a decrease in photovoltaic power.

[0053] In the specific implementation, the light intensity is extracted from the environmental feature data , ambient temperature and the coverage rate of obstructions , and obtain the output power of the photovoltaic at this time , and determine the photovoltaic environment model as:

[0054] in, is a model parameter, which represents the weight of each environmental factor on the output power. is the noise term. When updating the photovoltaic environment model, the recursive least squares method (RLS) can be used to dynamically update the model parameters, which can be specifically expressed as:

[0055]

[0056] in, is the parameter vector, is the input feature vector, is the Kalman gain, It is the forgetting factor, which is usually set to 0.95~0.99 and is used to reduce the weight of historical data.

[0057] For example, historical parameters , which means that for every 1 unit increase in light intensity, the power increases by 0.6 units; for every 1°C increase in temperature, the power decreases by 0.2 units; for every 1% increase in shading rate, the power decreases by 0.1 units. The new data is: , measured power , predicted power , and update the parameters according to the RLS algorithm , making the model more suitable for new data.

[0058] However, changes in environmental factors will have a certain impact on the actual output power of photovoltaics. Therefore, it is necessary to determine the connection between environmental changes and photovoltaic impacts. This can be expressed by quantifying the contribution of environmental factors to photovoltaic anomalies. Among them, the abnormal impact factor is a normalized correlation coefficient with a value range of [-1,1], which represents the linear correlation strength between environmental factors and photovoltaic anomalies. A value close to 1 indicates a strong positive correlation, a value close to -1 indicates a strong negative correlation, and a value close to 0 indicates no significant correlation. When determining the abnormal impact factor, it can be determined through the covariance matrix, which is specifically expressed as:

[0059]

[0060] in, is the abnormal impact factor, is the light intensity and power deviation The covariance of Indicates the standard deviation of light intensity and power deviation.

[0061] In a feasible implementation manner, the step of updating the photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining the abnormal impact factor of the current inspection task according to the photovoltaic environment model includes: Determine the environmental factors and photovoltaic factors in the current photovoltaic environmental model; Updating the environmental factor to the environmental characteristic data, and updating the photovoltaic factor to the photovoltaic characteristic data, to obtain a new environmental factor and a new photovoltaic factor; Performing a two-dimensional mapping between the new environmental factor and the new photovoltaic factor to construct a two-dimensional joint probability distribution matrix, and determining an entanglement value of the new environmental factor and the new photovoltaic factor according to the marginal distribution of the new environmental factor and the new photovoltaic factor, wherein the entanglement value is the degree of influence of the new environmental factor on the new photovoltaic factor; The entanglement value is determined as the abnormal impact factor of the current inspection task.

[0062] In the specific implementation, the environmental factor and the photovoltaic factor are used to describe the environmental status and the photovoltaic panel status respectively. The new environmental factor and the new photovoltaic factor are the factors after the environmental factor and the photovoltaic factor are updated. Among them, the environmental factor can include the light intensity ,temperature , occlusion rate etc. The photovoltaic factor can include the output power , component temperature , surface contamination index Then, the latest data collected during the inspection is input into the model to replace the old environmental factors and old photovoltaic factors. Then, a two-dimensional mapping is performed on the new environmental factors and the new photovoltaic factors to construct a two-dimensional joint probability distribution matrix to quantify the correlation between the environmental factors and the photovoltaic factors and analyze the joint distribution law of the two. For example, the environmental factors and photovoltaic factor The value range of is discretized into intervals, joint probability distribution matrix for:

[0063] in, Indicates the interval number, For the interval.

[0064] Refer to Table 1, which shows the statistical joint probability based on historical data examples:

[0065] Based on the joint distribution and marginal distribution, the influence of environmental factors on photovoltaic factors is quantified, that is, the entanglement value of the new environmental factors and the new photovoltaic factors.

[0066] Marginal distribution of environmental factors:

[0067] Photovoltaic factor marginal distribution:

[0068] Entanglement value :

[0069] The abnormal impact factor is the normalized entanglement value, so the entanglement value and abnormal impact factors The relationship between can be expressed as:

[0070] Step S30, inputting the photovoltaic characteristic data and the abnormal influencing factors into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; It should be noted that the photovoltaic inspection model is a convolutional neural network (CNN), which can process photovoltaic characteristic data, dynamically adjust the convolution kernel parameters based on abnormal influencing factors, and determine abnormal characteristics through pooling operations.

[0071] In a specific implementation, when photovoltaic characteristic data and abnormal influencing factors are input into a photovoltaic inspection model, they can be input into a first processing module of the photovoltaic inspection model. The first processing module is a convolutional neural network. The initial convolution kernel and initial pooling factor of the convolutional neural network are updated according to the abnormal influencing factors to obtain a real-time convolution kernel and a real-time pooling factor; image convolution is performed on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data; pooling processing is performed on the photovoltaic convolution data based on the real-time pooling factor to obtain photovoltaic image features; abnormality detection is performed on the photovoltaic image features to obtain abnormal photovoltaic features.

[0072] Assume that the photovoltaic characteristic data is a vector , the abnormal impact factor is the vector , the two need to be concatenated into a joint input vector before inputting the model.

[0073]

[0074] in, It is a vector concatenation operation.

[0075] The initial convolution kernel weight of the model is , abnormal impact factor Control weight adjustment range:

[0076] in, is the scaling factor, is the impact factor of the corresponding channel.

[0077] For example, the initial convolution kernel , impact factor , then the updated convolution kernel is:

[0078] The pooling factor includes the pooling window size and step length , the adjustment formula is:

[0079]

[0080] in, is the adjustment coefficient, which is taken as .

[0081] For example, initialize the window size , step length , impact factor , then the updated parameters are:

[0082]

[0083] When determining the real-time convolution kernel and the real-time pooling factor, image convolution and pooling processing can be performed. The specific implementation method can be:

[0084] in, is the real-time convolution kernel, is the bias term.

[0085] Then the convolution result Perform maximum pooling, the pooling process is:

[0086] After obtaining the pooling results, the pooled features can be classified to determine the type of anomaly. At this time, the anomaly probability is output through the fully connected layer and the Softmax function:

[0087] in, is the weight of the fully connected layer, is the bias term.

[0088] For example, the weights of the fully connected layer , pooling features , then the calculated y is close to , so it can be determined as a hot spot anomaly.

[0089] In addition, when performing a pooling operation, the pooling window size, step size and pooling type of the pooling operation can be determined according to the real-time pooling factor, and the photovoltaic convolution data can be processed in blocks to obtain multiple pooling input blocks. The pooling operation is performed on the pooling input blocks according to the pooling window size, step size and pooling type to obtain pooling output blocks. The pooling output blocks are spliced ​​to obtain photovoltaic image features.

[0090] In a feasible implementation manner, the step of performing image convolution on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data includes: Normalizing and denoising the photovoltaic characteristic data to obtain preprocessed photovoltaic characteristic data; performing convolution processing on the pre-processed photovoltaic characteristic data based on the real-time convolution kernel, multiplying a convolution window corresponding to the real-time convolution kernel by a characteristic value of the photovoltaic characteristic data to obtain a photovoltaic characteristic map; Photovoltaic convolution data is obtained according to the photovoltaic characteristic map and the activation function.

[0091] In the specific implementation, after normalizing and denoising the photovoltaic features, pre-processed photovoltaic feature data can be obtained, and the obtained data is convenient for model processing. When performing convolution processing, after multiplying the eigenvalues ​​of the photovoltaic data by the convolution window corresponding to the real-time convolution kernel, the photovoltaic feature map can be obtained, which can be activated by the ReLU activation function. Assume that the convolution result is:

[0092] The activation function is:

[0093] Then the convolution result after the ReLU activation function is:

[0094] Step S40: determining an abnormal target according to the abnormal photovoltaic characteristics, and outputting abnormal information of the abnormal target.

[0095] It should be noted that the abnormal target refers to the photovoltaic system with abnormal conditions, and the abnormal information includes the location information of the abnormal photovoltaic system and the corresponding abnormal type.

[0096] It can be understood that when determining the abnormal target and abnormal information, the abnormal type of the abnormal photovoltaic feature can be determined; the coordinate value of the abnormal photovoltaic feature is mapped in the three-dimensional point cloud coordinate system according to the position information; the abnormal target is determined according to the mapped coordinate value, the abnormal information is obtained according to the abnormal target and the abnormal type, and the abnormal information is output.

[0097] In the specific implementation, after obtaining the abnormal photovoltaic features, the abnormal photovoltaic features can be classified to determine the type of anomaly they belong to. For example, the output probability obtained is [0.65, 0.25, 0.10], indicating that the anomaly types are hot spots (probability 65%), cracks (probability 25%), and contamination (probability 10%). The type with the highest probability is selected as the final judgment result, that is, for this case, the determined anomaly type is a hot spot anomaly. The position of the abnormal feature in the image coordinate system When it is mapped to the 3D point cloud coordinate system , the mapping formula is:

[0098] The system then locates the abnormal target in the 3D point cloud model based on the mapped coordinates. By comparing the mapped coordinates with the center coordinates of the photovoltaic panel, the nearest photovoltaic panel is identified as the abnormal target. After determining the abnormal target and the type of abnormality, detailed abnormality information can be generated and output.

[0099] This embodiment provides a method for detecting abnormal targets during drone inspections based on deep learning. The method involves obtaining inspection data collected by a drone in a photovoltaic inspection area along an inspection route, separating environmental characteristic data and photovoltaic characteristic data based on the inspection data, updating a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, determining abnormal impact factors for the current inspection task based on the photovoltaic environment model, inputting the photovoltaic characteristic data and the abnormal impact factors into the photovoltaic inspection model, obtaining abnormal photovoltaic characteristics of the photovoltaic panel, determining abnormal targets based on the abnormal photovoltaic characteristics, and outputting abnormal information about the abnormal targets. In this manner, the connection between environmental changes and photovoltaics is used to determine the working conditions of photovoltaics, thereby improving the accuracy of drones in identifying abnormal targets during photovoltaic inspections.

[0100] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the deep learning-based drone inspection abnormal target detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0101] This application also provides a deep learning-based drone inspection abnormal target detection device, please refer to Figure 3 ,The abnormal target detection device for UAV inspection based on deep learning includes: The data acquisition module 10 is used to obtain the inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separate the inspection data into environmental characteristic data and photovoltaic characteristic data; A model updating module 20 is configured to update a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determine an abnormal impact factor of a current inspection task according to the photovoltaic environment model; The inspection and detection module 30 is used to input the photovoltaic characteristic data and the abnormal influencing factors into the photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; The abnormality output module 40 is configured to determine an abnormal target according to the abnormal photovoltaic characteristics and output abnormality information of the abnormal target.

[0102] In a feasible embodiment, the data acquisition module 10 is also used to obtain a three-dimensional cloud map of the photovoltaic inspection area, map the inspection route to the three-dimensional cloud map, and obtain a drone inspection route; obtain the inspection data collected by the drone along the drone inspection route, separate the inspection data based on the data type, and obtain three-dimensional point cloud data and optical inspection image data; determine the environmental area and the photovoltaic area according to the three-dimensional point cloud data, and fuse the optical inspection image data with the environmental area and the photovoltaic area respectively to obtain environmental feature data and photovoltaic feature data.

[0103] In a feasible embodiment, the data acquisition module 10 is also used to generate a scene fence based on the photovoltaic position in the photovoltaic inspection area, and divide the photovoltaic inspection area into an environmental scene and a photovoltaic scene by the scene fence; superimpose the three-dimensional point cloud data with the scene fence to determine the overlapping area of ​​the scene fence in the three-dimensional point cloud data; obtain the environmental area and the photovoltaic area according to the scene type of the overlapping area and the three-dimensional point cloud data respectively; and fuse the photovoltaic inspection image with the environmental area and the photovoltaic area respectively to obtain environmental feature data and photovoltaic feature data.

[0104] In a feasible embodiment, the data acquisition module 10 is also used to generate a three-dimensional point cloud coordinate system with the three-dimensional point cloud data, map the three-dimensional point cloud data to the three-dimensional point cloud coordinate system, and obtain three-dimensional point cloud coordinate values; determine the three-dimensional point cloud coordinate value sequence corresponding to the area range of the environmental area and the photovoltaic area; map the photovoltaic inspection image to the three-dimensional point cloud coordinate system, align the pixel values ​​of the photovoltaic inspection image with the three-dimensional point cloud coordinate system, fuse the photovoltaic inspection image with the three-dimensional point cloud coordinate system, and based on the three-dimensional point cloud coordinate value sequence, divide the photovoltaic inspection image into environmental feature data and photovoltaic feature data.

[0105] In a feasible implementation manner, the model update module 20 is also used to determine the environmental factors and photovoltaic factors in the current photovoltaic environment model; update the environmental factors to the environmental characteristic data, and update the photovoltaic factors to the photovoltaic characteristic data to obtain new environmental factors and new photovoltaic factors; perform two-dimensional mapping on the new environmental factors and the new photovoltaic factors to construct a two-dimensional joint probability distribution matrix, and determine the entanglement value of the new environmental factors and the new photovoltaic factors based on the marginal distribution of the new environmental factors and the new photovoltaic factors, and the entanglement value is the degree of influence of the new environmental factors on the new photovoltaic factors; and determine the entanglement value as the abnormal influencing factor of the current inspection task.

[0106] In a feasible embodiment, the inspection detection module 30 is also used to input the photovoltaic characteristic data and the abnormal influencing factor into the first processing module in the photovoltaic inspection model, and the first processing module is a convolutional neural network; the initial convolution kernel and the initial pooling factor of the convolutional neural network are updated according to the abnormal influencing factor to obtain a real-time convolution kernel and a real-time pooling factor; image convolution is performed on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data; pooling processing is performed on the photovoltaic convolution data based on the real-time pooling factor to obtain photovoltaic image features; and abnormality detection is performed on the photovoltaic image features to obtain abnormal photovoltaic features.

[0107] In a feasible embodiment, the inspection and detection module 30 is further used to normalize and denoise the photovoltaic characteristic data to obtain preprocessed photovoltaic characteristic data; convolve the preprocessed photovoltaic characteristic data based on the real-time convolution kernel, so that the convolution window corresponding to the real-time convolution kernel is multiplied by the eigenvalue of the photovoltaic characteristic data to obtain a photovoltaic characteristic map; and obtain photovoltaic convolution data according to the photovoltaic characteristic map and the activation function.

[0108] In a feasible embodiment, the inspection detection module 30 is also used to determine the pooling window size, step size and pooling type of the pooling operation according to the real-time pooling factor; block the photovoltaic convolution data to obtain multiple pooling input blocks; perform pooling operations on the pooling input blocks according to the pooling window size, the step size and the pooling type to obtain pooling output blocks; and splice the pooling output blocks to obtain photovoltaic image features.

[0109] In a feasible embodiment, the abnormal output module 40 is also used to determine the abnormal type of the abnormal photovoltaic feature; map the coordinate value of the position information of the abnormal photovoltaic feature in the three-dimensional point cloud coordinate system; determine the abnormal target according to the mapped coordinate value, obtain abnormal information according to the abnormal target and the abnormal type, and output the abnormal information.

[0110] The deep learning-based abnormal target detection device for drone inspections provided in this application adopts the deep learning-based abnormal target detection method for drone inspections in the above-mentioned embodiments, and can solve the technical problem of low accuracy of abnormal photovoltaic output during drone inspections. Compared with the prior art, the beneficial effects of the deep learning-based abnormal target detection device for drone inspections provided in this application are the same as the beneficial effects of the deep learning-based abnormal target detection method for drone inspections provided in the above-mentioned embodiments, and the other technical features of the deep learning-based abnormal target detection device for drone inspections are the same as the features disclosed in the above-mentioned embodiments and are not further described here.

[0111] The present application provides a deep learning-based drone inspection abnormal target detection device, and the deep learning-based drone inspection abnormal target detection device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep learning-based drone inspection abnormal target detection method in the above-mentioned embodiment one.

[0112] Reference below Figure 4, which shows a schematic diagram of the structure of a deep learning-based drone inspection abnormal target detection device suitable for implementing the embodiments of the present application. The deep learning-based drone inspection abnormal target detection device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The deep learning-based drone inspection abnormal target detection device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0113] like Figure 4 As shown, the deep learning-based drone inspection abnormal target detection device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in ROM (Read Only Memory) 1002 or programs loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the deep learning-based drone inspection abnormal target detection device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication devices 1009 can allow the deep learning-based drone inspection abnormal target detection device to communicate wirelessly or wired with other devices to exchange data. While the figure shows a deep learning-based drone inspection abnormal target detection device with various systems, it should be understood that implementation or presence of all the illustrated systems is not required. More or fewer systems may alternatively be implemented or present.

[0114] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0115] The deep learning-based drone inspection abnormal target detection device provided by this application adopts the deep learning-based drone inspection abnormal target detection method in the above-mentioned embodiment, which can solve the technical problems of deep learning-based drone inspection abnormal target detection. Compared with the existing technology, the beneficial effects of the deep learning-based drone inspection abnormal target detection device provided by this application are the same as the beneficial effects of the deep learning-based drone inspection abnormal target detection method provided by the above-mentioned embodiment, and the other technical features of the deep learning-based drone inspection abnormal target detection device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0116] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0117] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0118] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, and the computer-readable program instructions are used to execute the deep learning-based drone inspection abnormal target detection method in the above-mentioned embodiment.

[0119] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0120] The above-mentioned computer-readable storage medium can be included in the deep learning-based drone inspection abnormal target detection device; or it can exist independently without being assembled into the deep learning-based drone inspection abnormal target detection device.

[0121] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the deep learning-based drone inspection abnormal target detection device, the deep learning-based drone inspection abnormal target detection device enables the following: to obtain the inspection data collected by the drone in the photovoltaic inspection area according to the inspection route, and to obtain environmental characteristic data and photovoltaic characteristic data based on the inspection data; to update the photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and to determine the abnormal influencing factors of the current inspection task based on the photovoltaic environment model; to input the photovoltaic characteristic data and the abnormal influencing factors into the photovoltaic inspection model to obtain the abnormal photovoltaic characteristics of the photovoltaic panel; to determine the abnormal target based on the abnormal photovoltaic characteristics, and to output the abnormal information of the abnormal target.

[0122] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0123] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0124] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0125] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned deep learning-based method for detecting abnormal targets during drone inspections. This computer-readable storage medium can address the technical issues surrounding deep learning-based abnormal target detection during drone inspections. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the deep learning-based method for detecting abnormal targets during drone inspections provided in the aforementioned embodiments, and are not further elaborated here.

[0126] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned deep learning-based drone inspection abnormal target detection method.

[0127] The computer program product provided in this application can solve the technical problem of detecting abnormal targets during drone inspections based on deep learning. Compared to the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the deep learning-based abnormal target detection method for drone inspections provided in the above-mentioned embodiments, and will not be elaborated here.

[0128] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for detecting abnormal targets in drone inspections based on deep learning, characterized in that: The deep learning-based UAV inspection abnormal target detection method includes: Obtaining inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separating the inspection data into environmental characteristic data and photovoltaic characteristic data; updating a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining an abnormal impact factor of a current inspection task according to the photovoltaic environment model; Inputting the photovoltaic characteristic data and the abnormal influencing factors into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; An abnormal target is determined according to the abnormal photovoltaic feature, and abnormal information of the abnormal target is output.

2. The method according to claim 1, wherein The steps of obtaining inspection data collected by the drone in the photovoltaic inspection area along the inspection route and separating the inspection data to obtain environmental characteristic data and photovoltaic characteristic data include: Obtain a three-dimensional cloud map of the photovoltaic inspection area, map the inspection route to the three-dimensional cloud map, and obtain the drone inspection route; Acquire inspection data collected by the drone along the drone inspection route, separate the inspection data based on data type, and obtain three-dimensional point cloud data and optical inspection image data; An environmental area and a photovoltaic area are determined according to the three-dimensional point cloud data, and the optical inspection image data are fused with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data.

3. The method according to claim 2, wherein The steps of determining the environmental area and the photovoltaic area according to the three-dimensional point cloud data, and fusing the optical inspection image data with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data include: generating a scene fence according to the photovoltaic position in the photovoltaic inspection area, and dividing the photovoltaic inspection area into an environmental scene and a photovoltaic scene by the scene fence; Overlaying the three-dimensional point cloud data with the scene fence to determine an overlapping area of ​​the scene fence in the three-dimensional point cloud data; Obtaining an environmental area and a photovoltaic area according to the scene type of the overlapping area and the three-dimensional point cloud data respectively; The photovoltaic inspection image is fused with the environmental area and the photovoltaic area respectively to obtain environmental characteristic data and photovoltaic characteristic data.

4. The method according to claim 3, wherein The step of fusing the photovoltaic inspection image with the environmental area and the photovoltaic area to obtain environmental characteristic data and photovoltaic characteristic data comprises: Generating a three-dimensional point cloud coordinate system using the three-dimensional point cloud data, and mapping the three-dimensional point cloud data to the three-dimensional point cloud coordinate system to obtain three-dimensional point cloud coordinate values; Determine a sequence of three-dimensional point cloud coordinate values ​​corresponding to the area ranges of the environmental area and the photovoltaic area; The photovoltaic inspection image is mapped into a three-dimensional point cloud coordinate system, the pixel values ​​of the photovoltaic inspection image are aligned with the three-dimensional point cloud coordinate system, the photovoltaic inspection image is fused with the three-dimensional point cloud coordinate system, and based on the three-dimensional point cloud coordinate value sequence, the photovoltaic inspection image is divided to obtain environmental feature data and photovoltaic feature data.

5. The method according to claim 1, wherein The step of updating the photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determining the abnormal impact factor of the current inspection task according to the photovoltaic environment model includes: Determine the environmental factors and photovoltaic factors in the current photovoltaic environmental model; Updating the environmental factor to the environmental characteristic data, and updating the photovoltaic factor to the photovoltaic characteristic data, to obtain a new environmental factor and a new photovoltaic factor; Performing a two-dimensional mapping between the new environmental factor and the new photovoltaic factor to construct a two-dimensional joint probability distribution matrix, and determining an entanglement value of the new environmental factor and the new photovoltaic factor according to the marginal distribution of the new environmental factor and the new photovoltaic factor, wherein the entanglement value is the degree of influence of the new environmental factor on the new photovoltaic factor; The entanglement value is determined as the abnormal impact factor of the current inspection task.

6. The method according to claim 1, wherein The step of inputting the photovoltaic characteristic data and the abnormal influencing factors into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel includes: Inputting the photovoltaic characteristic data and the abnormal influencing factors into a first processing module in a photovoltaic inspection model, wherein the first processing module is a convolutional neural network; Updating the initial convolution kernel and the initial pooling factor of the convolutional neural network according to the abnormal impact factor to obtain a real-time convolution kernel and a real-time pooling factor; Performing image convolution on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data; performing pooling processing on the photovoltaic convolution data based on the real-time pooling factor to obtain photovoltaic image features; Anomaly detection is performed on the photovoltaic image features to obtain abnormal photovoltaic features.

7. The method according to claim 6, wherein The step of performing image convolution on the photovoltaic characteristic data based on the real-time convolution kernel to obtain photovoltaic convolution data includes: Normalizing and denoising the photovoltaic characteristic data to obtain preprocessed photovoltaic characteristic data; performing convolution processing on the pre-processed photovoltaic characteristic data based on the real-time convolution kernel, multiplying a convolution window corresponding to the real-time convolution kernel by a characteristic value of the photovoltaic characteristic data to obtain a photovoltaic characteristic map; Photovoltaic convolution data is obtained according to the photovoltaic characteristic map and the activation function.

8. The method according to claim 6, wherein The step of performing pooling processing on the photovoltaic convolution data based on the real-time pooling factor to obtain photovoltaic image features includes: Determining a pooling window size, a step size, and a pooling type for a pooling operation according to the real-time pooling factor; Performing block processing on the photovoltaic convolution data to obtain a plurality of pooled input blocks; Performing a pooling operation on the pooling input block according to the pooling window size, the step size, and the pooling type to obtain a pooling output block; The pooled output blocks are spliced ​​to obtain photovoltaic image features.

9. The method according to claim 1, wherein The step of determining an abnormal target according to the abnormal photovoltaic characteristics and outputting abnormal information of the abnormal target includes: determining an abnormal type of the abnormal photovoltaic characteristic; Mapping coordinate values ​​of the position information of the abnormal photovoltaic feature in a three-dimensional point cloud coordinate system; An abnormal target is determined according to the mapped coordinate value, abnormal information is obtained according to the abnormal target and the abnormal type, and the abnormal information is output.

10. A deep learning-based drone inspection abnormal target detection device, characterized in that: The deep learning-based drone inspection abnormal target detection device includes: The data acquisition module is used to obtain the inspection data collected by the drone in the photovoltaic inspection area along the inspection route, and separate the environmental characteristic data and photovoltaic characteristic data according to the inspection data; A model updating module, configured to update a photovoltaic environment model based on the environmental characteristic data and the photovoltaic characteristic data, and determine an abnormal impact factor of a current inspection task according to the photovoltaic environment model; An inspection and detection module, configured to input the photovoltaic characteristic data and the abnormal influencing factors into a photovoltaic inspection model to obtain abnormal photovoltaic characteristics of the photovoltaic panel; The abnormality output module is used to determine an abnormal target according to the abnormal photovoltaic characteristics and output abnormal information of the abnormal target.

Images