An inspection picture intelligent analysis method and system for a drone

Abstract

The application provides a kind of inspection picture intelligent analysis method and system for unmanned plane, it is related to image data analysis technical field, including: according to the frame period split of inspection video picture collected in real time by unmanned plane real-time inspection, obtain corresponding real-time frame image data;Real-time frame image data is extracted to feature, and the texture feature set of corresponding real-time frame image data is obtained;Similarity analysis is carried out to the texture feature set, and the abnormal type of corresponding regional photovoltaic panel is obtained, and then the corresponding abnormal processing scheme is executed;Based on deep learning algorithm, and according to the texture feature set of historical acquisition period, construct fast feature analysis model, and then the abnormal type of corresponding regional photovoltaic panel is quickly identified, so as to maintain photovoltaic power station in time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data analysis, in particular to a method and system for intelligent analysis of inspection pictures of a UAV. BACKGROUND

[0002] In the operation and maintenance process of a photovoltaic power station, the inspection of photovoltaic panels is a key link to ensure the power generation efficiency and equipment safety of the power station. Traditional photovoltaic panel inspection relies on manual inspection or simple unmanned aerial vehicle image acquisition, which has obvious limitations and cannot meet the needs of timely maintenance of photovoltaic power stations. Moreover, there is a lack of an integrated system that can integrate data acquisition, processing, analysis, rapid identification and storage, and the links are loose and the operation convenience is poor.

[0003] Therefore, the present application provides a method and system for intelligent analysis of inspection pictures of a UAV. SUMMARY

[0004] To solve the above technical problems, the purpose of the present application is to provide a method and system for intelligent analysis of inspection pictures of a UAV.

[0005] To achieve the above purpose, the present application provides the following technical solution: a method for intelligent analysis of inspection pictures of a UAV, the method comprising:

[0006] frame period splitting of the inspection video pictures collected by the UAV in real time to obtain corresponding real-time frame image data;

[0007] feature extraction of the real-time frame image data to obtain a texture feature set of the corresponding real-time frame image data; similarity analysis of the texture feature set to obtain the abnormal type of the corresponding area photovoltaic panel, and then execution of the corresponding abnormal processing scheme;

[0008] based on a deep learning algorithm and according to the texture feature set of the historical collection period, a rapid feature analysis model is constructed to quickly identify the abnormal type of the corresponding area photovoltaic panel, so as to facilitate timely maintenance of the photovoltaic power station.

[0009] Further, the process of frame period splitting of the inspection video pictures collected by the UAV in real time to obtain corresponding real-time frame image data comprises:

[0010] According to the preset inspection route, the UAV synchronously collects the inspection video pictures of the corresponding photovoltaic panel on the inspection route through the camera sensor carried by the UAV, the inspection video pictures including inspection thermal imaging video pictures and inspection visual video pictures; and a collection period is set, and a unique identifier is matched for the inspection video pictures collected in each collection period.

[0011] Further, the process of frame period splitting on the inspection thermal imaging video frame and the inspection visual video frame to obtain the real-time frame thermal imaging image data and the real-time frame surface image data comprises:

[0012] According to the unique identifier, the corresponding inspection thermal imaging video frame and the inspection visual video frame are obtained, and based on the size data of each photovoltaic panel on the corresponding inspection route and the real-time flight speed of the unmanned aerial vehicle, the collection period of the corresponding inspection thermal imaging video frame and the inspection visual video frame is divided into sub-collection periods;

[0013] A real-time flight speed threshold is set. If the real-time flight speed of the unmanned aerial vehicle in inspecting the corresponding photovoltaic panel is greater than the real-time flight speed threshold, the sub-collection period corresponding to the photovoltaic panel is frame period split according to the high-speed inspection frame splitting period. If the real-time flight speed of the unmanned aerial vehicle in inspecting the corresponding photovoltaic panel is less than or equal to the real-time flight speed threshold, the sub-collection period corresponding to the photovoltaic panel is frame period split according to the low-speed inspection frame splitting period.

[0014] According to the high-speed inspection frame splitting period and the low-speed inspection frame splitting period, the real-time frame thermal imaging image data and the real-time frame surface image data under the corresponding flight state are obtained.

[0015] Further, the process of feature extraction on the real-time frame image data to obtain the texture feature set corresponding to the real-time frame image data comprises:

[0016] Based on the temperature feature extraction technology and the gray level co-occurrence matrix, the texture feature set corresponding to the real-time frame image data is obtained. The texture feature set includes a thermal imaging texture feature set and a surface image texture feature set.

[0017] Different real-time frame image data of the same photovoltaic panel are respectively subjected to feature extraction to obtain the texture feature value of each real-time frame image data, and the mean value of the texture feature values corresponding to different real-time frame image data is taken as the texture feature set of the photovoltaic panel.

[0018] Further, the process of feature extraction on the real-time frame thermal imaging image data and the real-time frame surface image data to obtain the corresponding thermal imaging texture feature set and the surface image texture feature set comprises:

[0019] Based on the temperature feature extraction technology, the gray value of the corresponding pixel point is obtained, and the gray value is converted into an absolute temperature value. Based on the temperature feature extraction technology, the temperature standard deviation of the corresponding real-time frame thermal imaging image data is obtained. A temperature anomaly threshold is set, and the absolute temperature values of the corresponding pixel points of the real-time frame thermal imaging image data are respectively subjected to threshold judgment, and then the high-temperature pixel ratio of the corresponding real-time frame thermal imaging image data is counted.

[0020] parallel to the width direction of the real-time frame thermal imaging image data is denoted as parallel to the height direction of the real-time frame thermal imaging image data is denoted as parallel to the height direction of the real-time frame thermal imaging image data is denoted as parallel to the height direction of the real-time frame thermal imaging image data is denoted as parallel to the height direction of the real-time frame thermal imaging image data is denoted as

[0021] the high-temperature pixel proportion, the temperature standard deviation, parallel to the height direction of the real-time frame thermal imaging image data is denoted as parallel to the height direction of the real-time frame thermal imaging image data is denoted as

[0022] For the obtained real-time frame surface image data, a gray level co-occurrence matrix around each pixel point of the real-time frame surface image data is calculated in a sliding window manner, a series of surface image texture features are extracted based on the calculated gray level co-occurrence matrix, and a surface image texture feature set is constructed. The surface image texture features in the surface image texture feature set include contrast, correlation, energy, homogeneity, entropy, and inverse variance.

[0023] Further, the process of performing similarity analysis on the texture feature set to obtain the abnormal type of the corresponding regional photovoltaic panel, and then executing the corresponding abnormal processing scheme includes:

[0024] According to the similarity analysis on the thermal imaging texture feature set and the surface image texture feature set respectively, the corresponding abnormal feature similarity is obtained. The mean value of the abnormal feature similarity of the corresponding photovoltaic panel is taken and denoted as the thermal imaging abnormal feature similarity mean value and the surface image abnormal feature similarity mean value respectively. The abnormal feature similarity includes the thermal imaging abnormal feature similarity and the surface image abnormal feature similarity.

[0025] The texture feature similarity threshold values of different abnormal types are preset, the thermal imaging abnormal feature similarity mean value and the surface image abnormal feature similarity mean value are sorted according to the size of the texture feature similarity threshold value, the abnormal type with the maximum thermal imaging abnormal feature similarity mean value and surface image abnormal feature similarity mean value is taken, and the real-time frame surface image data and the real-time frame thermal imaging image data of the corresponding sub-acquisition period are called to compare and judge the abnormal type. If the comparison and judgment is successful, the corresponding abnormal processing scheme is executed. If the comparison and judgment fails, manual review and judgment are performed.

[0026] Further, the process of performing similarity analysis on the thermal imaging texture feature set and the surface image texture feature set includes:

[0027] According to the temperature gradient in the direction of the temperature gradient in the direction of a temperature gradient in the direction, calculating a gradient amplitude; further determining the abnormal type according to the gradient amplitude; and composing corresponding thermal imaging abnormal feature similarity according to the high-temperature pixel proportion, the temperature standard deviation and the gradient amplitude;

[0028] obtaining other pixel local regions adjacent to each pixel local region in the real-time frame surface image data, comparing the surface image texture feature set of each pixel local region in the real-time frame surface image data with the surface image texture feature set of the adjacent other pixel local region corresponding to each pixel local region, and obtaining surface image abnormal feature similarity of each pixel local region and the adjacent other pixel local region.

[0029] Further, based on the deep learning algorithm and according to the texture feature set of the historical acquisition period, a rapid feature analysis model is constructed, and the process of rapidly identifying the abnormal type of the corresponding region photovoltaic panel comprises:

[0030] obtaining a plurality of sets of historical acquisition period thermal imaging texture feature sets and surface image texture feature sets; obtaining a plurality of sets of historical acquisition period thermal imaging abnormal feature similarity and surface image abnormal feature similarity according to the thermal imaging texture feature set and the surface image texture feature set;

[0031] constructing a training sample set according to the thermal imaging abnormal feature similarity, the surface image abnormal feature similarity of a plurality of sets of historical acquisition periods and the texture feature similarity threshold of different abnormal types;

[0032] constructing a standard analysis model based on the deep learning algorithm; and inputting the training sample set into the standard analysis model to train the standard analysis model, and recording the standard analysis model after training as a rapid feature analysis model, and then rapidly identifying the abnormal type of the corresponding region photovoltaic panel.

[0033] The second aspect of the present application also provides an intelligent analysis system for a UAV inspection picture, which comprises a data acquisition module, a data processing module, an intelligent analysis module, a rapid processing module and a data storage module.

[0034] The data acquisition module is used for acquiring an inspection video picture during real-time inspection of the UAV.

[0035] The data processing module is used for frame period splitting according to the acquired inspection video picture, and obtaining corresponding real-time frame image data.

[0036] The intelligent analysis module is used for feature extraction of the real-time frame image data, obtaining a texture feature set of the corresponding real-time frame image data; similarity analysis of the texture feature set, obtaining an abnormal type of the corresponding region photovoltaic panel, and then executing a corresponding abnormal processing scheme.

[0037] The rapid processing module is based on a deep learning algorithm and constructs a rapid feature analysis model according to a texture feature set of a historical acquisition cycle, and then rapidly identifies an abnormal type of the corresponding regional photovoltaic panel, so as to timely maintain the photovoltaic power station.

[0038] The data storage module is used for storing real-time frame thermal imaging image data of the abnormal regional photovoltaic panel.

[0039] Compared with the prior art, the beneficial effects of the present application are: the present application effectively solves many deficiencies in the existing photovoltaic panel inspection technology through the specifically designed inspection picture intelligent analysis method and system: first, through the dynamically adaptive frame period splitting mode, the effectiveness and rationality of the real-time frame image data are ensured, the information redundancy or missing problem caused by the fixed splitting period is avoided, and a reliable data foundation is laid for subsequent feature extraction and abnormality recognition; second, the construction of the multi-dimensional texture feature set and the targeted similarity analysis comprehensively cover the thermal and optical features of the photovoltaic panel abnormality, significantly improve the accuracy of abnormal type recognition, and reduce the missed detection and false detection; third, the deep learning rapid feature analysis model based on historical data greatly improves the response speed of abnormality recognition, and helps the photovoltaic power station to realize timely maintenance; fourth, the integrated system module integrates the data acquisition, processing, analysis, rapid identification and storage functions, simplifies the inspection and operation process, improves the overall operation efficiency and ease of use, and provides effective support for the intelligent inspection of large-scale photovoltaic power stations. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0041] Figure 1 It is a step schematic diagram of a kind of inspection picture intelligent analysis method for unmanned aerial vehicle.

[0042] Figure 2 It is a module schematic diagram of a kind of inspection picture intelligent analysis system for unmanned aerial vehicle. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described in detail below. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope protected by the present application.

[0044] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and in the above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such a process, method, product, or apparatus.

[0045] As shown in Figure 1 A method for intelligent analysis of inspection pictures of a UAV, the method comprising the following steps:

[0046] Step 1: frame period splitting is performed according to an inspection video picture collected by real-time inspection of a UAV, to obtain corresponding real-time frame image data;

[0047] Step 2: feature extraction is performed on the real-time frame image data, to obtain a texture feature set of the corresponding real-time frame image data; similarity analysis is performed on the texture feature set, to obtain an abnormal type of a corresponding regional photovoltaic panel, and then a corresponding abnormal processing scheme is executed;

[0048] Step 3: a rapid feature analysis model is constructed based on a deep learning algorithm and according to a texture feature set of a historical collection period, and then an abnormal type of the corresponding regional photovoltaic panel is rapidly identified, so as to facilitate timely maintenance of a photovoltaic power station.

[0049] It should be further noted that, in the specific implementation process, the specific process of frame period splitting according to an inspection video picture collected by real-time inspection of a UAV to obtain corresponding real-time frame image data comprises:

[0050] Optionally, in the embodiments of the present application, according to a preset inspection route, a UAV synchronously and in real time collects inspection video pictures of corresponding photovoltaic panels on the inspection route through a camera sensor carried by the UAV, the inspection video pictures comprising inspection thermal imaging video pictures and inspection visual video pictures; a collection period is set, the collection period referring to a time length corresponding to complete collection of an inspection video picture of a regional photovoltaic panel, a unique identifier being matched to the inspection video picture collected in each collection period, and the inspection video picture corresponding to each collection period being obtained through the unique identifier.

[0051] It should be noted that the camera sensor described above includes but is not limited to various different types of cameras. The same collection cycle corresponds to a unique identifier of the inspection thermal imaging video picture and the inspection visual video picture.

[0052] It should be further noted that in the specific implementation process, the specific process of frame period splitting of the inspection thermal imaging video picture to obtain the real-time frame thermal imaging image data includes:

[0053] Optionally, in the embodiment of the present application, according to the unique identifier, the corresponding inspection thermal imaging video picture is obtained, and based on the size data of each photovoltaic panel on the corresponding inspection route and the real-time flight speed of the unmanned aerial vehicle, the collection cycle of the corresponding inspection thermal imaging video picture is divided into sub-collection cycles;

[0054] A real-time flight speed threshold is set. If the real-time flight speed of the unmanned aerial vehicle in inspecting the corresponding photovoltaic panel is greater than the real-time flight speed threshold, the sub-collection cycle corresponding to the photovoltaic panel is frame period split according to the high-speed inspection frame splitting period. If the real-time flight speed of the unmanned aerial vehicle in inspecting the corresponding photovoltaic panel is less than or equal to the real-time flight speed threshold, the sub-collection cycle corresponding to the photovoltaic panel is frame period split according to the low-speed inspection frame splitting period.

[0055] According to the high-speed inspection frame splitting period and the low-speed inspection frame splitting period, the real-time frame thermal imaging image data under the corresponding flight state is obtained.

[0056] It should be noted that the inspection route is determined by the geometric center of each photovoltaic panel, which is not a supplement to the content of the present application. Since the unmanned aerial vehicle is affected by factors such as wind speed during inspection, the flight speed of each photovoltaic panel is not consistent, and the average speed of the unmanned aerial vehicle in inspecting the photovoltaic panel is taken as the real-time flight speed.

[0057] It should be further noted that in the specific implementation process, the specific process of frame period splitting of the inspection visual video picture to obtain the real-time frame surface image data includes:

[0058] Optionally, in the embodiment of the present application, according to the unique identifier, the corresponding inspection visual video picture is obtained, and based on the size data of each photovoltaic panel on the corresponding inspection route and the real-time flight speed of the unmanned aerial vehicle, the collection cycle of the corresponding inspection visual video picture is divided into sub-collection cycles;

[0059] setting a real-time flight speed threshold; if the real-time flight speed of the UAV when inspecting the corresponding photovoltaic panel is greater than the real-time flight speed threshold, then the sub-acquisition period corresponding to the photovoltaic panel is frame period split according to the high-speed inspection frame split period; if the real-time flight speed of the UAV when inspecting the corresponding photovoltaic panel is less than or equal to the real-time flight speed threshold, then the sub-acquisition period corresponding to the photovoltaic panel is frame period split according to the low-speed inspection frame split period;

[0060] According to the high-speed inspection frame split period and the low-speed inspection frame split period, real-time frame surface image data corresponding to the flight state is obtained.

[0061] It needs to be further explained that, in the specific implementation process, the specific process of performing feature extraction on the real-time frame image data to obtain a texture feature set corresponding to the real-time frame image data includes:

[0062] Optionally, in the embodiment of the present application, the real-time frame image data includes real-time frame surface image data and real-time frame thermal imaging image data, and a texture feature set corresponding to the real-time frame image data is obtained based on temperature feature extraction technology and a gray level co-occurrence matrix; the texture feature set includes a thermal imaging texture feature set and a surface image texture feature set.

[0063] The different real-time frame image data of the same photovoltaic panel are respectively subjected to feature extraction, the texture feature values of each real-time frame image data are obtained, and the mean value of the texture feature values corresponding to the different real-time frame image data is taken as the texture feature set of the photovoltaic panel.

[0064] It needs to be further explained that, in the specific implementation process, the specific process of performing feature extraction on the real-time frame thermal imaging image data to obtain a thermal imaging texture feature set corresponding to the real-time frame thermal imaging image data includes:

[0065] Optionally, in the embodiment of the present application, the real-time frame thermal imaging image data is first subjected to gray scale processing based on temperature feature extraction technology to obtain a gray scale value corresponding to a pixel point, and the gray scale value is converted into an absolute temperature value;

[0066] Based on temperature feature extraction technology, the absolute temperature values of the pixel points corresponding to the real-time frame thermal imaging image data are subjected to standardization processing to obtain a real-time frame thermal imaging image data temperature standard deviation.

[0067] A temperature anomaly threshold is set, the absolute temperature values of the pixel points corresponding to the real-time frame thermal imaging image data are respectively subjected to threshold judgment, and then a high-temperature pixel ratio corresponding to the real-time frame thermal imaging image data is counted.

[0068] The width direction parallel to the real-time frame thermal imaging image data is denoted as x direction, and the length direction perpendicular to the x direction is denoted as y direction. the height direction of the real-time frame thermal imaging image data is denoted as the temperature gradient in the height direction of the real-time frame thermal imaging image data is calculated based on a Sobel operator the temperature gradient in the height direction, the temperature gradient in the height direction ;

[0069] It should be noted that the temperature gradient in the height direction is: ; wherein represents a horizontal direction Sobel kernel; is an absolute temperature value of the pixel point ; the temperature gradient in the height direction is positive, indicating that the temperature on the right side of the pixel is higher than that on the left side; the temperature gradient in the height direction is negative, indicating that the temperature on the left side is higher than that on the right side; the temperature gradient in the height direction is greater, the temperature change in the height direction is more intense.

[0070] The temperature gradient in the height direction is: ; wherein represents a vertical direction Sobel kernel; the temperature gradient in the height direction is positive, indicating that the temperature below the pixel is higher than that above the pixel; the temperature gradient in the height direction is negative, indicating that the temperature above the pixel is higher than that below the pixel; the greater the absolute value, the temperature change in the height direction is more intense.

[0071] Optionally, in the embodiments of the present application, the high-temperature pixel proportion, the temperature standard deviation, the temperature gradient in the height direction and the temperature gradient in the height direction are denoted as a thermal imaging texture feature set corresponding to the real-time frame thermal imaging image data.

[0072] It should be noted that the thermal imaging texture feature set includes but is not limited to the high-temperature pixel proportion, the temperature standard deviation, the temperature gradient in the height direction and the temperature gradient in the height direction.

[0073] It should be further noted that, in the specific implementation process, the specific process of performing feature extraction on the real-time frame surface image data to obtain a surface image texture feature set corresponding to the real-time frame surface image data includes:

[0074] ​Optionally, in the embodiments of the present application, for the acquired real-time frame surface image data, the gray level co-occurrence matrix around each pixel point of the real-time frame surface image data is calculated in a sliding window manner, and specific details are as follows: the gray levels of the real-time frame surface image data are quantized into discrete gray levels, the gray value range is divided into several levels, for example, an 8-bit image can be divided into 16, 32, 64 levels, and parameters required for defining the gray level co-occurrence matrix, including distance and direction, are determined according to actual application requirements, and specific steps are not described herein again. Subsequently, a series of surface image texture features are extracted based on the calculated gray level co-occurrence matrix, and a surface image texture feature set is constructed, and the surface image texture features in the surface image texture feature set include but are not limited to:

[0075] Contrast: describes the statistical characteristics of the contrast of different gray level pixels in the real-time frame surface image data, is a feature for measuring the roughness of image texture, reflects the definition and texture groove depth of the image, the deeper the texture groove, the greater the contrast, and the clearer the visual effect, and vice versa, the smaller the contrast, the shallower the groove, and the effect is blurred;

[0076] Correlation: describes the correlation degree of different gray level pixels in the real-time frame surface image data, which measures the similarity of the spatial gray level co-occurrence matrix elements in the row or column direction. Therefore, the correlation value reflects the local gray correlation in the real-time frame surface image data. When the matrix element values are uniform and equal, the correlation value is large, and vice versa. If the matrix element values differ greatly, the correlation value is small. If the real-time frame surface image data has a horizontal direction texture, the correlation of the horizontal direction matrix is greater than that of the remaining matrices.

[0077] Energy: describes the uniformity of the pixel gray scale distribution in the real-time frame surface image data, measures the randomness contained in the real-time frame surface image data, and reflects the complexity of the real-time frame surface image data. When all values of the co-occurrence matrix are equal or the pixel values exhibit maximum randomness, the entropy is maximum.

[0078] Homogeneity: describes the similarity of the gray levels of adjacent pixels in the real-time frame surface image data; reflects the homogeneity of the texture of the real-time frame surface image data, and measures how much the local changes of the texture of the real-time frame surface image data. The larger the value is, the more uniform the different regions of the texture of the real-time frame surface image data are.

[0079] Entropy: describes the uncertainty degree of the texture of the real-time frame surface image data, measures the randomness contained in the real-time frame surface image data, and reflects the complexity of the real-time frame surface image data. When all values of the co-occurrence matrix are equal or the pixel values exhibit maximum randomness, the entropy is maximum.

[0080] Inverse divergence: reflects the clarity and regularity of the texture of the real-time frame surface image data, and the texture is clear and regular.

[0081] It should be further explained that, in the specific implementation process, the specific process of performing similarity analysis on the texture feature set to obtain the abnormal type of the corresponding regional photovoltaic panel, and then executing the corresponding abnormal processing scheme includes:

[0082] Optionally, in the embodiment of the present application, according to the similarity analysis on the thermal imaging texture feature set and the surface image texture feature set respectively, the corresponding abnormal feature similarity is obtained; the mean value of the abnormal feature similarity of the corresponding photovoltaic panel is taken, and is respectively recorded as the thermal imaging abnormal feature similarity mean value and the surface image abnormal feature similarity mean value.

[0083] It should be noted that the abnormal feature similarity includes the thermal imaging abnormal feature similarity and the surface image abnormal feature similarity.

[0084] The texture feature similarity threshold of different abnormal types is preset, the thermal imaging abnormal feature similarity mean value and the surface image abnormal feature similarity mean value are sorted according to the size of the texture feature similarity threshold, the abnormal type with the maximum thermal imaging abnormal feature similarity mean value and surface image abnormal feature similarity mean value is taken, and the real-time frame surface image data and real-time frame thermal imaging image data of the corresponding sub-acquisition period are called to compare and judge the abnormal type, if the comparison and judgment is successful, the corresponding abnormal processing scheme is executed; if the comparison and judgment fails, the artificial audit judgment is performed.

[0085] It should be noted that the abnormal type of the corresponding regional photovoltaic panel includes but is not limited to shielding, dirt and rupture, etc. Each different abnormal type corresponds to a corresponding abnormal processing scheme.

[0086] It should be further explained that, in the specific implementation process, the specific process of performing similarity analysis on the thermal imaging texture feature set includes:

[0087] Optionally, in the embodiment of the present application, according to Temperature gradient in the direction And Temperature gradient in the direction , the gradient amplitude is calculated; the abnormal type is further judged according to the gradient amplitude.

[0088] It should be noted that the gradient amplitude is:

[0089] ; wherein, is the gradient amplitude.

[0090] It is further illustrated that the gradient amplitude reflects the degree of temperature change; for example, the edge of the hot spot: the gradient amplitude is large, and the radiation distribution presents a low center and a high edge. The boundary of the low-temperature area is shielded: the gradient amplitude is small, and the transition presents a gentle feature.

[0091] According to the high-temperature pixel ratio, the temperature standard deviation, and the gradient amplitude , the corresponding thermal imaging anomaly feature similarity is composed.

[0092] It needs to be further explained that in the specific implementation process, the specific process of performing similarity analysis on the surface image texture feature set includes:

[0093] Optionally, in the embodiment of the present application, the local region adjacent to each pixel point in the real-time frame surface image data is obtained, and the surface image texture feature set of each pixel point local region in the real-time frame surface image data is compared with the surface image texture feature set of the adjacent other pixel point local region corresponding to the each pixel point local region, to obtain the surface image anomaly feature similarity between each pixel point local region and the adjacent other pixel point local region.

[0094] It needs to be explained that the calculation formula of the surface image anomaly feature similarity between each pixel point local region and the adjacent other pixel point local region is:

[0095] ;

[0096] Wherein, represents the surface image anomaly feature similarity between the i-th pixel point local region and the j-th pixel point local region, represents the contrast between the i-th pixel point local region and the j-th pixel point local region, represents the energy of the i-th pixel point local region, represents the energy of the j-th pixel point local region, represents the entropy of the i-th pixel point local region, represents the entropy of the j-th pixel point local region, represents the inverse variance of the i-th pixel point local region, represents the inverse variance of the j-th pixel point local region, 、 、 、 and represent weight factors.

[0097] It needs to be further explained that, in the specific implementation process, based on the deep learning algorithm, and according to the texture feature set of the historical acquisition period, a rapid feature analysis model is constructed, and then the specific process of quickly identifying the abnormal type of the corresponding regional photovoltaic panel includes:

[0098] Optionally, in the embodiment of the application, a plurality of sets of historical acquisition period thermal imaging texture feature sets and surface image texture feature sets are obtained; according to the thermal imaging texture feature sets and the surface image texture feature sets, a plurality of sets of historical acquisition period thermal imaging abnormal feature similarities and surface image abnormal feature similarities are obtained.

[0099] According to the thermal imaging abnormal feature similarities, the surface image abnormal feature similarities and the texture feature similarity thresholds of different abnormal types of a plurality of sets of historical acquisition periods, the specific process of constructing the training sample set includes:

[0100] Grouping and labeling the thermal imaging abnormal feature similarities, the surface image abnormal feature similarities and the texture feature similarity thresholds of different abnormal types of a plurality of sets of historical acquisition periods, denoted as is a natural number;

[0101] The thermal imaging abnormal feature similarities, the surface image abnormal feature similarities and the texture feature similarity thresholds of different abnormal types of a plurality of sets of historical acquisition periods are used as sample data, and is a natural number less than and the sample data mean is obtained by using the sample data, denoted as sample set;

[0102] The thermal imaging abnormal feature similarities, the surface image abnormal feature similarities and the texture feature similarity thresholds of different abnormal types of the remaining sets of historical acquisition periods are used as a test set; according to the sample set and the test set, a training sample set is formed;

[0103] Based on the deep learning algorithm, a standard analysis model is constructed; and the training sample set is input into the standard analysis model, the standard analysis model is trained, and the trained standard analysis model is denoted as a rapid feature analysis model, and then the abnormal type of the corresponding regional photovoltaic panel is quickly identified.

[0104] It should be noted that the deep learning algorithm can adopt, but is not limited to, convolutional neural network algorithm, recurrent neural network, long short-term memory network, etc.

[0105] As shown in Figure 2 , a kind of for unmanned plane's inspection picture intelligent analysis system, the system includes: data acquisition module, data processing module, intelligent analysis module, rapid processing module and data storage module;

[0106] ​The data acquisition module is configured to acquire a real-time inspection video picture during the real-time inspection of the UAV.

[0107] The data processing module is configured to perform frame period splitting on the acquired real-time inspection video picture to obtain corresponding real-time frame image data.

[0108] The intelligent analysis module is configured to perform feature extraction on the real-time frame image data to obtain a texture feature set of the real-time frame image data, perform similarity analysis on the texture feature set, obtain an abnormal type of the corresponding regional photovoltaic panel, and then execute a corresponding abnormal processing scheme.

[0109] The fast processing module is configured to construct a fast feature analysis model based on a deep learning algorithm and according to a texture feature set of a historical acquisition period, and then quickly identify an abnormal type of the corresponding regional photovoltaic panel, so as to facilitate timely maintenance of the photovoltaic power station.

[0110] The data storage module is configured to store real-time frame image data of the abnormal regional photovoltaic panel.

[0111] Optionally, in the embodiment, a person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-described embodiments can be completed by programs instructing the hardware of the terminal device, and the programs can be stored in a computer-readable storage medium, and the storage medium can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0112] The serial numbers of the above-described embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0113] The integrated units in the above-described embodiments can be stored in the above-described computer-readable storage medium if the integrated units are realized in the form of software function units and sold or used as independent products. Based on such understanding, the technical solutions of the present application can be embodied in the form of software products, the computer software products are stored in the storage medium, and include a plurality of instructions for causing one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0114] In the above-described embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0115] In several embodiments provided in the present application, it should be understood that the disclosed application can be implemented in other manners. Of course, the embodiments described above are merely schematic and the division of units is only a logical function division. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, units or modules, and can be in electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place, or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0117] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0118] The above is only the preferred embodiment of the present application. It should be pointed out that for ordinary skilled persons in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method for intelligent analysis of inspection images from unmanned aerial vehicles (UAVs), characterized in that, The method includes: The real-time frame image data is obtained by splitting the inspection video footage collected by the drone in real time. Feature extraction is performed on the real-time frame image data to obtain the texture feature set corresponding to the real-time frame image data, including: Based on temperature feature extraction technology and gray-level co-occurrence matrix, a texture feature set corresponding to real-time frame image data is obtained; the texture feature set includes thermal imaging texture feature set and surface image texture feature set. Feature extraction is performed on different real-time frame image data of the same photovoltaic panel to obtain the texture feature value of each real-time frame image data, and the average of the corresponding texture feature values ​​of different real-time frame image data is taken as the texture feature set of the photovoltaic panel. Similarity analysis is performed on the texture feature set to obtain the anomaly type of the photovoltaic panel in the corresponding region, and then the corresponding anomaly handling scheme is executed, including: By performing similarity analysis on the thermal imaging texture feature set and the surface image texture feature set respectively, the corresponding abnormal feature similarity is obtained; the mean of the abnormal feature similarity of the corresponding photovoltaic panel is taken and recorded as the mean of thermal imaging abnormal feature similarity and the mean of surface image abnormal feature similarity respectively; the abnormal feature similarity includes thermal imaging abnormal feature similarity and surface image abnormal feature similarity. A preset texture feature similarity threshold is set for different anomaly types. Based on the texture feature similarity threshold, the mean similarity of thermal imaging anomaly features and the mean similarity of surface image anomaly features are sorted by size. The anomaly type with the largest mean similarity of thermal imaging anomaly features and the mean similarity of surface image anomaly features is selected. Real-time frame surface image data and real-time frame thermal imaging image data of the corresponding sub-acquisition period are retrieved and compared to determine the anomaly type. If the comparison is successful, the corresponding anomaly handling scheme is executed; if the comparison fails, it is judged by manual review. Based on deep learning algorithms and texture feature sets from historical collection periods, a fast feature analysis model is constructed to quickly identify the anomaly types of photovoltaic panels in the corresponding area, so as to facilitate timely maintenance of photovoltaic power stations.

2. The intelligent analysis method for inspection images of unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of obtaining corresponding real-time frame image data by splitting the inspection video footage collected by the drone in real-time includes: According to the preset inspection route, the drone synchronously and in real time collects inspection video images of the corresponding photovoltaic panels along the inspection route through its onboard camera sensors. The inspection video images include inspection thermal imaging video images and inspection visualization video images. The acquisition cycle is set, and a unique identifier is matched to the inspection video images acquired in each acquisition cycle.

3. The intelligent analysis method for inspection images of unmanned aerial vehicles (UAVs) according to claim 2, characterized in that, The process of splitting the inspection thermal imaging video and the inspection visualization video into frames to obtain real-time frame thermal imaging image data and real-time frame surface image data includes: Based on the unique identifier, the corresponding inspection thermal imaging video and inspection visualization video are acquired. Based on the size data of each photovoltaic panel along the corresponding inspection route and the real-time flight speed of the drone, the acquisition period for the corresponding inspection thermal imaging video and inspection visualization video is divided into... Individual collection cycle; Set a real-time flight speed threshold; if the real-time flight speed of the UAV inspecting the corresponding photovoltaic panel is greater than the real-time flight speed threshold, then the sub-collection period corresponding to the photovoltaic panel is split into frame periods according to the high-speed inspection frame splitting period; if the real-time flight speed of the UAV inspecting the corresponding photovoltaic panel is less than or equal to the real-time flight speed threshold, then the sub-collection period corresponding to the photovoltaic panel is split into frame periods according to the low-speed inspection frame splitting period. Based on the high-speed inspection frame splitting period and the low-speed inspection frame splitting period, real-time frame thermal imaging image data and real-time frame surface image data of the corresponding flight state are obtained.

4. The intelligent analysis method for inspection images of unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, The process of extracting features from the real-time frame thermal imaging image data and the real-time frame surface image data to obtain the corresponding thermal imaging texture feature set and surface image texture feature set includes: Based on temperature feature extraction technology, the grayscale value of the corresponding pixel is obtained and the grayscale value is converted into an absolute temperature value; based on temperature feature extraction technology, the temperature standard deviation of the corresponding real-time frame thermal imaging image data is obtained; a temperature anomaly threshold is set, and the absolute temperature value of the corresponding pixel of the real-time frame thermal imaging image data is judged by the threshold, thereby calculating the proportion of high temperature pixels in the corresponding real-time frame thermal imaging image data. The width direction parallel to the real-time frame thermal imaging image data is denoted as... Direction; the height direction parallel to the real-time frame thermal imaging image data is denoted as... Direction: Based on the Sobel operator, calculate real-time frame thermal imaging image data. Temperature gradient in direction and Temperature gradient in the direction; The percentage of high-temperature pixels, temperature standard deviation, Temperature gradient in direction and The temperature gradient in the direction is denoted as the thermal imaging texture feature set of the corresponding real-time frame thermal imaging image data; For the acquired real-time frame surface image data, the gray-level co-occurrence matrix around each pixel of the real-time frame surface image data is calculated by using a sliding window. Based on the calculated gray-level co-occurrence matrix, a series of surface image texture features are extracted to construct a surface image texture feature set. The surface image texture features in the surface image texture feature set include contrast, correlation, energy, homogeneity, entropy, and inverse variance.

5. The intelligent analysis method for inspection images of unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, The process of performing similarity analysis on the thermal imaging texture feature set and the surface image texture feature set includes: according to Temperature gradient in direction and The temperature gradient in the direction is calculated, and the gradient magnitude is further determined based on the gradient magnitude. The anomaly type is then determined, and the corresponding thermal imaging anomaly feature similarity is formed based on the proportion of high-temperature pixels, the temperature standard deviation, and the gradient magnitude. Obtain the local regions of other pixels adjacent to the local regions of each pixel in the real-time frame surface image data. Compare the similarity between the surface image texture feature set of each pixel's local region and the surface image texture feature set of the other adjacent local regions of each pixel's local region to obtain the surface image abnormality feature similarity between each pixel's local region and the other adjacent local regions of each pixel.

6. The intelligent analysis method for inspection images of unmanned aerial vehicles (UAVs) according to claim 5, characterized in that, The process of quickly identifying anomaly types of photovoltaic panels in a corresponding area by constructing a fast feature analysis model based on deep learning algorithms and texture feature sets from historical collection periods includes: Acquire several sets of thermal imaging texture feature sets and surface image texture feature sets from historical acquisition periods; based on the thermal imaging texture feature sets and surface image texture feature sets, obtain the similarity of thermal imaging anomaly features and surface image anomaly features from several sets of historical acquisition periods. A training sample set is constructed based on the similarity of thermal imaging anomaly features, surface image anomaly features, and texture feature similarity thresholds for different anomaly types from several historical acquisition cycles. A standard analysis model is constructed based on deep learning algorithms. The training sample set is then input into the standard analysis model to train it. The trained standard analysis model is recorded as a fast feature analysis model, which can then quickly identify the anomaly types of photovoltaic panels in the corresponding area.

7. An intelligent analysis system for inspection images of unmanned aerial vehicles (UAVs), implementing the intelligent analysis method for inspection images of UAVs as described in any one of claims 1 to 6, characterized in that, The system includes: a data acquisition module, a data processing module, an intelligent analysis module, a rapid processing module, and a data storage module; The data acquisition module is used to acquire inspection video footage during real-time drone inspections. The data processing module performs frame periodicity segmentation based on the collected inspection video footage to obtain the corresponding real-time frame image data. The intelligent analysis module is used to extract features from the real-time frame image data to obtain the texture feature set of the corresponding real-time frame image data, including: Based on temperature feature extraction technology and gray-level co-occurrence matrix, a texture feature set corresponding to real-time frame image data is obtained; the texture feature set includes thermal imaging texture feature set and surface image texture feature set. Feature extraction is performed on different real-time frame image data of the same photovoltaic panel to obtain the texture feature value of each real-time frame image data. The average of the corresponding texture feature values ​​of different real-time frame image data is taken as the texture feature set of the photovoltaic panel. Similarity analysis is performed on the texture feature set to obtain the anomaly type of the photovoltaic panel in the corresponding region, and then the corresponding anomaly handling scheme is executed, including: By performing similarity analysis on the thermal imaging texture feature set and the surface image texture feature set respectively, the corresponding abnormal feature similarity is obtained; the mean of the abnormal feature similarity of the corresponding photovoltaic panel is taken and recorded as the mean of thermal imaging abnormal feature similarity and the mean of surface image abnormal feature similarity respectively; the abnormal feature similarity includes thermal imaging abnormal feature similarity and surface image abnormal feature similarity. A preset texture feature similarity threshold is set for different anomaly types. Based on the texture feature similarity threshold, the mean similarity of thermal imaging anomaly features and the mean similarity of surface image anomaly features are sorted by size. The anomaly type with the largest mean similarity of thermal imaging anomaly features and the mean similarity of surface image anomaly features is selected. Real-time frame surface image data and real-time frame thermal imaging image data of the corresponding sub-acquisition period are retrieved and compared to determine the anomaly type. If the comparison is successful, the corresponding anomaly handling scheme is executed; if the comparison fails, it is judged by manual review. The rapid processing module, based on deep learning algorithms and texture feature sets from historical collection periods, constructs a rapid feature analysis model to quickly identify the anomaly types of photovoltaic panels in the corresponding area, so as to facilitate timely maintenance of the photovoltaic power station. The data storage module is used to store real-time frame thermal imaging image data of photovoltaic panel anomalies in the corresponding area.

Images