Unmanned aerial vehicle inspection photo processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an unmanned aerial vehicle inspection photo processing method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring EXIF information of an exchangeable image file of a target inspection photo, wherein the EXIF information comprises first position information of an unmanned aerial vehicle when the target inspection photo is shot; determining a target inspection object from a plurality of inspection objects based on the first position information; and naming the target inspection photo according to the identifier of the target inspection object. By adopting the method, the inspection photo processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, readable storage medium and program product for processing drone inspection photos. Background Art

[0002] With the continuous development and expansion of the power system, some power equipment in the power system has become increasingly large and widely distributed, such as overhead transmission lines (including towers) and wind turbine blades in wind farms. In order to ensure the safe and stable operation of power equipment, regular inspections have become a crucial maintenance measure.

[0003] Drone inspection technology has been widely used in power system inspections due to its advantages such as high efficiency, flexibility, and ability to obtain information in hard-to-reach areas. During the inspection process, drones can take a large number of inspection photos.

[0004] In traditional technology, these inspection photos are named manually, but this manual processing method has the problem of low efficiency in inspection photo processing. Summary of the invention

[0005] Based on this, it is necessary to provide a drone inspection photo processing method, device, computer equipment, readable storage medium and program product that can improve the efficiency of inspection photo processing in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for processing drone inspection photos, the method comprising: obtaining EXIF ​​information of a target inspection photo, the EXIF ​​information including the first position information of the drone when taking the target inspection photo; based on the first position information, determining a target inspection object from multiple inspection objects; and naming the target inspection photo according to an identification of the target inspection object.

[0007] In one of the embodiments, based on the first position information, a target inspection object is determined from multiple inspection objects, including: determining an initial inspection object from multiple inspection objects; determining a distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object; if the distance value is less than or equal to a distance threshold, determining the initial inspection object as the target inspection object; if the distance value is greater than the distance threshold, re-executing the step of determining the initial inspection object from multiple inspection objects until the distance value is less than or equal to the distance threshold.

[0008] In one of the embodiments, before determining the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object, the method also includes: verifying the first position information based on a verification function; if the verification passes, executing the step of determining the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object; if the verification fails, repairing the first position information; correspondingly, determining the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object, including: determining the distance value between the drone and the initial inspection object based on the repaired first position information and the second position information of the initial inspection object.

[0009] In one of the embodiments, the first location information includes the longitude and latitude coordinates of the UAV, and the second location information of the target inspection object includes the longitude and latitude coordinates of the target inspection object; the method also includes: converting the longitude and latitude coordinates of the UAV and the longitude and latitude coordinates of the target inspection object respectively to obtain the xy coordinates of the UAV and the xy coordinates of the target inspection object in a plane rectangular coordinate system; determining the target orientation of the UAV relative to the target inspection object when taking the target inspection photo based on the xy coordinates of the UAV and the xy coordinates of the target inspection object; correspondingly, naming the target inspection photo according to the identification of the target inspection object, including: naming the target inspection photo according to the target orientation and the identification of the target inspection object.

[0010] In one of the embodiments, the target position of the drone relative to the target inspection object when taking a target inspection photo is determined based on the xy coordinates of the drone and the xy coordinates of the target inspection object, including: obtaining the xy coordinates of the first inspection object and the xy coordinates of the second inspection object, the first inspection object and the second inspection object being two inspection objects adjacent to the target inspection object; obtaining a first vector between the target inspection object and the first inspection object based on the xy coordinates of the target inspection object and the xy coordinates of the first inspection object; obtaining a second vector between the target inspection object and the second inspection object based on the xy coordinates of the target inspection object and the xy coordinates of the second inspection object; obtaining a third vector between the target inspection object and the drone based on the xy coordinates of the target inspection object and the xy coordinates of the drone; performing a cross product calculation on the first vector and the third vector to obtain a first cross product value; performing a cross product calculation on the second vector and the third vector to obtain a second cross product value; and determining the target position based on the sign of the first cross product value and the sign of the second cross product value.

[0011] In one embodiment, the method further includes: performing feature extraction on the target inspection photo to obtain a first feature vector of the target inspection photo; performing feature extraction on the reference inspection photo to obtain a second feature vector of the reference inspection photo; wherein the reference inspection photo is an inspection photo that is taken at a different time but the same location as the target inspection photo; performing similarity calculation on the first feature vector and the second feature vector to obtain the similarity between the target inspection photo and the reference inspection photo; based on the similarity, performing abnormality analysis on the inspection object in the target inspection photo, and outputting the analysis result.

[0012] In the second aspect, the present application also provides a drone inspection photo processing device, which includes: an acquisition module for acquiring the EXIF ​​information of the target inspection photo, the EXIF ​​information including the first position information of the drone when taking the target inspection photo; a determination module for determining the target inspection object from multiple inspection objects based on the first position information; and a naming module for naming the target inspection photo according to the identification of the target inspection object.

[0013] In one of the embodiments, the determination module is specifically used to determine an initial inspection object from multiple inspection objects; determine the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object; if the distance value is less than or equal to the distance threshold, determine the initial inspection object as the target inspection object; if the distance value is greater than the distance threshold, re-execute the step of determining the initial inspection object from multiple inspection objects until the distance value is less than or equal to the distance threshold.

[0014] In one of the embodiments, the determination module is also used to verify the first position information based on the verification function; if the verification passes, the step of determining the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object is executed; if the verification fails, the first position information is repaired; correspondingly, the determination module is specifically used to determine the distance value between the drone and the initial inspection object based on the repaired first position information and the second position information of the initial inspection object.

[0015] In one of the embodiments, the first position information includes the longitude and latitude coordinates of the UAV, and the second position information of the target inspection object includes the longitude and latitude coordinates of the target inspection object; the device also includes an orientation determination module, which is used to convert the longitude and latitude coordinates of the UAV and the longitude and latitude coordinates of the target inspection object respectively to obtain the xy coordinates of the UAV and the xy coordinates of the target inspection object in a plane rectangular coordinate system; according to the xy coordinates of the UAV and the xy coordinates of the target inspection object, determine the target orientation of the UAV relative to the target inspection object when taking the target inspection photo; correspondingly, a naming module is specifically used to name the target inspection photo according to the target orientation and the identification of the target inspection object.

[0016] In one of the embodiments, the orientation determination module is specifically used to obtain the xy coordinates of a first inspection object and the xy coordinates of a second inspection object, where the first inspection object and the second inspection object are two inspection objects adjacent to the target inspection object; obtain a first vector between the target inspection object and the first inspection object based on the xy coordinates of the target inspection object and the xy coordinates of the first inspection object; obtain a second vector between the target inspection object and the second inspection object based on the xy coordinates of the target inspection object and the xy coordinates of the second inspection object; obtain a third vector between the target inspection object and the drone based on the xy coordinates of the target inspection object and the xy coordinates of the drone; perform a cross product calculation on the first vector and the third vector to obtain a first cross product value; perform a cross product calculation on the second vector and the third vector to obtain a second cross product value; determine the target orientation based on the sign of the first cross product value and the sign of the second cross product value.

[0017] In one embodiment, the device also includes an analysis module, which is used to extract features from the target inspection photo to obtain a first feature vector of the target inspection photo; extract features from the reference inspection photo to obtain a second feature vector of the reference inspection photo; wherein the reference inspection photo is an inspection photo that is taken at a different time but the same location as the target inspection photo; calculate the similarity between the first feature vector and the second feature vector to obtain the similarity between the target inspection photo and the reference inspection photo; based on the similarity, perform an abnormality analysis on the inspection object in the target inspection photo and output the analysis result.

[0018] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0019] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0020] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0021] The above-mentioned drone inspection photo processing method, device, computer equipment, readable storage medium and program product obtain the EXIF ​​information of the target inspection photo, which includes the first position information of the drone when taking the target inspection photo, and then determine the target inspection object from multiple inspection objects based on the first position information. Then, the target inspection photo is named according to the identification of the target inspection object to realize automatic processing of the inspection photo, thereby improving the processing efficiency of the inspection photo. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] FIG1 is a diagram of an application environment of a method for processing drone inspection photos in one embodiment;

[0024] FIG2 is a schematic diagram of a process of a method for processing drone inspection photos in one embodiment;

[0025] FIG3 is a flow chart of a method for processing drone inspection photos in another embodiment;

[0026] FIG4 is a schematic diagram showing the principle of a method for processing drone inspection photos in one embodiment;

[0027] FIG5 is a structural block diagram of a drone inspection photo processing device in one embodiment;

[0028] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] The drone inspection photo processing method provided in the embodiment of the present application can be applied to the application environment as shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains the exchangeable image file EXIF ​​information of the target inspection photo, and the EXIF ​​information includes the first position information of the drone when taking the target inspection photo; based on the first position information, the target inspection object is determined from multiple inspection objects; and the target inspection photo is named according to the identifier of the target inspection object.

[0031] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0032] In an exemplary embodiment, as shown in FIG2 , a method for processing drone inspection photos is provided, which is described by taking the method applied to the terminal in FIG1 as an example, and includes the following steps 201 to 203, wherein:

[0033] Step 201, obtaining EXIF ​​information of the target inspection photo, wherein the EXIF ​​information includes the first position information of the drone when taking the target inspection photo.

[0034] Among them, target inspection photos refer to inspection photos to be named taken by drones, and the inspection photos include inspection objects, which can be transmission lines or wind turbine blades in wind farms; transmission lines include at least one of poles, towers, insulators and wires.

[0035] EXIF (Exchangeable Image File) information is a series of information collected by the camera on the drone during the shooting process, which is placed in the header of the JPEG or TIFF file. The first location information includes the geodetic coordinates when the drone takes the inspection photo of the target, that is, the longitude, latitude and altitude of the drone.

[0036] For the inspection of power transmission lines, the drone flies according to the predetermined inspection route and takes photos of the inspection objects. During the shooting process, the EXIF ​​information of each inspection photo taken will be automatically recorded. Among them, the drone used to shoot the inspection object is a drone with a high-precision GPS (Global Positioning System), a high-definition camera and sufficient range. In addition, the parameters of the camera on the drone can be set in advance to obtain clear inspection photos. During the inspection process, the drone will also use GPS to collect the second position information of each inspection object. The second position information includes the geodetic coordinates of the inspection object (that is, GPS coordinates), that is, the longitude, latitude and altitude of the inspection object. It should be noted that the second position information can be the position information obtained after filtering.

[0037] For the inspection of wind turbine blades, the drone is equipped with a high-definition camera and flies around the wind turbine according to a predetermined flight path. The drone takes pictures of the wind turbine blades at different angles and distances to obtain a large number of photos containing detailed information about the blades. These photos cover the overall appearance, surface texture, and surrounding environment of the blades, ensuring comprehensive coverage of the wind turbine blades and providing a sufficient data source for accurate detection of blade status.

[0038] Step 202: Determine a target inspection object from multiple inspection objects based on the first location information.

[0039] The target inspection object refers to the inspection object that is the same as the inspection object in the target inspection photo.

[0040] Optionally, based on the first position information and the second position information of each inspection object, a target inspection object is determined from multiple inspection objects.

[0041] Step 203: Name the target inspection photo according to the identifier of the target inspection object.

[0042] Among them, the identification of the target inspection object is unique.

[0043] In one example, the transmission lines include ordinary lines and multi-circuit AC / DC lines on the same tower. For ordinary lines, the target orientation relative to the target inspection object when the drone takes the target inspection photo can be ignored, and the target inspection photo can be named directly according to the identification of the target inspection object.

[0044] For multi-circuit AC / DC lines on the same tower, it may be necessary to consider the target orientation relative to the target inspection object when the drone takes the target inspection photo, and name the target inspection photo according to the target orientation and the identification of the target inspection object.

[0045] In another example, for wind turbine blades in a wind farm, the target orientation relative to the target inspection object when the drone takes the target inspection photo may or may not be considered. This can be set as needed and is not limited here.

[0046] In addition, when naming the target inspection photo, at least one of the following can be considered: the shooting time of the target inspection photo, the route mark of the drone when shooting the target inspection photo, the wind turbine mark, and the category number. For example, the name of the wind turbine blade is in the format of "wind turbine mark-blade mark-shooting time-category number".

[0047] The above-mentioned drone inspection photo processing method obtains the EXIF ​​information of the target inspection photo, which includes the first position information of the drone when taking the target inspection photo, and then determines the target inspection object from multiple inspection objects based on the first position information. Then, the target inspection photo is named according to the identifier of the target inspection object to realize automatic processing of the inspection photo, thereby improving the processing efficiency of the inspection photo.

[0048] In one of the embodiments, determining a target inspection object from a plurality of inspection objects based on the first location information includes:

[0049] Step 11: determine an initial inspection object from multiple inspection objects.

[0050] Optionally, an inspection object may be randomly selected from a plurality of inspection objects as the initial inspection object, or the initial inspection object may be selected from a plurality of inspection objects in a preset order.

[0051] Step 12: Determine the distance value between the drone and the initial inspection object according to the first position information and the second position information of the initial inspection object.

[0052] Step 13, if the distance value is less than or equal to the distance threshold, the initial inspection object is determined as the target inspection object; if the distance value is greater than the distance threshold, the step of determining the initial inspection object from multiple inspection objects is re-executed until the distance value is less than or equal to the distance threshold.

[0053] The first position information is the geodetic coordinates when the drone takes the inspection photo i of the target , you can use It indicates that, , , They respectively represent the longitude, latitude and altitude when the UAV takes the inspection photo i of the target.

[0054] The second position information of the initial inspection object can be the geodetic coordinates of tower j , you can use It indicates that, , , Represent the longitude, latitude and height of tower j respectively.

[0055] The first position information and the second position information are respectively converted into coordinates, for example, into a spatial rectangular coordinate system, to obtain the first spatial coordinate corresponding to the target inspection photo. The second spatial coordinate corresponding to the initial inspection object .

[0056] in, , , ; , , . R represents the earth radius.

[0057] Calculate the distance between the first space coordinate and the second space coordinate for:

[0058]

[0059] The distance between the first spatial coordinate and the second spatial coordinate is the distance value between the UAV and the initial inspection object.

[0060] If the distance value is less than or equal to the distance threshold, it means that the target inspection object in the target inspection photo is the initial inspection object selected this time, that is, tower j; if the distance value is greater than the distance threshold, it means that the target inspection object in the target inspection photo is not the initial inspection object selected this time, and the initial inspection needs to be reselected, that is, the step of determining the initial inspection object from multiple inspection objects is re-executed until the distance value is less than or equal to the distance threshold.

[0061] In one embodiment, before determining the distance value between the drone and the initial inspection object according to the first position information and the second position information of the initial inspection object, the method further includes:

[0062] The first position information is verified based on the verification function; if the verification passes, the step of determining the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object is executed; if the verification fails, the first position information is repaired.

[0063] The verification function can be:

[0064]

[0065] Where a and b are both verification parameters, which are obtained by comparing the error distribution of the initial EXIF ​​information data with the coordinate data in the actual EXIF ​​information. The verification parameter affects the slope of the verification function curve, which determines how fast the function value changes when the data approaches the verification threshold.

[0066] Optionally, the longitude, latitude and altitude in the first location information are respectively input into the verification function to obtain the longitude verification value, latitude verification value and altitude verification value. When the longitude verification value is greater than or equal to the longitude verification threshold, the latitude verification value is greater than or equal to the latitude verification threshold, and the altitude verification value is greater than or equal to the latitude verification threshold, the verification is passed; when the longitude verification value is less than the longitude verification threshold, the latitude verification value is less than the latitude verification threshold, and / or the altitude verification value is less than the latitude verification threshold, the verification fails. Among them, the longitude verification threshold, the latitude verification threshold and the altitude verification value are all obtained by statistically analyzing the coordinate data in the EXIF ​​information of a large number of historical inspection photos, and observing the distribution range and concentration trend of the data.

[0067] If the verification is successful, it means that the first location information is valid data; if the verification is unsuccessful, it means that the first location information is invalid data. When the first location information is invalid data, the first location information needs to be repaired.

[0068] The first location information may be repaired by interpolation repair based on historical data (repairing by finding data in historical coordinate data that is highly similar to the data as interpolation) or reference object repair (recognizing the relative position relationship between the landmark features with known precise coordinates in the photo, and correcting the invalid coordinates accordingly).

[0069] Correspondingly, determining the distance value between the drone and the initial inspection object according to the first position information and the second position information of the initial inspection object includes: determining the distance value between the drone and the initial inspection object according to the repaired first position information and the second position information of the initial inspection object. The specific implementation method of this step refers to the above-mentioned distance value calculation method, which will not be repeated here.

[0070] In one embodiment, the first location information includes the longitude and latitude coordinates of the drone, and the second location information of the target inspection object includes the longitude and latitude coordinates of the target inspection object; the method further includes:

[0071] Step 21 , convert the longitude and latitude coordinates of the UAV and the longitude and latitude coordinates of the target inspection object respectively to obtain the xy coordinates of the UAV and the xy coordinates of the target inspection object in a plane rectangular coordinate system.

[0072] Among them, the latitude and longitude coordinates of the drone can be used Indicates that the latitude and longitude coordinates of the target inspection object can be used express.

[0073] After coordinate conversion, the xy coordinates of the drone for:

[0074]

[0075]

[0076] After coordinate conversion, the xy coordinates of the target inspection object for:

[0077]

[0078]

[0079] in, and Represents the latitude and longitude coordinates of the reference point; R represents the earth radius.

[0080] Step 22, determining the target orientation of the drone relative to the target inspection object when taking the target inspection photo according to the xy coordinates of the drone and the xy coordinates of the target inspection object;

[0081] Optionally, step 22 includes the following steps 221 to 222.

[0082] Step 221 : obtaining the xy coordinates of a first inspection object and the xy coordinates of a second inspection object, where the first inspection object and the second inspection object are two inspection objects adjacent to the target inspection object.

[0083] Among them, the xy coordinates of the first inspection object are It means the previous inspection object of the target inspection object, such as the previous tower j-1 of tower j; the xy coordinates of the second inspection object are represented by It means the next inspection object after the target inspection object, for example, the next tower j+1 after tower j.

[0084] Step 222, based on the xy coordinates of the target inspection object and the xy coordinates of the first inspection object, obtain the first vector between the target inspection object and the first inspection object; based on the xy coordinates of the target inspection object and the xy coordinates of the second inspection object, obtain the second vector between the target inspection object and the second inspection object; based on the xy coordinates of the target inspection object and the xy coordinates of the drone, obtain the third vector between the target inspection object and the drone.

[0085] Among them, the first vector of the target inspection object and the first inspection object ; The second vector of the target inspection object and the second inspection object ; The third vector between the inspection target and the drone .

[0086] Step 223, performing a cross product calculation on the first vector and the third vector to obtain a first cross product value; performing a cross product calculation on the second vector and the third vector to obtain a second cross product value; and determining the target orientation according to the sign of the first cross product value and the sign of the second cross product value.

[0087] Among them, the first cross product value and the second cross product value They are:

[0088]

[0089]

[0090] If the first cross product value and the second cross product value have the same sign, the target orientation is the first orientation; if the first cross product value and the second cross product value have opposite signs, the target orientation is the second orientation. The first orientation and the second orientation are two opposite orientations of the target inspection object, and the first orientation and the second orientation can be represented by predetermined state values, for example, the first orientation can be represented by 1, and the second orientation can be represented by -1.

[0091] Correspondingly, the target inspection photo is named according to the identification of the target inspection object, including: naming the target inspection photo according to the target orientation and the identification of the target inspection object.

[0092] In addition, the target inspection photos can be classified based on the target orientation. For example, if the target inspection photos are photos taken of transmission lines, they can be classified into corresponding lines based on the target orientation; if the target inspection photos are photos taken of wind turbine blades, they can be classified into corresponding blades based on the target orientation.

[0093] In one embodiment, as shown in FIG3 , a method for processing drone inspection photos is provided. The method further includes:

[0094] Step 301, perform feature extraction on the target inspection photo to obtain a first feature vector of the target inspection photo; perform feature extraction on the reference inspection photo to obtain a second feature vector of the reference inspection photo, wherein the reference inspection photo is an inspection photo taken at a different time but at the same location as the target inspection photo.

[0095] For the target inspection photo being an inspection photo taken of a power transmission line, in an optional embodiment, in step 301, feature extraction is performed on the target inspection photo to obtain a first feature vector of the target inspection photo, including the following steps 3011a to 3014a, wherein:

[0096] Step 3011a: Obtain a first shooting position feature vector of the target inspection photo according to the first position information.

[0097] Among them, the first shooting position feature vector can be used express.

[0098] Step 3012a, extracting features of the geometric shapes of the tower outline and the line direction in the target inspection photo to obtain a first geometric shape feature; quantifying the first geometric shape feature to obtain a first geometric shape feature vector of the target inspection photo.

[0099] Among them, for the tower contour, its geometric shape characteristics such as roundness and rectangularity are quantified; for the line direction, its geometric shape characteristics such as angle and curvature are quantified.

[0100] The first geometric shape eigenvector can be used Represents, where n1 is the vector dimension of the first geometric shape feature vector. n1 can be 3.

[0101] Optionally, feature extraction of the geometric shapes of the tower contours and line directions in the target inspection photos may be performed using a contour extraction algorithm.

[0102] Step 3013a, extracting features of the textures of the tower surface and the insulator surface in the target inspection photo to obtain first texture features; quantifying the first texture features to obtain a first texture feature vector of the target inspection photo.

[0103] Among them, insulators are components for hanging and fixing wires. For the surface texture of the tower, its roughness, contrast and other characteristics are quantified; for the surface texture of the insulator, its texture direction, density and other characteristics are quantified.

[0104] Optionally, feature extraction of the texture of the tower surface and the insulator surface in the target inspection photo may be achieved by using an image feature extraction algorithm. Quantification of texture features may be achieved by using a texture analysis algorithm.

[0105] The first texture feature vector can be used Represents, where m1 is the vector dimension of the first texture feature vector. m1 can be 3.

[0106] Step 3014a, performing weighted summation on the first shooting position feature vector, the first geometric shape feature vector and the first texture feature vector to obtain a first feature vector of the target inspection photo.

[0107] Among them, the first shooting position feature vector Corresponding to the first weight ; The first geometric shape feature vector Corresponding to the second weight ; Texture feature vector Corresponding to the third weight The sum of the first weight, the second weight, and the third weight is 1. The first eigenvector .

[0108] The first shooting position feature is used to determine whether the shooting is of different angles for the same inspection object; the first geometric shape feature vector is used to determine whether the inspection object is deformed and displaced; and the first texture feature vector is used to determine the change in the surface state of the inspection object.

[0109] Similarly, for the target inspection photo being an inspection photo taken of a wind turbine blade, in an optional embodiment, feature extraction is performed on the target inspection photo in step 301 to obtain a first feature vector of the target inspection photo, including the following steps 3011b to 3014b, wherein:

[0110] Step 3011b: Obtain a second shooting position feature vector of the target inspection photo according to the first position information.

[0111] Among them, the second shooting position feature vector can be used express.

[0112] Step 3012b, extracting features of the geometric shape of the wind turbine blade in the target inspection photo to obtain a second geometric shape feature; quantifying the second geometric shape feature to obtain a second geometric shape feature vector of the target inspection photo.

[0113] Among them, the geometric shape of the fan blade in the target inspection photo is extracted by using an image processing algorithm to extract the geometric shape features such as the contour and curvature of the fan blade. The second geometric shape feature is quantified, which can be the curvature of a certain cross section of the fan blade, the curvature change rate of the edge of the fan blade, etc.

[0114] The second geometric shape eigenvector can be used Represents, where n2 is the vector dimension of the second geometric shape feature vector. n2 can be 3.

[0115] Step 3013b, extracting features of the surface texture of the wind turbine blade in the target inspection photo to obtain a second texture feature vector.

[0116] The feature extraction of the surface texture of the wind blade in the target inspection photo may be to extract texture features such as the roughness and texture direction of the surface of the wind blade.

[0117] The second texture feature vector can be used Represents, where m2 is the vector dimension of the first texture feature vector. m2 can be 3.

[0118] Step 3014b, performing weighted summation on the second shooting position feature vector, the second geometric shape feature vector, and the second texture feature vector to obtain a first feature vector of the target inspection photo.

[0119] Among them, the second shooting position feature vector Corresponding to the fourth weight ; The second geometric shape feature vector Corresponding to the fifth weight ; The second texture feature vector Corresponding to the sixth weight The sum of the fourth weight, the fifth weight, and the sixth weight is 1. The first eigenvector .

[0120] The second shooting position feature is used to determine whether the same wind blade is shot at different angles; the second geometric shape feature vector is used to determine whether the wind blade is deformed and displaced; and the second texture feature vector is used to determine changes in the surface state of the wind blade.

[0121] In an optional embodiment, in step 301, feature extraction is performed on the reference inspection photo to obtain a second feature vector of the reference inspection photo. The above steps 3011a to 3014a or 3011b to 3014b may be referred to and will not be repeated here.

[0122] Step 302, calculating the similarity between the first feature vector and the second feature vector to obtain the similarity between the target inspection photo and the reference inspection photo;

[0123] The similarity calculation may adopt a cosine similarity calculation method.

[0124] Similarity between target inspection photo i and reference inspection photo k for:

[0125]

[0126] in, represents the first eigenvector; Represents the first eigenvector. The similarity is between 0 and 1, where 0 means completely dissimilar and 1 means completely similar.

[0127] Step 303: perform abnormal analysis on the inspection object in the target inspection photo according to the similarity, and output the analysis result.

[0128] Optionally, if the similarity is greater than or equal to the abnormality judgment threshold, the inspection object in the target inspection photo is normal; if the similarity is less than the abnormality judgment threshold, the inspection object in the target inspection photo is abnormal.

[0129] Among them, the abnormal judgment threshold , represents the mean similarity; represents the similarity standard deviation; Represents the abnormality judgment coefficient.

[0130] Taking the inspection object as a wind turbine blade as an example, abnormal situations with similarity less than the abnormal judgment threshold may include blade cracks (judged by observing changes in geometric shape and texture features), corrosion (judged by changes in texture features) or deformation (judged by changes in geometric shape features), etc. When it is judged that there is an abnormality in the wind turbine blade, a detailed abnormality report is generated, including the abnormal location (determined by coordinate matching), the type of problem, and the recommended inspection and maintenance measures. The drone continuously inspects the wind turbine blades. When new photos are generated or the data of existing photos change (such as the status of the wind turbine blade changes after maintenance), the processing results and corresponding classification and analysis conclusions are automatically updated. This ensures real-time monitoring of the blade status, timely detection of new problems, and safe and stable operation of the wind turbine. Through the continuous accumulation of data, the analysis parameters can be further optimized to improve the accuracy and reliability of detection.

[0131] In this embodiment, through similarity calculation, abnormal analysis is performed on the inspection object in the target inspection photo, and the analysis result is output, providing monitoring and maintenance information.

[0132] In summary, a schematic diagram of the principle of a method for processing drone inspection photos is provided as shown in Figure 4. The drone inspection photo processing method includes two stages, a naming and classification stage and a similarity calculation stage.

[0133] In the naming and classification stage, for the target inspection photos, the EXIF ​​information of the target inspection photos is obtained, and the EXIF ​​information includes the first position information of the drone when the target inspection photos were taken; the distance is calculated based on the first position information, and the target inspection object is determined from multiple inspection objects. For multiple AC and DC lines on the same tower, coordinate conversion, vector calculation, vector cross product calculation, and target orientation are performed in sequence. Based on the target orientation and the identification of the target inspection object, the target inspection photos are named and classified, and the named and classified inspection photos are output.

[0134] For the similarity calculation stage, for the target inspection photo, feature extraction is performed on the target inspection photo to obtain the shooting position feature, geometric shape feature and texture feature; the first feature is obtained based on the shooting position feature, geometric shape feature and texture feature; the second feature of the reference inspection photo with a different shooting time and the same shooting position as the target inspection photo is obtained; based on the first feature and the second feature, the similarity between the target inspection photo and the reference inspection photo is calculated.

[0135] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0136] Based on the same inventive concept, the embodiment of the present application also provides a drone inspection photo processing device for implementing the drone inspection photo processing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more drone inspection photo processing device embodiments provided below can refer to the limitations of the drone inspection photo processing method above, and will not be repeated here.

[0137] In an exemplary embodiment, as shown in FIG5 , a drone inspection photo processing device is provided. The drone inspection photo processing device 500 includes: an acquisition module 501, a determination module 502, and a naming module 503, wherein:

[0138] An acquisition module 501 is used to acquire the exchangeable image file EXIF ​​information of the target inspection photo, wherein the EXIF ​​information includes the first position information of the drone when taking the target inspection photo;

[0139] A determination module 502 is used to determine a target inspection object from multiple inspection objects based on the first location information;

[0140] The naming module 503 is used to name the target inspection photo according to the identifier of the target inspection object.

[0141] In one embodiment, the determination module 502 is specifically used to determine an initial inspection object from multiple inspection objects; determine the distance value between the drone and the initial inspection object based on the first position information and the second position information of the initial inspection object; if the distance value is less than or equal to the distance threshold, determine the initial inspection object as the target inspection object; if the distance value is greater than the distance threshold, re-execute the step of determining the initial inspection object from multiple inspection objects until the distance value is less than or equal to the distance threshold.

[0142] In one embodiment, the determination module 502 is also used to verify the first position information based on the verification function; if the verification passes, the step of determining the distance value between the UAV and the initial inspection object based on the first position information and the second position information of the initial inspection object is executed; if the verification fails, the first position information is repaired; correspondingly, the determination module is specifically used to determine the distance value between the UAV and the initial inspection object based on the repaired first position information and the second position information of the initial inspection object.

[0143] In one of the embodiments, the first position information includes the longitude and latitude coordinates of the UAV, and the second position information of the target inspection object includes the longitude and latitude coordinates of the target inspection object; the device also includes an orientation determination module, which is used to convert the longitude and latitude coordinates of the UAV and the longitude and latitude coordinates of the target inspection object respectively to obtain the xy coordinates of the UAV and the xy coordinates of the target inspection object in a plane rectangular coordinate system; according to the xy coordinates of the UAV and the xy coordinates of the target inspection object, determine the target orientation of the UAV relative to the target inspection object when taking the target inspection photo; correspondingly, a naming module is specifically used to name the target inspection photo according to the target orientation and the identification of the target inspection object.

[0144] In one of the embodiments, the orientation determination module is specifically used to obtain the xy coordinates of a first inspection object and the xy coordinates of a second inspection object, where the first inspection object and the second inspection object are two inspection objects adjacent to the target inspection object; obtain a first vector between the target inspection object and the first inspection object based on the xy coordinates of the target inspection object and the xy coordinates of the first inspection object; obtain a second vector between the target inspection object and the second inspection object based on the xy coordinates of the target inspection object and the xy coordinates of the second inspection object; obtain a third vector between the target inspection object and the drone based on the xy coordinates of the target inspection object and the xy coordinates of the drone; perform a cross product calculation on the first vector and the third vector to obtain a first cross product value; perform a cross product calculation on the second vector and the third vector to obtain a second cross product value; determine the target orientation based on the sign of the first cross product value and the sign of the second cross product value.

[0145] In one embodiment, the device also includes an analysis module, which is used to extract features from the target inspection photo to obtain a first feature vector of the target inspection photo; extract features from the reference inspection photo to obtain a second feature vector of the reference inspection photo; wherein the reference inspection photo is an inspection photo that is taken at a different time but the same location as the target inspection photo; calculate the similarity between the first feature vector and the second feature vector to obtain the similarity between the target inspection photo and the reference inspection photo; based on the similarity, perform an abnormality analysis on the inspection object in the target inspection photo and output the analysis result.

[0146] Each module in the above-mentioned drone inspection photo processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0147] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for processing drone inspection photos is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0148] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method described in the above method embodiment are implemented.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the above method embodiment are implemented.

[0151] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the method described in the above method embodiment when executed by a processor.

[0152] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0153] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0154] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing drone inspection photos, characterized in that: The method comprises: Acquire the exchangeable image file EXIF ​​information of the target inspection photo, wherein the EXIF ​​information includes the first position information of the drone when taking the target inspection photo; Based on the first position information, determining a target inspection object from a plurality of inspection objects; The target inspection photo is named according to the identification of the target inspection object.

2. The method according to claim 1, characterized in that The determining a target inspection object from a plurality of inspection objects based on the first position information includes: Determining an initial inspection object from the plurality of inspection objects; Determine a distance value between the drone and the initial inspection object according to the first position information and the second position information of the initial inspection object; If the distance value is less than or equal to the distance threshold, the initial inspection object is determined as the target inspection object; If the distance value is greater than the distance threshold, the step of determining an initial inspection object from the plurality of inspection objects is performed again until the distance value is less than or equal to the distance threshold.

3. The method according to claim 2, characterized in that Before determining the distance value between the drone and the initial inspection object according to the first position information and the second position information of the initial inspection object, the method further includes: Verifying the first location information based on a verification function; If the verification is successful, the step of determining the distance value between the drone and the initial inspection object according to the first position information and the second position information of the initial inspection object is performed; If the verification fails, the first position information is repaired; correspondingly, the distance value between the UAV and the initial inspection object is determined based on the first position information and the second position information of the initial inspection object, including: determining the distance value between the UAV and the initial inspection object based on the repaired first position information and the second position information of the initial inspection object.

4. The method according to claim 2, characterized in that: The first location information includes the longitude and latitude coordinates of the drone, and the second location information of the target inspection object includes the longitude and latitude coordinates of the target inspection object; the method further includes: The longitude and latitude coordinates of the UAV and the longitude and latitude coordinates of the target inspection object are respectively converted to obtain the xy coordinates of the UAV and the xy coordinates of the target inspection object in a plane rectangular coordinate system; Determine the target orientation of the drone relative to the target inspection object when taking the target inspection photo according to the xy coordinates of the drone and the xy coordinates of the target inspection object; Correspondingly, naming the target inspection photo according to the identifier of the target inspection object includes: The target inspection photo is named according to the target orientation and the identification of the target inspection object.

5. The method according to claim 4, characterized in that The determining, based on the xy coordinates of the drone and the xy coordinates of the target inspection object, the target orientation of the drone relative to the target inspection object when taking the target inspection photo comprises: Acquire the xy coordinates of a first inspection object and the xy coordinates of a second inspection object, wherein the first inspection object and the second inspection object are two inspection objects adjacent to the target inspection object; According to the xy coordinates of the target inspection object and the xy coordinates of the first inspection object, a first vector between the target inspection object and the first inspection object is obtained; according to the xy coordinates of the target inspection object and the xy coordinates of the second inspection object, a second vector between the target inspection object and the second inspection object is obtained; according to the xy coordinates of the target inspection object and the xy coordinates of the drone, a third vector between the target inspection object and the drone is obtained; Performing a cross product calculation on the first vector and the third vector to obtain a first cross product value; performing a cross product calculation on the second vector and the third vector to obtain a second cross product value; The target orientation is determined according to a sign of the first cross-product value and a sign of the second cross-product value.

6. The method according to claim 1, characterized in that The method further comprises: Performing feature extraction on the target inspection photo to obtain a first feature vector of the target inspection photo; Performing feature extraction on the reference inspection photo to obtain a second feature vector of the reference inspection photo; wherein the reference inspection photo is an inspection photo that is taken at a different time and the same location as the target inspection photo; Calculating the similarity between the first feature vector and the second feature vector to obtain the similarity between the target inspection photo and the reference inspection photo; According to the similarity, an abnormality analysis is performed on the inspection object in the target inspection photo, and the analysis result is output.

7. A drone inspection photo processing device, characterized in that: The device comprises: An acquisition module is used to acquire EXIF ​​information of a target inspection photo, wherein the EXIF ​​information includes first position information of the drone when taking the target inspection photo; A determination module, configured to determine a target inspection object from a plurality of inspection objects based on the first location information; A naming module is used to name the target inspection photo according to the identifier of the target inspection object.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images