Image automatic management method and device based on unmanned aerial vehicle inspection and medium

Abstract

The invention discloses an automatic image management method and device based on unmanned aerial vehicle inspection and a medium, and relates to the technical field of electronic digital data processing, and the method mainly comprises the steps: receiving a visible light image and a matched thermal infrared image, which are obtained through adding metadata information corresponding to an image into an EXIF field of the image by an unmanned aerial vehicle; based on the visual attribute information of the images, performing image rejection of the images based on health degree scores; by calculating the perceptual hash value of each rejected image, in combination with metadata information, deleting low-health-degree-score repeated images among similar images based on perceptual hash value distance judgment; extracting each feature point in the deleted thermal infrared image, and delimiting a target area corresponding to each feature point in the matched visible light image; and carrying out relative position deviation correction and storage on the feature points in the thermal infrared image and the target area in the visible light image through geometric transformation. The operation and maintenance efficiency and precision of the photovoltaic power station are improved, and powerful support is provided for safe operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic digital data processing, and particularly relates to an image automatic management method and device based on unmanned aerial vehicle (UAV) inspection and a medium. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle (UAV) technology and its wide application in photovoltaic power station inspection, the UAV has become an important tool for photovoltaic power station operation and management. The UAV can efficiently cover a large area and provide high-resolution visible light and thermal infrared images to help detect potential problems and faults. However, with the dramatic increase in the amount of UAV inspection image data, managing and analyzing these image data becomes more and more complex and difficult.

[0003] The existing UAV inspection image management system mainly faces the following challenges. First, due to the lack of unified naming rules, the image file name is often not standardized, which leads to subsequent management and retrieval difficulties. This not only increases the workload of manual processing, but also may cause important information to be lost or misused. Second, the problem of duplicate images is also a significant problem. During multiple inspection processes, a large number of similar or identical images may be generated, which not only wastes storage space, but also increases the burden of data analysis. In addition, due to factors such as shooting environment, device performance, etc., some images may have problems such as insufficient clarity, inappropriate brightness, or too low contrast, which will directly affect the usability of the images and the accuracy of the analysis results. Finally, the existing system cannot ensure the consistency and synchronization of thermal infrared and visible light images, which affects further data processing and fault diagnosis. SUMMARY

[0004] In order to improve the image management efficiency in the existing UAV photovoltaic power station inspection process, the present application proposes an image automatic management method based on UAV inspection, which comprises the following steps: S1: receiving visible light images and matched thermal infrared images uploaded at the same time during the UAV inspection of the photovoltaic power station, and adding image corresponding metadata information to the EXIF field of the images; S2: scoring the health degree of each image based on the visual attribute information of the image, and eliminating images with a health degree score lower than a preset score threshold; S3: calculating the perceptual hash value of each image after elimination, and combining the metadata information to determine the deletion of low health degree score duplicate images between similar images based on the perceptual hash value distance; S4: extracting each feature point in the deleted thermal infrared image, and demarcating the corresponding target area of each feature point in the matched visible light image; S5: Correcting the relative position deviation between the feature points in the thermal infrared image and the target region in the visible light image through geometric transformation and storing.

[0005] Further, in the S1 step, the unmanned aerial vehicle adopts a preset naming rule to name the image, and the preset naming rule is: site abbreviation_date_device model_image type abbreviation_resolution_sequence number.

[0006] Further, in the S2 step, the visual attribute information includes image definition, image brightness and image contrast.

[0007] Further, in the S2 step, the health score is obtained by the following formula: In the formula, is the health score, is the image definition, is the weight of the image definition, is the image brightness, is the weight of the image brightness, is the image contrast, is the weight of the image contrast.

[0008] Further, in the S2 and S3 steps, when a type of image is removed, the other type of image matched with the image is also removed accordingly.

[0009] Further, in the S3 step, the perceptual hash value is obtained based on the discrete cosine transform, and the formula expression is as follows: In the formula, is the coefficient after the discrete cosine transform, is the size of the image after size unification, is the size of the low-frequency part after the discrete cosine transform, is the image pixel value, when , otherwise , are the frequency indexes in the horizontal and vertical directions in the frequency domain respectively, is the average value of the coefficient after the discrete cosine transform, is the perceptual hash value corresponding to the region , and a string of binary numbers is obtained based on the perceptual hash values at each region to represent the perceptual hash value of the image.

[0010] ​Further, in the S3 step, when the same metadata information exists between the same type of images, and the Hamming distance of the perceptual hash values between the images is less than a preset distance threshold, the current two images are determined as repeated images.

[0011] Further, in the S4 step, affine transformation or perspective transformation is used to correct the relative position deviation.

[0012] The application also includes a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image automatic management method based on unmanned aerial vehicle inspection.

[0013] It also includes a data processing device, comprising: a memory having a computer program stored thereon; a processor configured to execute the computer program in the memory to implement the steps of the image automatic management method based on unmanned aerial vehicle inspection.

[0014] Compared with the prior art, the application has at least the following beneficial effects: (1) The image automatic management method based on unmanned aerial vehicle inspection proposed by the application generates a file name in a preset format by introducing a standardized file naming rule, and embeds key metadata into the EXIF field of the image, which not only facilitates subsequent tracking and retrieval, but also ensures the consistency and integrity of the image data; (2) Each image is health scored according to visual attribute information such as definition, brightness and contrast, and pictures that cannot be opened or have obvious errors are automatically identified and removed, greatly reducing the workload of manual review and improving the accuracy of data analysis; (3) The perceptual hash value of each image is calculated using the perceptual hash algorithm, and combined with the metadata information to determine whether there are repeated images, which can accurately find and delete repeated images with high similarity, thereby reducing the waste of storage space and improving the data processing efficiency; (4) When it is detected that a certain image needs to be removed, the corresponding image of another type will also be deleted, maintaining the consistency of the data, and this synchronous management method provides reliable data support for subsequent fault analysis. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a step diagram of the image automatic management method based on unmanned aerial vehicle inspection. DETAILED DESCRIPTION

[0016] In order to more clearly illustrate the technical solutions of the present application, the present application will be described in detail below in conjunction with the drawings and specific embodiments. It should be understood that these embodiments are only used for illustrative purposes and do not limit the application range of the present application. The protection scope of the present application is defined by the claims, and is not limited to the following specific embodiments.

[0017] The present application proposes an image automatic management method based on unmanned aerial vehicle inspection. Through a series of innovative technical means, the problems of naming confusion, repeated images, image quality and synchronous management in the prior art are solved. The following will introduce the working process of each module of the system through specific embodiments, as shown in Figure 1 The main steps include:

[0018] S1: receiving the visible light image and the matched thermal infrared image uploaded at the same time during the unmanned aerial vehicle inspection of the photovoltaic power station, and adding the image corresponding metadata information to the EXIF field of the image; S2: scoring the health degree of each image based on the visual attribute information of the image, and eliminating the images with a health degree score lower than a preset score threshold; S3: calculating the perceptual hash value of each image after elimination, combining the metadata information to determine the deletion of low health degree score repeated images between similar images based on the perceptual hash value distance; S4: extracting each feature point in the thermal infrared image after deletion, and marking the corresponding target area in the matched visible light image; S5: correcting the relative position deviation of the feature points in the thermal infrared image and the target area in the visible light image by geometric transformation and storing.

[0019] Specifically, the system first acquires the original image file (including visible light and thermal infrared images) through the unmanned aerial vehicle flight control system or image acquisition device, and immediately starts the naming standardization process. The core of this process is to build a set of extensible and analyzable naming rules, and at the same time, embed the key metadata (such as time, location, device information, etc.) into the EXIF field of the image file.

[0020] In the naming rule design, the system adopts a multi-level structure of "site abbreviation_date_device model_image type abbreviation_resolution_sequence number" to ensure that the file name of each image contains complete positioning information and acquisition parameters. For example, "PV01_20231015_DJI-3000_IR_1280x720_001.jpg" represents: site number PV01, acquisition date October 15, 2023, unmanned aerial vehicle device model DJI-3000, image type thermal infrared (IR), resolution 1280x720 pixels, and sequence number 001. This naming method not only facilitates manual identification of the basic attributes of the image, but also enables automatic retrieval and classification through programming tools (such as regular expressions).

[0021] At the same time, the system automatically writes metadata (such as GPS coordinates, shooting time, flight height, camera parameters, etc.) obtained during the acquisition process into the EXIF field. For example, GPS latitude and longitude information is embedded in the GPSLatitude and GPSLongitude tags of EXIF, the shooting time follows the ISO 8601 standard format (such as "2023:10:15 14:30:22"), and the device model and sensor parameters are recorded through custom EXIF tags (such as "DRONE_MODEL" and "SENSOR_RESOLUTION"). This process is achieved by calling standard image metadata operation libraries (such as ExifTool or OpenCV's EXIF interface), ensuring the compatibility and readability of the metadata.

[0022] In addition, the system also sets up a dynamic verification mechanism to ensure strict implementation of the naming rules. For example, during the file name generation process, if the site abbreviation does not match the preset list, the date format is incorrect, or the device model is missing, the system will automatically trigger an alarm and suspend the naming process until the parameters are manually corrected. For metadata embedding, the system uses double verification: first, verify the integrity of the EXIF field, and second, ensure the consistency of the written data with the original acquisition data through hash verification (such as MD5). This double verification mechanism effectively avoids subsequent analysis bias caused by naming errors or metadata loss.

[0023] It is worth noting that the standardized naming in step S1 and metadata embedding are not isolated operations, but form a closed loop with subsequent steps (such as image quality assessment, repeated image detection). For example, the "image type abbreviation" field in the file name can be directly associated with the subsequent classification logic, and the GPS coordinates in EXIF provide a spatial reference for subsequent image synchronization management (such as thermal infrared and visible light image registration). Therefore, step S1 not only solves the problem of file naming confusion and metadata loss in the prior art, but also provides a reliable data foundation for the entire system's automated management process.

[0024] After the file naming and metadata embedding, the system enters the image quality assessment and screening phase. This phase aims to quantify the visual attributes (such as sharpness, brightness, contrast) of the images and the metadata information, score the healthiness of each image, and automatically filter out low-quality or damaged images (healthiness score below the preset threshold) based on preset thresholds, thereby reducing redundant data in subsequent processing.

[0025] Specifically, the system first calls the image processing module to preprocess the input image, including grayscale, scaling, or noise filtering operations, to ensure the accuracy of the evaluation. Subsequently, the system conducts a comprehensive evaluation of the image from three core dimensions.

[0026] Sharpness C: measured by calculating the average or sum of image gradients. For grayscale images , the gradient amplitude can be calculated using the Sobel operator or other edge detection methods :

[0027] where and are the gradients along the X and Y axes, respectively. The sharpness C can be defined as the average or sum of all pixel gradient amplitudes.

[0028] Brightness B: obtained by calculating the average of all pixel values in the image: where is the size of the image.

[0029] Contrast D: measured by calculating the standard deviation of the image, where a larger standard deviation indicates a higher contrast: Then the healthiness score of the image is calculated by the following formula: where is the healthiness score, is the weight of image sharpness, is the weight of image brightness, is the weight of image contrast.

[0030] Preferably, the system can also introduce a dynamic weight mechanism to adaptively adjust the evaluation criteria according to the image type and scene. For example, for thermal infrared images, since their contrast is usually low, the system will appropriately reduce the weight of contrast, while increasing the sensitivity to abnormally high or low temperature areas. For visible light images, the system will dynamically adjust the evaluation criteria for brightness and contrast in combination with the white balance parameter in EXIF.

[0031] Through the above technical means, the application realizes intelligent evaluation and screening of image quality, effectively reduces the interference of low-quality or damaged images on the subsequent processing flow, provides high-quality data input for repeated image detection, synchronous management and other modules, and significantly improves the overall efficiency of photovoltaic power station unmanned aerial vehicle inspection and the reliability of fault diagnosis.

[0032] After completing the image quality evaluation, the system will enter the detection and deletion stage of repeated images. The system accurately identifies and deletes repeated or highly similar images through perceptual hashing algorithm combined with metadata association analysis, and the core method is discrete cosine transform (DCT) and perceptual hash value generation formula. The specific implementation process is as follows.

[0033] The system first pre-processes the images screened by quality: uniformly scales the input images (including visible light and thermal infrared images) to a fixed size (W x H) ), and converts them to grayscale images to reduce computational complexity. Then, two-dimensional discrete cosine transform (DCT) is applied to convert the image from spatial domain to frequency domain. The mathematical expression of DCT is:

[0034] Where, is the coefficient after discrete cosine transform, when , otherwise , is the pixel value of the image. Here, DCT decomposes the image into different frequency basis functions, with the low-frequency component (upper left corner) representing the global structure of the image and the high-frequency component (lower right corner) representing the details.

[0035] Based on the results of DTC transformation, the low-frequency part of the upper left corner is extracted and quantized. Here we use the average value excluding the direct current component as the quantization result, and the formula is as follows:

[0036] Where, is the average value of the coefficient after discrete cosine transform, is the size of the low-frequency part after discrete cosine transform. Based on this average value, a binary hash sequence is generated:

[0037] wherein is the region corresponding perceptual hash value, and finally a string of binary numbers is formed by each region in order, representing the perceptual hash value of the image.

[0038] Then the system compares the newly generated hash value with the hash values in the historical database, and calculates the Hamming distance: wherein, is the Hamming distance, is the length of the hash string, is the XOR function, is the index of the hash string, is the hash string of image 1, is the hash string of image 2. If (preset distance threshold), it is preliminarily determined that the two images are similar, at which time the system will further combine the metadata information for secondary verification. Mainly including:

[0039] Time correlation: if the time stamp interval of the two images is less than the threshold (such as 5 minutes), and the device model and site number in EXIF are consistent, then the repeated determination is supported; Spatial correlation: through the GPS coordinates of EXIF, if the difference between the longitude and latitude is less than the preset range (such as 10 meters), then the repeated relationship is further confirmed.

[0040] On the basis of determining that the two images are indeed repeated images, if there is an image with a higher health score among the two repeated images, then the image with a lower health score is selected for deletion. At the same time, using the naming rule mentioned earlier, the system will also quickly locate and synchronously delete another type of image at the same time and location. That is to say, if an image is determined to be a repeated image and is deleted, the other image corresponding to the image (whether it is repeated or not) will also be synchronously deleted to ensure data consistency. For example, if the visible light image is deleted due to repetition, the thermal infrared image corresponding to the time stamp and GPS coordinates will also be removed.

[0041] The system stores the retained images according to the site and date, and updates the hash index table in the metadata database. If the image is deleted, its metadata record will be marked as "deleted", ensuring data traceability.

[0042] In a preferred embodiment, for the determination of repeated images, a dynamic Hamming distance threshold adjustment strategy is also introduced. For example, in cloudy or variable light conditions, the Hamming distance threshold is appropriately relaxed to avoid misjudgment caused by light differences.

[0043] For the final selected images, the system will automatically identify and select the points with significant features in the thermal infrared image as reference points using advanced algorithms. These feature points often represent the key information or object boundaries in the image, which are crucial for subsequent image processing. Once these reference points are determined, the system will search for the corresponding feature points in the corresponding visible light image. In this way, the common target area boundary in the two images can be effectively identified to ensure the correspondence of the target object under different imaging modes.

[0044] Next, in order to eliminate the relative position deviation caused by imaging methods, viewing angle differences and other factors, it is particularly important to apply geometric transformation techniques such as affine transformation or perspective transformation for image correction. Affine transformation can maintain the parallelism of straight lines in the image unchanged, and is suitable for processing deformation caused by rotation and translation; while perspective transformation is further allowed to process spatial distortion problems caused by viewpoint changes. By accurately adjusting the parameters of these two types of transformations, the consistency and integrity of the target between different modal images can be effectively guaranteed, thereby providing a solid foundation for subsequent image fusion, analysis and interpretation.

[0045] After the above series of operations, the high similarity repeated images are accurately found and removed, the thermal infrared and visible light images are aligned one by one, the target area of interest is accurately cropped, and the geometric correction is performed to ensure the consistency between images, thereby realizing the high efficiency and high quality of unmanned aerial vehicle inspection image automatic management.

[0046] The present application also includes a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the steps of the unmanned aerial vehicle inspection-based image automatic management method.

[0047] It also includes a data processing device, comprising: a memory having a computer program stored thereon; a processor configured to execute the computer program in the memory to implement the steps of the unmanned aerial vehicle inspection-based image automatic management method.

[0048] In summary, the unmanned aerial vehicle inspection-based image automatic management method proposed by the present application introduces standardized file naming rules, automatically generates file names according to the preset format, and embeds key metadata into the EXIF field of the image. This not only facilitates subsequent tracking and retrieval, but also ensures the consistency and integrity of the image data. According to the visual attribute information such as the clarity, brightness and contrast of the image, each image is given a health score, and the images that cannot be opened or have obvious errors are automatically identified and removed, greatly reducing the workload of manual review and improving the accuracy of data analysis.

[0049] The perceptual hash value of each image is calculated by using the perceptual hash algorithm, and the metadata information is combined to determine whether there are duplicate images, which can accurately find and delete the high similarity duplicate images, thereby reducing the waste of storage space and improving the data processing efficiency. At the same time, when detecting that a certain image needs to be removed, the corresponding other type of image will also be deleted together, keeping the consistency of the data, and this synchronous management mode provides reliable data support for subsequent fault analysis.

[0050] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, motion condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications will also change accordingly.

[0051] In addition, the descriptions such as "first", "second", "one" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically limited.

[0052] In the present application, unless otherwise specifically defined and limited, the terms "connection", "fixing" and the like should be understood in a broad sense, for example, "fixing" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through an intermediate medium; can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise specifically limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0053] In addition, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the scope of protection required by the present application.

Claims

1. An automated image management method based on UAV inspection, characterized in that, Including the following steps: S1: Receive the visible light image and the matching thermal infrared image uploaded at the same time during the drone's inspection of the photovoltaic power station, and add the corresponding metadata information of the image to the EXIF ​​field of the image; S2: Based on the visual attribute information of the images, score the health of each image and remove images with a health score lower than the preset score threshold; S3: By calculating the perceptual hash value of each image after removal, and combining it with metadata information, duplicate images with low health scores are deleted based on the distance between images of the same type according to the perceptual hash value. S4: Extract each feature point from the deleted thermal infrared image and delineate the target area corresponding to each feature point in the matched visible light image; S5: Correct and store the relative positional deviation between feature points in thermal infrared images and target areas in visible light images through geometric transformation.

2. The image automation management method based on UAV inspection as described in claim 1, characterized in that, In step S1, the drone names the images using a preset naming rule, which is: site abbreviation_date_device model_image type abbreviation_resolution_serial number.

3. The image automation management method based on UAV inspection as described in claim 1, characterized in that, In step S2, the visual attribute information includes image sharpness, image brightness, and image contrast.

4. The image automation management method based on UAV inspection as described in claim 3, characterized in that, In step S2, the health score is obtained using the following formula: In the formula, Rate your health. Image clarity, As a weight for image sharpness, For image brightness, As a weight for image brightness, For image contrast, The weights for image contrast.

5. The image automation management method based on UAV inspection as described in claim 1, characterized in that, In steps S2 and S3, when one type of image is removed, another type of image that matches that image is also removed accordingly.

6. The image automation management method based on UAV inspection as described in claim 1, characterized in that, In step S3, the perceptual hash value is obtained based on the discrete cosine transform, and its formula is expressed as follows: In the formula, These are the coefficients after the discrete cosine transform. This refers to the size of the image after standardization. This represents the magnitude of the low-frequency component after discrete cosine transform. For image pixel values, when hour ,otherwise , These are the frequency indices in the horizontal and vertical directions, respectively, in the frequency domain. This represents the average value of the coefficients after the discrete cosine transform. For the region The perceptual hash value corresponding to each location is obtained, and a string of binary numbers is obtained based on the perceptual hash value of each region to represent the perceptual hash value of the image.

7. The image automation management method based on UAV inspection as described in claim 6, characterized in that, In step S3, when images of the same type have the same metadata information and the Hamming distance between the perceived hash values ​​of the images is less than a preset distance threshold, the two images are determined to be duplicate images.

8. The image automation management method based on UAV inspection as described in claim 1, characterized in that, In step S4, affine transformation or perspective transformation is used to correct the relative position deviation.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of an image automation management method based on unmanned aerial vehicle (UAV) inspection as described in any one of claims 1 to 8.

10. A data processing apparatus, characterized in that, include: A memory on which computer programs are stored; A processor for executing a computer program in the memory to implement the steps of the image automation management method based on unmanned aerial vehicle (UAV) inspection as described in any one of claims 1 to 8.

Images