A global defect positioning method and system for inspection images

Abstract

The application discloses a kind of global defect positioning method and system of inspection image, wherein the method obtains the inspection image collected in the process of unmanned aerial vehicle inspection, and the inspection image includes several inspection images containing embedded metadata;The inspection image is input into the trained defect detection model for image detection, and the first structured data output by the trained defect detection model is obtained;The first structured data includes the defect detection result of several inspection images, and the space-time index of the defect detection result;The first structured data is subjected to spatial defect point deduplication processing, and the second structured data is obtained;Global spatial analysis is carried out on the second structured data, and the global defect positioning result of the inspection image is obtained.The method can improve the accuracy of inspection defect positioning, and is beneficial to reduce the operation and maintenance cost after inspection.The application relates to the technical field of unmanned aerial vehicle inspection.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to a global defect localization method and system for inspection images. Background Technology

[0002] With the widespread application of drone technology in industrial fields such as power line inspection, photovoltaic power station monitoring, water conservancy equipment inspection, and tunnel inspection, defect identification of inspection images collected during drone inspections has become one of the key focuses of relevant personnel.

[0003] Currently, the relevant technology typically involves drones flying along pre-planned routes and collecting inspection images by hovering at fixed points during flight. To ensure no missed inspections, this method often allows some overlap in the inspection images collected by the drone at adjacent hovering positions. This can easily lead to the same defect being repeatedly identified and located from different angles, resulting in unsatisfactory defect localization accuracy and increased subsequent maintenance costs.

[0004] Therefore, the problems with the relevant technologies still need to be solved and optimized. Summary of the Invention

[0005] The purpose of this invention is to at least partially solve one of the technical problems existing in the related art.

[0006] Therefore, one objective of this invention is to provide a global defect localization method and system for inspection images, wherein the method can improve the accuracy of inspection defect localization and help reduce the subsequent operation and maintenance costs of inspection.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include: In a first aspect, embodiments of this application provide a method for global defect localization of inspection images, including: Acquire inspection images collected during the drone inspection process, wherein the inspection images include several inspection images containing embedded metadata; The inspection images are input into a trained defect detection model for image detection, resulting in first structured data output by the trained defect detection model. The first structured data includes defect detection results of several inspection images and spatiotemporal indexes of the defect detection results. Each spatiotemporal index corresponds to the embedded metadata of an inspection image. The first structured data is deduplicated by removing spatial defect points to obtain the second structured data; Global spatial analysis is performed on the second structured data to obtain the global defect location results of the inspection image.

[0008] In addition, the method according to the above embodiments of this application may also have the following additional technical features: Furthermore, in one embodiment of this application, the step of inputting the inspection image into a trained defect detection model for image detection to obtain the first structured data output by the trained defect detection model includes: Metadata parsing is performed on the embedded metadata of the inspection image to obtain the spatiotemporal index, which includes spatial coordinate information and timestamp information; The inspection image is input into the trained defect detection model to perform defect detection, and the defect detection result is obtained. Based on the spatiotemporal index, the defect detection results are subjected to structured association processing to obtain the first structured data.

[0009] Furthermore, in one embodiment of this application, the step of performing spatial defect point deduplication on the first structured data to obtain the second structured data includes: Spatial clustering is performed on the first structured data to obtain several first spatial clusters; All first spatial clusters are deduplicated to obtain several second spatial clusters, and each second spatial cluster corresponds to a first spatial cluster after deduplication. The second structured data is obtained based on all the second spatial clusters.

[0010] Furthermore, in one embodiment of this application, the step of spatially clustering the first structured data to obtain a plurality of first spatial clusters includes: Obtain the neighborhood radius and minimum number of points threshold; Based on the neighborhood radius, all spatiotemporal indexes in the first structured data are clustered in a neighborhood to obtain several intermediate clusters; Based on the minimum number of points threshold, all intermediate clusters are filtered by cluster point threshold to obtain several first spatial clusters.

[0011] Furthermore, in one embodiment of this application, the step of deduplicating cluster points of all the first spatial clusters to obtain a plurality of second spatial clusters includes: Obtain the time interval threshold; Based on the time interval threshold, the timestamps of all cluster points in the first spatial cluster are analyzed to obtain several first cluster groups; the first cluster group includes several first cluster points, and the timestamp interval between the second cluster point and the third cluster point is less than the time interval threshold; the second cluster point is any one of the first cluster points in the first cluster group; the third cluster point is the first cluster point in the first cluster group that is adjacent to the second cluster point in the time series, excluding the second cluster point. Defect analysis is performed on all the first cluster groups to obtain several second cluster groups; Based on the second cluster group, the first spatial cluster is updated to obtain the second spatial cluster.

[0012] Furthermore, in one embodiment of this application, the defect analysis of all the first clusters to obtain a plurality of second clusters includes: Obtain a similarity threshold and several defect feature vectors, where each defect feature vector is a feature vector of the defect region of the corresponding first cluster point; Based on the similarity threshold, a defect similarity analysis is performed on all the defect feature vectors to obtain the defect analysis results. The defect analysis results are used to characterize whether all cluster points in the first cluster group belong to the spatial points of the same defect. If the defect analysis result indicates that all cluster points within the first cluster group belong to the spatial points of the same defect, then the first cluster group is updated to obtain the second cluster group.

[0013] Furthermore, in one embodiment of this application, the step of updating the first cluster group to obtain the second cluster group includes: Obtain the confidence information and spatial coordinate information of each first cluster point in the first cluster group; Filter all the confidence information to obtain the target confidence level, which is the highest confidence level among all the confidence information. Coordinate optimization is performed on all the aforementioned spatial coordinate information to obtain the target coordinate information; Based on the target confidence level and target coordinate information, the first cluster group is deduplicated and updated to obtain the second cluster group.

[0014] Furthermore, in one embodiment of this application, the method further includes: Heatmap analysis was performed on the second structured data to obtain the inspection heatmap; Based on the inspection heatmap, the global defect location results are rendered and updated to obtain the updated global defect location results.

[0015] Secondly, embodiments of this application provide a global defect localization system for inspection images, comprising: The first processing unit is used to acquire inspection images collected during the inspection process of the UAV, the inspection images including several inspection images containing embedded metadata. The second processing unit is used to input the inspection image into a trained defect detection model for image detection, and obtain the first structured data output by the trained defect detection model; the first structured data includes defect detection results of several inspection images, and spatiotemporal indexes of the defect detection results; each spatiotemporal index corresponds to the embedded metadata of an inspection image. The third processing unit is used to perform spatial defect point deduplication processing on the first structured data to obtain the second structured data; The fourth processing unit is used to perform global spatial analysis on the second structured data to obtain the global defect location result of the inspection image.

[0016] Thirdly, embodiments of this application also provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0017] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by the processor, is used to implement the above-described method.

[0018] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application: This application discloses a method and system for global defect localization of inspection images. The method acquires inspection images collected during UAV inspections, including several inspection images containing embedded metadata. The inspection images are input into a trained defect detection model for image detection, yielding first structured data output by the model. This first structured data includes defect detection results for several inspection images and spatiotemporal indices of the defect detection results. Each spatiotemporal index corresponds to embedded metadata for one inspection image. The first structured data undergoes spatial defect point deduplication to obtain second structured data. Global spatial analysis is then performed on the second structured data to obtain the global defect localization result of the inspection images. This method, by deduplicating spatial defect points in the first structured data containing defect detection results and spatiotemporal indices, reduces duplicate defect localization due to different angles, thereby improving the accuracy of subsequent inspection defect localization and reducing subsequent maintenance costs. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 A flowchart illustrating a global defect localization method for inspection images provided in an embodiment of this application; Figure 2 A schematic diagram of the framework of a global defect localization system for inspection images provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] With the widespread application of drone technology in industrial fields such as power inspection (e.g., damage detection of insulators, vibration dampers, equalizing rings, shielding rings, etc. in power systems, as well as bird nest detection attached to high-voltage towers), photovoltaic power station monitoring (e.g., detection of surface defects such as cracks, bird nests, stains, and hot spots on photovoltaic modules), water conservancy equipment inspection, and tunnel inspection (e.g., detection of cracks, seepage, and deformation in tunnels), defect identification of inspection images collected during drone inspections has become one of the key focuses of relevant personnel.

[0024] Currently, the relevant technology typically involves drones flying along pre-planned routes and collecting inspection images during flight by hovering at fixed points. To ensure no missed inspections, this method often allows some overlap in the inspection images collected by the drone at adjacent hovering positions. This can easily lead to the same defect being repeatedly identified and located from different angles / positions. Furthermore, since the drone's location information is usually obtained through GPS or BeiDou systems, the positioning signal is fragile in complex and special environments such as canyons, tunnels, and warehouses, resulting in a significant decrease in positioning accuracy. This makes the accuracy of defect location unsatisfactory, and it also requires maintenance personnel to process duplicate defect reports, which is not intuitive and increases the workload for maintenance personnel, thereby increasing subsequent maintenance costs.

[0025] Furthermore, due to challenges such as complex lighting interference (e.g., strong midday sunlight causing photovoltaic panel reflections, and dark tunnel environments making defects such as cracks less noticeable) and confusion of multiple defect types (e.g., stains and bird droppings having similar appearance to real hot spots), drone inspections have a high false detection rate, making it difficult to meet practical application needs.

[0026] It should be noted that the aforementioned related technologies are only used to assist in understanding the technical solutions of this application and do not mean that they belong to the publicly disclosed prior art.

[0027] In view of this, embodiments of this application provide a global defect localization method and system for inspection images. The method performs spatial defect point deduplication on the first structured data containing defect detection results and spatiotemporal indexes. Specifically, it performs defect analysis on the first cluster group obtained by analyzing in the time dimension. This method can analyze whether the same defect captured from different angles has been repeatedly identified and located, effectively improving the accuracy of defect localization and helping to reduce the operation and maintenance costs of inspections.

[0028] Furthermore, when all cluster points within the first cluster group belong to the same spatial point of the defect, this method updates the first cluster group by optimizing the spatial coordinates of all cluster points in the first cluster group to determine the spatial coordinates. This deduplication update of the first cluster group is achieved by using duplicate defect location data from different angles / positions in the spatial dimension to update the corresponding defect location data. This helps to improve the accuracy of defect location and further enhances the precision of defect location.

[0029] Furthermore, when all cluster points within the first cluster group belong to the same spatial point of the defect, the method updates the first cluster group by using the target confidence score (i.e., the highest confidence score among all the confidence scores of the first cluster points) to remove duplicates. This allows the deduplication update to retain the cluster points corresponding to the defect detection results with the highest confidence score output by the defect detection model in inspection images taken from different angles / positions. This is beneficial for improving the adaptability of UAV inspection to challenges such as complex lighting interference and effectively reducing the false detection rate of UAV inspection.

[0030] Reference Figure 1 In this embodiment of the application, a method for global defect localization of inspection images includes: Step 110: Acquire inspection images collected during the UAV inspection process, wherein the inspection images include several inspection images containing embedded metadata; In this embodiment of the application, the drone inspection process can be the inspection process of drones in industrial fields such as power, photovoltaic, water conservancy, and tunnels. The inspection image can be a video image containing the object to be inspected, which is captured by the drone through a camera device during the inspection process. The video image is composed of several inspection image frames containing embedded metadata. Alternatively, the inspection image can be an inspection image containing embedded cloud data, which is captured by the drone through a camera device at several specific locations during the inspection process.

[0031] It is understood that the embedded metadata can be information data embedded into the inspection image in metadata format. Specifically, the embedded metadata can be the EXIF ​​information of the inspection image, which includes the latitude and longitude coordinates and timestamp when the inspection image was taken; or, in another embodiment, the EXIF ​​information may include the shooting altitude, UAV serial number, camera parameters (such as focal length and sensor) in addition to the latitude and longitude coordinates and timestamp when the inspection image was taken.

[0032] Step 120: Input the inspection image into the trained defect detection model for image detection to obtain the first structured data output by the trained defect detection model; the first structured data includes defect detection results of several inspection images and spatiotemporal indexes of the defect detection results; each spatiotemporal index corresponds to the embedded metadata of an inspection image; In this embodiment, the defect detection model can be a YOLO series target detection model, such as the YOLO-v8 model. There are various training methods for YOLO series models, which will not be elaborated here. Image detection can be performed by using the trained defect detection model to detect defect regions in the inspection image, thereby outputting defect detection results with spatiotemporal indexes (i.e., the first structured data).

[0033] In some embodiments, inputting the inspection image into a trained defect detection model for image detection to obtain the first structured data output by the trained defect detection model includes: Metadata parsing is performed on the embedded metadata of the inspection image to obtain the spatiotemporal index, which includes spatial coordinate information and timestamp information; The inspection image is input into the trained defect detection model to perform defect detection, and the defect detection result is obtained. Based on the spatiotemporal index, the defect detection results are subjected to structured association processing to obtain the first structured data.

[0034] In this embodiment, for any inspection image in the inspection imagery, image detection can be performed by parsing the embedded metadata in the inspection image. Specifically, the latitude and longitude coordinates in the embedded metadata are determined as spatial coordinate information, and the timestamps in the embedded metadata are determined as timestamp information, thereby obtaining the spatiotemporal index corresponding to the inspection image. Defect detection can be performed by inputting the inspection image into a trained defect detection model, and then using the trained defect detection model to detect several defects present in the inspection image, thereby obtaining the defect detection result of the inspection image. The defect detection result includes the defect type, confidence level, severity level of the defect, and image ID of the several defects in the inspection image.

[0035] It is understood that, in one alternative implementation, before the inspection image is input into the trained defect detection model, the inspection image can be preprocessed. Specifically, the preprocessing can be based on the camera parameters embedded in the metadata, performing operations such as distortion correction, Clahe contrast enhancement, and image pixel value normalization on the inspection image.

[0036] Structured association processing can involve binding and associating defect detection results (such as defect type, confidence level, defect severity level, and associated image ID) of inspection images with spatiotemporal indexes (such as latitude and longitude coordinates and timestamps). Specifically, this can be achieved by binding the defect detection results and spatiotemporal index information into a structured JSON record. After obtaining the JSON record for each inspection image, the first structured data can be generated based on all the JSON records.

[0037] Step 130: Perform spatial defect point deduplication on the first structured data to obtain the second structured data; In this embodiment of the application, the spatial defect point deduplication process can be performed on the first structured data in the spatial dimension to reduce the repeated positioning caused by the UAV taking pictures of the same defect from different angles / positions, thereby obtaining the second structured data.

[0038] In some embodiments, the process of deduplicating spatial defect points in the first structured data to obtain the second structured data includes: Spatial clustering is performed on the first structured data to obtain several first spatial clusters; Furthermore, the spatial clustering of the first structured data to obtain several first spatial clusters includes: Obtain the neighborhood radius and minimum number of points threshold; Based on the neighborhood radius, all spatiotemporal indexes in the first structured data are clustered in a neighborhood to obtain several intermediate clusters; Based on the minimum number of points threshold, all intermediate clusters are filtered by cluster point threshold to obtain several first spatial clusters.

[0039] In this embodiment, the deduplication of spatial defect points can first involve spatial clustering of the first structured data. Specifically, since the first structured data includes a JSON record for each inspection image, and each JSON record contains the latitude and longitude coordinates of the corresponding inspection image at the time of capture, spatial clustering can first convert the spatiotemporal index of each inspection image in the first structured data into a spatial point in a spatial coordinate system, and then perform cluster analysis on the neighborhood points of several spatial points based on the neighborhood radius in the Dbscan clustering algorithm, thereby determining several intermediate clusters. The specific radius value of the neighborhood radius can be set according to the actual situation.

[0040] Understandably, cluster point threshold filtering can be based on a minimum point count threshold to filter the number of cluster points in each intermediate cluster. These cluster points are spatial points within the intermediate cluster, resulting in several first spatial clusters. Specifically, for any intermediate cluster, the minimum point count threshold can be compared with the number of cluster points in that intermediate cluster. If the minimum point count threshold is greater than or equal to the number of cluster points in the intermediate cluster, then that intermediate cluster can be deleted; or, if the minimum point count threshold is less than the number of cluster points in the intermediate cluster, then that intermediate cluster can be identified as a first spatial cluster.

[0041] All first spatial clusters are deduplicated to obtain several second spatial clusters, and each second spatial cluster corresponds to a first spatial cluster after deduplication. Further, the process of deduplicating cluster points in all the first spatial clusters to obtain several second spatial clusters includes: Obtain the time interval threshold; Based on the time interval threshold, the timestamps of all cluster points in the first spatial cluster are analyzed to obtain several first cluster groups; the first cluster group includes several first cluster points, and the timestamp interval between the second cluster point and the third cluster point is less than the time interval threshold; the second cluster point is any one of the first cluster points in the first cluster group; the third cluster point is the first cluster point in the first cluster group that is adjacent to the second cluster point in the time series, excluding the second cluster point. In this embodiment of the application, for any first spatial cluster, cluster point deduplication can first be performed by performing interval analysis on the timestamps of all cluster points of the first spatial cluster according to the time interval threshold, thereby obtaining several first cluster groups divided according to the time dimension. The specific value of the time interval threshold can be set according to the actual situation.

[0042] Specifically, interval analysis can determine the timestamp of each cluster point based on the spatiotemporal index corresponding to each cluster point in the first spatial cluster, and sort the timestamps of each cluster point by time to obtain a time series of cluster points, where each time point in the time series represents a cluster point. Then, the timestamp interval between each time point in the time series and the next time point is calculated in a cyclical manner. If the timestamp interval is less than the time interval threshold, it indicates that the cluster point corresponding to the current time point and the cluster point corresponding to the next time point are from continuously captured inspection images, which is highly likely to be repeated images of the same object. In this case, the next time point can be assigned to the cluster group of the current time point, and the next cyclic process continues until the end of the cycle. Alternatively, if the timestamp interval is greater than or equal to the time interval threshold, the next time point can be assigned to a new cluster group, and the next cyclic process continues until the end of the cycle. A specific cycle termination condition could be that the current time point in the time series is the last time point in the time series.

[0043] For example, for a time series {T1, T2, T3, T4, T5, T6...Tn-1, Tn}, if time points T4, T5, and T6 belong to the same cluster group during the cycle, a first cluster group can be constructed based on the cluster points corresponding to time points T4, T5, and T6. Furthermore, if time points T4, T5, and T6 belong to the same cluster group during the cycle, and the current second cluster point is the cluster point corresponding to time point T5, then the third cluster point adjacent to the cluster point corresponding to time point T5 in the time series can be either the cluster point time point corresponding to time point T4 or the cluster point corresponding to time point T6.

[0044] Defect analysis is performed on all the first cluster groups to obtain several second cluster groups; Furthermore, the defect analysis performed on all the first clusters yields several second clusters, including: Obtain a similarity threshold and several defect feature vectors, where each defect feature vector is a feature vector of the defect region of the corresponding first cluster point; Based on the similarity threshold, a defect similarity analysis is performed on all the defect feature vectors to obtain the defect analysis results. The defect analysis results are used to characterize whether all cluster points in the first cluster group belong to the spatial points of the same defect. In this embodiment, the similarity threshold is used to characterize the minimum similarity threshold between the defect feature vectors of each cluster point within a cluster group, and its specific value can be set according to the actual situation. For any first cluster group, since the defect detection model often adds a detection box at the defect location in the inspection image when performing defect detection on the inspection image, and attaches the defect type, confidence level, and severity level of the defect indicated by the detection box to the detection box, so as to obtain the defect detection result of the inspection image; therefore, the defect feature vector in this embodiment can be the feature vector of the detection box in the defect region indicated by the inspection image in the defect detection result corresponding to the cluster point.

[0045] Understandably, in the first feasible implementation, defect similarity analysis can involve calculating the cosine similarity between each defect feature vector in the first cluster and the remaining defect feature vectors, and comparing each calculated cosine similarity with a similarity threshold. Specifically, for any two defect feature vectors within the first cluster, if the calculated cosine similarity is greater than or equal to the similarity threshold, it indicates that the defects corresponding to these two feature vectors belong to the same defect; or, if the calculated cosine similarity is less than the similarity threshold, it indicates that the defects corresponding to these two feature vectors do not belong to the same defect. Based on the comparison results of all cosine similarities and the similarity threshold within the first cluster, the defect analysis results can be easily obtained.

[0046] In the second feasible implementation, the defect similarity analysis can first randomly select the defect feature vector of a cluster point from all cluster points in the first cluster group and determine it as the baseline feature vector. Then, the cosine similarity between the baseline feature vector and the defect feature vectors of the remaining cluster points is calculated and analyzed, and compared with the similarity threshold. The subsequent content is similar to the content of the first implementation of the defect similarity analysis and can be easily deduced by analogy.

[0047] In the third feasible implementation, the defect similarity analysis can first be based on the time series corresponding to each cluster point in the first cluster group. The defect feature vectors of the cluster points that are adjacent in the time series are calculated pairwise with cosine similarity, and the obtained cosine similarity is compared with the similarity threshold. The subsequent content is similar to the content of the first implementation of defect similarity analysis and can be simply deduced by analogy.

[0048] It should be noted that, because the defect similarity analysis in the third implementation method introduces the sequential relationship of each cluster point in the first cluster group in the time dimension, and the inspection images continuously captured by the UAV during the inspection process have extremely high similarity, for any inspection image corresponding to any cluster point in the first cluster group, the defect similarity analysis in the third implementation method can quickly and accurately determine the inspection image most similar to the current inspection image from all the inspection images corresponding to the cluster points and perform cosine similarity calculation. This ensures the accuracy of subsequent global defect localization while reducing the computational load required for defect similarity analysis, effectively improving the efficiency of subsequent global defect localization.

[0049] If the defect analysis result indicates that all cluster points within the first cluster group belong to the spatial points of the same defect, then the first cluster group is updated to obtain the second cluster group.

[0050] Further, the step of updating the first cluster group to obtain the second cluster group includes: Obtain the confidence information and spatial coordinate information of each first cluster point in the first cluster group; Filter all the confidence information to obtain the target confidence level, which is the highest confidence level among all the confidence information. Coordinate optimization is performed on all the aforementioned spatial coordinate information to obtain the target coordinate information; Based on the target confidence level and target coordinate information, the first cluster group is deduplicated and updated to obtain the second cluster group.

[0051] In this embodiment of the application, the confidence information and spatial coordinate information of each cluster point in the first cluster group can be determined based on the defect detection result and / or spatiotemporal index corresponding to each cluster point. Specifically, for any cluster point, the confidence information of the cluster point can be determined by the confidence of the defect detection result corresponding to that cluster point, and the latitude and longitude coordinates in the spatiotemporal index corresponding to that cluster point can be determined.

[0052] Understandably, confidence level selection can be achieved by choosing the highest confidence level from all confidence levels, denoted as the target confidence level. Coordinate optimization can be performed by optimizing the spatial coordinates of each cluster point in the first cluster group to obtain the spatial coordinates of the cluster points corresponding to the target confidence level. Specifically, this can be done by calculating the average coordinates of all spatial coordinates to obtain the target coordinates. Deduplication and updating can be performed by deleting all cluster points in the first cluster group except those corresponding to the target confidence level, and then updating the remaining cluster points with the target coordinates to obtain the second cluster group. Based on the second cluster group, the first spatial cluster is updated to obtain the second spatial cluster.

[0053] The second structured data is obtained based on all the second spatial clusters.

[0054] In the embodiments of this application, for any first spatial cluster, after obtaining all second cluster groups, the original first cluster group corresponding to the first spatial cluster can be replaced and updated based on each second cluster group, thereby obtaining the updated first spatial cluster (i.e., the second spatial cluster).

[0055] Understandably, after updating each first spatial cluster, the second structured data can be easily determined based on all the resulting second spatial clusters. Specifically, it can be constructed based on the information inherited or updated during the deduplication process of spatial defect points in the second spatial clusters. For example, the cluster points of the second spatial clusters inherit defect types, defect severity levels, timestamps, etc., as well as the target confidence and target coordinate information obtained by updating the cluster points of the second spatial clusters. This second structured data is similar in content to the first structured data and can be easily deduced by analogy.

[0056] Step 140: Perform global spatial analysis on the second structured data to obtain the global defect location result of the inspection image.

[0057] In this embodiment of the application, global spatial analysis can be performed by converting each JSON record in the second structured data into spatial point data and then plotting it on the DOM (Digital Orthophoto Map) of the inspection plan to obtain global defect location results. Each spatial point in the global defect location results includes the inspection image and its defect type, timestamp, target coordinate information, target confidence, etc.

[0058] In some embodiments, the method further includes: Heatmap analysis was performed on the second structured data to obtain the inspection heatmap; Based on the inspection heatmap, the global defect location results are rendered and updated to obtain the updated global defect location results.

[0059] In this embodiment, heatmap analysis first involves converting each JSON record in the second structured data into spatial point data and plotting it on a GIS map, then converting the latitude and longitude coordinates of the spatial points in the GIS map into projected coordinates. Next, a kernel density algorithm is used to analyze the density of the spatial point distribution in the GIS map, and the resulting density distribution is then visualized as a continuous heatmap representation (i.e., an inspection heatmap). Rendering updates can convert the inspection heatmap into a semi-transparent layer and then overlay this semi-transparent layer onto the DOM base map of the global defect location results, thereby obtaining a visualized and updated global defect location result. This visualized and updated global defect location result can intuitively display the inspection status to maintenance personnel, which helps reduce their subsequent workload.

[0060] The following describes in detail, with reference to the accompanying drawings, a global defect localization system for inspection images proposed according to an embodiment of this application.

[0061] Reference Figure 2 The global defect localization system for inspection images proposed in this application includes: The first processing unit 101 is used to acquire inspection images collected during the inspection process of the UAV, the inspection images including a number of inspection images containing embedded metadata. The second processing unit 102 is used to input the inspection image into a trained defect detection model for image detection, and obtain the first structured data output by the trained defect detection model; the first structured data includes defect detection results of several inspection images, and spatiotemporal indexes of the defect detection results; each spatiotemporal index corresponds to the embedded metadata of an inspection image. The third processing unit 103 is used to perform spatial defect point deduplication processing on the first structured data to obtain the second structured data. The fourth processing unit 104 is used to perform global spatial analysis on the second structured data to obtain the global defect location result of the inspection image.

[0062] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0063] Reference Figure 3 This application also provides an electronic device, including: At least one processor 201; At least one memory 202 is used to store at least one program; When the at least one program is executed by the at least one processor 201, the at least one processor 201 implements the method embodiment described above.

[0064] Similarly, it can be understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0065] This application also provides a computer-readable storage medium storing a program executable by a processor 201, which, when executed by the processor 201, is used to implement the above-described method embodiments.

[0066] Similarly, the content of the above method embodiments is applicable to the present computer-readable storage medium embodiments. The specific functions implemented by the present computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0067] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0069] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0070] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0071] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0073] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0074] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0075] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0076] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0077] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for global defect localization in inspection images, characterized in that, include: Acquire inspection images collected during the drone inspection process, wherein the inspection images include several inspection images containing embedded metadata; The inspection images are input into a trained defect detection model for image detection, resulting in first structured data output by the trained defect detection model. The first structured data includes defect detection results of several inspection images and spatiotemporal indexes of the defect detection results. Each spatiotemporal index corresponds to the embedded metadata of an inspection image. The first structured data is deduplicated by removing spatial defect points to obtain the second structured data; Global spatial analysis is performed on the second structured data to obtain the global defect location results of the inspection image; The step of inputting the inspection image into a trained defect detection model for image detection, and obtaining the first structured data output by the trained defect detection model, includes: Metadata parsing is performed on the embedded metadata of the inspection image to obtain the spatiotemporal index, which includes spatial coordinate information and timestamp information; The inspection image is input into the trained defect detection model to perform defect detection, and the defect detection result is obtained. Based on the spatiotemporal index, the defect detection results are subjected to structured correlation processing to obtain the first structured data; The step of performing spatial defect point deduplication on the first structured data to obtain the second structured data includes: Spatial clustering is performed on the first structured data to obtain several first spatial clusters; All first spatial clusters are deduplicated to obtain several second spatial clusters, and each second spatial cluster corresponds to a first spatial cluster after deduplication. The second structured data is obtained based on all the second spatial clusters; The process of deduplicating cluster points in all the first spatial clusters yields several second spatial clusters, including: Obtain the time interval threshold; Based on the time interval threshold, the timestamps of all cluster points in the first spatial cluster are analyzed to obtain several first cluster groups; the first cluster group includes several first cluster points, and the timestamp interval between the second cluster point and the third cluster point is less than the time interval threshold; the second cluster point is any one of the first cluster points in the first cluster group; the third cluster point is the first cluster point in the first cluster group that is adjacent to the second cluster point in the time series, excluding the second cluster point. Defect analysis is performed on all the first cluster groups to obtain several second cluster groups; Based on the second cluster group, the first spatial cluster is updated to obtain the second spatial cluster.

2. The method according to claim 1, characterized in that, The spatial clustering of the first structured data yields several first spatial clusters, including: Obtain the neighborhood radius and minimum number of points threshold; Based on the neighborhood radius, all spatiotemporal indexes in the first structured data are clustered in a neighborhood to obtain several intermediate clusters; Based on the minimum number of points threshold, all intermediate clusters are filtered by cluster point threshold to obtain several first spatial clusters.

3. The method according to claim 1, characterized in that, The defect analysis performed on all the first clusters yields several second clusters, including: Obtain a similarity threshold and several defect feature vectors, where each defect feature vector is a feature vector of the defect region of the corresponding first cluster point; Based on the similarity threshold, a defect similarity analysis is performed on all the defect feature vectors to obtain the defect analysis results. The defect analysis results are used to characterize whether all cluster points in the first cluster group belong to the spatial points of the same defect. If the defect analysis result indicates that all cluster points within the first cluster group belong to the spatial points of the same defect, then the first cluster group is updated to obtain the second cluster group.

4. The method according to claim 3, characterized in that, The step of updating the first cluster group to obtain the second cluster group includes: Obtain the confidence information and spatial coordinate information of each first cluster point in the first cluster group; Filter all the confidence information to obtain the target confidence level, which is the highest confidence level among all the confidence information. Coordinate optimization is performed on all the aforementioned spatial coordinate information to obtain the target coordinate information; Based on the target confidence level and target coordinate information, the first cluster group is deduplicated and updated to obtain the second cluster group.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Heatmap analysis was performed on the second structured data to obtain the inspection heatmap; Based on the inspection heatmap, the global defect location results are rendered and updated to obtain the updated global defect location results.

6. A global defect localization system for inspection images, characterized in that, include: The first processing unit is used to acquire inspection images collected during the inspection process of the UAV, the inspection images including several inspection images containing embedded metadata. The second processing unit is used to input the inspection image into a trained defect detection model for image detection, and obtain the first structured data output by the trained defect detection model; the first structured data includes defect detection results of several inspection images, and spatiotemporal indexes of the defect detection results; each spatiotemporal index corresponds to the embedded metadata of an inspection image. The third processing unit is used to perform spatial defect point deduplication processing on the first structured data to obtain the second structured data; The fourth processing unit is used to perform global spatial analysis on the second structured data to obtain the global defect location result of the inspection image; The step of inputting the inspection image into a trained defect detection model for image detection, and obtaining the first structured data output by the trained defect detection model, includes: Metadata parsing is performed on the embedded metadata of the inspection image to obtain the spatiotemporal index, which includes spatial coordinate information and timestamp information; The inspection image is input into the trained defect detection model to perform defect detection, and the defect detection result is obtained. Based on the spatiotemporal index, the defect detection results are subjected to structured correlation processing to obtain the first structured data; The step of performing spatial defect point deduplication on the first structured data to obtain the second structured data includes: Spatial clustering is performed on the first structured data to obtain several first spatial clusters; All first spatial clusters are deduplicated to obtain several second spatial clusters, and each second spatial cluster corresponds to a first spatial cluster after deduplication. The second structured data is obtained based on all the second spatial clusters; The process of deduplicating cluster points in all the first spatial clusters yields several second spatial clusters, including: Obtain the time interval threshold; Based on the time interval threshold, the timestamps of all cluster points in the first spatial cluster are analyzed to obtain several first cluster groups; the first cluster group includes several first cluster points, and the timestamp interval between the second cluster point and the third cluster point is less than the time interval threshold; the second cluster point is any one of the first cluster points in the first cluster group; the third cluster point is the first cluster point in the first cluster group that is adjacent to the second cluster point in the time series, excluding the second cluster point. Defect analysis is performed on all the first cluster groups to obtain several second cluster groups; Based on the second cluster group, the first spatial cluster is updated to obtain the second spatial cluster.

7. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1-5.

Images