An unmanned aerial vehicle target detection and positioning method based on visual intelligence

Abstract

This invention relates to the field of UAV inspection technology and discloses a UAV target detection and localization method based on visual intelligence. The method includes: acquiring visible light images, infrared thermal maps, laser depth data, inertial measurement attitude data, and topological data of the inspection object; performing spatiotemporal registration on the above data to establish a correspondence between image frames and the inspection object; performing defect detection to obtain defect categories, target bounding boxes, and three types of candidate anchor points; determining the primary anchor point based on the defect category; verifying the primary anchor point based on the topological data of the inspection object and determining the defect entity anchor point; and calculating the geodetic latitude and longitude coordinates of the defect target based on the defect entity anchor point. By generating multimodal candidate anchor points and combining them with the topological data of the inspection object for consistency verification, the defect entity anchor point used for coordinate calculation is determined, avoiding direct positioning using the center of the target bounding box or a single modal point, and reducing the deviation of the defect location point from the actual fault location.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, specifically to a UAV target detection and localization method based on visual intelligence. Background Technology

[0002] Drone inspection refers to the operation of using drones to collect images, identify the status, and record the location of objects such as power lines, photovoltaic power stations, pipeline facilities, and park equipment along a preset route. With the development of machine vision and multi-source sensing technology, drone inspection has gradually evolved from manually viewing images to automatically identifying the objects through visible light images, infrared thermal maps, laser depth data, and attitude data, and converting the identified defective targets into geographic coordinates that can be used for inspection records.

[0003] In current UAV inspection processes, defect target boxes are typically identified in images first using a detection model. Then, the center of the target box, a single-modal detection point, or a ranging point is used as the defect location point, and the target coordinates are calculated by combining attitude data, laser depth data, or digital elevation maps. However, in practical applications, industrial inspection defects often exhibit localized and eccentric characteristics. For example, the locations of thermal anomalies, external damage, and depth anomalies may not align with the center of the target box. Directly using the center of the target box or a single-modal point for coordinate calculation can easily cause the output defect coordinates to deviate from the actual fault location. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a visual intelligence-based method for UAV target detection and localization, solving the problem that defect location points in UAV inspections are prone to deviating from the actual fault locations.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a visual intelligence-based method for UAV target detection and localization, comprising: S1. Acquire visible light images, infrared thermal maps, laser depth data, inertial measurement attitude data, and topological data of the inspection object collected by the inspection drone. The topological data of the inspection object includes the object type, object identifier, and spatial range of the inspection object. S2. Perform spatiotemporal registration on the visible light image, infrared thermal map, laser depth data, and inertial measurement attitude data to establish the correspondence between image frames and inspection objects; S3. Perform defect detection on the registered visible light image, infrared thermal map and laser depth data to obtain the defect category and target box, and generate visible light candidate anchor points, infrared candidate anchor points and laser candidate anchor points within the target box; S4. Based on the defect category and the preset correspondence between defect category and anchor point type, determine the primary anchor point from the visible light candidate anchor point, infrared candidate anchor point and laser candidate anchor point; S5. Perform consistency verification on the primary anchor point based on the topology data of the inspected object; when the primary anchor point fails the consistency verification, determine the candidate anchor point that passes the consistency verification from the remaining candidate anchor points as the defect entity anchor point; S6. Based on the anchor points of the defective entity, inertial measurement attitude data, laser depth data, and digital elevation map, calculate the geodetic latitude and longitude coordinates of the defective target.

[0006] Preferably, establishing the correspondence between image frames and inspection objects in step S2 includes the following steps: According to the acquisition timestamp, the visible light image, infrared thermal map, laser depth data and inertial measurement attitude data are synchronized to the image frame at the same acquisition time. Based on the pre-calibrated coordinate transformation parameters between the sensors, the infrared thermal map and laser depth data are projected onto the image coordinate system of the visible light image; Based on the spatial range in the topological data of the inspected object, the inspected object region identified in the image coordinate system is spatially matched with the corresponding object identifier to establish an association.

[0007] Preferably, the defect detection in step S3 of the registered visible light image, infrared thermogram, and laser depth data includes the following steps: Extract appearance anomaly features from the visible light image, extract thermal anomaly features from the infrared thermogram, and extract depth anomaly features from the laser depth data; Based on the preset correspondence between industrial defect types and modal weights, the appearance anomaly features, thermal anomaly features, and depth anomaly features are weighted and fused to obtain fused detection features; The correspondence between the industrial defect types and modal weights includes: the infrared modal weight of thermal anomaly defects is greater than the visible light modal weight and the laser modal weight; the visible light modal weight of appearance damage defects is greater than the infrared modal weight and the laser modal weight; and the laser modal weight of geometric deformation defects is greater than the visible light modal weight and the infrared modal weight. The fused detection features are input into the airborne defect detection model to obtain the defect category and target box.

[0008] Preferably, step S3, generating visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the target box, includes the following steps: Within the target box, a visible light abnormal region is determined based on the feature map spatial response region corresponding to the appearance abnormality feature, and a point is selected from the edge center, skeleton center and region centroid of the visible light abnormal region as a visible light candidate anchor point according to the defect category. Within the target box, an infrared anomaly region is determined based on the feature map spatial response region corresponding to the thermal anomaly feature, and a point is selected as an infrared candidate anchor point from the temperature rise peak point and the centroid of the region according to the defect category. Within the target bounding box, a depth anomaly region is determined based on the feature map spatial response region corresponding to the depth anomaly feature, and the maximum deviation point of the depth value in the depth anomaly region from the mean of the neighborhood, or the centroid of the depth anomaly region, is determined as a candidate laser anchor point.

[0009] Preferably, the process of determining the primary anchor point from the visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points in step S4 includes the following steps: The aforementioned defect categories are classified into thermal anomaly defects, appearance damage defects, or geometric deformation defects; The primary anchor point type is determined based on the defect category corresponding to the defect type. Specifically, thermal anomaly defects correspond to infrared anchor point types, appearance damage defects correspond to visible light anchor point types, and geometric deformation defects correspond to laser anchor point types. The candidate anchor points corresponding to the primary anchor point type are determined as primary anchor points.

[0010] Preferably, classifying the defect into thermal anomaly defects, appearance damage defects, or geometric deformation defects includes the following steps: When the detection result corresponding to the defect category contains a temperature rise area in the infrared thermogram, the defect category is classified as a thermal anomaly defect. When the detection result corresponding to the defect category includes edge breakage area, texture abnormal area or surface color abnormal area in the visible light image, the defect category is classified as appearance damage defect. When the detection result corresponding to the defect category contains a depth abrupt change region or a geometric deviation region in the laser depth data, the defect category is classified as a geometric deformation defect.

[0011] Preferably, step S5, which involves performing a consistency check on the primary anchor point based on the topology data of the inspected object, includes the following steps: Based on the position of the target box where the main anchor point is located in the image coordinate system, combined with the camera projection matrix, inertial measurement attitude data and digital elevation map elevation, the target box is projected onto the local coordinate system of the digital elevation map to determine the inspection object area and object identifier corresponding to the target box; Based on the position of the primary anchor point in the image coordinate system, the laser depth data of the corresponding position, and the inertial measurement attitude data, the primary anchor point is converted into a spatial point to be verified through coordinate transformation. The coordinate transformation includes, in sequence, the transformation from pixel coordinate system to camera coordinate system, from camera coordinate system to body coordinate system, from body coordinate system to navigation coordinate system, and from navigation coordinate system to local coordinate system of digital elevation map. The spatial point to be verified is compared with the topology data of the inspected object. When the spatial point to be verified is located within the spatial range corresponding to the object identifier, and the object identifier corresponding to the spatial point to be verified is consistent with the object identifier of the inspected object area, the primary anchor point is determined to have passed the consistency verification.

[0012] Preferably, when the primary anchor point fails the consistency check, determining the candidate anchor points that pass the consistency check from the remaining candidate anchor points as defective entity anchor points includes the following steps: When there are other candidate anchor points besides the primary anchor point, each candidate anchor point other than the primary anchor point is converted into a candidate spatial point through coordinate transformation, and each candidate spatial point is verified according to the consistency verification method. The candidate anchor points corresponding to the candidate spatial points that pass the consistency check are determined as candidate entity anchor points; When there are multiple candidate entity anchor points, the candidate entity anchor point with the highest matching priority is determined as the defect entity anchor point according to the matching priority between the modal type corresponding to each candidate anchor point and the defect category. When the primary anchor point and the remaining candidate anchor points fail the consistency check, the primary anchor point is marked as an anchor point to be reviewed, and the defect target corresponding to the target box is marked as a defect target to be reviewed.

[0013] Preferably, the process of calculating the geodetic latitude and longitude coordinates of the defective target in step S6 includes the following steps: Obtain the pixel position of the defect entity anchor point in the image coordinate system, and determine the spatial viewing direction corresponding to the defect entity anchor point based on the pixel position and the camera projection matrix; Based on the laser depth data of the spatial line of sight and the corresponding position of the defect entity anchor point, the spatial position of the defect entity anchor point in the camera coordinate system is determined. Based on the inertial measurement attitude data and the pre-calibrated coordinate transformation parameters, the spatial position of the defect entity anchor point in the camera coordinate system is sequentially transformed to the body coordinate system, the navigation coordinate system, and the local coordinate system of the digital elevation map. Based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map is converted into the geodetic latitude and longitude coordinates of the defect target.

[0014] Preferably, step S6 further includes the following step: When the laser depth data corresponding to the defect entity anchor point is not available, or the spatial position calculated based on the laser depth data is not within the spatial range of the corresponding object identifier in the topology data of the inspected object, the spatial line of sight corresponding to the defect entity anchor point is determined based on the pixel position of the defect entity anchor point in the image coordinate system and the camera projection matrix. The intersection of the spatial line of sight with the elevation of the digital elevation map and the spatial range in the topology data of the inspected object is obtained to obtain the compensated spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map. Based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the compensation spatial location is converted into the geodetic latitude and longitude coordinates of the defective target.

[0015] This invention provides a visual intelligence-based method for target detection and localization in unmanned aerial vehicles (UAVs). It offers the following advantages: 1. This invention generates visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the target box, and performs consistency verification on the primary anchor points by combining the topology data of the inspected object. This avoids directly using the center of the target box or a single modal point for positioning, and ensures that the defect entity anchor points on which the coordinate calculation is based correspond to the actual inspected object, thereby reducing the deviation of the defect positioning point from the actual fault location.

[0016] 2. This invention determines the primary anchor point type based on the defect category, prioritizing the association of thermal anomaly defects with infrared anchor points, appearance damage defects with visible light anchor points, and geometric deformation defects with laser anchor points, thereby enabling different industrial defects to adopt positioning criteria that match their physical characteristics.

[0017] 3. When laser depth data is unavailable or the calculation results do not match the spatial range of the inspected object, this invention uses the elevation of the digital elevation map and the topological data of the inspected object to determine the compensation spatial location, so that the defective target can still be converted into geodetic latitude and longitude coordinates, reducing the impact of depth data anomalies on the positioning results. Attached Figure Description

[0018] Figure 1 This is a three-dimensional flowchart of a visual intelligence-based unmanned aerial vehicle (UAV) target detection and localization method according to the present invention. Figure 2 A flowchart for anchor point consistency verification and defective entity anchor point determination in this invention; Figure 3 This is a schematic diagram of the planar structure of an embodiment of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see the appendix Figure 1 -Appendix Figure 3 This invention provides a visual intelligence-based method for unmanned aerial vehicle (UAV) target detection and localization, comprising: S1. Acquire visible light images, infrared thermal maps, laser depth data, inertial measurement attitude data, and topology data of the inspection object collected by the inspection drone. The topology data of the inspection object includes the object type, object identifier, and spatial range of the inspection object. Specifically, during the inspection process, the inspection drone collects visible light images, infrared thermal maps, laser depth data, and inertial measurement attitude data. It also reads the topology data of the inspection objects corresponding to the current inspection task. This topology data is pre-established according to the inspection objects within the inspection area and includes at least object type, object identifier, and spatial range. The object type distinguishes different categories of inspection objects, the object identifier distinguishes different inspection objects within the same inspection area, and the spatial range indicates the location of the corresponding inspection object under the coordinate reference of the inspection area. This data is cached by the onboard processing unit of the inspection drone and serves as the data basis for subsequent spatiotemporal registration, defect detection, candidate anchor point generation, and coordinate calculation.

[0021] S2. Perform spatiotemporal registration on visible light images, infrared thermal maps, laser depth data, and inertial measurement attitude data to establish the correspondence between image frames and inspection objects; Furthermore, establishing the correspondence between image frames and inspected objects in S2 includes the following steps: According to the acquisition timestamp, the visible light image, infrared thermal map, laser depth data and inertial measurement attitude data are synchronized to the image frame at the same acquisition time. Based on the pre-calibrated coordinate transformation parameters between the sensors, the infrared thermal map and laser depth data are projected onto the image coordinate system of the visible light image; Based on the spatial range in the topological data of the inspected objects, the inspected object regions identified in the image coordinate system are spatially matched with the corresponding object identifiers to establish a correlation.

[0022] Specifically, when performing spatiotemporal registration on various types of data, the acquisition timestamps carried by each data are first read, and the frame time of the visible light image is used as the reference time. The infrared thermal map, laser depth data, and inertial measurement attitude data are synchronized to the image frame at the same acquisition time. For data with different acquisition times, the data frame with the smallest time difference can be selected for matching, or interpolation can be performed based on data from adjacent acquisition times, so that the same image frame contains visible light image, infrared temperature distribution, laser depth value, and attitude information. After this processing, each frame of visible light image corresponds to a set of infrared thermal map data, laser depth data, and inertial measurement attitude data. Subsequent processing can analyze the appearance, thermal state, spatial distance, and UAV attitude of the inspection object at the same acquisition time. After time synchronization is completed, the pre-calibrated coordinate transformation parameters between each sensor are called to perform spatial mapping on the infrared thermal map and laser depth data. Specifically, the temperature pixels in the infrared thermal map are projected into the image coordinate system of the visible light image according to the coordinate transformation relationship between the infrared data and the visible light image. The laser depth data is projected into the corresponding pixel position or pixel area of ​​the visible light image according to the coordinate transformation relationship between the laser depth acquisition direction and the visible light image. Thus, the same image coordinate position can be associated with appearance pixels, infrared temperature values ​​and laser depth values. Taking the inspection of power transmission lines as an example, when an insulator string area appears in the visible light image, the corresponding infrared temperature distribution and laser depth value in that area are mapped to the same image coordinate range, which is convenient for extracting candidate anchor points in the target box later. Based on the spatial range in the topology data of the inspected objects, the inspected object regions identified in the image coordinate system are spatially matched with the corresponding object identifiers. In practice, the inspected objects that can be projected into the current field of view can be determined first based on the pose information of the current image frame and the spatial range in the topology data of the inspected objects. Then, the spatial ranges of these inspected objects are projected into the image coordinate system of the visible light image and compared with the inspected object regions identified in the image. If a certain identified region and the projected region corresponding to a certain object identifier meet the spatial overlap condition, then the identified region and the object identifier are associated. After the association is established, the target boxes obtained by subsequent defect detection not only have the image position, but also correspond to the specific inspected object and its spatial range, thus providing the object identifier and spatial range basis for the consistency verification of the subsequent primary anchor points.

[0023] S3. Perform defect detection on the registered visible light image, infrared thermal map and laser depth data to obtain the defect category and target box, and generate visible light candidate anchor points, infrared candidate anchor points and laser candidate anchor points within the target box; Furthermore, in S3, defect detection of the registered visible light image, infrared thermogram, and laser depth data includes the following steps: Extract appearance anomaly features from visible light images, extract thermal anomaly features from infrared thermograms, and extract depth anomaly features from laser depth data; Based on the preset correspondence between industrial defect types and modal weights, appearance anomaly features, thermal anomaly features, and depth anomaly features are weighted and fused to obtain fused detection features; Among them, the correspondence between industrial defect types and modal weights includes: the infrared modal weight of thermal anomaly defects is greater than the visible light modal weight and the laser modal weight; the visible light modal weight of appearance damage defects is greater than the infrared modal weight and the laser modal weight; and the laser modal weight of geometric deformation defects is greater than the visible light modal weight and the infrared modal weight. The fused detection features are input into the airborne defect detection model to obtain the defect category and target bounding box.

[0024] Specifically, when processing the registered visible light image, infrared thermal map, and laser depth data, three types of anomaly features are first constructed within the same target area. The visible light image is used to extract information on the edges, textures, color abrupt changes, and contour breaks of the surface of the inspected object, thereby forming appearance anomaly features. The infrared thermal map is used to extract temperature differences, areas of concentrated temperature rise, and thermal boundary distribution between the target area and the surrounding area, thereby forming thermal anomaly features. The laser depth data is used to extract information on depth continuity changes, local depth abrupt changes, and surface geometric deviations within the target area, thereby forming depth anomaly features. All three types of anomaly features are limited to the image frames that have been registered, so that the same pixel area or corresponding image area can simultaneously have appearance, thermal distribution, and depth attributes, avoiding the mixing of features from different inspected objects during subsequent fusion. After obtaining the appearance anomaly features, thermal anomaly features, and depth anomaly features, a weighted fusion is performed according to the preset correspondence between industrial defect types and modal weights. The fused detection features can be determined according to the following formula: ; in, Indicates the defect category The corresponding fusion detection features, This represents the appearance anomaly features extracted from visible light images. This represents the thermal anomaly features extracted from the infrared thermogram. This represents depth anomaly features extracted from laser depth data. Indicates the defect category The corresponding visible light mode weights, Indicates the defect category The corresponding infrared mode weights, Indicates the defect category The corresponding laser mode weights, the three types of weights satisfy When the defect category is thermal anomaly, the infrared mode weight is greater than the visible light mode weight and the laser mode weight. When the defect category is appearance damage, the visible light mode weight is greater than the infrared mode weight and the laser mode weight. When the defect category is geometric deformation, the laser mode weight is greater than the visible light mode weight and the infrared mode weight. Taking joint overheating defects as an example, the infrared mode weight can be set to 0.6, the visible light mode weight to 0.3, and the laser mode weight to 0.1. Taking insulator crack defects as an example, the visible light mode weight can be set to 0.6, the infrared mode weight to 0.2, and the laser mode weight to 0.2. The above weights are used to determine the proportion of different modal features participating in the fusion detection features, so that the fusion detection features are consistent with the main manifestation of the current defect type. After obtaining the fused detection features, the fused detection features are sent to the airborne defect detection model for target region localization and category determination. During implementation, the airborne defect detection model performs multi-scale feature extraction on the fused detection features and calculates the position offset, target existence confidence, and defect category confidence of the candidate target region on the feature maps at different scales. For each candidate target region, the model determines the position of the target box in the image coordinate system based on the position offset, determines the corresponding defect category based on the defect category confidence, and filters overlapping candidate target boxes, retaining the target boxes that match the defect category. The resulting defect category and target box are then used to generate visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the target box.

[0025] Furthermore, generating visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the target bounding box in S3 includes the following steps: Within the target bounding box, the visible light abnormal region is determined based on the feature map spatial response region corresponding to the appearance abnormality features. Then, a point is selected from the edge center, skeleton center, and region centroid of the visible light abnormal region as a visible light candidate anchor point according to the defect category. Within the target box, the infrared anomaly region is determined based on the feature map spatial response region corresponding to the thermal anomaly features. Then, a point is selected from the temperature rise peak point and the centroid of the region in the infrared anomaly region as an infrared candidate anchor point according to the defect category. Within the target bounding box, the depth anomaly region is determined based on the spatial response region of the feature map corresponding to the depth anomaly feature, and the point where the depth value in the depth anomaly region deviates most from the mean of the neighborhood, or the centroid of the depth anomaly region, is determined as the candidate anchor point for laser.

[0026] Specifically, after the target bounding box is determined, the generation of visible light candidate anchor points takes the response region of the feature map space corresponding to the appearance anomaly feature as the processing object. In practice, the target bounding box is mapped onto the feature map where the appearance anomaly feature is located, the response region corresponding to the target bounding box is extracted, and the visible light anomaly region is determined according to the response intensity. The visible light anomaly region can be formed by edge abrupt changes, texture breaks, or surface color anomalies. For crack-like defects, the skeleton line can be extracted along the anomaly region, and the center of the skeleton line can be used as the visible light candidate anchor point. For broken strands or edge damage defects, the edge center can be determined according to the edge distribution of the anomaly region. For corrosion or surface peeling defects, the centroid of the region can be determined according to the pixel distribution within the anomaly region. The visible light candidate anchor point can be represented as: ; in, Indicates candidate anchor points for visible light. This represents the feature map spatial response region corresponding to the abnormal appearance features. Indicates the center of the edge of the visible light anomaly region. Indicates the skeletal center of the visible light anomaly region. The centroid of the visible light anomaly region is represented. In actual selection, a point is determined from the above points according to the defect category as a candidate anchor point for visible light, and this point is retained in the image coordinate system of the target box for subsequent primary anchor point selection. The generation of infrared candidate anchor points uses the spatial response region of the feature map corresponding to the thermal anomaly features as the processing object. Specifically, the target box is mapped onto the feature map containing the thermal anomaly features, and continuous regions within the target box with higher thermal responses than the surrounding areas are extracted to form infrared anomaly regions. In one embodiment, referring to... Figure 3 For thermal anomaly defects such as joint overheating, photovoltaic hot spots, or leakage temperature differences, when a clear temperature rise peak point exists within the infrared anomaly region, the temperature rise peak point is selected as the infrared candidate anchor point. When the infrared anomaly region is distributed in a patchy pattern and the peaks are not concentrated, the centroid of the region is selected as the infrared candidate anchor point. The infrared candidate anchor point can be represented as: ; in, Indicates infrared candidate anchor points, This represents the spatial response region of the feature map corresponding to the thermal anomaly characteristics. This indicates the peak temperature rise point in the infrared anomaly region. The centroid of the infrared anomaly region is indicated. For example, within the target frame of a photovoltaic module, if the highest temperature point corresponding to the infrared anomaly region is located at image coordinates (238, 156), and the centroid of the region is located at image coordinates (241, 160), when the defect type is a hot spot defect and the temperature rise peak is concentrated, (238, 156) will be used as the infrared candidate anchor point. When the temperature rise region is relatively uniformly distributed, (241, 160) will be used as the infrared candidate anchor point. The generation of laser candidate anchor points focuses on the spatial response region of the feature map corresponding to the depth anomaly features. Specifically, the target bounding box is mapped onto the feature map containing the depth anomaly features. Regions within the target bounding box where the depth continuity changes are extracted, forming depth anomaly regions. For cases with significant local deformation or component offset, the difference between the depth value of each pixel in the depth anomaly region and the average depth of its neighbors is calculated, and the pixel with the largest difference is selected as the laser candidate anchor point. For cases where the depth anomaly region exhibits a continuous planar change, the centroid of the depth anomaly region is selected as the laser candidate anchor point. The laser candidate anchor point can be represented as: ; in, Indicates the candidate anchor point for laser. This represents the spatial response region of the feature map corresponding to the deep anomaly features. The centroid of the region representing the deep anomaly. This represents the point where the depth value in the depth anomaly region deviates most from the neighborhood mean. This maximum deviation point is determined by the following formula: ; in, Indicates a deep anomaly region Pixels within, Represents pixels The corresponding laser depth value, Represents pixels The average laser depth within the neighborhood is processed as described above to form visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the same target box. These three types of candidate anchor points correspond to the locations of appearance anomalies, thermal anomalies, and depth anomalies, respectively, and will participate in the determination of the primary anchor point in subsequent steps based on the defect category.

[0027] S4. Based on the defect category and the preset correspondence between defect category and anchor point type, determine the primary anchor point from visible light candidate anchor points, infrared candidate anchor points and laser candidate anchor points; Furthermore, in S4, determining the primary anchor point from visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points includes the following steps: The defects are categorized into thermal anomaly defects, appearance damage defects, or geometric deformation defects. The primary anchor point type is determined based on the defect category corresponding to the defect type. Specifically, thermal anomaly defects correspond to infrared anchor point types, appearance damage defects correspond to visible light anchor point types, and geometric deformation defects correspond to laser anchor point types. The candidate anchor points corresponding to the primary anchor point type are determined as the primary anchor points.

[0028] Specifically, after obtaining the defect category and the visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the same target box, the defect category is first classified into the corresponding defect category according to its manifestation. In practice, the detection results corresponding to thermal anomaly defects usually show a temperature rise area in the infrared thermogram, the detection results corresponding to appearance damage defects usually show edge breakage, texture abnormality, or surface color abnormality in the visible light image, and the detection results corresponding to geometric deformation defects usually show depth abrupt change or geometric deviation in the laser depth data. This classification result is used to determine which modality's corresponding candidate anchor point should be used first, rather than directly using the center of the target box as the positioning point. The relationship between defect categories and primary anchor point types can be determined through a preset mapping relationship, which can be expressed as follows: ; in, Indicates the defect category, Indicates the defect category The corresponding primary anchor point type, Indicates the type of infrared anchor point. Indicates the type of visible light anchor point. Indicates the type of laser anchor point. Represents a set of defects of the thermal anomaly type. This represents a set of defects related to physical damage. This represents the set of geometric deformation defects. According to this mapping relationship, thermal anomaly defects correspond to infrared anchor point types, appearance damage defects correspond to visible light anchor point types, and geometric deformation defects correspond to laser anchor point types. After determining the primary anchor point type, the primary anchor point is selected from the candidate anchor points of the corresponding type. The primary anchor point can be represented as: ; in, Indicates the primary anchor point. Indicates the defect category For candidate anchor points under the corresponding primary anchor point type, if the defect category is joint overheating defect, this defect category will be classified into the thermal anomaly defect set. ,but Correspondingly, infrared candidate anchor points are determined as primary anchor points. If the defect category is insulator crack defect, this defect category is classified into the set of appearance damage defects. ,but The visible light candidate anchor points are determined as the primary anchor points. If the defect category is pipeline deformation defect, this defect category is classified into the geometric deformation defect set. ,but Correspondingly, the laser candidate anchor points are determined as the primary anchor points. Thus, the primary anchor points correspond to the main physical manifestations of the defect categories and serve as the objects to be verified in subsequent consistency checks.

[0029] Furthermore, classifying defects into thermal anomaly defects, appearance damage defects, or geometric deformation defects includes the following steps: When the detection result corresponding to the defect category contains the temperature rise area in the infrared thermogram, the defect category is classified as a thermal anomaly defect. When the detection results corresponding to the defect category include edge breakage areas, texture abnormal areas, or surface color abnormal areas in the visible light image, the defect category will be classified as appearance damage defects. When the detection results corresponding to a defect category contain a region of abrupt depth change or geometric deviation in the laser depth data, the defect category is classified as a geometric deformation defect.

[0030] Specifically, when the detection result contains a temperature rise area in the infrared thermal map, the corresponding defect category is classified as a thermal anomaly defect. In practice, the temperature distribution is first extracted in the infrared thermal map area corresponding to the target box, and the temperature value of the target area is compared with the temperature value of the surrounding background area. If there is a continuous temperature rise area in the target area, and the continuous temperature rise area overlaps with the detected defect target in the image coordinate system, then the defect category is classified as a thermal anomaly defect. Taking photovoltaic module inspection as an example, when a local infrared temperature rise area appears in the target box, and the temperature rise area is located in the identified photovoltaic module area, the corresponding defect category can be classified as a thermal anomaly defect. Subsequently, the infrared anchor point type is preferred. When the detection results contain edge breakage areas, texture abnormal areas, or surface color abnormal areas in the visible light image, the corresponding defect category will be classified as appearance damage defects. During implementation, the edge continuity, texture distribution, and surface color distribution will be extracted in the visible light image area corresponding to the target box, and compared with the normal appearance area in the same inspection object area. If there are edge interruptions, crack-like textures, local peeling, corrosion discoloration, or color abrupt changes in the target area, the defect category will be classified as appearance damage defects. Taking insulator inspection as an example, when a through or locally extended crack texture appears in the visible light image area within the target box, and the texture is located within the insulator area, the corresponding defect category can be classified as appearance damage defects. The visible light anchor point type will be preferred in subsequent primary anchor point selection. When the detection results contain depth abrupt changes or geometric deviations in the laser depth data, the corresponding defect category is classified as a geometric deformation defect. That is, the depth value distribution is extracted within the laser depth region corresponding to the target box, and the deviation of the depth value within the target region from the average depth of the neighborhood or the normal surface of the same inspected object is calculated. If there are continuous depth abrupt changes, local protrusions, local depressions, or spatial orientation deviations within the target region, the defect category is classified as a geometric deformation defect. Taking pipeline inspection as an example, when the laser depth data within the target box indicates that the pipeline surface has local protrusions or bending deviations relative to adjacent pipe sections, the corresponding defect category can be classified as a geometric deformation defect, and the laser anchor point type is preferred for subsequent primary anchor point selection.

[0031] Reference Figure 2 S5. Perform consistency verification on the primary anchor point based on the topology data of the inspected object; if the primary anchor point fails the consistency verification, select the candidate anchor point that passes the consistency verification from the remaining candidate anchor points as the defect entity anchor point. Furthermore, the consistency verification of the primary anchor point based on the topology data of the inspected object in S5 includes the following steps: Based on the position of the target box where the main anchor point is located in the image coordinate system, combined with the camera projection matrix, inertial measurement attitude data and digital elevation map elevation, the target box is projected onto the local coordinate system of the digital elevation map to determine the inspection object area and object identifier corresponding to the target box. Based on the position of the primary anchor point in the image coordinate system, the laser depth data of the corresponding position, and the inertial measurement attitude data, the primary anchor point is converted into a spatial point to be verified through coordinate transformation. The coordinate transformation includes the transformation from pixel coordinate system to camera coordinate system, from camera coordinate system to body coordinate system, from body coordinate system to navigation coordinate system, and from navigation coordinate system to local coordinate system of digital elevation map. The spatial point to be verified is compared with the topology data of the inspected object. When the spatial point to be verified is located within the spatial range corresponding to the object identifier, and the object identifier corresponding to the spatial point to be verified is consistent with the object identifier of the inspected object area, the primary anchor point is determined to have passed the consistency verification.

[0032] Specifically, first, based on the position of the target bounding box where the primary anchor point is located in the image coordinate system, the image region corresponding to the target bounding box is determined. Then, combined with the camera projection matrix, inertial measurement attitude data, and digital elevation map elevation, this image region is projected onto the local coordinate system of the digital elevation map. During projection, the pixel region within the target bounding box forms a spatial line-of-sight range according to the camera projection matrix. The inertial measurement attitude data is used to determine the attitude direction of the UAV at the time of acquisition, and the digital elevation map elevation is used to constrain the landing point of this spatial line-of-sight range in the inspection area. After projection, the landing point area is matched with the spatial range in the topology data of the inspection object to determine the inspection object region and object identifier corresponding to the target bounding box. For example, in photovoltaic inspection, if the area after the target bounding box is projected falls within the spatial range of a photovoltaic module, then a correspondence is established between the target bounding box and the object identifier of the photovoltaic module. Subsequently, the selected anchor point is converted from its image coordinate position to a spatial point to be verified. Specifically, the pixel position of the selected anchor point in the image coordinate system and the corresponding laser depth data are read. The pixel position is then transformed to the camera coordinate system according to the camera projection matrix. Combined with pre-calibrated coordinate transformation parameters, the position is transformed from the camera coordinate system to the body coordinate system. Based on the inertial measurement attitude data, the position in the body coordinate system is transformed to the navigation coordinate system. Finally, based on the correspondence between the local coordinate system of the digital elevation map and the navigation coordinate system, the point is transformed to the local coordinate system of the digital elevation map to obtain the spatial point to be verified. This spatial point represents the position of the selected anchor point under the spatial reference of the inspection area and is used for comparison with the topology data of the inspection object. During consistency verification, the spatial point to be verified is compared with the spatial range and object identifier in the topology data of the inspected object. This verification relationship can be represented as: If and only if: ;and: ;otherwise: ; in, This indicates the consistency check result of the primary anchor point. A value of 1 indicates that the consistency check passed, and a value of 0 indicates that the consistency check failed. Indicates the primary anchor point; This represents the spatial point to be verified obtained after coordinate transformation from the primary anchor point. Represents the object identifier in the topology data of the inspected object. The corresponding spatial range, This represents the object identifier corresponding to the spatial point to be verified. This represents the object identifier of the inspection object area corresponding to the target box where the primary anchor point is located. This represents the target bounding box containing the primary anchor point, serving as a feasible example if the spatial point to be verified... In the local coordinate system of the digital elevation map, the position is x = 12.4 meters, y = 8.6 meters, and z = 1.2 meters. The spatial range of the corresponding object identifier is x from 10 meters to 15 meters, y from 6 meters to 10 meters, and z from 0.8 meters to 1.6 meters. Furthermore, the object identifier corresponding to the spatial point to be verified and the object identifier corresponding to the target box are both the same photovoltaic module number. Therefore, the primary anchor point is determined to have passed the consistency verification. This verification result is used to subsequently determine the defective entity anchor point, avoiding the use of candidate points located in adjacent inspection objects or background areas for coordinate calculation.

[0033] Furthermore, when the primary anchor point fails the consistency check, determining the candidate anchor points that pass the consistency check from the remaining candidate anchor points as the defective entity anchor points includes the following steps: When there are other candidate anchor points besides the primary anchor point, each candidate anchor point other than the primary anchor point is converted into a candidate spatial point according to the aforementioned coordinate transformation method, and each candidate spatial point is verified according to the consistency verification method. The candidate anchor points corresponding to the candidate spatial points that pass the consistency check are determined as candidate entity anchor points; When there are multiple candidate entity anchor points, the candidate entity anchor point with the highest matching priority is determined as the defect entity anchor point according to the matching priority between the modal type and the defect category of each candidate anchor point. When the primary anchor point and the remaining candidate anchor points fail the consistency check, the primary anchor point is marked as an anchor point to be reviewed, and the defect target corresponding to the target box is marked as a defect target to be reviewed.

[0034] Specifically, when the primary anchor point fails the consistency check, the other candidate anchor points within the same target box, excluding the primary anchor point, are processed. In practice, the set of other candidate anchor points is first determined. This set consists of visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points that have not been determined as primary anchor points. For each candidate anchor point in the set, the aforementioned conversion methods from pixel coordinate system to camera coordinate system, camera coordinate system to body coordinate system, body coordinate system to navigation coordinate system, and navigation coordinate system to local coordinate system of digital elevation map are used to convert the candidate anchor point into a candidate spatial point. Subsequently, each candidate spatial point is checked for consistency with the spatial range and object identifier in the topology data of the inspected object. It is determined whether the candidate spatial point falls within the spatial range of the inspected object corresponding to the target box, and whether the object identifier corresponding to the candidate spatial point is consistent with the object identifier corresponding to the target box. Candidate spatial points that pass the consistency check are identified as candidate anchor points. During implementation, the candidate entity anchor points retain their original modal types, such as visible light anchor point type, infrared anchor point type, or laser anchor point type, and retain their image coordinate positions, corresponding spatial point positions, and consistency check results. This process enables the selection of a defect entity anchor point for coordinate calculation among multiple available candidate anchor points, instead of directly abandoning the defect target corresponding to the target box after the primary anchor point fails. When multiple candidate entity anchor points exist, the defect entity anchor point is determined based on the matching priority between the modal type and defect category corresponding to each candidate anchor point. This process can be represented as: ; in, Indicates the anchor point of the defective entity. Indicates the primary anchor point. This indicates the consistency check result of the primary anchor point. This represents the set of candidate anchor points other than the primary anchor point. This represents any candidate anchor point in the remaining set of candidate anchor points. Indicates candidate anchor points The consistency check results Indicates candidate anchor points Modal types and defect categories Matching priority This indicates the primary anchor point marked as the anchor point to be reviewed. The matching priority can be determined according to the modal type corresponding to the defect category. For example, in thermal anomaly defects, the infrared anchor point type has a higher priority than the visible light anchor point type and the laser anchor point type. In appearance damage defects, the visible light anchor point type has a higher priority than the infrared anchor point type and the laser anchor point type. In geometric deformation defects, the laser anchor point type has a higher priority than the visible light anchor point type and the infrared anchor point type. As a feasible example, the defect category corresponding to the target box is photovoltaic hot spot defect. The initially determined primary anchor point is the infrared candidate anchor point. If the infrared candidate anchor point falls into the spatial range of the adjacent photovoltaic module after coordinate transformation, resulting in the object identifier being inconsistent with the object identifier corresponding to the target box, then the infrared candidate anchor point fails the consistency check. In this case, the visible light candidate anchor point and the laser candidate anchor point in the same target box are converted into candidate spatial points and subjected to consistency check. If both the visible light candidate anchor point and the laser candidate anchor point pass the consistency check, then according to the matching priority of photovoltaic hot spot defect and each mode type, the candidate anchor point with higher priority is selected as the defect entity anchor point from the candidate entity anchor points that have passed the check. When the primary anchor point and other candidate anchor points fail the consistency check, the defect entity anchor point that can be directly used for coordinate calculation will no longer be determined from the current target box. Instead, the primary anchor point will be marked as an anchor point to be checked, and the defect target corresponding to the target box will be marked as a defect target to be checked. The marking result is retained along with the target box, defect category and the verification results of each candidate anchor point for use in subsequent inspection result review or re-collection. Through this process, anchor points that do not meet the topological constraints of the inspection object will not directly enter the subsequent geodetic latitude and longitude coordinate calculation process.

[0035] S6. Based on the defect entity anchor points, inertial measurement attitude data, laser depth data, and digital elevation map, calculate the geodetic latitude and longitude coordinates of the defect target.

[0036] Furthermore, the process of calculating the geodetic latitude and longitude coordinates of the defective target in S6 includes the following steps: Obtain the pixel position of the defect entity anchor point in the image coordinate system, and determine the spatial viewing direction corresponding to the defect entity anchor point based on the pixel position and the camera projection matrix; Based on the laser depth data of the spatial line of sight and the corresponding position of the defect entity anchor point, the spatial position of the defect entity anchor point in the camera coordinate system is determined. Based on the inertial measurement attitude data and the pre-calibrated coordinate transformation parameters, the spatial position of the defect entity anchor point in the camera coordinate system is sequentially transformed to the body coordinate system, the navigation coordinate system, and the local coordinate system of the digital elevation map. Based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map is converted into the geodetic latitude and longitude coordinates of the defect target.

[0037] Specifically, first read the pixel position of the defect entity anchor point in the image coordinate system. This pixel position comes from the defect entity anchor point determined after the consistency check. Based on the pixel position and the camera projection matrix, determine the spatial line of sight from the camera optical center and passing through the pixel position. This spatial line of sight is used to represent the orientation of the defect entity anchor point in the camera field of view. It is still under the camera coordinate system and does not yet contain the actual distance information of the target point. After determining the spatial line-of-sight direction, the laser depth data corresponding to the defect entity anchor point is read. This laser depth data is used to determine the scale in the spatial line-of-sight direction, converting the defect entity anchor point from a two-dimensional pixel position to a spatial position. Specifically, the spatial line-of-sight direction is combined with the corresponding laser depth data to obtain the three-dimensional spatial position of the defect entity anchor point in the camera coordinate system. If the pixel position of a defect entity anchor point in the image coordinate system is x=320, y=240, the corresponding spatial line-of-sight direction points to the surface of the inspected object within the target box, and the laser depth data corresponding to this position is 18 meters, then this 18-meter depth is used as the distance scale in the spatial line-of-sight direction to determine the spatial position of the defect entity anchor point in the camera coordinate system. After obtaining the spatial position in the camera coordinate system, a step-by-step coordinate transformation is performed based on the inertial measurement attitude data and the pre-calibrated coordinate transformation parameters. In practice, the defect entity anchor point is first transformed from the camera coordinate system to the body coordinate system using the coordinate transformation parameters between the camera coordinate system and the body coordinate system. Then, based on the inertial measurement attitude data, the spatial position in the body coordinate system is transformed to the navigation coordinate system. Subsequently, based on the correspondence between the navigation coordinate system and the local coordinate system of the digital elevation map, the spatial position is transformed to the local coordinate system of the digital elevation map. Through this step-by-step transformation, the position of the defect entity anchor point is transformed from the local position under the image acquisition viewpoint to the spatial position under the coordinate reference of the inspection area. Finally, based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map is converted into the geodetic latitude and longitude coordinates of the defect target. This transformation relationship can be determined by the geodetic coordinates corresponding to the origin of the local coordinate system of the digital elevation map and the direction of the local coordinate axes. After the transformation is completed, the defect target not only has image coordinates and corresponding inspection object identifiers, but also geodetic latitude and longitude coordinates that can be used for inspection records. These geodetic latitude and longitude coordinates are generated by the defect entity anchor point that has passed the consistency verification, rather than being directly generated from the center of the target box or unverified candidate points.

[0038] Furthermore, S6 also includes the following steps: When the laser depth data for the location corresponding to the defective entity anchor point is not available, or the spatial location calculated based on the laser depth data is not within the spatial range of the corresponding object identifier in the topology data of the inspected object, the spatial line of sight corresponding to the defective entity anchor point is determined based on the pixel position of the defective entity anchor point in the image coordinate system and the camera projection matrix. The spatial line of sight is intersected with the elevation of the digital elevation map and the spatial range in the topology data of the inspected object to obtain the compensated spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map. Based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the spatial location of the compensation is converted into the geodetic latitude and longitude coordinates of the defective target.

[0039] Specifically, first determine whether there is valid laser depth data at the location corresponding to the defect anchor point. If there is no laser depth value at the location, or if the spatial location calculated based on the laser depth value is not within the spatial range of the corresponding object identifier in the topology data of the inspected object, then the laser depth value is not used as the spatial scale. Instead, the spatial line of sight is re-determined based on the pixel position of the defect anchor point in the image coordinate system and the camera projection matrix. This spatial line of sight represents the direction in which the defect anchor point extends from the image plane to the space of the inspected area, and serves as the basis for subsequent compensation spatial location calculation. After obtaining the spatial line of sight direction, intersect this spatial line of sight direction with the elevation of the digital elevation map and the spatial range in the topology data of the inspected object. The compensation spatial location can be determined according to the following relationship: ; in, This indicates the compensated spatial location of the defect entity anchor point in the local coordinate system of the digital elevation map. Indicates anchor point of defective entity The spatial line of sight is determined by the pixel position and the camera projection matrix. Indicates the anchor point of the defective entity. This represents the elevation constraint surface formed by the elevation of the digital elevation map. This represents the spatial range of the corresponding object identifier in the topology data of the inspected object. When finding the intersection, first find the position that intersects with the elevation constraint surface of the digital elevation map along the spatial line of sight, and then determine whether the position is within the spatial range of the corresponding object identifier. When there is an intersection point that satisfies the spatial range constraint, the intersection point is used as the compensation spatial position. If there are multiple intersection points between the spatial line of sight and the elevation constraint surface, the intersection point located within the spatial range of the corresponding object identifier is selected as the compensation spatial position. Taking photovoltaic module inspection as an example, if the laser depth data corresponding to the defective entity anchor point is missing, or the spatial position obtained based on the laser depth data falls into the area of ​​an adjacent module, then the spatial line of sight is regenerated based on the pixel position of the defective entity anchor point. If the spatial line of sight intersects with the elevation constraint surface of the digital elevation map, and the intersection point falls within the spatial range of the photovoltaic module corresponding to the target box, then the intersection point is used as the compensation spatial position. In this way, subsequent coordinate transformation no longer depends on the missing or inconsistent laser depth data with the object topology, but uses the compensation spatial position that meets the elevation and object spatial range constraints. After obtaining the compensation spatial location, based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the compensation spatial location is converted into the geodetic latitude and longitude coordinates of the defect target. This transformation can be expressed as: ; in, The geodetic latitude and longitude coordinates of the defective target. This indicates the transformation relationship from the local coordinate system to the geodetic coordinate system in a digital elevation map. This indicates the spatial location of the compensation. After the transformation, the obtained geodetic latitude and longitude coordinates are used as the coordinates of the defect target, and are retained along with the defect entity anchor point, object identifier, and compensation source information.

[0040] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for target detection and localization of unmanned aerial vehicles (UAVs) based on visual intelligence, characterized in that, include: S1. Acquire visible light images, infrared thermal maps, laser depth data, inertial measurement attitude data, and topological data of the inspection object collected by the inspection drone. The topological data of the inspection object includes the object type, object identifier, and spatial range of the inspection object. S2. Perform spatiotemporal registration on the visible light image, infrared thermal map, laser depth data, and inertial measurement attitude data to establish the correspondence between image frames and inspection objects; S3. Perform defect detection on the registered visible light image, infrared thermal map and laser depth data to obtain the defect category and target box, and generate visible light candidate anchor points, infrared candidate anchor points and laser candidate anchor points within the target box; S4. Based on the defect category and the preset correspondence between defect category and anchor point type, determine the primary anchor point from the visible light candidate anchor point, infrared candidate anchor point and laser candidate anchor point; S5. Perform consistency verification on the primary anchor point based on the topology data of the inspected object; when the primary anchor point fails the consistency verification, determine the candidate anchor point that passes the consistency verification from the remaining candidate anchor points as the defect entity anchor point; S6. Based on the anchor points of the defective entity, inertial measurement attitude data, laser depth data, and digital elevation map, calculate the geodetic latitude and longitude coordinates of the defective target.

2. The method for UAV target detection and localization based on visual intelligence according to claim 1, characterized in that, Establishing the correspondence between image frames and inspection objects in S2 includes the following steps: According to the acquisition timestamp, the visible light image, infrared thermal map, laser depth data and inertial measurement attitude data are synchronized to the image frame at the same acquisition time. Based on the pre-calibrated coordinate transformation parameters between the sensors, the infrared thermal map and laser depth data are projected onto the image coordinate system of the visible light image; Based on the spatial range in the topological data of the inspected object, the inspected object region identified in the image coordinate system is spatially matched with the corresponding object identifier to establish an association.

3. The method for UAV target detection and localization based on visual intelligence according to claim 2, characterized in that, The defect detection process in step S3, which involves registering the visible light image, infrared thermogram, and laser depth data, includes the following steps: Extract appearance anomaly features from the visible light image, extract thermal anomaly features from the infrared thermogram, and extract depth anomaly features from the laser depth data; Based on the preset correspondence between industrial defect types and modal weights, the appearance anomaly features, thermal anomaly features, and depth anomaly features are weighted and fused to obtain fused detection features; The correspondence between the industrial defect types and modal weights includes: the infrared modal weight of thermal anomaly defects is greater than the visible light modal weight and the laser modal weight; the visible light modal weight of appearance damage defects is greater than the infrared modal weight and the laser modal weight; and the laser modal weight of geometric deformation defects is greater than the visible light modal weight and the infrared modal weight. The fused detection features are input into the airborne defect detection model to obtain the defect category and target box.

4. The UAV target detection and localization method based on visual intelligence according to claim 3, characterized in that, The step S3, which generates visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points within the target box, includes the following steps: Within the target box, a visible light abnormal region is determined based on the feature map spatial response region corresponding to the appearance abnormality feature, and a point is selected from the edge center, skeleton center and region centroid of the visible light abnormal region as a visible light candidate anchor point according to the defect category. Within the target box, an infrared anomaly region is determined based on the feature map spatial response region corresponding to the thermal anomaly feature, and a point is selected as an infrared candidate anchor point from the temperature rise peak point and the centroid of the region according to the defect category. Within the target bounding box, a depth anomaly region is determined based on the feature map spatial response region corresponding to the depth anomaly feature, and the maximum deviation point of the depth value in the depth anomaly region from the mean of the neighborhood, or the centroid of the depth anomaly region, is determined as a candidate laser anchor point.

5. The UAV target detection and localization method based on visual intelligence according to claim 4, characterized in that, The process of determining the primary anchor point from the visible light candidate anchor points, infrared candidate anchor points, and laser candidate anchor points in step S4 includes the following steps: The aforementioned defect categories are classified into thermal anomaly defects, appearance damage defects, or geometric deformation defects; The primary anchor point type is determined based on the defect category corresponding to the defect type. Specifically, thermal anomaly defects correspond to infrared anchor point types, appearance damage defects correspond to visible light anchor point types, and geometric deformation defects correspond to laser anchor point types. The candidate anchor points corresponding to the primary anchor point type are determined as primary anchor points.

6. The method for unmanned aerial vehicle (UAV) target detection and localization based on visual intelligence according to claim 5, characterized in that, The process of classifying the defect into thermal anomaly defects, appearance damage defects, or geometric deformation defects includes the following steps: When the detection result corresponding to the defect category contains a temperature rise area in the infrared thermogram, the defect category is classified as a thermal anomaly defect. When the detection result corresponding to the defect category includes edge breakage area, texture abnormal area or surface color abnormal area in the visible light image, the defect category is classified as appearance damage defect. When the detection result corresponding to the defect category contains a depth abrupt change region or a geometric deviation region in the laser depth data, the defect category is classified as a geometric deformation defect.

7. The method for UAV target detection and localization based on visual intelligence according to claim 1, characterized in that, The consistency verification of the primary anchor point based on the topology data of the inspected object in step S5 includes the following steps: Based on the position of the target box where the main anchor point is located in the image coordinate system, combined with the camera projection matrix, inertial measurement attitude data and digital elevation map elevation, the target box is projected onto the local coordinate system of the digital elevation map to determine the inspection object area and object identifier corresponding to the target box; Based on the position of the primary anchor point in the image coordinate system, the laser depth data of the corresponding position, and the inertial measurement attitude data, the primary anchor point is converted into a spatial point to be verified through coordinate transformation. The coordinate transformation includes, in sequence, the transformation from pixel coordinate system to camera coordinate system, from camera coordinate system to body coordinate system, from body coordinate system to navigation coordinate system, and from navigation coordinate system to local coordinate system of digital elevation map. The spatial point to be verified is compared with the topology data of the inspected object. When the spatial point to be verified is located within the spatial range corresponding to the object identifier, and the object identifier corresponding to the spatial point to be verified is consistent with the object identifier of the inspected object area, the primary anchor point is determined to have passed the consistency verification.

8. The UAV target detection and localization method based on visual intelligence according to claim 7, characterized in that, When the primary anchor point fails the consistency check, determining the candidate anchor point that passes the consistency check from the remaining candidate anchor points as the defective entity anchor point includes the following steps: When there are other candidate anchor points besides the primary anchor point, each candidate anchor point other than the primary anchor point is converted into a candidate spatial point through coordinate transformation, and each candidate spatial point is verified according to the consistency check. The candidate anchor points corresponding to the candidate spatial points that pass the consistency check are determined as candidate entity anchor points; When there are multiple candidate entity anchor points, the candidate entity anchor point with the highest matching priority is determined as the defect entity anchor point according to the matching priority between the modal type corresponding to each candidate anchor point and the defect category. When the primary anchor point and the remaining candidate anchor points fail the consistency check, the primary anchor point is marked as an anchor point to be reviewed, and the defect target corresponding to the target box is marked as a defect target to be reviewed.

9. A method for unmanned aerial vehicle (UAV) target detection and localization based on visual intelligence according to claim 8, characterized in that, The steps for calculating the geodetic latitude and longitude coordinates of the defective target in S6 include: Obtain the pixel position of the defect entity anchor point in the image coordinate system, and determine the spatial viewing direction corresponding to the defect entity anchor point based on the pixel position and the camera projection matrix; Based on the laser depth data of the spatial line of sight and the corresponding position of the defect entity anchor point, the spatial position of the defect entity anchor point in the camera coordinate system is determined. Based on the inertial measurement attitude data and the pre-calibrated coordinate transformation parameters, the spatial position of the defect entity anchor point in the camera coordinate system is sequentially transformed to the body coordinate system, the navigation coordinate system, and the local coordinate system of the digital elevation map. Based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map is converted into the geodetic latitude and longitude coordinates of the defect target.

10. A method for UAV target detection and localization based on visual intelligence according to claim 9, characterized in that, S6 further includes the following steps: When the laser depth data corresponding to the defect entity anchor point is not available, or the spatial position calculated based on the laser depth data is not within the spatial range of the corresponding object identifier in the topology data of the inspected object, the spatial line of sight corresponding to the defect entity anchor point is determined based on the pixel position of the defect entity anchor point in the image coordinate system and the camera projection matrix. The intersection of the spatial line of sight with the elevation of the digital elevation map and the spatial range in the topology data of the inspected object is obtained to obtain the compensated spatial position of the defect entity anchor point in the local coordinate system of the digital elevation map. Based on the transformation relationship between the local coordinate system and the geodetic coordinate system of the digital elevation map, the compensation spatial location is converted into the geodetic latitude and longitude coordinates of the defective target.

Images