Obstruction recognition-based inspection strategy generation method and device, medium

Abstract

The present disclosure provides an inspection strategy generation method and device based on occlusion identification, medium, related to unmanned aerial vehicle inspection technology field. The method comprises: acquiring the inspection image corresponding to the target area and determining the occlusion area and its corresponding classification features, determining the occlusion source confidence corresponding to the occlusion area, and combining the highest confidence and confidence difference to determine the target occlusion source type; determining the target compensation rule according to the target occlusion source type and generating the occlusion event record, determining the repeated occlusion area, the main occlusion source type and the historical compensation situation based on the occlusion event record corresponding to a plurality of inspection tasks, and generating the target inspection strategy. The present scheme can reduce the probability of misjudgment of the occlusion source and the occurrence of unsuitable compensation, reduce repeated collection and invalid compensation, and reduce the probability of the same target area being repeatedly affected by the occlusion in the multiple inspection processes, improve the proportion of effective inspection image acquisition and the efficiency of inspection task execution.

Description

Technical Field

[0001] This disclosure relates to the field of unmanned aerial vehicle (UAV) inspection technology, and more specifically, to a method, device, and medium for generating inspection strategies based on occlusion recognition. Background Technology

[0002] With the development of drone inspection technology, drones are increasingly being used for inspection of engineering objects such as dam facades, plant exteriors, slopes, reservoir banks, and equipment platforms in hydropower projects. Drones can collect inspection images of target areas according to preset routes and waypoints, and identify and interpret risk conditions such as structural cracks, leakage marks, rust areas, and equipment anomalies based on the inspection images.

[0003] Currently, when dealing with occlusion in inspection images, related technologies typically only determine whether the target area is occluded and use a uniform reshoot or image processing method. It is difficult to select an appropriate processing method based on the differences in spatial stability, duration, and motion state of different occlusion sources. At the same time, for occlusion situations that repeatedly occur in multiple inspection tasks for the same target area, the related technologies lack continuous utilization of historical occlusion situations and processing results, which makes it easy for the same occlusion problem to recur in subsequent inspections.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a method, device, and medium for generating inspection strategies based on occlusion recognition, thereby reducing the probability of misjudgment of occlusion sources and inappropriate compensation, reducing repeated acquisition and invalid re-shooting, and reducing the probability that the same target area will be repeatedly affected by occlusion during multiple inspections, thereby improving the proportion of effective inspection images and the efficiency of inspection task execution.

[0006] According to a first aspect of the present disclosure, a method for generating inspection strategies based on occlusion recognition is provided, comprising: acquiring an inspection image corresponding to a target area, and determining occlusion areas in the inspection image that overlap with the target area; extracting classification features corresponding to the occlusion areas, the classification features including at least two of geometric morphological features, color and texture features, temporal motion features, and on-site facility matching features; determining occlusion source confidence levels corresponding to multiple preset occlusion source types based on the classification features, and determining a target occlusion source type corresponding to the occlusion area based on the highest occlusion source confidence level and the confidence difference between the highest occlusion source confidence level and the second highest occlusion source confidence level, wherein, if the highest occlusion source confidence level and / or the confidence difference does not meet the preset classification conditions, the target occlusion source type is determined as unconfirmed occlusion. Multiple preset occlusion source types include vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion. Based on the target occlusion source type, the target compensation rule corresponding to the occlusion area is determined, and an occlusion event record corresponding to the occlusion area is generated. The occlusion event record includes at least the target area identifier, the target occlusion source type, and the compensation result corresponding to the target compensation rule. Based on the occlusion event records corresponding to multiple inspection tasks, repeated occlusion areas are identified, and the main occlusion source type and historical compensation status corresponding to the repeated occlusion areas are determined. Based on the main occlusion source type and historical compensation status, a target inspection strategy corresponding to the repeated occlusion areas is generated. The target inspection strategy includes at least one of the following: backup waypoint, backup shooting direction, gimbal tilt angle adjustment, shooting distance adjustment, delayed shooting time, and number of consecutive acquisition frames.

[0007] In some example embodiments of this disclosure, based on the aforementioned scheme, determining the occlusion area overlapping with the target area in the inspection image includes: determining the target area in the inspection image based on the target area identifier, and identifying candidate occlusion areas covering the target area; determining at least two of the following: the overlap ratio between the candidate occlusion area and the target area, the continuous coverage width of the candidate occlusion area within the target area, whether the candidate occlusion area covers the key monitoring sub-area, and the length of the missing boundary of the target area caused by the candidate occlusion area; and determining the candidate occlusion area as an occlusion area if at least two of the above conditions are met.

[0008] In some example embodiments of this disclosure, based on the aforementioned scheme, classification features corresponding to the occluded area are extracted, including: extracting at least one of the boundary complexity, porosity, rectangularity, and straight edge ratio of the occluded area as geometric morphological features; extracting at least one of the green vegetation index, color mean, color variance, and texture direction consistency of the occluded area as color texture features; determining at least one of the centroid displacement, area change rate, number of consecutive occurrence frames, local optical flow direction dispersion, and occurrence duration based on the positional change of the occluded area in consecutive frames as temporal motion features; and determining on-site facility matching features based on the matching relationship between the occluded area and the projected positions of fixed facilities in the engineering facility ledger.

[0009] In some example embodiments of this disclosure, based on the aforementioned scheme and based on classification features, the occlusion source confidence levels corresponding to multiple preset occlusion source types are determined, including: for each preset occlusion source type, determining the feature scores corresponding to geometric morphology features, color texture features, temporal motion features, and on-site facility matching features; determining the occlusion source confidence level corresponding to the preset occlusion source type based on the feature scores and the weights corresponding to each feature score; wherein, when determining the occlusion source confidence level corresponding to vegetation occlusion, the weights corresponding to boundary complexity in color texture features and geometric morphology features are greater than the weights corresponding to other classification features; when determining the occlusion source confidence level corresponding to fixed facility occlusion, the weights corresponding to the proportion of straight edges in on-site facility matching features and geometric morphology features are greater than the weights corresponding to other classification features; when determining the occlusion source confidence level corresponding to short-term dynamic occlusion, the weights corresponding to temporal motion features are greater than the weights corresponding to other classification features.

[0010] In some example embodiments of this disclosure, based on the aforementioned scheme, the target occlusion source type corresponding to the occlusion area is determined according to the confidence level of the highest occlusion source and the confidence difference between the highest occlusion source confidence level and the second highest occlusion source confidence level. This includes: when the confidence level of the highest occlusion source is not less than a preset classification confidence threshold and the confidence difference is not less than a preset confidence difference threshold, the preset occlusion source type corresponding to the highest occlusion source confidence level is determined as the target occlusion source type; when the confidence level of the highest occlusion source is less than the preset classification confidence threshold and / or the confidence difference is less than the preset confidence difference threshold, the target occlusion source type is determined as unconfirmed occlusion.

[0011] In some example embodiments of this disclosure, based on the foregoing scheme, the target compensation rule corresponding to the occlusion area is determined according to the target occlusion source type, including: when the target occlusion source type is vegetation occlusion, retrieving unoccluded candidate images from the historical inspection images corresponding to the target area, and mapping the unoccluded area corresponding to the occluded area in the unoccluded candidate images to the occluded area in the inspection images; and / or, determining a backup shooting direction that is laterally deflected relative to the original shooting direction according to the original shooting direction corresponding to the inspection images, and adjusting the shooting direction of the UAV and the gimbal pitch angle according to the backup shooting direction; when the target occlusion source type is fixed facility occlusion, determining the fixed occlusion direction according to the projection direction of the fixed facility in the target area, and in Avoid locations with fixed obstruction directions to generate backup waypoints, and control the UAV to re-acquire inspection images corresponding to the target area based on the backup waypoints; when the target obstruction source type is short-term dynamic obstruction, and when the safe hovering condition is met, control the UAV to re-acquire inspection images corresponding to the target area after a preset delay at the current waypoint; and / or, control the UAV to continuously acquire multiple frames of inspection images at the same waypoint or in the same flight segment, and select the inspection image with the smallest obstruction ratio from the multiple frames of inspection images; when the target obstruction source type is unconfirmed obstruction, according to the obstruction ratio corresponding to the obstructed area, retrieve unobstructed candidate images from the historical inspection images corresponding to the target area and generate compensation results, or generate backup waypoint reshoot instructions or manual review prompts.

[0012] In some example embodiments of this disclosure, based on the aforementioned scheme and based on the occlusion event records corresponding to multiple inspection tasks, determining the repeated occlusion area includes: within a preset statistical period, based on multiple occlusion event records corresponding to the same target area, determining at least one of the following: occlusion occurrence frequency, occlusion event proportion of the same occlusion source type, and compensation failure number; if the occlusion occurrence frequency reaches a preset frequency threshold, the occlusion event proportion reaches a preset proportion threshold, and / or the compensation failure number reaches a preset number threshold, the target area is determined as a repeated occlusion area; based on the target occlusion source type and compensation result in each occlusion event record corresponding to the repeated occlusion area, determining the main occlusion source type and historical compensation status corresponding to the repeated occlusion area.

[0013] In some example embodiments of this disclosure, based on the aforementioned scheme, a target inspection strategy corresponding to a repeatedly occluded area is generated according to the main occlusion source type and historical compensation information. This includes: when the main occlusion source type is vegetation occlusion, determining the target lateral shooting angle based on the occlusion ratio corresponding to different lateral shooting angles during historical inspections, and generating at least one of backup waypoints, backup shooting directions, and gimbal pitch angle adjustment based on the target lateral shooting angle; when the main occlusion source type is fixed facility occlusion, determining the occlusion direction based on the projection direction of the fixed facility in the target area, and generating a backup shooting direction and / or backup waypoint on the opposite side of the occlusion direction; when the main occlusion source type is short-term dynamic occlusion, determining the concentrated occlusion period based on the occlusion occurrence time in the occlusion event record corresponding to the repeatedly occluded area, and generating at least one of an inspection time adjustment strategy, a delayed shooting strategy, and a continuous acquisition strategy to avoid the concentrated occlusion period; and when the main occlusion source type is unconfirmed occlusion, generating a conservative inspection strategy including backup waypoints and manual verification markers.

[0014] According to a second aspect of the present disclosure, an electronic device is provided, including: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the inspection strategy generation method based on occlusion recognition in the first aspect.

[0015] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the inspection strategy generation method based on occlusion recognition in the first aspect.

[0016] The technical solutions provided in this disclosure can include the following beneficial effects: The inspection strategy generation method based on occlusion recognition in the example embodiments of this disclosure, on the one hand, comprehensively judges the credibility of the occlusion source by combining the differences in occlusion areas in different dimensions such as shape, texture, temporal changes, and correlation with on-site facilities. When different occlusion sources are difficult to distinguish clearly, unconfirmed processing is adopted, enabling occlusion judgment to adapt to situations in complex engineering scenarios where different occluded objects have similar appearances and insufficient single-feature differentiation capabilities. This reduces the probability of misjudging occlusion sources, minimizes inappropriate compensation and invalid inspection adjustments caused by misclassification, and improves the reliability of occlusion processing. On the other hand, by assigning different compensation processing logics to different occlusion sources, the compensation method can be matched with the positional stability, motion state, and continuous characteristics of the occluded object, avoiding... For different occlusion situations, a unified approach of reshooting or image processing is adopted to reduce duplicate acquisitions, invalid reshoots, and unnecessary adjustments to inspection paths, thereby improving the effective processing rate of occluded images and the efficiency of UAV inspection operations. On the other hand, by continuously utilizing the occlusion situations and corresponding compensation results formed by multiple inspection tasks, the system identifies situations where the same target area is repeatedly affected by occlusion. Based on the main sources of occlusion and previous compensation situations, the system adjusts the waypoints, shooting directions, gimbal attitudes, shooting distances, or shooting sequences in subsequent inspections. This allows subsequent inspections to avoid the impact of occlusions with recurring patterns, reduces the recurrence of the same occlusion problem in different inspection tasks, increases the proportion of effective inspection images of the target area, reduces the number of repeated reshoots, and improves the execution efficiency of periodic inspection tasks.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0019] Figure 1 The illustration shows a flowchart of an inspection strategy generation method based on occlusion recognition according to some embodiments of the present disclosure.

[0020] Figure 2 The illustration shows a flowchart of determining an occlusion area that overlaps with a target area in an inspection image according to some embodiments of the present disclosure.

[0021] Figure 3 The illustration shows a flowchart of determining the target compensation rule corresponding to the occlusion area according to some embodiments of the present disclosure.

[0022] Figure 4 The illustration shows a flowchart of a target inspection strategy for generating repeated occlusion areas according to some embodiments of the present disclosure.

[0023] Figure 5 The schematic diagram illustrates the structural schematic of a computer system of an electronic device according to some embodiments of the present disclosure.

[0024] Figure 6 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is shown.

[0025] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0026] In actual inspection environments, the target area is typically surrounded by objects such as tree branches and leaves, fixed pipelines, railings, equipment components, personnel, vehicles, and construction machinery. These objects may be located between the drone and the target area, preventing some textures, boundaries, or key monitoring locations of the target area from being fully displayed in the inspection images. Related technologies for drone inspections typically involve image acquisition along a preset inspection route, followed by quality checks on the acquired images after the inspection task is completed. When obstructions are found in the target area, the drone is usually reassigned for re-shooting, or the obstructed images are marked as unusable.

[0027] However, the presence of different occluding objects varies considerably in inspection scenarios. For example, vegetation may sway locally with environmental disturbances, fixed facilities are usually in a relatively stable position for a long time, while people, vehicles, and other objects may only pass through the target area for a short period of time. At the same time, some occluding objects of different natures may appear with similar colors, outlines, or shadow features in a single inspection image, making it difficult to determine the actual source of the occlusion based solely on the stability of the current image.

[0028] Current inspection image processing methods typically focus on determining whether there is occlusion in the target area or whether the target content is missing. Upon detecting occlusion, a uniform re-shooting or image processing approach is used. However, because the spatial stability, duration, and movement of different occlusion sources vary, using the same processing method is difficult to adapt to different occlusion scenarios. For example, for occluded objects that remain in a fixed position for a long time, repeatedly capturing images using the original inspection method may still result in re-occlusion; for moving objects that only exist for a short period, directly replanning the re-shooting location may increase unnecessary inspection work. These methods lack a basis for targeted processing of different occlusion situations.

[0029] Furthermore, hydropower project inspections are typically cyclical and repetitive, with the same target area being captured multiple times in different inspection tasks. After processing a single instance of occlusion, these inspection methods often lack the ability to continuously utilize the occlusion patterns and processing results from different inspection tasks, making it difficult to determine whether the same target area is repeatedly affected by a specific occlusion object. Therefore, subsequent inspection tasks may still capture images of the target area using the same waypoints, shooting directions, or shooting sequences, leading to the recurrence of the same or similar occlusion problems.

[0030] Based on one or more problems existing in related technologies, this example embodiment first provides a method for generating inspection strategies based on occlusion recognition. This method for generating inspection strategies based on occlusion recognition can be applied to terminal devices or servers, and this embodiment does not make any special limitations on it. Figure 1 The illustration schematically shows a flowchart of an inspection strategy generation method based on occlusion recognition according to some embodiments of the present disclosure. (Reference) Figure 1 As shown, the inspection strategy generation method based on occlusion recognition may include the following steps: Step S110: Obtain the inspection image corresponding to the target area, and determine the occlusion area in the inspection image that overlaps with the target area; Step S120: Extract the classification features corresponding to the occluded area. The classification features include at least two of the following: geometric morphology features, color and texture features, temporal motion features, and on-site facility matching features. Step S130: Based on the classification features, determine the occlusion source confidence levels corresponding to multiple preset occlusion source types, and determine the target occlusion source type corresponding to the occlusion area based on the highest occlusion source confidence level and the confidence level difference between the highest occlusion source confidence level and the second highest occlusion source confidence level. If the highest occlusion source confidence level and / or the confidence level difference does not meet the preset classification conditions, the target occlusion source type is determined as unconfirmed occlusion. The multiple preset occlusion source types include vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion. Step S140: Based on the target occlusion source type, determine the target compensation rule corresponding to the occlusion area, and generate an occlusion event record corresponding to the occlusion area. The occlusion event record includes at least the target area identifier of the target area, the target occlusion source type, and the compensation result corresponding to the target compensation rule. Step S150: Based on the occlusion event records corresponding to multiple inspection tasks, determine the repeated occlusion area, and determine the main occlusion source type and historical compensation status corresponding to the repeated occlusion area; Step S160: Based on the main occlusion source type and the historical compensation situation, generate a target inspection strategy corresponding to the repeated occlusion area. The target inspection strategy includes at least one of the following: backup waypoint, backup shooting direction, gimbal pitch angle adjustment, shooting distance adjustment, delayed shooting time, and number of consecutive acquisition frames.

[0031] According to the inspection strategy generation method based on occlusion recognition in this example embodiment, on the one hand, by combining the differences in occlusion areas in different dimensions such as shape, texture, temporal changes, and correlation with on-site facilities, the credibility of the occlusion source is comprehensively judged. When different occlusion sources are difficult to distinguish clearly, unconfirmed processing is adopted, so that the occlusion judgment can adapt to the situation in complex engineering scenarios where different occluded objects have similar appearances and insufficient single feature discrimination ability, thereby reducing the probability of misjudgment of occlusion sources, reducing unsuitable compensation and invalid inspection adjustments caused by misclassification, and improving the reliability of occlusion processing. On the other hand, by assigning different compensation processing logic to different occlusion sources, the compensation method can be matched with the positional stability, motion state, and continuous characteristics of the occluded object, avoiding For different occlusion situations, a unified approach of reshooting or image processing is adopted to reduce duplicate acquisitions, invalid reshoots, and unnecessary adjustments to inspection paths, thereby improving the effective processing rate of occluded images and the efficiency of UAV inspection operations. On the other hand, by continuously utilizing the occlusion situations and corresponding compensation results formed by multiple inspection tasks, the system identifies situations where the same target area is repeatedly affected by occlusion. Based on the main sources of occlusion and previous compensation situations, the system adjusts the waypoints, shooting directions, gimbal attitudes, shooting distances, or shooting sequences in subsequent inspections. This allows subsequent inspections to avoid the impact of occlusions with recurring patterns, reduces the recurrence of the same occlusion problem in different inspection tasks, increases the proportion of effective inspection images of the target area, reduces the number of repeated reshoots, and improves the execution efficiency of periodic inspection tasks.

[0032] The following will further explain the inspection strategy generation method based on occlusion recognition in this example embodiment.

[0033] In step S110, the inspection image corresponding to the target area is acquired, and the occlusion area in the inspection image that overlaps with the target area is determined.

[0034] In one example embodiment of this disclosure, the target area refers to the engineering area that needs to be identified or risk assessed during the inspection task. For example, it may include the outer facade of the dam, the outer wall of the powerhouse, the slope surface, the reservoir bank area, the power transmission facility area, the equipment platform area, or a pre-defined key monitoring area. The target area can be pre-configured with a corresponding target area identifier, which can be used to associate inspection images collected for the same target area in different inspection tasks.

[0035] Inspection images refer to image data collected by UAVs on a target area during inspection missions. These can be visible light images or image frames from inspection videos. Optionally, each inspection image can be associated with corresponding acquisition attributes, such as acquisition time, waypoint coordinates, flight altitude, shooting distance, gimbal pitch angle, heading angle, or inspection mission number, to establish a relationship between the inspection image and the corresponding target area and inspection mission. Inspection images of the same target area collected under different inspection missions, different waypoints, or different shooting attitudes can be grouped into a collection of inspection images corresponding to the same target area.

[0036] In the specific implementation process, the position of the target region in the inspection image can be determined based on the target region identifier. For example, the pre-marked spatial position of the target region can be projected onto the inspection image, or the target region can be determined through template matching, target detection, semantic segmentation, or image registration based on historical inspection images. The target region in the inspection image can be represented by a rectangular box, a polygonal region, or a pixel-level region mask; this embodiment does not limit this.

[0037] An occlusion region refers to an area that covers at least a portion of the image area of ​​a target region, preventing the corresponding textures, boundaries, or other image content from being directly presented in the current inspection image. For example, when tree branches, fixed pipelines, railings, supports, personnel, vehicles, or mobile construction equipment are located between a drone and the target region, their projected areas in the inspection image can form occlusion regions. Candidate occlusion content in inspection images can be identified using methods such as foreground region segmentation, target region edge continuity analysis, semantic segmentation, historical background subtraction, or multi-frame motion segmentation. The occlusion region is then determined based on the spatial overlap between the candidate occlusion content and the target region.

[0038] For example, a target region mask and an occlusion mask corresponding to the candidate occlusion content can be generated separately. Pixel-level intersection is performed on the target region mask and the occlusion mask, and the overlapping part is determined as the occlusion region that overlaps with the target region. By limiting the occlusion region to have an actual overlap relationship with the target region, the number of background objects or moving objects outside the target region that enter the subsequent occlusion source determination process can be reduced.

[0039] In step S120, the classification features corresponding to the occluded area are extracted. The classification features include at least two of the following: geometric morphology features, color and texture features, temporal motion features, and on-site facility matching features.

[0040] In one example embodiment of this disclosure, classification features are used to characterize the image representation and change state of the occluded object corresponding to the occluded area from different dimensions. Since different occluded objects may exhibit similar features in terms of single color, outline, or local texture, the occluded area can be described from multiple dimensions such as geometric shape, color texture, temporal motion, and association with on-site facilities, and at least two of the above classification features are selected for subsequent determination of the confidence level of the occlusion source.

[0041] Geometric morphological features are used to characterize the spatial shape or boundary morphology of occluded areas. For example, the contour point set of the occluded area can be extracted based on the area mask corresponding to the occluded area, and geometric morphological features can be formed based on the contour point set, area, circumscribed region, or internal connectivity. Different occluded objects have different geometric structures. For example, the boundaries of areas corresponding to natural vegetation usually have a more fragmented or irregular shape, while areas corresponding to fixed facilities usually have more obvious regular edges or linear structures. Therefore, geometric morphological features can be used to describe these morphological differences.

[0042] Color texture features are used to characterize the color distribution and pixel space variation patterns within an occluded area. The channel values ​​corresponding to the occluded area can be statistically analyzed in the Red Green Blue (RGB) color space, or the inspection image can be converted to the Hue Saturation Value (HSV) color space or other color spaces before extracting color features. Texture features can be determined based on grayscale variations, directional variations, or local repetitive structures between adjacent pixels within the occluded area. In some implementations, brightness normalization or local contrast adjustment can be performed on the inspection image before extracting color texture features to reduce the impact of ambient lighting changes on the color texture features.

[0043] Temporal motion features are used to characterize the positional or morphological changes of occluded regions across consecutive frames. In practice, multiple consecutive image frames acquired before and after the current inspection image can be obtained, and the correspondence between occluded regions in different image frames can be established. Temporal motion features are formed based on changes in the position, extent, or local pixel motion of the occluded regions. For example, target tracking or optical flow estimation can be used to determine the motion changes of the image content corresponding to the occluded region. For overall image changes caused by the drone's own motion, the overall motion relationship between inspection images can be estimated based on the stable structure in the target region, and the motion changes of the occluded regions can be corrected to obtain the temporal change state of the occluded region relative to the target region.

[0044] On-site facility matching features are used to characterize the correspondence between obstructed areas and existing fixed facilities at the construction site. A facility ledger can be pre-established at the construction site, recording the location data of fixed facilities such as pipelines, railings, supports, platform components, equipment boxes, or monitoring equipment. Based on the UAV's waypoint location, shooting direction, and camera parameters, the locations of fixed facilities can be projected onto the inspection images, and the spatial correspondence between the projected locations of the fixed facilities and the obstructed areas can be compared. In some implementations, the projected locations of fixed facilities can be obtained from the 3D engineering model corresponding to the engineering object, or fixed structure images from historical inspection images can be used as reference content to determine the on-site facility matching features.

[0045] In step S130, based on the classification features, the occlusion source confidence levels corresponding to multiple preset occlusion source types are determined respectively. The target occlusion source type corresponding to the occlusion area is determined according to the highest occlusion source confidence level and the confidence difference between the highest and second-highest occlusion source confidence levels. If the highest occlusion source confidence level and / or the confidence difference does not meet the preset classification conditions, the target occlusion source type is determined as unconfirmed occlusion. The multiple preset occlusion source types include vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion.

[0046] In one example embodiment of this disclosure, the preset occlusion source type refers to the occlusion source category pre-set in conjunction with the UAV inspection scenario. Among them, vegetation occlusion can include occlusion formed by natural vegetation such as tree branches and leaves, shrubs or vines; fixed facility occlusion can include occlusion formed by fixed pipelines, railings, supports, monitoring equipment boxes, fixed platform components or other relatively stable engineering facilities; short-term dynamic occlusion can include occlusion formed by objects such as personnel, vehicles, mobile construction machinery or temporary work objects that change position or appear briefly in continuous frames of the inspection image.

[0047] The occlusion source confidence score characterizes the degree to which an occluded area belongs to a corresponding preset occlusion source type. For the same occluded area, the occlusion source confidence scores for vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion can be calculated separately. The occlusion source confidence score can be represented by a normalized value between 0 and 1, or as a percentage or other numerical form that can compare the confidence levels of different preset occlusion source types.

[0048] In the specific implementation process, classification features can be formed into classification feature vectors, which are then input into a pre-trained classification model. The classification model outputs classification scores or predicted probabilities corresponding to multiple preset occlusion source types, and the occlusion source confidence is determined based on the classification scores or predicted probabilities. The classification model can employ support vector machines, random forests, gradient boosting models, or neural networks, among others. Alternatively, corresponding feature evaluation relationships can be established for different preset occlusion source types, and different classification features can be scored and combined to obtain the corresponding occlusion source confidence. This embodiment does not limit the specific calculation method for the occlusion source confidence.

[0049] After obtaining the confidence scores for multiple occlusion sources, they can be sorted from highest to lowest. The occlusion source confidence score with the highest value is determined as the highest occlusion source confidence score, and the second highest occlusion source confidence score is determined as the second highest occlusion source confidence score. The highest occlusion source confidence score reflects the degree of support of the current classification feature for the most likely occlusion source type, while the difference in confidence scores between the highest and second highest occlusion source confidence scores reflects the degree of distinction between the most likely occlusion source type and other occlusion source types.

[0050] Preset classification conditions can simultaneously constrain the confidence level and confidence difference of the highest occlusion source. When both the confidence level and confidence difference of the highest occlusion source satisfy the preset classification conditions, the preset occlusion source type corresponding to the confidence level of the highest occlusion source can be determined as the target occlusion source type. When the confidence level of the highest occlusion source is insufficient, it indicates that the current classification features cannot fully support any preset occlusion source type; when the confidence difference is insufficient, it indicates that at least two preset occlusion source types have similar levels of confidence for the current occluded region. In either of these cases, the target occlusion source type can be determined as unconfirmed occlusion.

[0051] Unconfirmed occlusion indicates that the target area has been identified as occluded, but based on the current classification features, the occluded area cannot be reliably classified into vegetation occlusion, fixed facility occlusion, or short-term dynamic occlusion. For example, branches and leaves may have similar color characteristics to green protective objects, regular construction components and fixed pipelines may have similar geometric shapes, and local shadow changes may also produce similar appearances to image changes of dynamic objects. By retaining unconfirmed occlusion, we can avoid forcibly determining the occlusion source type solely based on the confidence level of the occlusion source with the highest value, thus reducing the probability of misclassification of occlusion source.

[0052] In step S140, based on the target occlusion source type, a target compensation rule corresponding to the occlusion area is determined, and an occlusion event record corresponding to the occlusion area is generated. The occlusion event record includes at least the target area identifier of the target area, the target occlusion source type, and the compensation result corresponding to the target compensation rule.

[0053] In one example embodiment of this disclosure, a target compensation rule is used to determine the occlusion processing method for the current occlusion area. Since the occluded objects corresponding to different target occlusion source types have different positional stability and temporal variation characteristics, the association between different target occlusion source types and candidate compensation rules can be established in advance. After determining the target occlusion source type, the target compensation rule is determined from the corresponding candidate compensation rules.

[0054] Target compensation rules can include methods for compensating the current inspection image, or methods for changing the UAV image acquisition conditions and reacquiring inspection images. For example, for occluded objects with relatively stable positions, the visibility of the target area can be improved by changing the image acquisition position or shooting direction; for occluded objects with short-term changing characteristics, inspection images with lower occlusion levels can be obtained by changing the image acquisition time or increasing the number of acquired images; for cases where the source of the target occlusion is unconfirmed, a relatively conservative compensation method can be adopted to avoid performing highly type-specific processing based on uncertain occlusion sources.

[0055] Target compensation rules can be pre-stored in the rule base of the inspection management platform, establishing a mapping relationship between target occlusion source types and target compensation rules. After determining the target occlusion source type, the corresponding candidate compensation rules can be queried, and the target compensation rule can be determined by combining the current inspection task's operating status, available historical inspection images, and the UAV's current operating conditions. In other implementations, the target occlusion source type and the current inspection task status can also be input into a pre-established strategy selection model, which will then output the corresponding target compensation rule.

[0056] The compensation result is used to characterize the processing status after executing the target compensation rule. For example, the compensation result can be used to indicate whether the compensation was successful or failed, or whether a usable inspection image was obtained after the compensation process. In the specific implementation, the compensation result can be generated based on the visibility status of the target area after the compensation process or the usability status of the obtained inspection image.

[0057] For the current occlusion area, an occlusion event record can be generated. An occlusion event record is a data record used to record an occlusion event and its corresponding processing result, and can be stored in association with the inspection task that generated the occlusion event. The occlusion event record includes at least the target area identifier, the target occlusion source type, and the compensation result corresponding to the target compensation rule. This allows occlusion events occurring in the same target area in different inspection tasks to be associated based on the target area identifier, and retains the corresponding occlusion source determination result and compensation processing result. The occlusion event record can be stored in a relational database, a document database, or an inspection task data file; this embodiment does not limit this.

[0058] In step S150, based on the occlusion event records corresponding to multiple inspection tasks, the repeated occlusion area is determined, and the main occlusion source type and historical compensation status corresponding to the repeated occlusion area are determined.

[0059] In one example embodiment of this disclosure, multiple inspection tasks can be inspection tasks performed on the same engineering object at different inspection cycles, on different dates, or in different work batches. Based on the target area identifier in the occlusion event records, the occlusion event records corresponding to different inspection tasks can be aggregated, and occlusion event records with the same target area identifier can be associated with the same target area.

[0060] A repeatedly occluded area refers to a target area that is repeatedly affected by occlusion in multiple inspection tasks. Specifically, statistical or time-series analysis can be performed on multiple occlusion event records corresponding to the same target area to determine whether the occlusion events for that target area occur repeatedly in different inspection tasks. For example, an occlusion event sequence corresponding to the target area can be formed, and the degree of repeated occlusion of the target area can be evaluated based on the occurrence of occlusion events in different inspection tasks; alternatively, the occlusion event records corresponding to the target area can be aggregated, and the aggregation result can be used to determine whether the target area belongs to a repeatedly occluded area. By making judgments based on multiple inspection tasks, the situation where a single, occasional occlusion directly affects subsequent inspection strategies can be reduced.

[0061] The primary occlusion source type is used to characterize the main target occlusion source type corresponding to repeatedly occluded areas in multiple inspection tasks. The primary occlusion source type can be determined by aggregating the occlusion event records corresponding to repeatedly occluded areas and analyzing the distribution of different target occlusion source types across multiple occlusion events. For example, the target occlusion source type with a large number of corresponding occlusion events or a high degree of repetition in multiple inspection tasks can be identified as the primary source type. In some implementations, the evaluation weight of occlusion events corresponding to recent inspection tasks can be increased to accommodate changes in the occlusion environment at the engineering site over time.

[0062] Historical compensation data is used to characterize the compensation processing status of repeatedly occluded areas in previous inspection tasks. Since each occlusion event record stores the compensation results corresponding to the target compensation rule, multiple compensation results corresponding to repeatedly occluded areas can be summarized to form historical compensation data. For example, historical compensation data can indicate whether previous compensation yielded usable inspection images, or it can retain a sequence of compensation results corresponding to multiple inspection tasks.

[0063] In step S160, a target inspection strategy corresponding to the repeated occlusion area is generated based on the main occlusion source type and the historical compensation situation. The target inspection strategy includes at least one of the following: backup waypoint, backup shooting direction, gimbal pitch angle adjustment, shooting distance adjustment, delayed shooting time, and number of consecutive acquisition frames.

[0064] In one example embodiment of this disclosure, a target inspection strategy is used to configure the image acquisition conditions for repeatedly occluded areas in subsequent inspection tasks. Unlike determining target compensation rules for the current occlusion event, the target inspection strategy is used to adjust the subsequent inspection process based on the occlusion event records formed by multiple inspection tasks, so that the UAV uses acquisition parameters adapted to historical occlusion conditions when inspecting repeatedly occluded areas again.

[0065] In the specific implementation process, the image acquisition parameters that need to be adjusted can be determined based on the main source of occlusion, and the historical compensation data can be used to determine whether changes in past acquisition conditions can yield usable inspection images. For example, if historical compensation data indicates that a change in a certain acquisition parameter can yield usable inspection images, a target inspection strategy can be generated based on the corresponding acquisition parameter; if past compensation fails to improve the visibility of repeatedly occluded areas, the corresponding acquisition parameters can be changed or other combinations of inspection parameters can be used.

[0066] Alternate waypoints refer to candidate UAV spatial locations set for areas with repeated obstruction, and can be represented by geographic coordinates or local engineering coordinates. Alternate waypoints can change their position relative to the original inspection waypoints to alter the observation positional relationship between the UAV and the repeatedly obstructed area. Alternate shooting directions refer to candidate observation directions formed by the UAV or its onboard image acquisition device for the repeatedly obstructed area, and can be represented by heading angle, camera optical axis direction, or the amount of directional change relative to the original shooting direction. By adjusting the alternate shooting direction, the projection relationship between the obstructed object and the repeatedly obstructed area in the imaging direction can be changed.

[0067] The gimbal pitch angle adjustment amount indicates the angle that the UAV gimbal needs to be adjusted relative to its original pitch angle. By changing the gimbal pitch attitude, the observation direction of the image acquisition device in the vertical direction is adjusted. The shooting distance adjustment amount indicates the change in shooting distance between the UAV and the repeatedly obscured area. It can be expressed as a distance difference or a distance change ratio to adjust the imaging range and field of view of the repeatedly obscured area in the inspection image.

[0068] The delayed capture time indicates the length of time during which image acquisition is postponed for areas with repeated occlusion. It can be written into the inspection task as a target area-level acquisition parameter. The number of consecutive acquisition frames limits the number of inspection images continuously acquired for areas with repeated occlusion, enabling the UAV to acquire inspection images at multiple adjacent acquisition times.

[0069] Target inspection strategies can include one or a combination of the aforementioned parameters. For example, alternative waypoints and alternative shooting directions can be adjusted simultaneously, or the delayed shooting time and the number of consecutive acquisition frames can be adjusted while keeping the UAV's spatial position unchanged. In the specific implementation process, a strategy mapping relationship between the main occlusion source type and adjustable inspection parameters can be established in advance. Candidate inspection parameters are determined based on the main occlusion source type, and the corresponding parameter adjustment direction or candidate parameter combination is determined by combining historical compensation data. When multiple candidate target inspection strategies are formed, candidate strategies that cannot be executed can be filtered out by combining UAV flight conditions and the adjustable range of the image acquisition device. The determined target inspection strategy is then written into the subsequent inspection task configuration, thereby reducing the possibility of obtaining occluded images again when repeatedly occluded areas are inspected under the original acquisition conditions.

[0070] The contents of steps S110 to S160 will be described in detail below.

[0071] In one example embodiment of this disclosure, it can be achieved through Figure 2 The steps in the process are used to identify the occluded areas in the inspection image that overlap with the target area, referring to... Figure 2 As shown, it can specifically include: Step S210: Determine the target area in the inspection image based on the target area identifier, and identify candidate occlusion areas covering the target area; Step S220: Determine at least two of the following: the overlap ratio between the candidate occlusion area and the target area, the continuous coverage width of the candidate occlusion area within the target area, whether the candidate occlusion area covers the key monitoring sub-area, and the length of the missing boundary of the target area caused by the candidate occlusion area. Step S230: If at least two conditions are met, the candidate occlusion region is determined as the occlusion region.

[0072] The target area identifier is used to uniquely or relatively uniquely indicate the engineering area to be inspected in the inspection task. It can be associated with dam areas, slope areas, equipment platform areas, or other pre-defined engineering areas. Specifically, the association between the target area identifier and the spatial location of the target area, reference images, or image templates can be established in advance. After the inspection images are acquired, the corresponding target area identifier is determined based on the waypoint location, shooting direction, or inspection task information associated with the inspection images, and the image range of the target area in the inspection images is further located.

[0073] After identifying the target region, occlusion detection can be performed on the image content covering the target region to obtain candidate occlusion regions. Candidate occlusion regions refer to image areas initially judged to potentially occlude the target region, but it is not yet determined whether the occlusion warrants further processing. Specifically, candidate occlusion regions can be identified based on foreground content, texture anomalies, boundary defects, or image differences between the target region and the reference image. For example, the target region in the current inspection image can be compared with the corresponding historical reference image to identify newly added foreground image regions within the target region; alternatively, foreground objects covering the target region can be identified through image segmentation, and the image range corresponding to the foreground objects can be used as candidate occlusion regions. Candidate occlusion regions can be represented using region masks, polygon boundaries, or pixel sets to further determine the spatial relationship between the candidate occlusion regions and the target region.

[0074] For identified candidate occlusion areas, whether they meet the effective occlusion criteria can be determined based on the degree of influence of the candidate occlusion area on the actual visibility of the target area. Specifically, at least two factors can be selected for judgment: the overlap ratio between the candidate occlusion area and the target area, the continuous coverage width of the candidate occlusion area within the target area, whether the candidate occlusion area covers a key monitoring sub-area, and the length of the missing boundary of the target area caused by the candidate occlusion area. Using at least two different occlusion impact factors can avoid judging the degree of occlusion solely based on a single area indicator while ignoring the impact of occlusion location or occlusion pattern on inspection interpretation.

[0075] The overlap ratio characterizes the extent to which a candidate occlusion region covers the target region. The overlap area between the candidate occlusion region and the target region can be determined, and the overlap ratio is obtained based on the ratio between the overlap area and the target region area. A larger overlap ratio indicates a larger area of ​​the image that cannot be directly displayed in the target region. In practice, the target region mask and the candidate occlusion region mask can be obtained separately, their intersection can be calculated at the pixel level, and the overlap ratio can be determined based on the number of intersection pixels and the number of pixels in the target region. When multiple candidate occlusion regions jointly cover the target region, the overlap ratio for each candidate occlusion region can be determined separately, or multiple candidate occlusion regions that are adjacent to each other or belong to the same occlusion object can be merged to determine the overlap ratio.

[0076] The continuous coverage width characterizes the width range of the continuous occlusion formed by the candidate occlusion region within the target region. Compared to the overlap ratio, the continuous coverage width further reflects whether the occlusion region continuously cuts off the image content in the target region along a certain direction. For example, although a thin pipe or railing corresponds to a small occlusion area, it may form a continuous coverage along the horizontal or vertical direction of the target region, segmenting the structural boundary or continuous texture of the target region. Therefore, the overlap range between the candidate occlusion region and the target region can be projected according to the main direction, width direction, or preset measurement direction of the target region, and the continuous coverage width can be determined according to the length of the continuous projection interval; alternatively, the continuous coverage width can be determined according to the maximum inscribed width or continuous pixel bandwidth formed by the candidate occlusion region within the target region, but this embodiment does not limit this.

[0077] A key monitoring sub-region refers to a localized area within the target area that receives a high level of attention during inspections. It can be pre-defined based on the requirements of the inspection task. For example, a key monitoring sub-region could be the location of historical cracks, leakage monitoring locations, structural joint areas, equipment connection locations, or other areas requiring focused image interpretation. Key monitoring sub-regions can be stored in association with the target area identifier and represented as local area coordinates or an image mask. To determine whether a candidate occlusion area covers a key monitoring sub-region, a comparison can be made to determine if there is spatial overlap between the candidate occlusion area and the key monitoring sub-region. In one implementation, coverage of the key monitoring sub-region can be determined simply by the presence of overlap between the candidate occlusion area and the key monitoring sub-region.

[0078] The target region boundary missing length characterizes the degree to which the visible boundary of the target region is interrupted or missing due to the candidate occlusion area. The target region boundary can be the outline of an engineering component, a structural boundary line, or a pre-marked region edge. When the candidate occlusion area covers the target region boundary, the target region boundary segments covered by the candidate occlusion area can be determined, and the lengths of the corresponding segments can be counted to obtain the target region boundary missing length. Specifically, the target region boundary can be represented as a set of boundary pixels or a contour curve, and the continuous boundary segments falling within the candidate occlusion area can be determined. Then, the target region boundary missing length can be determined based on the pixel length of the corresponding boundary segment or the length after mapping to the actual engineering space. For cases with multiple discontinuous missing boundary segments, the lengths of each missing boundary segment can be accumulated, or the judgment can be made based on the longest continuous missing boundary segment.

[0079] After obtaining at least two of the above pieces of information, corresponding effective occlusion conditions can be set for each piece of information. For example, a ratio judgment condition can be set for the overlap ratio, a width judgment condition can be set for the continuous coverage width, a coverage judgment condition can be set for the key monitoring sub-area, and a length judgment condition can be set for the missing length of the target area boundary. Different effective occlusion conditions can be pre-configured according to the size of the inspected object, the type of the target area, or the requirements of the inspection task, and different judgment parameters can be used for different target areas.

[0080] When at least two of the selected conditions satisfy the corresponding valid occlusion criteria, the candidate occlusion region can be determined as an occlusion region. For example, if the candidate occlusion region has a large degree of overlap with the target region and forms continuous coverage within the target region, it can be determined that the candidate occlusion region has a substantial impact on the image integrity of the target region; or, if the candidate occlusion region covers the key monitoring sub-region and causes obvious loss of the target region boundary, it can also be determined as an occlusion region.

[0081] By judging the impact of occlusion from multiple aspects such as the degree of overlap between candidate occlusion areas and target areas, the degree of continuous coverage, the coverage of key monitoring sub-regions, and the degree of missing boundaries of target areas, and determining the occlusion area when at least two of the corresponding conditions are met, the situation where local noise, slight foreground changes, or occasional image anomalies are mistakenly identified as effective occlusion can be reduced, thereby improving the accuracy of occlusion area determination and making subsequent occlusion source analysis more focused on the image areas that actually affect the inspection and interpretation.

[0082] In one example embodiment of this disclosure, the extraction of classification features corresponding to the occluded region can be achieved through the following steps, which may specifically include: At least one of the following can be extracted from the occluded area: boundary complexity, porosity, rectangularity, and proportion of straight edges, as geometric morphological features; at least one of the following can be extracted from the occluded area: green vegetation index, color mean, color variance, and texture direction consistency, as color texture features; based on the positional changes of the occluded area in consecutive frames, at least one of the following can be determined as temporal motion features: centroid displacement, area change rate, number of consecutive occurrence frames, local optical flow direction dispersion, and occurrence duration; and based on the matching relationship between the occluded area and the projected positions of fixed facilities in the engineering facility ledger, on-site facility matching features can be determined.

[0083] Geometric morphological features characterize the external contour, internal connectivity, and overall regularity of the occluded area. Boundary complexity describes the degree of curvature or fragmentation of the occluded area's boundary. For example, it can be determined based on the ratio between the boundary length and area of ​​the occluded area, or on information such as changes in boundary curvature and the number of boundary inflection points; the more irregular the boundary, the higher the boundary complexity. Porosity describes the degree to which unoccluded content penetrates or forms gaps within the occluded area. It can be determined based on the ratio between the area of ​​voids within the occluded area and the overall area of ​​the occluded area. Natural vegetation typically has many discontinuous gaps between branches and leaves, thus the corresponding occluded area may have a high porosity. Rectangularity describes the degree to which the occluded area closely resembles a regular rectangle. It can be determined based on the ratio between the area of ​​the occluded area and the area of ​​its smallest circumscribed rectangle. Regular equipment boxes, components, or other fixed facilities typically have relatively high rectangularity. The straight edge ratio is used to describe the proportion of approximately straight line segments in the boundary of the occluded area. The boundary of the occluded area can be obtained by edge detection, and straight edges can be extracted by Hough transform or line segment detection. The straight edge ratio is then determined based on the proportional relationship between the length of the straight edge and the total boundary length of the occluded area.

[0084] Color texture features are used to characterize the color distribution and local pixel variation patterns within the occluded area. The green vegetation index can be used to characterize the prominence of the green component relative to other color components in the occluded area. For example, the green vegetation index can be calculated based on the pixel values ​​corresponding to the red, green, and blue channels in the Red-Green-Blue (RGB) color space. In one implementation, the average value of each color channel in the occluded area can be determined separately, and the green vegetation index can be formed based on the differences between the green channel and the red and blue channels. In other optional implementations, the green vegetation index can also be replaced by the normalized green-red difference index or hue distribution information in the HueSaturation Value (HSV) color space, as long as it reflects the green vegetation color characteristics corresponding to the occluded area.

[0085] Color mean is used to characterize the overall color level of an occluded area, and can be calculated by averaging the pixel values ​​of one or more color channels. Color variance characterizes the dispersion of color values ​​within an occluded area, and can be determined based on the deviation of each pixel's color value from its corresponding color mean. Color mean and color variance together describe the overall hue and color distribution stability of an occluded area. For example, regular structures with relatively uniform surface color and natural vegetation with more dispersed color variations can exhibit different color statistical characteristics.

[0086] Texture orientation consistency characterizes the degree to which local textures within an occluded region are distributed along the same or similar directions. In practice, the gradient direction of each pixel or image patch within the occluded region can be calculated, and the distribution of different gradient directions can be statistically analyzed. When most texture directions are concentrated within one or a limited number of directions, texture orientation consistency is considered high. Texture analysis methods such as gray-level co-occurrence matrix, local directional filtering, or local binary mode can also be used to obtain texture information of the occluded region, and then texture orientation consistency can be determined based on the directional distribution of the texture response.

[0087] Temporal motion features can be determined based on the positional changes of occluded regions across consecutive frames. Consecutive frames can be multiple inspection images acquired by the UAV at the same waypoint, the same flight segment, or adjacent acquisition times. To ensure that the same occluded region can be compared across different image frames, image alignment can be performed on the consecutive frames first, and the correspondence between occluded regions in consecutive frames can be established through target tracking, region matching, or optical flow estimation. When the UAV undergoes displacement or the camera attitude changes, the global image motion can be estimated using stable structures in the target region, and the observed motion corresponding to the occluded region can be compensated to reduce the impact of the UAV's own motion on the temporal motion features.

[0088] Centroid displacement is used to characterize the degree of change in the overall position of an occluded region across consecutive frames. The centroid of the occluded region can be calculated separately in each image frame, and the centroid displacement can be determined based on the distance between centroid coordinates in adjacent frames or within a preset time range. Centroid displacement can be represented as pixel distance or normalized relative to the width or height of the target region. For moving personnel, vehicles, or mobile construction equipment, the corresponding occluded region typically exhibits a significant centroid displacement; for relatively stable fixed facilities, the centroid position usually remains relatively stable.

[0089] The area change rate is used to characterize the degree of change in the coverage area of ​​an occluded region across consecutive frames. The area of ​​the occluded region in adjacent image frames can be determined separately, and the area change rate is determined based on the proportional relationship between the area difference and the area of ​​the occluded region in the reference frame. When a dynamic object enters or leaves the target region, the area of ​​the occluded region may change significantly; therefore, the area change rate can be used to describe the entry, departure, or scale change of the occluded object relative to the target region.

[0090] The consecutive occurrence frame count characterizes the number of image frames in which an occluded area is continuously identified in continuous inspection images. It can be started from the first image frame in which an occluded area is detected, continuously tracking the corresponding occluded area until the occluded area disappears or the region correspondence condition is no longer met, and then counting the corresponding image frames. The occurrence duration can be determined based on the consecutive occurrence frame count and the frame rate of the inspection images, or directly based on the time difference between the first and last occurrence times of the occluded area. The consecutive occurrence frame count and occurrence duration reflect the degree of continuous presence of the occluded object in the current inspection acquisition process.

[0091] Local optical flow direction dispersion is used to characterize the differences in motion direction at different local locations within an occluded region. Optical flow estimation can be performed on adjacent image frames to obtain optical flow vectors corresponding to multiple pixels or feature points within the occluded region, and the degree of dispersion between the directions of each optical flow vector can be statistically analyzed. When the occluded object moves along similar directions as a whole, the optical flow directions at different locations within the occluded region are usually relatively consistent; however, when tree branches and leaves sway locally due to wind disturbance, different branch and leaf areas may form local motions in different directions, thus exhibiting higher optical flow direction dispersion.

[0092] On-site facility matching features can be determined based on the matching relationship between the obstructed area and the projected location of fixed facilities in the engineering facility ledger. The engineering facility ledger records relatively fixed facility objects in the engineering site, which may include fixed pipelines, railings, supports, platform frames, equipment boxes, monitoring equipment, and other fixed facilities. The engineering facility ledger can record the facility identification, spatial coordinates, geometric dimensions, or three-dimensional contour data corresponding to the fixed facilities.

[0093] In the specific implementation process, based on the waypoint positions, camera parameters, and shooting attitude of the UAV when collecting inspection images, fixed facilities located near the target area in the engineering facility register can be projected onto the inspection images to obtain the projected positions of the fixed facilities. Camera parameters may include focal length, principal point position, and other intrinsic camera parameters, while shooting attitude may include heading angle, pitch angle, or extrinsic camera parameters. Subsequently, the spatial relationship between the occluded area and the projected positions of the fixed facilities can be compared, such as determining the degree of regional overlap, boundary distance, or center position deviation, and on-site facility matching features can be formed based on the corresponding spatial relationship.

[0094] For example, the boundaries of the occluded area and the projected area of ​​the fixed facility can be extracted separately, and the distances from multiple sampling points in the occluded area boundary to the projected area boundary of the fixed facility can be calculated. The degree of boundary matching can be determined based on the average or median of these distances. Alternatively, the intersection-union ratio or overlap ratio between the occluded area and the projected area of ​​the fixed facility can be calculated directly. The closer their spatial locations or the higher the degree of overlap, the higher the degree of matching between the occluded area and the corresponding fixed facility.

[0095] Optionally, when the engineering facility ledger does not directly provide spatial data that can be used for image projection, the spatial location of fixed facilities can be obtained by using the three-dimensional engineering model corresponding to the engineering object and the projection location of fixed facilities can be generated; alternatively, fixed structure areas with long-term stable locations can be extracted from historical inspection images, fixed facility reference templates can be established, and on-site facility matching features can be determined based on the image matching results between the occluded area and the fixed facility reference template.

[0096] By extracting classification features from the morphological structure, color texture, continuous frame changes, and correspondence with the spatial location of engineering facilities of the occluded area, we can use the differences in boundary regularity, internal hole status, color distribution, motion changes, and facility association degree of different occluded objects to describe them, reduce the type confusion caused by relying solely on a single image appearance feature, and improve the ability to distinguish between vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion.

[0097] In one example embodiment of this disclosure, the following steps can be used to determine the occlusion source confidence levels corresponding to multiple preset occlusion source types based on classification features, specifically including: For each preset occlusion source type, feature scores can be determined for geometric morphology features, color and texture features, temporal motion features, and on-site facility matching features. Based on the feature scores and their corresponding weights for each preset occlusion source type, the occlusion source confidence level for that type is determined. Specifically, when determining the occlusion source confidence level for vegetation occlusion, the weight corresponding to boundary complexity in color and texture features and geometric morphology features is greater than the weight corresponding to other classification features. When determining the occlusion source confidence level for fixed facility occlusion, the weight corresponding to the proportion of straight edges in on-site facility matching features and geometric morphology features is greater than the weight corresponding to other classification features. When determining the occlusion source confidence level for short-term dynamic occlusion, the weight corresponding to temporal motion features is greater than the weight corresponding to other classification features.

[0098] The feature score characterizes the degree to which the corresponding classification feature supports the current occlusion region belonging to the corresponding preset occlusion source type. Different classification features may have different original data units and numerical ranges. Therefore, before determining the feature score, each classification feature can be normalized or standardized to convert different classification features to a comparable numerical range. For example, the classification feature can be mapped to between 0 and 1 based on a pre-defined feature value range, or the standardization parameter can be determined based on the feature distribution corresponding to historical inspection data. This embodiment does not limit this approach.

[0099] For different preset occlusion source types, the same classification feature can correspond to different feature evaluation relationships. For example, when determining feature scores for vegetation occlusion, occluded areas with higher greenness, more fragmented boundaries, or significant differences in local movement directions can obtain higher corresponding feature scores; when determining feature scores for fixed facility occlusion, occluded areas with more regular shapes, more straight edges, or a high degree of matching with the projected position of the fixed facility can obtain higher corresponding feature scores; when determining feature scores for short-term dynamic occlusion, changes in position or area in consecutive frames, or the occurrence of a continuous state that matches a dynamic object, can obtain higher temporal motion feature scores. Therefore, each feature score can be determined based on the degree of matching between the corresponding classification feature and the feature distribution of the corresponding preset occlusion source type.

[0100] Optionally, a corresponding feature scoring function can be pre-established for each preset occlusion source type. For example, categorical features can be input into a piecewise mapping function, a probability distribution function, or an evaluation model built based on training samples, and the corresponding feature score can be output. Alternatively, one or more reference intervals can be set for each categorical feature, and the corresponding feature score can be determined based on the positional relationship between the categorical feature and the reference interval. For categorical features that include multiple specific feature quantities, such as geometric morphology features that include multiple morphological parameters, the sub-feature scores corresponding to each morphological parameter can be determined first, and then the feature score corresponding to the geometric morphology feature can be formed based on the scores of multiple sub-features.

[0101] After obtaining the feature scores corresponding to each classification feature, the occlusion source confidence corresponding to the preset occlusion source type can be determined based on the feature scores and weights corresponding to each feature score. The weights are used to characterize the contribution of different classification features in the judgment process of the corresponding preset occlusion source type. Since different occlusion source types have different image representations and variation patterns, different weight combinations can be configured for vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion.

[0102] In one implementation, the scores of each feature can be weighted and summed, and the confidence level of the occlusion source can be determined based on the weighted result. For example, the scores of geometric shape features, color and texture features, temporal motion features, and on-site facility matching features can be multiplied by their corresponding weights, and then the results can be combined. Each weight can be normalized so that the sum of the weights is a preset value. It should be noted that the weight combinations corresponding to each preset occlusion source type can be set independently, and the weights corresponding to the same classification feature can be different under different preset occlusion source types.

[0103] When determining the confidence level of occlusion sources corresponding to vegetation occlusion, the weight of boundary complexity in color texture features and geometric morphology features is greater than the weight of other classification features. Vegetation objects usually have relatively obvious color distribution characteristics, and the boundaries of occlusion areas formed by natural vegetation such as branches, shrubs, or vines are usually irregular. Therefore, color texture features and boundary complexity can directly reflect the image characteristics of vegetation occlusion. By increasing the weight of the above classification features, the confidence level of occlusion sources corresponding to vegetation occlusion can place more emphasis on the color performance and boundary morphology of the occluded area, while temporal motion features or on-site facility matching features can still be used as auxiliary judgment information in the calculation.

[0104] When determining the occlusion source confidence for fixed facility occlusion, the weight corresponding to the proportion of straight edges in the on-site facility matching features and geometric morphology features is greater than the weight corresponding to other classification features. Fixed facilities such as pipelines, railings, supports, or platform components typically have relatively stable spatial locations, and their image contours may contain many straight edges. Therefore, the degree of matching between the occluded area and the projected position of the fixed facility, as well as the distribution of straight edges in the occluded area, can directly reflect the characteristics of fixed facility occlusion. By increasing the weight corresponding to the on-site facility matching features and the proportion of straight edges, the influence of color changes or local lighting conditions on the determination of fixed facility occlusion can be reduced.

[0105] When determining the confidence level of occlusion sources for short-term dynamic occlusion, the weight of temporal motion features is greater than that of other classification features. Short-term dynamic occlusion is typically caused by people, vehicles, mobile construction equipment, or temporary work objects. Its main characteristic is that the occlusion area changes position or range in consecutive frames, or only appears for a limited duration. Therefore, temporal information such as centroid displacement, area change, continuous occurrence, or local motion direction can directly reflect the changing characteristics of short-term dynamic occlusion. By increasing the weight of temporal motion features, the possibility of misclassifying dynamic objects as other occlusion source types based solely on single-frame color or shape information can be reduced.

[0106] In some implementations, the weights corresponding to each feature score can be determined through training or statistics based on pre-labeled historical inspection samples, or they can be pre-configured by inspection management personnel according to the engineering scenario. For example, for inspection areas with dense vegetation distribution, the contribution of classification features related to vegetation characteristics in vegetation occlusion judgment can be increased; for engineering areas with a large number of fixed facilities and complete facility records, the contribution of on-site facility matching features in fixed facility occlusion judgment can be increased.

[0107] By setting differentiated feature contribution relationships for different preset occlusion source types, vegetation occlusion judgment focuses more on color texture and boundary complexity, fixed facility occlusion judgment focuses more on facility matching degree and straight edge distribution, and short-time dynamic occlusion judgment focuses more on motion changes in continuous frames. This can match the confidence of each occlusion source with the main physical characteristics of the corresponding occluded object, reduce classification bias caused by uniformly using the same feature weights for different types, and improve the representation accuracy of occlusion source confidence.

[0108] In one example embodiment of this disclosure, the target occlusion source type corresponding to the occlusion area can be determined by the following steps based on the confidence level of the highest occlusion source and the confidence difference between the highest and second-highest occlusion source. Specifically, this may include: If the confidence level of the highest occlusion source is not less than the preset classification confidence threshold and the confidence difference is not less than the preset confidence difference threshold, the preset occlusion source type corresponding to the highest occlusion source confidence level can be determined as the target occlusion source type; if the confidence level of the highest occlusion source is less than the preset classification confidence threshold and / or the confidence difference is less than the preset confidence difference threshold, the target occlusion source type can be determined as unconfirmed occlusion.

[0109] This process involves comparing the confidence levels of multiple preset occlusion source types, identifying the highest confidence level as the top occlusion source confidence level, and designating the next highest confidence level as the second-highest occlusion source confidence level. The top occlusion source confidence level characterizes the degree of match between the current occluded area and the most likely preset occlusion source type, while the second-highest confidence level characterizes the preset occlusion source type whose match with the current occluded area is second only to the type corresponding to the top occlusion source confidence level. Furthermore, a difference calculation can be performed between the top and second-highest confidence levels to obtain the confidence level difference.

[0110] The preset classification confidence threshold is used to determine whether the confidence level of the highest occlusion source reaches a level sufficient to support the corresponding preset occlusion source type. The preset confidence difference threshold is used to determine whether there is sufficient distinction between the preset occlusion source type corresponding to the highest occlusion source confidence level and the preset occlusion source type corresponding to the second highest occlusion source confidence level. The preset classification confidence threshold and the preset confidence difference threshold can be preset based on the occlusion classification results corresponding to historical inspection images, or they can be determined statistically or through model validation based on labeled occlusion samples. For example, based on samples with multiple known occlusion source types, the highest occlusion source confidence level and the confidence difference distribution corresponding to correctly classified samples can be statistically analyzed, and the corresponding thresholds can be determined accordingly.

[0111] When the confidence level of the highest occlusion source is not less than a preset classification confidence threshold, it indicates that the current classification feature has a high degree of support for the preset occlusion source type corresponding to the highest occlusion source confidence level. Simultaneously, when the confidence difference is not less than a preset confidence difference threshold, it indicates that this preset occlusion source type has a significant distinguishing advantage over other preset occlusion source types. When both of these conditions are met, the preset occlusion source type corresponding to the highest occlusion source confidence level can be determined as the target occlusion source type. For example, when the occlusion source corresponding to vegetation occlusion has the highest confidence level, and this occlusion source confidence level reaches a preset classification confidence threshold, and the confidence difference between it and the second highest occlusion source confidence level corresponding to fixed facility occlusion or short-term dynamic occlusion reaches a preset confidence difference threshold, the target occlusion source type can be determined as vegetation occlusion.

[0112] When the highest occlusion source confidence is less than the preset classification confidence threshold, although the occlusion source confidence corresponding to a certain preset occlusion source type is relatively the highest among multiple occlusion source confidences, the current classification features still do not adequately support this preset occlusion source type. For example, an occluded area may simultaneously possess some green color features and regular edge features, but none of these features are prominent, resulting in the vegetation occlusion having the highest relative occlusion source confidence but still a low overall value. In this case, the target occlusion source type can be determined as unconfirmed occlusion, rather than directly outputting the determined occlusion source type based on the relative maximum value.

[0113] When the confidence difference is less than a preset confidence difference threshold, it indicates that the confidence of the highest occlusion source and the second highest occlusion source are relatively close, meaning that the current occlusion area simultaneously possesses similar characteristics of two or more preset occlusion source types. For example, green protective facilities may have a distinct green color feature, along with regular straight edges and features matching the location of fixed facilities, making the occlusion source confidence of vegetation-type occlusion and fixed facility-type occlusion relatively close. Similarly, wind-blown branches and leaves may move in consecutive frames, making both vegetation-type occlusion and short-term dynamic occlusion have high occlusion source confidence. In this case, the target occlusion source type can be determined as unconfirmed occlusion to avoid forcibly performing type confirmation when different preset occlusion source types are difficult to reliably distinguish.

[0114] It should be noted that the target occlusion source type can be determined as unconfirmed occlusion when at least one of the following occurs: the highest occlusion source confidence level is less than the preset classification confidence level threshold, or the confidence level difference is less than the preset confidence level difference threshold. In other words, a specific preset occlusion source type is determined only when both the absolute confidence level of the occlusion source and the relative distinguishability between different occlusion sources meet the corresponding requirements.

[0115] By simultaneously judging the absolute confidence level of the highest occlusion source and the degree of distinction between the confidence levels of the highest and second-highest occlusion sources, and determining the target occlusion source type as unconfirmed occlusion when either condition is insufficient, we can avoid forcibly outputting a specific occlusion type based solely on the confidence ranking result. This reduces the probability of occluded objects with similar appearances or mixed features being misclassified, thereby reducing the mismatch compensation caused by the transmission of incorrect occlusion sources to subsequent compensation processing.

[0116] In one example embodiment of this disclosure, it can be achieved through Figure 3 The steps in the document determine the target compensation rule corresponding to the occlusion area based on the type of target occlusion source, referring to... Figure 3 As shown, it can specifically include: Step S310: If the target occlusion source type is vegetation occlusion, retrieve unobstructed candidate images from the historical inspection images corresponding to the target area, and map the unobstructed area in the unobstructed candidate images corresponding to the occluded area to the occluded area in the inspection images; and / or, determine an alternative shooting direction that is laterally deflected relative to the original shooting direction according to the original shooting direction corresponding to the inspection images, and adjust the shooting direction of the UAV and the gimbal pitch angle according to the alternative shooting direction; Step S320: When the target occlusion source type is the fixed facility type occlusion, determine the fixed occlusion direction according to the projection direction of the fixed facility in the target area, generate an alternate waypoint at a position avoiding the fixed occlusion direction, and control the UAV to re-acquire the inspection image corresponding to the target area based on the alternate waypoint; Step S330: When the target occlusion source type is the short-term dynamic type occlusion, and when the safe hovering condition is met, control the UAV to re-acquire the inspection image corresponding to the target area after a preset delay at the current waypoint; and / or, control the UAV to continuously acquire multiple frames of inspection images at the same waypoint or the same flight segment, and select the inspection image with the smallest occlusion ratio from the multiple frames of inspection images. Step S340: If the source type of the target occlusion is the unconfirmed occlusion, retrieve unoccluded candidate images from the historical inspection images corresponding to the target area according to the occlusion ratio corresponding to the occlusion area and generate compensation results, or generate backup waypoint reshoot instructions or manual review prompts.

[0117] Historical inspection images refer to inspection images collected and saved for the same target area before the current inspection task. A correspondence between the current inspection image and historical inspection images can be established based on the target area identifier. Since the same target area usually has a relatively stable engineering structure in different inspection tasks, images that are not obstructed by vegetation or have a low degree of obstruction can be retrieved from historical inspection images as unobstructed candidate images.

[0118] In the implementation process, historical inspection images belonging to the same target area can be first filtered based on the target area identifier. Then, unobstructed candidate images are further filtered by considering factors such as shooting direction, shooting distance, gimbal attitude, or the degree of matching between the target area image content and the target area. Unobstructed candidate images refer to historical inspection images in which the content of the target area corresponding to the current obstructed area can be directly presented. Image matching can be performed between the unobstructed areas in the current inspection image and the corresponding areas in the historical inspection images, and a spatial correspondence between the current inspection image and the unobstructed candidate images can be established based on the matching results.

[0119] After identifying unoccluded candidate images, the unoccluded region corresponding to the current occluded region in the candidate images can be determined based on spatial correspondence. This unoccluded region is then mapped to the occluded region in the inspection image. Spatial correspondence can be determined based on feature point matching, structure line matching, target region contour matching, or image registration results, and region mapping can be achieved using homography transformation, affine transformation, or local geometric mapping. After completing region mapping, brightness consistency adjustment or edge fusion can be performed between the mapped region and the current inspection image to reduce brightness differences and abrupt changes in region boundaries at different inspection times. It should be noted that mapping using historical inspection images is mainly used to form compensated image content for the target region and does not change the original inspection image. The original inspection image can be retained for subsequent data tracing.

[0120] In addition to using historical inspection images for compensation, an alternative shooting direction that is laterally deflected relative to the original shooting direction can be determined based on the original shooting direction corresponding to the inspection image. The original shooting direction can be determined based on the UAV's heading angle, the optical axis direction of the image acquisition device, or the shooting posture recorded during the inspection mission. Since vegetation such as branches, shrubs, or vines usually only cover part of the target area from the observation direction, changing the image acquisition direction can alter the imaging projection relationship between the vegetation and the target area.

[0121] For example, with the target area as the observation center, candidate shooting directions can be generated to one or both sides of the original shooting direction. The direction that reduces vegetation obstruction can then be selected as the backup shooting direction. After determining the backup shooting direction, the drone's heading or the orientation of the image acquisition device can be adjusted, and the gimbal's pitch angle can be adjusted accordingly to re-point the image acquisition device towards the target area. In one example implementation, the backup shooting direction can have a lateral angle change relative to the original shooting direction, while the gimbal's pitch angle is adjusted synchronously according to the drone's position and the spatial position of the target area to ensure that the target area remains within the image acquisition range. By changing the lateral observation direction and the gimbal's attitude, some vegetation obstruction can be bypassed, and the inspection image corresponding to the target area can be reacquired.

[0122] When the target occlusion source is a fixed facility, the fixed occlusion direction can be determined based on the projection direction of the fixed facility in the target area. The projection direction of the fixed facility in the target area characterizes the spatial direction in which the fixed facility occludes the target area relative to the target area when viewed from the current image acquisition position. For example, when a fixed pipeline is located on the left front side of the target area and covers part of the image content of the target area, the corresponding fixed occlusion direction can be determined based on the relative position between the projection area of ​​the fixed pipeline and the target area.

[0123] Specifically, the direction from the projected area of ​​the fixed facility to the target area or the direction of the fixed facility relative to the target area can be determined based on the relative positional relationship between the projected location of the fixed facility and the target area, and a fixed occlusion direction can be formed accordingly. The projected location of the fixed facility can be determined based on the spatial location of the fixed facility in the engineering facility ledger and the current shooting posture of the drone, or it can be determined based on the fixed facility areas already identified in the current inspection images.

[0124] After determining the fixed occlusion direction, alternative waypoints can be generated at locations that avoid this direction. Alternative waypoints are candidate spatial positions that alter the UAV's observation position relative to the original waypoint. By changing the spatial relationship between the UAV and the target area, fixed facilities are no longer located between the image acquisition device and the target area, or the coverage area of ​​the fixed facilities over the target area is reduced. For example, alternative waypoints can be generated along the opposite side of the fixed occlusion direction, or multiple candidate waypoints can be generated around the target area, and alternative waypoints can be selected based on the relationship between the observation direction of each candidate waypoint and the fixed occlusion direction.

[0125] After generating a backup waypoint, the UAV can be controlled to move from the current waypoint or inspection route to the backup waypoint, and inspection images corresponding to the target area can be re-acquired based on the backup waypoint. During the UAV's movement to or after reaching the backup waypoint, the UAV's heading and gimbal attitude can be adjusted according to the spatial relationship between the backup waypoint and the target area, ensuring the image acquisition device faces the target area. The backup waypoint can be represented using geographic coordinates or local engineering coordinates; this embodiment does not limit the specific representation of the backup waypoint. By changing the UAV's spatial observation position relative to a fixed obstruction direction, the situation where fixed facilities continuously obstruct the target area when repeatedly acquiring images from the original waypoint can be reduced.

[0126] When the target occlusion source is a short-term dynamic occlusion, the current flight status of the UAV can be used to determine whether the safe hovering conditions are met. The safe hovering conditions determine whether the UAV can maintain a stable hover for a preset duration at the current waypoint. These conditions can be determined based on at least one of the following: whether the current location is within a permitted hovering area, the distance between the UAV and surrounding obstacles, remaining battery power, or the current flight mission status. When the safe hovering conditions are met, the UAV can be kept at the current waypoint, and after a preset delay, the corresponding inspection image of the target area can be re-acquired.

[0127] The preset delay duration can be understood as the waiting time from the moment a short-term dynamic occlusion is determined or the current image acquisition is stopped, until the corresponding inspection image of the target area is acquired again. Personnel, vehicles, or mobile construction equipment typically have a certain degree of continuous movement and may leave the imaging direction of the target area during the waiting period. Therefore, by delaying the acquisition time, the image of the target area can be reacquired without changing the spatial position of the UAV. The preset duration can be pre-configured or determined based on the movement state of the occluded area in consecutive frames. For example, the delay duration can be estimated based on the moving direction and speed of the occluded object, or the time required for the occluded area to exit the target area's image range.

[0128] The occlusion ratio can be determined based on the overlap area between the occluded area and the target area in the corresponding inspection image. After obtaining the occlusion ratios corresponding to multiple frames of inspection images, the occlusion ratios can be compared, and the inspection image with the smallest occlusion ratio can be selected as the valid inspection image corresponding to the current target area. When multiple inspection images have the same or similar minimum occlusion ratios, an inspection image can also be selected based on the clarity of the target area or the acquisition time. By continuously acquiring inspection images corresponding to multiple acquisition times, the positional changes of short-term dynamic objects in the time dimension can be utilized to filter inspection images with higher visibility of the target area from multiple images.

[0129] When the target occlusion source type is unconfirmed occlusion, the corresponding processing method can be determined based on the occlusion ratio of the occluded area. Since unconfirmed occlusion indicates that the current classification features are insufficient to reliably distinguish the specific occlusion source, it is possible to avoid directly using compensation processing methods that only apply to a certain occlusion source type, and instead select the appropriate processing method based on the actual coverage of the target area by the occluded area.

[0130] When the occlusion ratio corresponds to a low degree of occlusion, unoccluded candidate images can be retrieved from historical inspection images corresponding to the target area, and compensation results can be generated based on these unoccluded candidate images. The retrieval method for unoccluded candidate images can be the same as the retrieval method for historical inspection images corresponding to vegetation occlusion. For example, a set of historical inspection images can be determined based on the target area identifier, and unoccluded candidate images suitable for compensation can be filtered based on the image matching relationship between the current inspection image and historical images. The compensation result can include a target area compensation image generated based on the unoccluded candidate images, or it can be used to indicate whether historical image content suitable for compensation was successfully obtained.

[0131] When the occlusion level is high or suitable unobstructed candidate images cannot be obtained, a backup waypoint re-capture command or a manual review prompt can be generated. The backup waypoint re-capture command instructs the UAV to change its current image acquisition position and re-capture the corresponding inspection image of the target area at the backup waypoint. Since the specific source of occlusion cannot be determined without confirming the occlusion, the backup waypoint can be generated based on the flyable space around the target area and the original waypoint position, without relying on the direction corresponding to a specific source of occlusion.

[0132] Manual verification prompts are used to instruct relevant personnel to further confirm the current obstructed area, the type of source of the obstruction, or the availability status of the inspection image. Manual verification prompts can be output through the inspection management platform, inspection control terminal, or other inspection information display interfaces, and can be associated with target area markers and the current inspection image, allowing relevant personnel to view the corresponding obstruction situation.

[0133] By employing compensation methods tailored to the spatial stability, temporal variation characteristics, or classification reliability of different types of occlusion—vegetation occlusion, fixed facility occlusion, short-term dynamic occlusion, and unconfirmed occlusion—invalid processing caused by uniform reshooting or image compensation for different occlusion situations can be reduced. Utilizing historical unoccluded images or changing the lateral shooting direction can reduce the impact of vegetation occlusion, changing waypoints can reduce continuous fixed facility occlusion, delayed shooting or continuous acquisition can utilize the temporal changes of dynamic objects to obtain low-occlusion images, and conservative processing of unconfirmed occlusion can reduce erroneous categorized compensation, thereby improving the effective processing rate of occluded images and reducing duplicate acquisition and invalid reshoots.

[0134] In one example embodiment of this disclosure, the determination of repeated occlusion areas based on occlusion event records corresponding to multiple inspection tasks can be achieved through the following steps, specifically including: Within a preset statistical period, based on multiple occlusion event records corresponding to the same target area, at least one of the following can be determined: occlusion frequency, proportion of occlusion events of the same occlusion source type, and number of compensation failures for the target area. If the occlusion frequency reaches a preset frequency threshold, the proportion of occlusion events reaches a preset proportion threshold, and / or the number of compensation failures reaches a preset number threshold, the target area is identified as a repeated occlusion area. Based on the target occlusion source type and compensation results in each occlusion event record corresponding to the repeated occlusion area, the main occlusion source type and historical compensation status corresponding to the repeated occlusion area are determined.

[0135] This feature allows for statistical analysis of multiple occlusion event records corresponding to the same target area within a preset statistical period. The preset statistical period is used to limit the scope of historical inspection tasks participating in the duplicate occlusion judgment and can be set according to the number of inspection tasks or a natural time range. For example, the preset statistical period can correspond to a number of recent inspection tasks, or to inspection tasks executed within a few days or months. The same preset statistical period can be used for different target areas, or different preset statistical periods can be configured according to the inspection frequency or importance of the target area.

[0136] In the specific implementation process, based on the target area identifier in the occlusion event log, occlusion event logs with the same target area identifier in different inspection tasks can be grouped into an event set corresponding to the same target area, and sorted according to the inspection task execution time or the occlusion event occurrence time. Based on this event set, at least one of the following can be determined: the occlusion occurrence frequency corresponding to the target area, the proportion of occlusion events of the same occlusion source type, and the number of compensation failures, to evaluate the degree of occlusion repetition of the target area in multiple inspection tasks.

[0137] Occlusion frequency is used to characterize how frequently a target area is occluded within a preset statistical period. The number of inspection tasks with effective occlusion in the target area within the preset statistical period can be counted, and the occlusion frequency can be determined based on the ratio between the number of inspection tasks with effective occlusion and the total number of inspection tasks corresponding to the target area. For example, if the same target area generates corresponding occlusion event records in multiple inspection tasks, its occlusion frequency is relatively high; if occlusion event records are generated only in a few inspection tasks, its occlusion frequency is relatively low. By statistically analyzing occlusion occurrences according to the inspection task dimension, the excessive amplification of occlusion redundancy can be avoided when multiple frames of inspection images are continuously acquired in the same inspection task, resulting in multiple related records.

[0138] The proportion of occlusion events of the same occlusion source type is used to characterize the degree of concentration of repeated occurrences of a certain target occlusion source type among multiple occlusion events corresponding to a target area. Specifically, the number of occlusion events corresponding to vegetation occlusion, fixed facility occlusion, short-term dynamic occlusion, and unconfirmed occlusion can be counted separately, and the corresponding occlusion event proportion can be determined according to the ratio between the number of occlusion events corresponding to each target occlusion source type and the total number of occlusion events corresponding to the target area. For example, when vegetation occlusion occurs continuously in multiple inspection tasks for the same target area, a high proportion of occlusion events corresponding to vegetation occlusion can indicate that the target area is repeatedly affected by the same type of occlusion. For the case of multiple occlusion areas in the same inspection task, corresponding occlusion event records can be generated separately, or multiple occlusion areas belonging to the same occlusion object can be associated and counted according to spatial location. This embodiment does not limit this.

[0139] The number of compensation failures represents the cumulative number of instances where the expected processing result was not obtained after executing the target compensation rule for the target area. The success of the corresponding target compensation rule can be determined based on the compensation results in the occlusion event log, and the number of occlusion events with failed compensation results within a preset statistical period can be counted. In one implementation, if no usable inspection image is obtained after executing the target compensation rule, the corresponding compensation result can be identified as a compensation failure. In another implementation, if the compensation process fails to reduce the occlusion level of the target area or requires re-enhancing the image, the corresponding compensation result can also be marked as a compensation failure. By counting the number of compensation failures, areas where the frequency of occlusion may not be high, but where existing compensation processes consistently fail to improve the visibility of the target area, can be identified.

[0140] After obtaining at least one of the following: occlusion frequency, the proportion of occlusion events of the same occlusion source type, and the number of compensation failures, corresponding preset frequency thresholds, preset proportion thresholds, and preset number of failures can be set respectively. These thresholds can be pre-configured based on historical inspection data statistics, target area type, or inspection task requirements. For example, a reference range for occlusion frequency can be determined based on the historical occlusion distribution of multiple target areas, and a preset frequency threshold can be set accordingly; a preset proportion threshold can be set based on the degree of recurrence of the same occlusion source; and the preset number of failures can be set based on the allowable degree of continuous compensation failures.

[0141] When the frequency of occlusion reaches a preset frequency threshold, the proportion of occlusion events reaches a preset proportion threshold, and / or the number of compensation failures reaches a preset number threshold, the target area can be identified as a repeatedly occluded area. Understandably, repeatedly occluded areas can be identified based on a single criterion or a combination of multiple criteria. For example, if a target area is frequently occluded in multiple inspection tasks, it can be identified as a repeatedly occluded area based on the frequency of occlusion occurrences; if a target area is primarily and continuously affected by the same type of occlusion source, it can be identified as a repeatedly occluded area based on the proportion of occlusion events; if the compensation processing for the target area fails multiple times, it can also be identified as a repeatedly occluded area based on the number of compensation failures.

[0142] After identifying the overlapping occlusion areas, the target occlusion source types in the occlusion event records corresponding to each overlapping occlusion area can be further summarized to determine the dominant occlusion source type. The dominant occlusion source type is used to characterize the occlusion source that dominates the overlapping occlusion area in multiple inspection tasks. Specifically, the number of occlusion events corresponding to each target occlusion source type can be counted separately, and the target occlusion source type with the most occlusion events can be determined as the dominant occlusion source type; alternatively, the proportion of occlusion events corresponding to each target occlusion source type can be compared, and the target occlusion source type with the highest proportion can be determined as the dominant occlusion source type.

[0143] In some alternative implementations, the primary occlusion source type can be determined by considering the temporal sequence of different inspection tasks. For example, a higher evaluation weight can be assigned to occlusion events corresponding to more recent inspection tasks, so that the primary occlusion source type can reflect the current occlusion status of the repeatedly occluded area. When the statistical results corresponding to multiple target occlusion source types are relatively close, further judgment can be made based on the differences in the number of corresponding occlusion events or their temporal distribution; if a clear dominant type still cannot be determined, the primary occlusion source type can be identified as unconfirmed occlusion.

[0144] Historical compensation information can be determined based on the compensation results recorded in the occlusion event logs corresponding to the overlapping occlusion areas. Specifically, multiple compensation results can be aggregated according to the order of inspection task execution to form a sequence of historical compensation results for overlapping occlusion areas; alternatively, the frequency or proportion of occurrence of different compensation results can be statistically analyzed to form historical compensation information. Historical compensation information can be used to indicate whether previous compensation processing successfully obtained usable inspection images, whether re-shooting is still necessary, or whether the compensation results across multiple inspection tasks remain stable.

[0145] For example, when the compensation results for the repeated occlusion area in multiple inspection tasks all indicate successful compensation, historical compensation data can show that the previous compensation process has relatively stable effectiveness; when multiple compensation results are consecutive failures, historical compensation data can show that the current compensation process has a low degree of improvement for the repeated occlusion area; when the compensation results differ in different inspection tasks, the corresponding compensation result sequence can be retained so that the target inspection strategy can be determined in combination with the main occlusion source type.

[0146] By comprehensively analyzing the frequency of occlusion occurrences, the proportion of occlusion events of the same occlusion source type, and the number of compensation failures within a preset statistical period, repeated occlusion areas can be identified from multiple dimensions such as the degree of occlusion repetition, the concentration of occlusion sources, and the effectiveness of compensation. This reduces the probability of occasional occlusion or single anomalies being misjudged as repeated occlusion. At the same time, by summarizing the target occlusion source type and compensation results corresponding to repeated occlusion areas, the main occlusion source type and historical compensation status can be determined more accurately, improving the accuracy of characterizing the historical occlusion characteristics and past compensation status of repeated occlusion areas.

[0147] In one example embodiment of this disclosure, it can be achieved through Figure 4 The intermediate steps implement the generation of target inspection strategies for areas with repeated occlusion based on the main occlusion source type and historical compensation data, referencing... Figure 4 As shown, it can specifically include: Step S410: When the main source of obstruction is vegetation obstruction, determine the target side shooting angle according to the obstruction ratio corresponding to different side shooting angles during historical inspections, and generate at least one of the following based on the target side shooting angle: backup waypoint, backup shooting direction, and gimbal pitch angle adjustment. Step S420: If the main source of obstruction is the fixed facility type of obstruction, determine the obstruction direction according to the projection direction of the fixed facility in the target area, and generate an alternative shooting direction and / or alternative waypoint on the opposite side of the obstruction direction. Step S430: When the main occlusion source type is the short-term dynamic occlusion, determine the concentrated occlusion time period according to the occlusion occurrence time in the occlusion event record corresponding to the repeated occlusion area, and generate at least one of the inspection time adjustment strategy, delayed shooting strategy and continuous acquisition strategy to avoid the concentrated occlusion time period. Step S440: If the main source of obstruction is the unconfirmed obstruction, generate a conservative inspection strategy that includes alternative waypoints and manual verification identifiers.

[0148] In cases where the primary source of obstruction is vegetation, the target's lateral shooting angle can be determined based on the obstruction ratio corresponding to different lateral shooting angles during historical inspections. The lateral shooting angle characterizes the observation angle after the shooting direction has shifted laterally relative to the original shooting direction of the target area. Different lateral shooting angles correspond to different line-of-sight directions, causing the projection overlap relationship between the vegetation and the target area to change accordingly.

[0149] In the specific implementation process, inspection images under different shooting directions can be obtained from historical inspection images corresponding to the repeatedly occluded areas, and the lateral shooting angle can be determined according to the shooting direction corresponding to each inspection image. For each lateral shooting angle, the occlusion ratio in the corresponding inspection image can be determined, and multiple occlusion ratios corresponding to the same lateral shooting angle or similar angle ranges can be summarized. For example, the average occlusion ratio, minimum occlusion ratio, or median occlusion ratio can be used to represent the historical occlusion situation of the corresponding lateral shooting angle, and then the occlusion ratios corresponding to different lateral shooting angles can be compared to determine the target lateral shooting angle.

[0150] The target lateral shooting angle can be the lateral shooting angle with the lowest occlusion rate during historical inspections, or it can be a lateral shooting angle with an occlusion rate lower than a preset value and corresponding to good historical compensation results. Since tree branches, shrubs, or vines may be concentrated in a specific observation direction relative to the target area, determining the target lateral shooting angle from historical lateral shooting angles with low occlusion rates allows for the use of existing inspection results to determine a more suitable lateral observation direction for observing repeatedly occluded areas. When historical data shows that a certain lateral angle corresponds to a low occlusion rate, that lateral angle can be preferentially used as a reference observation angle for subsequent inspections.

[0151] After determining the target's lateral shooting angle, at least one of the following can be generated based on the target area's location and the target's lateral shooting angle: a backup waypoint, a backup shooting direction, and a gimbal pitch angle adjustment. The backup waypoint can be set laterally to the original waypoint, allowing the UAV to observe the repeatedly obscured area from the position corresponding to the target's lateral shooting angle. The backup shooting direction can be determined based on the deflection relationship between the target's lateral shooting angle and the original shooting direction. The gimbal pitch angle adjustment can be determined based on the backup waypoint, the target area's spatial location, and the camera's optical axis direction, ensuring that the adjusted image acquisition range continues to cover the repeatedly obscured area. These inspection parameters can be adjusted individually or in combination. For example, after the UAV moves to the backup waypoint, the backup shooting direction and gimbal pitch angle can be adjusted simultaneously to change the imaging projection relationship between vegetation and the repeatedly obscured area.

[0152] When the primary source of obstruction is fixed infrastructure, the obstruction direction can be determined based on the projection direction of the fixed infrastructure in the target area. The projection direction characterizes the directional relationship of the fixed infrastructure relative to the target area when observing the target area from the viewpoint corresponding to historical inspection images. Since fixed pipelines, railings, supports, or fixed platform components typically maintain relatively stable positions on the construction site, the obstruction direction they create on the target area exhibits strong repeatability under similar inspection conditions.

[0153] In practice, the spatial location of fixed facilities in the engineering facility ledger can be obtained, and the projection position of the fixed facilities in the target area can be determined by combining the waypoint positions, shooting directions, and camera parameters corresponding to historical inspection images. Alternatively, the occlusion direction can be determined directly based on the relative position between the occluded area corresponding to the fixed facilities in historical inspection images and the target area. For example, when the projection area of ​​the fixed facilities is mainly located on the left side of the target area and extends into the target area, the corresponding direction can be determined as the occlusion direction.

[0154] After determining the obstruction direction, alternative shooting directions and / or alternative waypoints can be generated on the opposite side of the obstruction direction. The opposite side of the obstruction direction refers to the side that allows the UAV's line of sight to move away from the current projection direction of the fixed facility. It is not limited to forming a strictly opposite angle to the obstruction direction; as long as it can change the spatial projection relationship between the fixed facility, the UAV, and the target area, it is acceptable. For example, one or more candidate waypoints can be set around the target area on the opposite side of the obstruction direction, and alternative waypoints can be selected based on whether the candidate waypoints are located in the permitted flight area and whether they can completely capture the target area.

[0155] Alternate shooting directions can be formed by pointing from alternative waypoints to the target area, or by changing the UAV's heading or the orientation of the image acquisition device while keeping the original waypoints unchanged. When multiple alternative shooting directions or waypoints exist, priority can be given to selecting the parameters corresponding to those that yield usable inspection images from historical compensation scenarios, or selecting candidate parameters with low overlap with the projected area of ​​fixed facilities. By changing the observation position or direction from the opposite side of the fixed obstruction direction, the situation where fixed facilities are continuously positioned in front of the target area's imaging line of sight can be reduced.

[0156] When the primary source of occlusion is short-term dynamic occlusion, the concentrated occlusion period can be determined based on the occlusion occurrence time in the occlusion event records corresponding to the repeatedly occluded areas. The occlusion occurrence time can be the time the occluded area was detected, the time the corresponding inspection image was acquired, or the time the occlusion event record was generated. Multiple occlusion occurrence times can be statistically analyzed according to the time of day, the inspection operation period, or a preset time interval to determine whether short-term dynamic occlusion occurs concentratedly within a specific time range.

[0157] For example, a day can be divided into multiple consecutive time intervals, and the number of short-term dynamic occlusion events occurring in each time interval can be counted. When the number of occlusion events or the proportion of occlusion events in a certain time interval is relatively high, the corresponding time interval can be identified as a concentrated occlusion period. Alternatively, time clustering can be used to cluster multiple occlusion occurrence times, and the time range where the occlusion occurrence times are relatively concentrated can be identified as the concentrated occlusion period. The concentrated occlusion period can correspond to the concentrated passage time of people, the time of vehicle operation, or the concentrated operation time of mobile construction equipment.

[0158] After identifying the concentrated occlusion period, an inspection time adjustment strategy can be generated to avoid this period. This strategy alters the planned acquisition time for subsequent inspection tasks targeting areas with repeated occlusion. For example, acquisition tasks originally scheduled within the concentrated occlusion period can be moved to before or after this period. When adjusting the acquisition time, the waypoints and shooting parameters corresponding to the target area can be kept constant, reducing the probability of short-term dynamic objects occluding the target area again over time.

[0159] In some implementations, a delayed acquisition strategy can also be generated. This strategy postpones the current image acquisition operation when the UAV has already reached the waypoint corresponding to a repeatedly occluded area and short-term dynamic occlusion still exists. The delay time can be determined based on the duration of historical occlusion events or historical compensation data. For example, a reference duration can be formed based on the durations of multiple short-term dynamic occlusion events, and the delayed acquisition time can be determined accordingly, ensuring that the dynamically occluded object has a higher probability of leaving the imaging range corresponding to the target area when acquisition resumes.

[0160] The continuous acquisition strategy instructs the UAV to continuously acquire multiple frames of inspection images for areas with repeated occlusion. The number of consecutive acquisition frames can be determined based on the persistence of short-term dynamic occlusion during historical inspections or the compensation results corresponding to previous continuous acquisitions. For example, if historical compensation results indicate that increasing the number of consecutive acquisition frames can yield inspection images with a lower occlusion ratio, a corresponding or more suitable number of consecutive acquisition frames can be adopted in subsequent inspection tasks. By acquiring inspection images at multiple consecutive acquisition moments, the positional changes of personnel, vehicles, or mobile construction equipment over time can be utilized to increase the likelihood that at least one frame of the inspection image shows an area with repeated occlusion in a low-occlusion state.

[0161] The inspection time adjustment strategy, delayed shooting strategy, and continuous acquisition strategy can be used individually or in combination. For example, the planned inspection time for areas with repeated occlusion can be adjusted to outside the concentrated occlusion period. If dynamic occlusion objects are still detected during actual acquisition, then delayed shooting or continuous acquisition can be performed, thereby adjusting the image acquisition sequence according to the temporal variation characteristics of short-term dynamic occlusion.

[0162] When the primary source of occlusion is unconfirmed occlusion, a conservative inspection strategy can be generated that includes alternative waypoints and manual verification markers. Unconfirmed occlusion indicates that repeatedly occluded areas cannot be consistently categorized into vegetation occlusion, fixed facility occlusion, or short-term dynamic occlusion across multiple occlusion events. Therefore, it is not advisable to directly generate a directional or time-specific inspection strategy based on a single characteristic of a definite occlusion source.

[0163] Backup waypoints can be generated within the flyable area surrounding the original inspection waypoint to provide additional observation positions for subsequent inspections. Specifically, one or more candidate waypoints can be generated based on the spatial location of the target area, the location of the original waypoint, and the UAV's permitted flight range. Candidate waypoints located in no-fly zones, insufficient distance from obstacles, or beyond the UAV's operational range are then eliminated, and backup waypoints are determined from the remaining candidate waypoints. Since the specific source of the unconfirmed occlusion is unclear, backup waypoints can be used to increase the observation perspective of the target area without pre-assuming that the occluding object has specific spatial or motion characteristics.

[0164] Manual verification markers are used to indicate areas with repeated occlusion that require further review by inspection personnel. These markers can be associated with the target area markers of the repeated occlusion areas and written into subsequent inspection tasks, inspection image records, or inspection result data. The inspection management platform can use these manual verification markers to distinguish corresponding repeated occlusion areas from areas where automatically generated inspection strategies can be directly applied. This allows inspection personnel to view historical inspection images, occlusion event records, and compensation results for that area. After manual verification, inspection personnel can confirm whether to use alternative waypoints for data collection or further adjust inspection parameters.

[0165] By adjusting the observation direction based on the occlusion ratio corresponding to historical lateral shooting angles for vegetation occlusion, changing the shooting direction or waypoints based on the fixed occlusion direction for fixed facility occlusion, adjusting the acquisition sequence based on the time of occlusion occurrence for short-term dynamic occlusion, and using backup waypoints and manual verification for unconfirmed occlusion, subsequent inspection strategies can be adapted to the main occlusion patterns in areas with repeated occlusion. This reduces the situation where the same area is repeatedly captured using the original acquisition conditions, thereby increasing the acquisition ratio of effective inspection images, reducing the number of retakes, and improving the execution efficiency of periodic inspection tasks.

[0166] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0167] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described inspection strategy generation method based on occlusion recognition is also provided.

[0168] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be embodied in the following forms: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0169] The following reference Figure 5 To describe an electronic device 500 according to such an embodiment of the present disclosure. Figure 5 The electronic device 500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0170] like Figure 5 As shown, the electronic device 500 is presented in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0171] The storage unit stores program code that can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0172] Storage unit 520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 521 and / or cache memory (Cache) 522, and may further include read-only memory (ROM) 523.

[0173] Storage unit 520 may also include a program / utility 524 having a set (at least one) program module 525, such program module 525 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0174] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0175] Electronic device 500 can also communicate with one or more external devices 570 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 500, and / or with any device that enables electronic device 500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 550. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 560. As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0176] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0177] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0178] refer to Figure 6 As shown, a program product 600 for implementing the above-described occlusion-based inspection strategy generation method according to embodiments of the present disclosure is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0179] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0180] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0181] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0182] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0183] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0184] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0185] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This specification is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. It is understood that the specification and embodiments are to be considered exemplary only and are intended to illustrate the true scope and technical concepts of this disclosure.

[0186] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for generating inspection strategies based on occlusion recognition, characterized in that, include: Acquire the inspection image corresponding to the target area, and determine the occlusion area in the inspection image that overlaps with the target area; Extract the classification features corresponding to the occluded area. The classification features include at least two of the following: geometric morphology features, color and texture features, temporal motion features, and on-site facility matching features. Based on the classification features, the occlusion source confidence levels corresponding to multiple preset occlusion source types are determined respectively. The target occlusion source type corresponding to the occlusion area is determined based on the highest occlusion source confidence level and the confidence difference between the highest and second-highest occlusion source confidence levels. If the highest occlusion source confidence level and / or the confidence difference does not meet the preset classification conditions, the target occlusion source type is determined as unconfirmed occlusion. The multiple preset occlusion source types include vegetation occlusion, fixed facility occlusion, and short-term dynamic occlusion. Based on the target occlusion source type, determine the target compensation rule corresponding to the occlusion area, and generate an occlusion event record corresponding to the occlusion area. The occlusion event record includes at least the target area identifier of the target area, the target occlusion source type, and the compensation result corresponding to the target compensation rule. Based on the occlusion event records corresponding to multiple inspection tasks, the repeated occlusion areas are identified, and the main occlusion source types and historical compensation status corresponding to the repeated occlusion areas are determined. Based on the main occlusion source type and the historical compensation situation, a target inspection strategy corresponding to the repeated occlusion area is generated. The target inspection strategy includes at least one of the following: backup waypoint, backup shooting direction, gimbal pitch angle adjustment, shooting distance adjustment, delayed shooting time, and number of consecutive acquisition frames.

2. The method according to claim 1, characterized in that, The step of determining the occlusion region in the inspection image that overlaps with the target region includes: The target region in the inspection image is determined based on the target region identifier, and candidate occlusion regions covering the target region are identified. Determine at least two of the following: the overlap ratio between the candidate occlusion area and the target area, the continuous coverage width of the candidate occlusion area within the target area, whether the candidate occlusion area covers the key monitoring sub-area, and the length of the missing boundary of the target area caused by the candidate occlusion area; If at least two of the corresponding valid occlusion conditions are met, the candidate occlusion region is determined as the occlusion region.

3. The method according to claim 1, characterized in that, The step of extracting the classification features corresponding to the occluded region includes: Extract at least one of the boundary complexity, porosity, rectangularity, and straight edge ratio of the occluded region as the geometric morphological feature; Extract at least one of the green vegetation index, color mean, color variance, and texture direction consistency of the occluded area as the color texture feature; Based on the positional change of the occlusion region in consecutive frames, at least one of the following is determined as the temporal motion feature: centroid displacement, area change rate, number of consecutive occurrence frames, local optical flow direction dispersion, and occurrence duration. Based on the matching relationship between the obscured area and the projected location of fixed facilities in the engineering facility ledger, the matching characteristics of the on-site facilities are determined.

4. The method according to claim 3, characterized in that, The step of determining the occlusion source confidence level corresponding to multiple preset occlusion source types based on the classification features includes: For each preset occlusion source type, the feature scores corresponding to the geometric shape features, color texture features, temporal motion features, and on-site facility matching features are determined respectively; The occlusion source confidence level corresponding to the preset occlusion source type is determined based on the feature scores and weights corresponding to each feature score. In determining the occlusion source confidence corresponding to the vegetation occlusion, the weights corresponding to the boundary complexity in the color texture feature and the geometric morphology feature are greater than the weights corresponding to other classification features. When determining the occlusion source confidence level corresponding to the fixed facility occlusion, the weights corresponding to the proportion of straight edges in the on-site facility matching features and the geometric morphology features are greater than the weights corresponding to other classification features. When determining the occlusion source confidence corresponding to the short-term dynamic occlusion, the weight corresponding to the temporal motion feature is greater than the weight corresponding to other classification features.

5. The method according to claim 1, characterized in that, The step of determining the target occlusion source type corresponding to the occlusion area based on the confidence level of the highest occlusion source and the confidence level difference between the highest and second-highest occlusion source includes: If the confidence level of the highest occlusion source is not less than a preset classification confidence level threshold, and the confidence level difference is not less than a preset confidence level difference threshold, then the preset occlusion source type corresponding to the confidence level of the highest occlusion source is determined as the target occlusion source type. If the confidence level of the highest occlusion source is less than the preset classification confidence level threshold, and / or the confidence level difference is less than the preset confidence level difference threshold, the target occlusion source type is determined to be the unconfirmed occlusion.

6. The method according to claim 1, characterized in that, The step of determining the target compensation rule corresponding to the occlusion area based on the target occlusion source type includes: When the source of the target occlusion is vegetation occlusion, retrieve unobstructed candidate images from the historical inspection images corresponding to the target area, and map the unobstructed area in the unobstructed candidate images corresponding to the occluded area to the occluded area in the inspection image; and / or, determine an alternative shooting direction that is laterally deflected relative to the original shooting direction according to the original shooting direction corresponding to the inspection image, and adjust the shooting direction of the UAV and the gimbal pitch angle according to the alternative shooting direction; When the source of the target obstruction is a fixed facility type obstruction, the fixed obstruction direction is determined according to the projection direction of the fixed facility in the target area, and an alternate waypoint is generated at a position that avoids the fixed obstruction direction. The UAV is then controlled to re-acquire the inspection image corresponding to the target area based on the alternate waypoint. When the target occlusion source type is the short-term dynamic type occlusion, and when the safe hovering condition is met, the UAV is controlled to re-acquire the inspection image corresponding to the target area after a preset time delay at the current waypoint; and / or, the UAV is controlled to continuously acquire multiple frames of inspection images at the same waypoint or the same flight segment, and select the inspection image with the smallest occlusion ratio from the multiple frames of inspection images. If the source of the target occlusion is the unconfirmed occlusion, then based on the occlusion ratio corresponding to the occlusion area, retrieve unoccluded candidate images from the historical inspection images corresponding to the target area and generate compensation results, or generate backup waypoint reshoot instructions or manual review prompts.

7. The method according to claim 1, characterized in that, The determination of repeatedly occluded areas based on occlusion event records corresponding to multiple inspection tasks includes: Within a preset statistical period, based on multiple occlusion event records corresponding to the same target area, at least one of the following is determined: the occlusion occurrence frequency, the proportion of occlusion events of the same occlusion source type, and the number of compensation failures for the target area. If the frequency of occlusion reaches a preset frequency threshold, the proportion of occlusion events reaches a preset proportion threshold, and / or the number of compensation failures reaches a preset number threshold, the target area is determined as the repeated occlusion area. Based on the target occlusion source type and compensation result in each occlusion event record corresponding to the repeated occlusion area, the main occlusion source type and the historical compensation situation corresponding to the repeated occlusion area are determined.

8. The method according to claim 1, characterized in that, The step of generating a target inspection strategy corresponding to the repeated occlusion area based on the main occlusion source type and the historical compensation information includes: When the main source of obstruction is vegetation obstruction, the target lateral shooting angle is determined according to the obstruction ratio corresponding to different lateral shooting angles during historical inspections, and at least one of the following is generated based on the target lateral shooting angle: backup waypoint, backup shooting direction, and gimbal pitch angle adjustment. When the main source of obstruction is the fixed facility type of obstruction, the obstruction direction is determined according to the projection direction of the fixed facility in the target area, and an alternative shooting direction and / or alternative waypoint is generated on the opposite side of the obstruction direction. When the main source of occlusion is the short-term dynamic type of occlusion, the concentrated occlusion period is determined according to the occlusion occurrence time in the occlusion event record corresponding to the repeated occlusion area, and at least one of the inspection time adjustment strategy, delayed shooting strategy and continuous acquisition strategy is generated to avoid the concentrated occlusion period. When the primary source of obstruction is the unconfirmed obstruction, a conservative inspection strategy is generated that includes alternative waypoints and manual verification identifiers.

9. An electronic device, characterized in that, include: processor; as well as A memory storing computer-readable instructions, which, when executed by the processor, implement the inspection strategy generation method based on occlusion recognition as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the inspection strategy generation method based on occlusion recognition as described in any one of claims 1 to 8.

Images