An automatic retake method for a patrol image collection task

Abstract

This disclosure provides an automatic reshooting method for inspection image acquisition tasks, relating to the field of UAV inspection technology. The method includes: acquiring the target area corresponding to the inspection task, standard shooting constraints, acquired inspection images, and the original shooting state; determining the imaging deviation index of the target area in the acquired inspection images, and determining imaging anomaly attributes when the imaging deviation index does not meet the corresponding standard shooting constraints; determining the target shooting control degrees of freedom based on the imaging anomaly attributes, and determining the corresponding reshooting correction parameters based on the imaging deviation index; determining the target reshooting state by combining the original shooting state and the reshooting correction parameters, generating a reshooting task, and controlling the UAV to acquire reshooting images according to the reshooting task. This solution can reduce the probability of recurring similar anomalies in reshooting images, improve the first-pass yield of reshooting images, and enhance the completeness and usability of inspection image results.

Description

Technical Field

[0001] This disclosure relates to the field of unmanned aerial vehicle (UAV) inspection technology, and more specifically, to an automatic reshooting method for inspection image acquisition tasks. Background Technology

[0002] With the development of drone inspection technology, drones can collect images of targets such as power transmission lines, photovoltaic power stations, hydropower stations, and bridges according to pre-planned inspection tasks. During the inspection process, the location, scale, and visibility of key parts of the target area in the inspection image directly affect whether the inspection image can meet the requirements for subsequent defect identification or manual review. Therefore, ensuring that the target area has an imaging state that meets the inspection requirements has become an important issue in drone inspection image acquisition.

[0003] Currently, when relevant technologies detect issues such as target misalignment, abnormal target scale, boundary cropping, or missing key parts in inspection images, they typically re-execute the original shooting process or rely on manual experience to adjust the shooting position and angle. Because different imaging anomalies correspond to different framing deviations, the reshooting methods in these technologies cannot reasonably adjust the reshooting process to address the actual imaging anomalies. This results in reshoot images potentially exhibiting the same framing anomalies repeatedly, leading to a low reshoot pass rate.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide an automatic reshooting method for inspection image acquisition tasks, thereby reducing the probability of the same type of anomaly recurring in the reshooting images, improving the first-pass yield of the reshooting images, and enhancing the integrity and usability of the inspection image results.

[0006] According to a first aspect of the present disclosure, an automatic reshooting method for an inspection image acquisition task is provided, comprising: acquiring a target area corresponding to the inspection task, standard shooting constraints corresponding to the target area, and an acquired inspection image corresponding to the target area; and acquiring the original shooting state when acquiring the acquired inspection image, the original shooting state including at least one of shooting position, gimbal angle, and shooting distance; determining an imaging deviation index of the target area in the acquired inspection image, and when at least one imaging deviation index does not meet the corresponding standard shooting constraint, determining an imaging anomaly attribute corresponding to the target area based on the imaging deviation index that does not meet the corresponding standard shooting constraint, the imaging anomaly attribute being determined. The characteristics include at least one of the following: offset direction, offset degree, cropping position, and orientation of missing key parts; based on the imaging anomaly attributes, the target shooting control degree of freedom is determined from multiple shooting control degrees of freedom, and the reshoot correction parameters corresponding to the target shooting control degree of freedom are determined based on the imaging deviation index. Among them, the multiple shooting control degrees of freedom include at least two of the following: gimbal rotation degree of freedom, shooting position movement degree of freedom, shooting distance adjustment degree of freedom, and reshoot angle adjustment degree of freedom; the target reshoot state is determined based on the original shooting state and the reshoot correction parameters, and a reshoot task corresponding to the target area is generated based on the target reshoot state; the UAV is controlled to collect reshoot images of the target area according to the reshoot task.

[0007] In an example embodiment of this disclosure, based on the aforementioned scheme, the imaging deviation index includes the center offset of the target projection frame center of the target area relative to the image center of the acquired inspection image; according to the imaging anomaly attributes, the target shooting control degree of freedom is determined from multiple shooting control degrees of freedom, and the reshoot correction parameters corresponding to the target shooting control degree of freedom are determined according to the imaging deviation index, including: when determining the gimbal angle adjustment method based on the center offset, the target shooting control degree of freedom is determined to be the gimbal rotation degree of freedom; when the center offset includes the horizontal center offset, the gimbal horizontal angle correction amount is determined according to the proportion of the horizontal center offset to the image width of the acquired inspection image and the horizontal field of view of the camera that acquired the acquired inspection image, and the gimbal horizontal angle is corrected according to the return direction corresponding to the horizontal offset direction of the center offset; when the center offset includes the vertical center offset, the gimbal pitch angle correction amount is determined according to the proportion of the vertical center offset to the image height of the acquired inspection image and the vertical field of view of the camera, and the gimbal pitch angle is corrected according to the return direction corresponding to the vertical offset direction of the center offset.

[0008] In an example embodiment of this disclosure, based on the aforementioned scheme, the imaging deviation index includes the target area ratio of the target region in the acquired inspection image; according to the imaging anomaly attributes, the target shooting control degree of freedom is determined from multiple shooting control degrees of freedom, and the reshoot correction parameter corresponding to the target shooting control degree of freedom is determined according to the imaging deviation index, including: determining the area ratio deviation between the target area ratio and the preset area ratio range corresponding to the standard shooting constraint, and determining the target shooting control degree of freedom as the shooting distance adjustment degree of freedom; when the target area ratio is less than the lower limit of the preset area ratio range, a first shooting distance correction amount for reducing the shooting distance is determined according to the area ratio deviation; when the target area ratio is greater than the upper limit of the preset area ratio range, a second shooting distance correction amount for increasing the shooting distance is determined according to the area ratio deviation; and the shooting distance in the original shooting state is corrected according to the first shooting distance correction amount or the second shooting distance correction amount.

[0009] In an example embodiment of this disclosure, based on the aforementioned scheme, the imaging deviation index includes the intersection position between the target segmentation mask of the target region and the image boundary of the acquired inspection image; determining the imaging anomaly attribute corresponding to the target region based on the imaging deviation index that does not meet the corresponding standard shooting constraints includes: determining the cropping position of the target region based on the intersection position of the target segmentation mask and the image boundary, and determining the cropping position as the imaging anomaly attribute corresponding to the target region; determining the target shooting control degree of freedom from multiple shooting control degrees of freedom based on the imaging anomaly attribute, and determining the reshoot correction parameter corresponding to the target shooting control degree of freedom based on the imaging deviation index, including: determining the target compensation direction that makes the cropped part corresponding to the target region enter the range of the reshoot image based on the cropping position; determining at least one of the gimbal rotation degree of freedom and the shooting position movement degree of freedom as the target shooting control degree of freedom based on the target compensation direction, and determining the reshoot correction parameter that adjusts the gimbal angle or shooting position along the target compensation direction based on the cropping position.

[0010] In an example embodiment of this disclosure, based on the aforementioned scheme, the imaging deviation index includes the visible proportion of key parts corresponding to the target area; when the visible proportion of key parts is lower than the minimum visible proportion corresponding to the standard shooting constraint, the missing key parts that do not meet the visibility requirements are identified, and the missing orientation of the missing key parts relative to the target area is determined; the original shooting direction when the collected inspection images are acquired is obtained, and the degree of freedom for adjusting the supplementary shooting angle is determined as the degree of freedom for target shooting control based on the missing orientation, and a lateral supplementary shooting position is generated along the direction corresponding to the missing orientation, which is different from the shooting position in the original shooting state; a lateral supplementary shooting angle is added based on the original shooting direction; at least one supplementary shooting viewpoint is generated based on the lateral supplementary shooting position and the shooting direction after adding the lateral supplementary shooting angle, and a corresponding supplementary shooting task is generated based on the supplementary shooting viewpoint; when there is occlusion of the missing key parts, at least two supplementary shooting viewpoints located in different lateral orientations of the target area are generated, and supplementary shooting tasks are generated based on the at least two supplementary shooting viewpoints respectively.

[0011] In an example embodiment of this disclosure, based on the aforementioned scheme, before determining the imaging deviation index of the target area in the acquired inspection image, the method further includes: performing target detection on the acquired inspection image to obtain a target detection result characterizing whether the target area exists in the acquired inspection image; determining the theoretical coverage range corresponding to the acquired inspection image based on the shooting position, gimbal angle, shooting distance, and spatial position of the target area when the acquired inspection image was captured; determining the acquired inspection image as a valid candidate image of the target area when the target detection result characterizes the target area in the acquired inspection image and the spatial position of the target area is within the theoretical coverage range; determining the acquired inspection image as an image to be quality reviewed when the target detection result does not characterize the target area in the acquired inspection image and the spatial position of the target area is not within the theoretical coverage range; and excluding the correspondence between the acquired inspection image and the target area when the target detection result does not characterize the target area in the acquired inspection image and the spatial position of the target area is not within the theoretical coverage range.

[0012] In an example embodiment of this disclosure, based on the aforementioned scheme, the standard shooting constraints are determined according to the inspection object type corresponding to the target area; the method further includes: obtaining at least two constraints among the target center allowable range, target area proportion range, maximum boundary cropping ratio, minimum visible ratio of key parts, and target area sharpness threshold corresponding to the inspection object type, and determining the at least two constraints as the standard shooting constraints corresponding to the target area; for each constraint, based on the imaging state of the target area in the acquired inspection image, determining at least two imaging deviation indicators among the target area center offset, target area proportion, boundary cropping ratio, visible ratio of key parts, and target area sharpness index; determining the corresponding sub-item quality score according to each imaging deviation index, the sub-item quality score including at least two of the target centering score, target scale score, boundary integrity score, key part visibility score, and target area sharpness score; determining the evaluation weight corresponding to each sub-item quality score according to the inspection object type or the importance of the target area, and determining the coverage quality score corresponding to the target area based on each sub-item quality score and the corresponding evaluation weight; determining whether the acquired inspection image meets the standard shooting constraints based on whether the coverage quality score and at least one imaging deviation index do not meet the corresponding constraints.

[0013] In an example embodiment of this disclosure, based on the aforementioned scheme, after controlling the drone to collect supplementary images of the target area according to the supplementary shooting task, the method further includes: determining the imaging deviation index of the currently collected supplementary image and the previously collected image corresponding to the target area, and determining the coverage quality score corresponding to the currently collected supplementary image and the previously collected image according to the corresponding imaging deviation index; when the currently collected supplementary image meets the standard shooting constraints, determining the currently collected supplementary image as a valid inspection image corresponding to the target area; when the currently collected supplementary image does not meet the standard shooting constraints, determining the score improvement of the coverage quality score corresponding to the currently collected supplementary image relative to the coverage quality score corresponding to the previously collected image; in a continuous preset number of supplementary shots, when the score improvement of each supplementary shot is lower than the supplementary shooting improvement threshold, stopping the automatic supplementary shooting of the target area and outputting a manual review prompt; when the score improvement is not lower than the supplementary shooting improvement threshold, re-determining the supplementary shooting correction parameters according to the imaging deviation index corresponding to the currently collected supplementary image, and generating the next supplementary shooting task.

[0014] In an example embodiment of this disclosure, based on the aforementioned scheme, when there are multiple target areas corresponding to reshooting tasks, the method further includes: determining the reshooting priority value corresponding to each reshooting task based on the importance level of each target area, the degree of deviation of the corresponding imaging deviation index from the standard shooting constraints, and the corresponding coverage quality score; determining the path cost corresponding to each reshooting task based on the distance between the current position of the UAV and the reshooting position corresponding to each reshooting task, the transfer distance between different reshooting positions, and the return distance after executing the corresponding reshooting task; determining the candidate execution order of multiple reshooting tasks based on the reshooting priority value and path cost corresponding to each reshooting task; sequentially accumulating the estimated reshooting power required to execute the corresponding reshooting task according to the candidate execution order, and determining the estimated return power required to return to the return point from the last reshooting position after executing the accumulated reshooting task for each accumulation; determining the target reshooting task and the target execution order of the target reshooting task from the candidate execution order based on the remaining power of the UAV, the estimated reshooting power, the estimated return power, and the safety margin; and executing each target reshooting task according to the target execution order.

[0015] In an example embodiment of this disclosure, based on the foregoing scheme, the method further includes: associating and storing the imaging anomaly attributes, reshoot correction parameters, and score improvement obtained after executing the corresponding reshoot correction parameters for each reshoot, forming a historical reshoot record corresponding to the target area; when re-determining the reshoot correction parameters, determining the current imaging anomaly attribute based on the imaging deviation index corresponding to the reshoot image acquired this time, and determining at least one historical reshoot record that matches the current imaging anomaly attribute from the historical reshoot records, and obtaining the historical reshoot correction parameters and the corresponding score improvement from the at least one historical reshoot record; determining the target historical reshoot correction parameter that satisfies the improvement condition based on the score improvement corresponding to each historical reshoot correction parameter from the historical reshoot correction parameters; and determining the reshoot correction parameter corresponding to the current reshoot based on the target historical reshoot correction parameter.

[0016] According to a second aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions, wherein when executed by the processor, the computer-readable instructions implement the automatic re-capture method for the inspection image acquisition task in the first aspect.

[0017] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the automatic reshooting method for the inspection image acquisition task in the first aspect.

[0018] The technical solutions provided in this disclosure can include the following beneficial effects: The automatic reshooting method for inspection image acquisition tasks in the example embodiments of this disclosure, on the one hand, determines the imaging deviation in the acquired inspection images based on the standard shooting constraints corresponding to the target area, and further distinguishes imaging anomaly attributes such as offset direction, offset degree, cropping position, or missing key parts. This allows problems such as target offset, target scale anomaly, boundary cropping, and missing key parts in the inspection images to be identified according to the actual framing state of the target area, reducing the situation of missing local framing anomalies based solely on the overall image quality judgment, and improving the accuracy and specificity of inspection image anomaly judgment. On the other hand, by selecting matching shooting control degrees of freedom based on imaging anomaly attributes and determining corresponding reshooting correction parameters in conjunction with imaging deviation indicators, different imaging anomalies can be addressed by rotating the gimbal, shooting, and reshooting respectively. Targeted corrections are made by moving the camera position, adjusting the shooting distance, or adjusting the reshooting angle to avoid repeatedly using the original shooting state or applying the same adjustment method to different anomalies during the reshooting process. This reduces the probability of the same framing anomalies reappearing in the reshooted images and improves the matching degree between the reshooting adjustments and the actual imaging anomalies. On the other hand, by combining the original shooting state and the reshooting correction parameters to determine the target reshooting state and generate reshooting tasks, the reshooting position, gimbal angle, shooting distance, or reshooting angle can be adjusted based on the actual anomalies reflected in the already acquired inspection images. This reduces the reliance on human experience for reshooting parameters, reduces repeated reshooting caused by inaccurate reshooting position or shooting angle settings, improves the first-pass yield of reshooting images, and reduces the need to reschedule drones for supplementary flight missions, thereby improving the integrity and usability of the inspection image results.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 The illustration shows a flowchart of an automatic re-capture method for an inspection image acquisition task according to some embodiments of the present disclosure.

[0022] Figure 2 The illustration schematically shows a process diagram for determining the target shooting control degrees of freedom and reshoot correction parameters according to some embodiments of the present disclosure.

[0023] Figure 3The illustration shows a flowchart of generating a reshoot task based on missing key parts according to some embodiments of the present disclosure.

[0024] Figure 4 The illustration shows a flowchart of a process for determining whether acquired inspection images meet standard shooting constraints according to some embodiments of the present disclosure.

[0025] Figure 5 The schematic diagram illustrates the structural schematic of a computer system of an electronic device according to some embodiments of the present disclosure.

[0026] Figure 6 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is shown.

[0027] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0029] Furthermore, the accompanying drawings are for illustrative purposes only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] In this example embodiment, an automatic reshooting method for inspection image acquisition tasks is first provided. This automatic reshooting method for inspection image acquisition tasks can be executed by an airborne computing device carried by a UAV, a ground control terminal, an inspection management platform, or a server. This embodiment does not limit the specific subject executing the method. Figure 1 The illustration schematically shows a flowchart of an automatic re-capture method for an inspection image acquisition task according to some embodiments of the present disclosure. (Reference) Figure 1 As shown, the automatic re-capture method for this inspection image acquisition task may include the following steps: Step S110: Obtain the target area corresponding to the inspection task, the standard shooting constraints corresponding to the target area, and the collected inspection images corresponding to the target area; and obtain the original shooting state when collecting the collected inspection images. The original shooting state includes at least one of shooting position, gimbal angle, and shooting distance. Step S120: Determine the imaging deviation index of the target area in the acquired inspection image, and when at least one of the imaging deviation indices does not meet the corresponding standard shooting constraints, determine the imaging abnormality attribute corresponding to the target area based on the imaging deviation index that does not meet the corresponding standard shooting constraints. The imaging abnormality attribute includes at least one of offset direction, offset degree, cropping position and missing key part orientation. Step S130: Based on the imaging anomaly attribute, determine the target shooting control degree of freedom from multiple shooting control degrees of freedom, and determine the reshoot correction parameter corresponding to the target shooting control degree of freedom based on the imaging deviation index. The multiple shooting control degrees of freedom include at least two of the following: gimbal rotation degree of freedom, shooting position movement degree of freedom, shooting distance adjustment degree of freedom, and reshoot angle adjustment degree of freedom. Step S140: Determine the target reshoot state based on the original shooting state and the reshoot correction parameters, and generate a reshoot task corresponding to the target area based on the target reshoot state; Step S150: Control the drone to collect supplementary images of the target area according to the supplementary shooting task.

[0031] According to the automatic reshooting method for the inspection image acquisition task in this example embodiment, on the one hand, the imaging deviation in the acquired inspection image is determined based on the standard shooting constraints corresponding to the target area, and the imaging anomaly attributes such as offset direction, offset degree, cropping position, or missing key parts are further distinguished. This allows problems such as target offset, target scale anomaly, boundary cropping, and missing key parts in the inspection image to be identified according to the actual framing state of the target area, reducing the situation of missing local framing anomalies based solely on the overall image quality judgment, and improving the accuracy and pertinence of inspection image anomaly judgment. On the other hand, by selecting the matching shooting control degrees of freedom based on the imaging anomaly attributes and determining the corresponding reshooting correction parameters in combination with the imaging deviation index, different imaging anomalies can be addressed by rotating the gimbal and shooting respectively. Targeted corrections are made by moving the position, adjusting the shooting distance, or adjusting the reshooting angle to avoid repeating the original shooting state or using the same adjustment method for different anomalies during the reshooting process. This reduces the probability of the same framing anomaly reappearing in the reshooted images and improves the matching degree between the reshooting adjustment and the actual imaging anomaly. On the other hand, by combining the original shooting state and the reshooting correction parameters to determine the target reshooting state and generate the reshooting task, the reshooting position, gimbal angle, shooting distance, or reshooting angle can be adjusted based on the actual anomalies reflected in the acquired inspection images. This reduces the reliance on human experience for reshooting parameters, reduces repeated reshooting caused by inaccurate reshooting position or shooting angle settings, improves the first-pass yield of reshooting images, and reduces the need to reschedule drones for supplementary flight missions, thereby improving the integrity and usability of the inspection image results.

[0032] The automatic reshooting method for the inspection image acquisition task in this example embodiment will be further explained below.

[0033] In step S110, the target area corresponding to the inspection task, the standard shooting constraints corresponding to the target area, and the collected inspection images corresponding to the target area are obtained, and the original shooting state when the collected inspection images are collected is obtained. The original shooting state includes at least one of the shooting position, gimbal angle, and shooting distance.

[0034] In one example embodiment of this disclosure, an inspection task refers to task data used to control a drone to perform image acquisition operations on a predetermined inspection target. The inspection task can be generated based on a pre-planned inspection route, inspection range, and inspection target, and may include multiple image acquisition task units arranged in a preset task order. Each image acquisition task unit may correspond to one or more target areas. During the execution of the inspection task, the drone can sequentially reach the corresponding acquisition positions and use its onboard imaging device to acquire inspection images of the corresponding target areas.

[0035] For example, the inspection task can be applied to inspection scenarios such as power transmission lines, photovoltaic power stations, hydropower stations, bridges, pipe corridors, building facades, or wind power generation equipment. Taking power transmission line inspection as an example, the inspection task can be used to control a drone to fly along the power transmission line and collect images of the towers and their auxiliary structures; taking photovoltaic power station inspection as an example, the inspection task can be used to control a drone to fly along the distribution area of ​​the photovoltaic array and collect images of the corresponding equipment area. This embodiment does not limit the specific inspection scenario corresponding to the inspection task.

[0036] The target area refers to the object region in an inspection task where images that meet the shooting requirements need to be obtained. The target area can correspond to the entire inspection object or a local structure within it. For example, for a large inspection object, it can be divided into multiple target areas based on its structural composition, with each target area corresponding to a specific image acquisition requirement. The target area can be described by target area identifiers, target component identifiers, spatial coordinates, spatial extent, or the relationship between the target object and its local parts.

[0037] In the specific implementation process, the correspondence between inspection points and target areas can be pre-established when generating inspection tasks. For example, inspection point identifiers and target area identifiers corresponding to each inspection point can be recorded in the task file. When the UAV reaches the corresponding inspection point, the target area to be collected is determined according to the association in the task file. Alternatively, the target area corresponding to the currently collected inspection image can be determined based on the spatial location of the target area and the actual flight position of the UAV. This embodiment does not limit the specific association method between the target area and the inspection task.

[0038] Standard imaging constraints refer to the constraint data used to define the imaging requirements that a target area should meet in inspection images. Since the imaging content of different target areas may differ in subsequent defect identification, condition analysis, or manual review processes, corresponding standard imaging constraints can be configured for different target areas. Standard imaging constraints can be used to describe the expected imaging state of the target area in inspection images and serve as a benchmark for judging whether the acquired inspection images meet the imaging requirements.

[0039] Standard shooting constraints can be represented by numerical thresholds, allowable intervals, spatial ranges, state conditions, or a set of constraints formed by combining multiple conditions. For example, a certain imaging state of the target area can be limited to a preset range, or the target area can be limited to not having preset framing anomalies. Multiple constraints can also be configured simultaneously, and the satisfaction of one or more constraints can be used to determine whether the acquired inspection images meet the shooting requirements. Standard shooting constraints can be pre-stored in the inspection task file, the inspection management platform, or the drone's local storage module, and associated with the corresponding target area through a target area identifier.

[0040] Optionally, standard shooting constraints can be pre-set by the task configuration personnel during the inspection task creation stage, or they can be pre-generated based on existing inspection specifications, historical inspection tasks, or shooting templates corresponding to the target area. Furthermore, standard shooting constraints can be distributed to the UAV as part of the inspection task configuration information along with the inspection task, or they can be obtained from an external data storage device based on the target area identifier when image analysis is required. This embodiment does not limit the generation method or storage location of standard shooting constraints.

[0041] Acquired inspection images refer to image data collected and generated by the UAV targeting a specific area. These images can be still images captured by the onboard imaging device, or video frames or keyframes extracted from inspection videos. A single target area can correspond to one or multiple acquired inspection images. When multiple acquired inspection images correspond to the same target area, each image can be processed separately, or at least one image can be selected for processing based on the task execution order or image acquisition time.

[0042] In one implementation, the drone can acquire collected inspection images in real time during the inspection mission and send the acquired images to the onboard computing module for processing after each image acquisition is completed. In another implementation, the collected inspection images can be acquired uniformly by the ground control terminal, inspection management platform, or server after the drone has completed part or all of the inspection mission.

[0043] Furthermore, the acquired inspection images can be associated and stored with corresponding task information. For example, the target area identifier, inspection point identifier, image acquisition time, or task execution sequence number can be written into the image record corresponding to the acquired inspection image. The aforementioned association information can be stored in image metadata, task logs, image index tables, or independent inspection result files, so as to determine the target area corresponding to the acquired inspection image based on the corresponding association information.

[0044] The original shooting state refers to the shooting state adopted by the UAV when acquiring the already acquired inspection images, reflecting the actual acquisition conditions that formed the currently acquired inspection images. In this embodiment, the original shooting state may include at least one of the following: shooting position, gimbal angle, and shooting distance.

[0045] The shooting location can be used to characterize the spatial position of the UAV or airborne imaging device when acquiring the collected inspection images. The shooting location can be represented by geographic coordinates, inspection scene coordinates, or relative position to the target area. For example, the spatial coordinates of the UAV can be obtained through the Global Navigation Satellite System (GNSS), or the shooting location can be obtained through Real-Time Kinematic (RTK), visual positioning, or inertial navigation.

[0046] The gimbal angle can be used to characterize the shooting orientation of the onboard imaging device when acquiring collected inspection images. The gimbal angle can be directly output by the gimbal control module, or it can be determined in combination with the UAV attitude and the installation relationship of the onboard imaging device. The shooting distance can be used to characterize the distance between the onboard imaging device and the target area when acquiring collected inspection images. It can be calculated based on the shooting position and the spatial position of the target area, or it can be determined using distance sensors or scene spatial data.

[0047] In practical applications, the original shooting state can be recorded simultaneously when the airborne imaging device triggers the image acquisition action, and then stored in association with the acquired inspection images. For example, when acquiring an inspection image, the drone's position and gimbal attitude at the corresponding time point can be recorded simultaneously, and the shooting distance can be determined based on the spatial location of the target area. In another implementation, flight logs, gimbal control logs, and image acquisition time can be recorded separately, and the original shooting state can be retrieved from the corresponding logs based on the image acquisition time during subsequent processing.

[0048] In step S120, the imaging deviation index of the target area in the acquired inspection image is determined, and when at least one of the imaging deviation indices does not meet the corresponding standard shooting constraints, the imaging abnormality attribute corresponding to the target area is determined according to the imaging deviation index that does not meet the corresponding standard shooting constraints. The imaging abnormality attribute includes at least one of offset direction, offset degree, cropping position and missing key part orientation.

[0049] In one example embodiment of this disclosure, the imaging deviation index refers to data that characterizes the difference between the actual imaging state of the target area in the acquired inspection image and the corresponding shooting requirements. The imaging deviation index takes the target area as the analysis object, rather than the entire image of the acquired inspection image as the sole analysis object. Therefore, even if there are no obvious quality abnormalities in the overall acquired inspection image, it is possible to determine whether there are any situations that do not meet the standard shooting constraints based on the imaging state of the target area itself.

[0050] Specifically, the image region corresponding to the target area in the acquired inspection images can be determined first, and an imaging deviation index can be generated based on the geometric features, spatial location, visibility status, or image quality status of the image region. The image region corresponding to the target area can be determined based on the image analysis results, or it can be determined based on pre-configured target area information or the correspondence between the image and the target area. This embodiment does not limit the specific method for determining the image region corresponding to the target area.

[0051] Imaging deviation indices can be represented by numerical values, vectors, positional data, or state data. For example, for imaging deviations with spatial directional characteristics, the imaging deviation index can be formed by combining the direction and the deviation amount; for imaging deviations with positional characteristics, they can be represented by abnormal locations or abnormal regions; and for imaging deviations with degree characteristics, they can be represented by the difference or proportion between the actual value and the corresponding boundary of the standard shooting constraint.

[0052] In one implementation, pre-defined image processing rules can be used to process the acquired inspection images, and an imaging deviation index can be calculated based on the processing results. For example, the position, range, or visibility status of the target area in the image coordinate system can be obtained, and then the corresponding imaging deviation index can be generated based on the relationship between the above data and standard shooting constraints. In another implementation, a pre-trained image analysis model can be used to process the acquired inspection images. The image analysis model outputs feature data to characterize the imaging status of the target area, and then the imaging deviation index is determined based on the corresponding feature data. Alternatively, a combination of model recognition and rule calculation can be used to obtain the imaging deviation index. For example, the image analysis model first determines the image content corresponding to the target area, and then a pre-defined calculation rule is used to form an imaging deviation index for comparison with standard shooting constraints.

[0053] After obtaining the imaging deviation index, the actual imaging state of the target area can be determined by comparing the imaging deviation index with the corresponding standard shooting constraints. A pre-established correspondence between the imaging deviation index and the standard shooting constraints can be configured, meaning that corresponding standard shooting constraints can be configured for different imaging deviation indices. When the imaging deviation index is in numerical form, threshold comparison or interval judgment can be used to determine whether the corresponding standard shooting constraints are met; when the imaging deviation index is in positional or state form, positional relationship judgment or state matching can be used to determine whether the corresponding standard shooting constraints are met.

[0054] When all imaging deviation indicators meet the corresponding standard shooting constraints, it can be assumed that the actual imaging state of the current target area has not triggered the corresponding reshoot correction process. When at least one imaging deviation indicator does not meet the corresponding standard shooting constraints, the imaging anomaly attribute is further determined based on the imaging deviation indicator that does not meet the corresponding standard shooting constraints.

[0055] Imaging anomaly attributes refer to data that further describes the abnormal features reflected by imaging deviations. These attributes reflect the spatial direction, degree, or location of the anomaly. In this embodiment, imaging anomaly attributes include at least one of offset direction, offset degree, cropping position, and the location of missing key parts. Through imaging anomaly attributes, based on the determination that an anomaly exists in the target area, the specific direction or location characteristics of the anomaly can be further described.

[0056] The offset direction is used to characterize the direction in which the actual imaging state of the target area deviates from the expected imaging state. The offset direction can be represented by the image coordinate direction corresponding to the acquired inspection image, or by the spatial direction corresponding to the inspection scene. For example, the offset direction can be represented by a horizontal direction, a vertical direction, or a direction vector formed relative to a preset reference direction. This embodiment does not limit the coordinate expression form of the offset direction.

[0057] The degree of offset is used to characterize the deviation of the actual imaging state of the target area from the standard shooting constraints. For example, the degree of offset can be determined based on the difference between the imaging deviation index and the constraint boundary of the standard shooting constraints, or it can be determined based on the ratio between the above difference and a preset reference value. In some implementations, multiple offset levels can be divided according to the deviation magnitude, and the corresponding offset level can be used to represent the degree of offset.

[0058] The cropping position is used to characterize the location where an anomaly occurs when part of the target area is not fully captured in the acquired inspection image. The cropping position can represent a single local location within the target area or multiple local locations. When multiple parts of the target area are not fully imaged, multiple cropping positions can be recorded separately to reflect the actual incompleteness of the target area's frame.

[0059] The orientation of missing critical areas is used to characterize the orientation of critical areas that do not meet the shooting requirements relative to the target area. Critical areas can be pre-designated target local areas that require focused acquisition during the inspection task. The orientation of missing critical areas can be represented by the structural orientation of the target area itself, or by its orientation relative to the current shooting direction. This "missing" can include critical areas not being included in the acquired inspection image, or critical areas being located within the acquired inspection image but whose actual imaging state does not meet the corresponding shooting requirements.

[0060] A target area can correspond to one or more imaging anomaly attributes. When multiple imaging deviation indicators simultaneously fail to meet the corresponding standard shooting constraints, the corresponding imaging anomaly attributes can be determined separately based on each imaging deviation indicator, and these multiple imaging anomaly attributes can be associated with the target area. In one implementation, all imaging anomaly attributes can be retained simultaneously; in another implementation, the anomaly processing order can be determined based on the degree of deviation of each imaging deviation indicator, with priority given to processing imaging anomaly attributes with larger deviations. This embodiment does not limit the processing order of multiple imaging anomaly attributes.

[0061] In step S130, based on the imaging anomaly attribute, a target shooting control degree of freedom is determined from multiple shooting control degrees of freedom, and a reshoot correction parameter corresponding to the target shooting control degree of freedom is determined based on the imaging deviation index. The multiple shooting control degrees of freedom include at least two of the following: gimbal rotation degree of freedom, shooting position movement degree of freedom, shooting distance adjustment degree of freedom, and reshoot angle adjustment degree of freedom.

[0062] In one example embodiment of this disclosure, the shooting control degree of freedom refers to the control dimension that can be adjusted by a drone or airborne imaging device during the reshooting process, thereby changing the actual imaging state of the target area. Different shooting control degrees of freedom correspond to different shooting state adjustment methods. By adjusting the corresponding shooting control degrees of freedom, the relative positional relationship, shooting orientation, or observation relationship between the airborne imaging device and the target area can be changed.

[0063] The gimbal rotational degree of freedom refers to the control dimension by rotating the gimbal mounted on the UAV to change the shooting orientation of the onboard imaging device. The gimbal can be used to mount cameras or other image acquisition devices. When the UAV's spatial position remains unchanged or changes only slightly, gimbal rotation can alter the optical axis direction of the imaging device, causing a corresponding change in the image's position of the target area. The gimbal rotational degree of freedom can correspond to one or more gimbal rotation directions; this embodiment does not limit the gimbal structure or the supported rotation directions.

[0064] The degree of freedom for shooting position movement refers to the control dimension by changing the spatial position of the drone to alter the relative positional relationship between the onboard imaging device and the target area. Shooting position movement can include horizontal movement, vertical movement, or a combination of these spatial directions. The drone can perform corresponding flight maneuvers based on the adjusted position and movement amount, re-capturing images of the target area from the new shooting position.

[0065] The shooting distance adjustment degree of freedom refers to the control dimension used to change the shooting distance between the airborne imaging device and the target area. The shooting distance can be adjusted by controlling the drone to move closer to or further away from the target area. At different shooting distances, the imaging range or scale of the target area in the image can change; therefore, the shooting distance adjustment degree of freedom can be used to adjust for imaging anomalies related to the shooting distance.

[0066] The degree of freedom for adjusting the reshooting angle refers to the control dimension of changing the viewing angle of the airborne imaging device observing the target area. The reshooting angle can be formed by both the shooting position and the shooting direction. In some inspection scenarios, even if the target area has been included in the acquired inspection image, some content in the target area may still not form a satisfactory imaging result due to the influence of the target structure or the current observation direction. In this case, the observation direction for reshooting can be changed to reshoot the target area from different angles.

[0067] The target shooting control degree of freedom refers to the shooting control degree of freedom selected from multiple shooting control degrees of freedom based on the imaging anomaly attributes corresponding to the current target area, and used for reshooting adjustments. There can be one or more target shooting control degrees of freedom. For example, when the imaging state corresponding to the current anomaly can be changed by only one shooting state adjustment method, one target shooting control degree of freedom can be determined; when multiple shooting states need to be adjusted in combination, multiple target shooting control degrees of freedom can be determined simultaneously.

[0068] In the specific implementation process, a mapping relationship between imaging anomaly attributes and shooting control degrees of freedom can be established in advance. For example, a mapping table, configuration file, or control rules can be used to record the optional shooting control degrees of freedom corresponding to different imaging anomaly attributes. After obtaining the imaging anomaly attributes corresponding to the current target area, the above mapping relationship can be queried, and the target shooting control degree of freedom can be determined from multiple shooting control degrees of freedom.

[0069] Optionally, when an imaging anomaly attribute corresponds to multiple selectable shooting control degrees of freedom, the target shooting control degree of freedom can be selected from the multiple selectable shooting control degrees of freedom based on the current operating status of the UAV. For example, the shooting control degree of freedom that can be executed can be determined by combining the current spatial position of the UAV, its movable range, the adjustable range of the gimbal, or the surrounding environment of the target area. This embodiment does not limit the specific selection rules for the target shooting control degree of freedom.

[0070] After determining the target shooting control degrees of freedom, the corresponding reshoot correction parameters are determined based on the imaging deviation index. The reshoot correction parameters are control parameters used to describe the adjustments made to the target shooting control degrees of freedom; they can represent the adjustment direction, adjustment amount, or the adjusted target parameters. Since the imaging deviation index reflects the deviation of the current imaging state from the standard shooting constraints, it can be used to further determine the specific degree of adjustment required for the target shooting control degrees of freedom.

[0071] For example, when the target shooting control degree of freedom corresponds to the adjustment of a certain shooting state parameter, the corresponding adjustment amount can be determined based on the degree of deviation reflected by the imaging deviation index. When the imaging deviation index also has directional information, the adjustment direction corresponding to the reshoot correction parameter can also be determined based on the corresponding directional information. In one implementation, the imaging deviation index can be converted according to a pre-established conversion relationship to obtain the reshoot correction parameter. In another implementation, multiple imaging deviation intervals can be pre-set, and corresponding reshoot correction parameters can be configured for each imaging deviation interval. After determining the current imaging deviation index, the corresponding parameter can be obtained by querying.

[0072] The reshoot correction parameters can also be represented using incremental parameters or target parameters. When using incremental parameters, the reshoot correction parameters represent the amount of adjustment that needs to be increased or decreased relative to the original shooting state. When using target parameters, the reshoot correction parameters can directly represent the parameter values ​​that the corresponding shooting state should achieve after the reshoot adjustment. Different parameter expression forms can be used for different target shooting control degrees of freedom.

[0073] By first determining the shooting control degrees of freedom that need to be adjusted based on the imaging anomaly attributes, and then combining the imaging deviation index to determine the corresponding reshoot correction parameters, a continuous data relationship can be established between the anomaly type, the adjustment object, and the degree of adjustment. This allows different imaging anomalies to be applied to their corresponding shooting states, reducing the need for indiscriminate adjustments to all shooting states or repeated use of the original shooting states for reshoots, and lowering the probability of the same type of framing anomaly reappearing in the reshot images.

[0074] In step S140, the target reshoot state is determined based on the original shooting state and the reshoot correction parameters, and a reshoot task corresponding to the target area is generated based on the target reshoot state.

[0075] In one example embodiment of this disclosure, the target re-capture state refers to the shooting state that the UAV needs to reach when it re-captures re-capture images of the target area. Unlike the original shooting state, which corresponds to the acquired inspection images, the target re-capture state is a shooting state determined based on the original shooting state and combined with re-capture correction parameters. This allows the target re-capture state to retain the shooting relationships established by the original inspection task and make corresponding adjustments for current imaging anomalies.

[0076] In the actual implementation process, the original shooting state parameters that need to be adjusted can be determined based on the target shooting control degrees of freedom, and then the reshoot correction parameters can be applied to the corresponding parameters. For example, the shooting position in the original shooting state can be used as a reference to adjust the shooting position accordingly; the original gimbal angle can be used as a reference to adjust the gimbal state accordingly; and the adjusted shooting distance can be determined based on the original shooting distance. For shooting state parameters that are not determined to need adjustment, the corresponding parameters in the original shooting state can continue to be used.

[0077] In one implementation, an incremental correction method can be used to determine the target reshoot state. For example, if the reshoot correction parameter represents the change in state relative to the original shooting state, then the reshoot correction parameter can be combined with the corresponding parameters in the original shooting state to obtain the target reshoot state. In another implementation, the reshoot correction parameter can directly represent the corrected shooting target value. Therefore, the target reshoot state can be determined based on the reshoot correction parameter, while retaining other state parameters from the original shooting state that do not need to be changed.

[0078] Furthermore, a re-capture task corresponding to the target area is generated based on the target re-capture status. A re-capture task refers to the task data used to control the UAV to re-capture the target area. It can be an additional task to the original inspection task or a task unit inserted during the execution of the inspection task. The re-capture task may include a target area identifier and control information to bring the UAV to the target re-capture status.

[0079] In one implementation, reshoot waypoints can be generated based on the target reshoot status, and the shooting parameters corresponding to the target reshoot status can be associated with the reshoot waypoints. After the UAV reaches the reshoot waypoint, it can perform image acquisition according to the corresponding shooting status. In another implementation, flight control commands, gimbal control commands, and image acquisition commands corresponding to the target reshoot status can be combined to form a reshoot task, and the reshoot task can be inserted into the UAV's current task sequence.

[0080] Reshooting tasks can be generated by the UAV's onboard computing module, or by the ground control terminal or inspection management platform. For example, the UAV can analyze the acquired inspection images locally and directly generate reshooting tasks to shorten the time interval between the discovery of image anomalies and the execution of reshooting tasks; alternatively, the acquired inspection images and the original shooting status can be sent to external processing equipment, which will then generate reshooting tasks and distribute them to the UAV.

[0081] Furthermore, the reshooting task can be linked to the target area, the already acquired inspection images, and the corresponding original shooting status. For example, the target area identifier and the already acquired inspection image identifier can be recorded in the reshooting task, and the relationship between the reshooting task and the original inspection task can be recorded in the task management data.

[0082] By using the original shooting state as a benchmark and forming the target shooting state based on the reshooting correction parameters, it is possible to avoid completely deviating from the original inspection task and replanning the shooting process. This preserves the basic relationship between the established target area and the shooting position, while also enabling adjustments to the shooting state related to the current imaging anomaly. This reduces parameter deviations caused by relying on manual experience to reset the reshooting parameters, and improves the matching degree between the reshooting task and the actual imaging anomaly.

[0083] In step S150, the drone is controlled to acquire supplementary images of the target area according to the supplementary shooting task.

[0084] In one example embodiment of this disclosure, the reshooting task can be sent to the flight control module, mission control module, or onboard computing module of the UAV. The corresponding module can parse the reshooting task and control the UAV and onboard imaging device to perform corresponding state adjustments and image acquisition actions according to the target reshooting state.

[0085] Specifically, when the target reshoot status involves changes in the UAV's spatial state, flight control commands can be generated based on the relationship between the UAV's current position and the corresponding position in the target reshoot status, and the UAV can be controlled to move to the corresponding position; when the target reshoot status involves changes in the shooting orientation of the airborne imaging device, the corresponding shooting mechanism can be controlled to adjust to the target reshoot status; when the target reshoot status involves changes in the shooting distance, the actual shooting distance can be adjusted to meet the corresponding requirements by adjusting the relative position between the UAV and the target area.

[0086] In one implementation, once the drone reaches the target reshoot state, the onboard imaging device can be directly triggered to perform image acquisition. In another implementation, before performing image acquisition, it can be determined whether the deviation between the drone's current actual shooting state and the target reshoot state is within a preset allowable error range. When the deviation is within the preset allowable error range, the image acquisition action is triggered; when the deviation is not within the preset allowable error range, the drone or the corresponding shooting mechanism can continue to be adjusted until the image acquisition conditions are met.

[0087] Recaptured images refer to images acquired by a drone again for a target area according to a recapture mission. Recaptured images can be still images or continuous image data. After acquiring recaptured images, they can be associated with the corresponding recapture mission and target area, and the actual shooting status at the time of acquisition can be recorded. For example, the target area identifier, recapture mission identifier, and recaptured image can be stored together so that the recaptured images can be incorporated into the inspection results data corresponding to the original inspection mission.

[0088] Optionally, the reshooting task can be inserted during the execution of the original inspection task, or it can be executed after the preset image acquisition process in the original inspection task is completed. For example, when the reshooting task is generated, the drone is still near the target area, and can be directly adjusted to the target reshooting state and reshoot images can be acquired according to the reshooting task; when the drone has left the target area, the reshooting task can be added to the subsequent task sequence and executed when the drone returns to the corresponding work area.

[0089] By re-acquiring the target area according to the re-shooting task based on the actual imaging anomaly, the UAV can complete the re-shooting in a shooting state that is different from simply repeating the original shooting process. This allows the actual framing state of the target area to be adjusted according to the current imaging anomaly, reducing the probability that the re-shot images will have the same or similar framing anomalies as the already acquired inspection images. It also reduces the need for repeated re-shooting of the same target area and the rescheduling of supplementary flight tasks, thereby improving the first-pass yield of the re-shot images and the completeness and usability of the inspection image results.

[0090] The contents of steps S110 to S150 will be described in detail below.

[0091] In one example embodiment of this disclosure, the imaging deviation index may include the center offset of the target projection frame center of the target area relative to the image center of the acquired inspection image; the target projection frame can be used to represent the imaging range of the target area in the acquired inspection image, and it can be determined by the target detection result or the target area contour information. The center of the target projection frame can be determined according to the geometric center of the target projection frame, and the image center can be determined according to the image width and image height of the acquired inspection image. The positional difference between the two is used to characterize the offset of the target area relative to the image center.

[0092] The center offset can include at least one of a horizontal center offset and a vertical center offset. The horizontal center offset characterizes the degree of offset of the target projection frame center relative to the image center in the image width direction, while the vertical center offset characterizes the degree of offset of the target projection frame center relative to the image center in the image height direction. The center offset can be represented by the number of pixels, normalized distance, or a proportion of the corresponding image size, and can be combined with the offset direction to form an imaging deviation index for describing the image distortion state of the target area.

[0093] It can be done Figure 2 The steps described in the document involve determining the target shooting control degree of freedom from multiple shooting control degrees of freedom based on imaging anomaly attributes, and determining the corresponding reshoot correction parameters for the target shooting control degree of freedom based on imaging deviation indices, with reference to... Figure 2 As shown, it can specifically include: Step S210: When determining the gimbal angle adjustment method based on the center offset, the target shooting control degree of freedom is determined to be the gimbal rotation degree of freedom; Step S220: When the center offset includes a horizontal center offset, determine the gimbal horizontal angle correction amount based on the ratio of the horizontal center offset to the image width of the acquired inspection image and the horizontal field of view of the camera that acquired the inspection image, and correct the gimbal horizontal angle according to the return direction corresponding to the horizontal offset direction of the center offset. Step S230: When the center offset includes a center offset in the vertical direction, determine the gimbal pitch angle correction amount based on the ratio of the center offset in the vertical direction to the image height of the acquired inspection image and the vertical field of view of the camera, and correct the gimbal pitch angle according to the return direction corresponding to the vertical offset direction of the center offset.

[0094] Among them, the gimbal angle adjustment method refers to changing the camera's shooting orientation to align the target area's image position in the re-enhanced image with the image center, without changing or minimally changing the drone's current spatial position. In one implementation, the gimbal angle adjustment method can be used when the imaging anomaly corresponding to the center offset can be corrected within the gimbal's adjustable angle range; alternatively, a correspondence can be directly established between the center offset anomaly and the gimbal's rotational degrees of freedom based on preset control rules.

[0095] When the center offset includes a horizontal center offset, the pan-tilt-zoom (PTZ) horizontal angle correction can be determined based on the proportion of the horizontal center offset to the width of the acquired inspection image and the horizontal field of view of the camera that acquired the inspection image. Specifically, the relative proportion of the horizontal center offset in the image width direction can be used to characterize the degree of offset of the target area relative to the horizontal field of view of the camera, and combined with the horizontal field of view of the camera, the positional offset in the image can be converted into the PTZ horizontal angle adjustment.

[0096] For example, the gimbal horizontal angle correction can be determined based on the ratio between the horizontal center offset and the image width, and the correspondence between this ratio and the horizontal field of view. In the specific implementation, the above correspondence can be achieved using a preset conversion relationship, a calibration relationship, or a lookup table relationship. This embodiment does not limit the specific calculation form of the gimbal horizontal angle correction, as long as the corresponding angle adjustment can be determined based on the horizontal center offset and the horizontal field of view.

[0097] After determining the pan-tilt-zoom (PTZ) horizontal angle correction amount, the PTZ horizontal angle can be corrected according to the return-to-center direction corresponding to the horizontal offset direction of the center offset. The return-to-center direction refers to the PTZ rotation direction that moves the image position of the target area in the re-shot image towards the image center. For example, when the center of the target projection frame shifts to one side relative to the image center, the PTZ can be controlled to rotate along the corresponding return-to-center direction by the PTZ horizontal angle correction amount to reduce the horizontal offset between the target projection frame center and the image center.

[0098] When the center offset includes the vertical center offset, the gimbal pitch correction can be determined based on the proportion of the vertical center offset to the image height of the acquired inspection image and the camera's vertical field of view. Similar to the horizontal processing method, the relative proportion of the vertical center offset in the image height direction can be used to characterize the degree of offset of the target area relative to the camera's vertical field of view, and this, combined with the camera's vertical field of view, can be used to determine the gimbal pitch correction.

[0099] The gimbal pitch angle correction can be determined based on a preset conversion relationship, or by obtaining the correspondence between the vertical position change of the image and the gimbal pitch angle change through camera calibration, and then determining the corresponding correction amount based on this correspondence. After determining the gimbal pitch angle correction amount, the gimbal pitch angle is corrected according to the straightening direction corresponding to the vertical offset direction of the center offset, so that the imaging position of the target area in the re-shot image moves towards the image center.

[0100] When the center offset includes both horizontal and vertical center offset, the gimbal horizontal angle correction and gimbal pitch angle correction can be determined separately, and then combined and corrected. This allows for adjusting the camera's shooting orientation based on the target area's offset in the two-dimensional image, reducing the likelihood of the target area deviating from the expected framing position in reshot images.

[0101] By determining the pan-tilt horizontal angle correction and / or pan-tilt pitch angle correction based on the center offset of the target projection frame relative to the image center, and correcting the pan-tilt angle according to the corresponding return direction, the positional deviation of the target area in the image can be converted into the corresponding pan-tilt angle adjustment. This reduces the failure of reshoots caused by the target area deviating from the expected framing position, improves the centering of the target area in the reshoot image, and enhances the accuracy of the reshoot framing.

[0102] In one example embodiment of this disclosure, the imaging deviation index may include the target area ratio of the target region in the acquired inspection image; the target area ratio is used to characterize the imaging scale occupied by the target region in the acquired inspection image, and can be determined based on the ratio between the image area corresponding to the target region and the effective image area of ​​the acquired inspection image. The image area corresponding to the target region can be determined based on the target projection frame, the target region outline, or other data that can characterize the imaging range of the target region.

[0103] The following steps can be used to determine the target shooting control degree of freedom from multiple shooting control degrees of freedom based on imaging anomaly attributes, and to determine the corresponding reshoot correction parameters for the target shooting control degree of freedom based on imaging deviation indices. Specifically, this may include: The deviation of the target area ratio from the preset area ratio range corresponding to the standard shooting constraint can be determined, and the target shooting control degree of freedom can be determined as the shooting distance adjustment degree of freedom. When the target area ratio is less than the lower limit of the preset area ratio range, a first shooting distance correction amount for reducing the shooting distance is determined based on the area ratio deviation. When the target area ratio is greater than the upper limit of the preset area ratio range, a second shooting distance correction amount for increasing the shooting distance is determined based on the area ratio deviation. The shooting distance in the original shooting state is corrected based on the first shooting distance correction amount or the second shooting distance correction amount.

[0104] The preset area proportion range is used to limit the appropriate imaging scale of the target area in the inspection image, and can be determined according to the standard shooting constraints corresponding to the target area. When the target area proportion is within the preset area proportion range, it can be considered that the imaging scale of the target area in the acquired inspection image meets the corresponding shooting requirements; when the target area proportion is less than the lower limit of the preset area proportion range or greater than the upper limit of the preset area proportion range, it can be considered that the target area has a scale deviation.

[0105] Furthermore, the area ratio deviation between the target area ratio and the preset area ratio range can be determined. The area ratio deviation characterizes the degree to which the current imaging scale of the target region deviates from the appropriate imaging scale. In one implementation, when the target area ratio is less than the lower limit of the preset area ratio range, the area ratio deviation can be determined based on the difference between the lower limit of the preset area ratio range and the target area ratio; when the target area ratio is greater than the upper limit of the preset area ratio range, the area ratio deviation can be determined based on the difference between the target area ratio and the upper limit of the preset area ratio range. The area ratio deviation can also be expressed as a normalized difference or a relative deviation ratio; this embodiment does not limit this.

[0106] When a scale deviation is determined in the target area based on the target area ratio, the target shooting control degree of freedom can be defined as the shooting distance adjustment degree of freedom. The shooting distance adjustment degree of freedom is used to adjust the imaging scale of the target area in the re-shot image by changing the shooting distance between the airborne imaging device and the target area. Specifically, the shooting distance can be changed by controlling the drone to move closer to or away from the target area.

[0107] When the target area percentage is less than the lower limit of the preset area percentage range, it indicates that the target area is too small in the acquired inspection images. In this case, a first shooting distance correction amount can be determined based on the area percentage deviation to reduce the shooting distance. The first shooting distance correction amount can be determined based on the preset correspondence between the area percentage deviation and the shooting distance adjustment amount, or it can be obtained by querying the deviation range where the area percentage deviation falls. Generally, the larger the area percentage deviation, the larger the first shooting distance correction amount can be set to allow the drone to get closer to the target area.

[0108] When the target area percentage exceeds the upper limit of the preset area percentage range, it indicates that the target area is too large in the acquired inspection images. In this case, a second shooting distance correction amount can be determined based on the area percentage deviation to increase the shooting distance. The second shooting distance correction amount can be obtained using the same or similar determination method as the first shooting distance correction amount and is used to control the drone to move away from the target area.

[0109] After determining the first or second shooting distance correction amount, the shooting distance can be corrected based on the shooting distance in the original shooting state. For example, when using the first shooting distance correction amount, the corresponding distance can be reduced from the original shooting distance; when using the second shooting distance correction amount, the corresponding distance can be increased from the original shooting distance, thereby obtaining the shooting distance corresponding to the target reshoot state.

[0110] In one optional implementation, a mapping relationship between area ratio deviation and shooting distance correction can be pre-established, and a corresponding first or second shooting distance correction can be determined based on the current area ratio deviation. Alternatively, different mapping relationships can be configured according to the imaging scale requirements of different target areas, adapting the shooting distance adjustment to the actual imaging needs of the target area. By adjusting the shooting distance based on the deviation of the target area ratio from a preset area ratio range, the situation where the target area is too small or too large in the re-shot image, thus failing to meet the shooting requirements again, can be reduced.

[0111] By determining the shooting distance correction amount based on the deviation between the target area ratio and the preset area ratio range, and reducing the shooting distance when the target area ratio is too small and increasing the shooting distance when the target area ratio is too large, the distance between the drone and the target area can be adjusted for abnormal imaging scale of the target area. This reduces the situation where the target area is too small, resulting in insufficient detail, or too large, resulting in insufficient field of view, and improves the imaging scale adaptability of the target area in the reshot image.

[0112] In one example embodiment of this disclosure, the imaging deviation index may include the intersection position between the target segmentation mask of the target region and the image boundary of the acquired inspection image; the target segmentation mask is used to characterize the actual imaging range of the target region in the acquired inspection image, and can be obtained through image segmentation processing, and represented in the form of pixel region, binary mask, or region contour. The image boundary may include the left boundary, right boundary, upper boundary, and lower boundary of the acquired inspection image. The intersection position between the target segmentation mask and the image boundary can be used to reflect whether the target region has incomplete imaging at the corresponding boundary position.

[0113] For example, it can detect whether the target segmentation mask extends to the image boundary and determine the boundary position where the target segmentation mask intersects with the image boundary. For instance, when the target segmentation mask intersects with the left boundary of the image, it can be considered that the left side of the target region may be cropped; when the target segmentation mask intersects with the right boundary of the image, it can be considered that the right side of the target region may be cropped; when the target segmentation mask intersects with the top or bottom boundary of the image, it can be determined that the target region may be cropped on the top or bottom side. When the target segmentation mask intersects with multiple image boundaries simultaneously, multiple corresponding intersection positions can be determined.

[0114] The following steps can be used to determine the imaging anomaly attribute of a target region based on an imaging deviation index that does not meet the corresponding standard shooting constraints. Specifically, this can include: determining the clipping position of the target region based on the intersection position of the target segmentation mask and the image boundary, and defining the clipping position as the imaging anomaly attribute corresponding to the target region; and further, the following steps can be used to determine the target shooting control degree of freedom from multiple shooting control degrees of freedom based on the imaging anomaly attribute, and determine the reshoot correction parameters corresponding to the target shooting control degree of freedom based on the imaging deviation index. Specifically, this can include: The target compensation direction can be determined based on the cropping position so that the cropped part corresponding to the target area can be included in the range of the reshot image; at least one of the gimbal rotation degree of freedom and the shooting position movement degree of freedom can be determined as the target shooting control degree of freedom based on the target compensation direction; and the reshot correction parameters for adjusting the gimbal angle or shooting position along the target compensation direction can be determined based on the cropping position.

[0115] The cropping position can be used to characterize the location where the target area does not fully enter the acquired inspection image, and can correspond to the left, right, top, or bottom of the target area. In one implementation, the cropping position can be determined directly based on the image boundary where they intersect; alternatively, the cropping position can be further confirmed by combining the contour distribution of the target segmentation mask near the image boundary.

[0116] After determining the cropping position, a target compensation direction can be determined based on the cropping position to bring the cropped portion of the target area into the range of the re-shot image. The target compensation direction refers to the adjustment direction by which the currently incomplete portion of the target area is moved into the effective imaging range of the re-shot image by adjusting the shooting state. The target compensation direction can be determined based on the positional relationship between the cropping position and the effective image area. For example, a mapping relationship between different cropping positions and their corresponding target compensation directions can be pre-established, and the corresponding target compensation direction can be obtained by querying based on the current cropping position.

[0117] Furthermore, at least one of the gimbal rotation degree of freedom and the shooting position movement degree of freedom can be determined as the target shooting control degree of freedom based on the target compensation direction. The gimbal rotation degree of freedom is suitable for adjusting the framing position of the target area in the reshot image by changing the camera's shooting orientation, and the shooting position movement degree of freedom is suitable for adjusting the relative position between the camera and the target area by changing the spatial position of the drone. In one implementation, it can be determined whether to use the gimbal rotation degree of freedom based on the current adjustable range of the gimbal; when the adjustable range of the gimbal is insufficient to complete the corresponding compensation, the shooting position movement degree of freedom can be used, or both the gimbal rotation degree of freedom and the shooting position movement degree of freedom can be used simultaneously.

[0118] The reshooting correction parameters can be determined based on the cropping location to adjust the gimbal angle or shooting position along the target compensation direction. These parameters can represent the gimbal's rotation direction and angle adjustment amount, or the drone's shooting position movement direction and displacement. In practice, the reshooting correction parameters can be determined based on the degree of anomaly corresponding to the cropping location. For example, the adjustment amount can be determined based on the intersection range and length of the target segmentation mask with the image boundary, or the estimated degree to which the target region exceeds the effective imaging range. Alternatively, the corresponding parameters can be obtained by querying the pre-defined correspondence between cropping locations and reshooting correction parameters.

[0119] When multiple cropping locations exist within a target area, the target compensation direction for each cropping location can be determined separately, and the target shooting control degrees of freedom and reshoot correction parameters can be comprehensively determined. For example, multiple target compensation directions can be combined to adjust the direction, while simultaneously adjusting the gimbal angle and shooting position, so that multiple cropped portions can re-enter the reshoot image range. By determining the target compensation direction based on the cropping location and further adjusting the corresponding shooting state, the occurrence of the same boundary cropping problem in the reshoot image can be reduced.

[0120] By determining the cropping position of the target region based on the intersection between the target segmentation mask and the image boundary, and further determining the target compensation direction that allows the cropped part to enter the range of the re-shot image, the gimbal angle and / or shooting position can be adjusted for the specific cropping direction, reducing the occurrence of cropping on the same side boundary in the re-shot image and improving the imaging integrity of the target region in the re-shot image.

[0121] In one example embodiment of this disclosure, the imaging deviation index may include the visibility ratio of key components corresponding to the target area; key components refer to local structures in the target area that need to be highlighted to form effective inspection images, and can be predetermined based on the structural characteristics of the inspection object, inspection requirements, or defect identification needs. For example, key components may be connecting parts, ends, fixing parts, or other local areas that need to be observed in detail. The visibility ratio of key components is used to characterize the proportion of key components that meet the visibility requirements in the acquired inspection images to all key components corresponding to the target area, and can be determined based on the ratio between the number of identified effective key components and the total number of pre-configured key components.

[0122] It can be done Figure 3 The steps in the document are used to generate the reshoot task. (Refer to...) Figure 3 As shown, it can specifically include: Step S310: When the visible ratio of the key part is lower than the minimum visible ratio corresponding to the standard shooting constraint, determine the missing key part that does not meet the visibility requirement, and determine the missing orientation of the missing key part relative to the target area. Step S320: Obtain the original shooting direction when the collected inspection image is acquired, determine the degree of freedom of the reshoot angle adjustment as the target shooting control degree of freedom based on the missing direction, and generate a lateral reshoot position that is different from the shooting position in the original shooting state along the direction corresponding to the missing direction. Step S330: Add a side-facing reshoot angle based on the original shooting direction; Step S340: Generate at least one reshoot viewpoint based on the lateral reshoot position and the shooting direction after increasing the lateral reshoot angle, and generate a corresponding reshoot task based on the reshoot viewpoint; Step S350: When the missing key part is occluded, at least two reshoot viewpoints located in different lateral directions of the target area are generated, and reshoot tasks are generated based on the at least two reshoot viewpoints respectively.

[0123] Specifically, when the visibility ratio of critical areas is lower than the minimum visibility ratio corresponding to the standard shooting constraints, it can be considered that some critical areas in the target area do not meet the shooting requirements. In this case, the missing critical areas that do not meet the visibility requirements are identified. Missing critical areas can be critical areas that are not included in the acquired inspection images, or critical areas that are occluded in the acquired inspection images and cannot form a satisfactory imaging result. Specifically, the key area identification results in the acquired inspection images can be compared with the pre-configured key area information of the target area to determine the missing critical areas.

[0124] Furthermore, the missing critical component's orientation relative to the target area can be determined. The missing orientation characterizes the relative position of the missing critical component within the target area and can be determined based on the preset structural position corresponding to the missing critical component or local coordinate relationships within the target area. For example, the missing orientation can be represented as one side, end, or other local orientation of the target area. This embodiment does not limit the specific representation of the missing orientation, as long as it can characterize the positional relationship of the missing critical component relative to the target area.

[0125] It can acquire the original shooting direction when acquiring the already acquired inspection images, and determine the degree of freedom for adjusting the reshoot angle based on the missing location as the degree of freedom for target shooting control. The original shooting direction is used to characterize the camera's observation direction relative to the target area when acquiring the already acquired inspection images, and can be determined based on the camera's optical axis direction, gimbal attitude, and the relative position between the UAV and the target area. Since the missing parts of key components may be related to the current observation direction, the missing key components can be brought into the effective imaging range of the reshoot image under the new observation direction by changing the reshoot angle.

[0126] Specifically, a lateral reshoot position, distinct from the original shooting position, can be generated along the direction corresponding to the missing location. The lateral reshoot position refers to a reshoot position that is laterally offset relative to the original shooting position, and it can be determined based on the missing location and the spatial position of the target area. For example, using the original shooting position as a reference, a new shooting position can be set along the lateral direction corresponding to the location of the missing critical part, allowing the camera to observe the target area from a spatial position different from the original shooting position.

[0127] Furthermore, a lateral supplementary shooting angle is added to the original shooting direction. The lateral supplementary shooting angle is used to change the camera's observation direction relative to the target area, and can represent the amount of angular change of the supplementary shooting direction relative to the original shooting direction. In one implementation, the lateral supplementary shooting angle can be set according to a preset supplementary shooting angle; alternatively, a corresponding angle can be selected from multiple preset lateral supplementary shooting angles based on the location of the missing key part. This embodiment does not limit the specific method for determining the lateral supplementary shooting angle.

[0128] At least one supplementary shooting viewpoint can be generated based on the lateral supplementary shooting position and the shooting direction after increasing the lateral supplementary shooting angle. The supplementary shooting viewpoint describes the spatial shooting position and shooting direction corresponding to the UAV performing supplementary shooting, and can serve as the basic data for generating supplementary shooting tasks. Specifically, the lateral supplementary shooting position can be used as the position parameter of the supplementary shooting viewpoint, and the shooting direction after increasing the lateral supplementary shooting angle can be used as the direction parameter of the supplementary shooting viewpoint, thereby forming a corresponding supplementary shooting viewpoint. Then, a supplementary shooting task for acquiring supplementary images of the target area can be generated based on the supplementary shooting viewpoint.

[0129] When critical missing parts are obstructed, at least two re-shooting viewpoints located at different lateral orientations within the target area can be generated. Obstruction can be determined based on the critical part identification results, target area structural information, or occlusion relationships in previously acquired inspection images. Different re-shooting viewpoints correspond to different lateral re-shooting positions or different shooting directions, enabling the UAV to re-shoot the target area from multiple observation directions. Generating re-shooting tasks based on at least two re-shooting viewpoints increases the likelihood of a valid image of the missing critical part in at least one re-shooting image, reducing the need for repeated re-shooting due to obstruction from a single re-shooting perspective.

[0130] By identifying the missing key parts and their missing locations when the visibility of key parts is insufficient, and generating lateral reshoot positions and angles along the corresponding directions, the spatial position and shooting direction of the camera observing the target area can be changed, allowing key parts that were not effectively presented under the original shooting direction to obtain a new imaging perspective. When there is obstruction, by setting multiple reshoot viewpoints with different lateral orientations, the probability of a single reshoot viewpoint continuing to be obstructed by the same obstruction can be reduced, thereby improving the effective visibility of key parts and the coverage integrity of key structures by inspection images.

[0131] In one example embodiment of this disclosure, before determining the imaging deviation index of the target area in the acquired inspection image, the correspondence between the acquired inspection image and the target area can be determined first to avoid calculating the imaging deviation index for an incorrect target area. Specifically, this can include: Target detection can be performed on the acquired inspection images to obtain target detection results indicating whether a target area exists in the acquired inspection images. Based on the shooting position, gimbal angle, shooting distance when the inspection images were acquired, and the spatial location of the target area, the theoretical coverage area corresponding to the acquired inspection images is determined. When the target detection results indicate that the target area exists in the acquired inspection images and the spatial location of the target area is within the theoretical coverage area, the acquired inspection images are determined as valid candidate images for the target area. When the target detection results do not indicate that the target area exists in the acquired inspection images and the spatial location of the target area is within the theoretical coverage area, the acquired inspection images are determined as images to be reviewed for quality. When the target detection results do not indicate that the target area exists in the acquired inspection images and the spatial location of the target area is not within the theoretical coverage area, the correspondence between the acquired inspection images and the target area is excluded.

[0132] The object detection can be achieved using a pre-trained object detection model, or by using recognition methods based on image feature matching, template matching, or object segmentation results. The object detection result can include at least one of the following: object category, object location, and object confidence score, and is used to determine whether a target object corresponding to the target region has been identified in the acquired inspection image.

[0133] In one implementation, the target category to be detected in the acquired inspection images can be determined based on the target area identifier or inspection object type recorded in the inspection task, and the target category in the target detection results can be matched. When a target object corresponding to the target area exists in the target detection results, it can be considered that the target detection results indicate that the target area exists in the acquired inspection images; when no corresponding target object is detected, it can be considered that the target detection results do not indicate that the target area exists in the acquired inspection images.

[0134] Furthermore, the theoretical coverage area corresponding to the acquired inspection images can be determined based on the shooting position, gimbal angle, shooting distance, and spatial location of the target area when the acquired inspection images are captured. The theoretical coverage area is used to characterize the spatial area that the camera can theoretically image under the actual shooting conditions corresponding to the acquired inspection images.

[0135] Specifically, the spatial position of the camera in the inspection scene can be determined based on the shooting position, the shooting orientation of the camera can be determined based on the gimbal angle, and the target imaging area corresponding to the camera can be determined based on the shooting distance. In one implementation, the theoretical coverage range can also be determined by combining the camera's field of view parameters. For example, the field of view cone range can be determined based on the camera's optical axis direction and the camera's field of view angle, and the theoretical imaging area at the corresponding distance position can be determined based on the shooting distance. This embodiment does not limit the specific calculation method of the theoretical coverage range.

[0136] The spatial location of the target area can be obtained from the target area coordinates in the inspection task, equipment log data, or the spatial model of the inspection scenario. After determining the theoretical coverage area, it can be determined whether the spatial location of the target area is within the theoretical coverage area. The spatial location of the target area can be determined by the center location of the target area, or by whether there is overlap between the spatial range of the target area and the theoretical coverage area.

[0137] When the target detection results indicate that the target area exists in the acquired inspection images and the spatial location of the target area is within the theoretical coverage range, the acquired inspection images are identified as valid candidate images for the target area. At this time, the spatial coverage relationship between the image content detection results and the actual shooting state both indicate that the acquired inspection images correspond to the target area. Therefore, the imaging deviation index of the target area can be further determined based on the acquired inspection images.

[0138] When the target detection results do not indicate that the target region exists in the acquired inspection image, but the spatial location of the target region is within the theoretical coverage area, the acquired inspection image can be identified as an image to be quality reviewed. This situation indicates that, based on the spatial coverage relationship corresponding to the actual shooting state, the target region should theoretically be within the imaging range of the acquired inspection image, but the target detection process failed to identify the target region. Reasons for this situation may include a small imaging scale of the target region, local occlusion, abnormal image quality, or unstable target detection results. Therefore, the correspondence between the acquired inspection image and the target region can be temporarily not ruled out, and it can be further processed as an image to be quality reviewed.

[0139] When the target detection results do not indicate that the target area exists in the acquired inspection image, and the spatial location of the target area is not within the theoretical coverage area, the correspondence between the acquired inspection image and the target area can be excluded. In this case, neither the target detection results nor the theoretical spatial coverage relationship indicate that the acquired inspection image corresponds to the target area. Therefore, it is possible to avoid calculating the imaging deviation index of the target area based on the acquired inspection image.

[0140] By combining the target detection results with the theoretical coverage range determined based on the shooting position, gimbal angle, shooting distance, and target area spatial location, the correspondence between the acquired inspection images and the target area can be jointly judged. This can reduce the missed detections and misjudgments caused by relying solely on target detection, and also avoid using images of actually uncovered target areas for imaging deviation analysis. This improves the accuracy of matching the target area with the inspection images and reduces the probability of triggering invalid reshoots due to incorrect image object matching.

[0141] In one example embodiment of this disclosure, the standard shooting constraints can be determined based on the inspection object type corresponding to the target area; the inspection object type is used to characterize the category of the inspection object to which the target area belongs. Different inspection objects may have different imaging content and imaging requirements during the image acquisition process. Therefore, corresponding standard shooting constraints can be set for different inspection object types.

[0142] It can be done Figure 4 The steps to determine whether the acquired inspection images meet the standard shooting constraints are as follows: Figure 4 As shown, it can specifically include: Step S410: Obtain at least two of the following constraints corresponding to the inspection object type: the allowable range of the target center, the range of the target area proportion, the maximum boundary cropping ratio, the minimum visible ratio of the key parts, and the target area clarity threshold; and determine the at least two constraints as the standard shooting constraints corresponding to the target area. Step S420: For each of the constraints, based on the imaging state of the target area in the acquired inspection image, determine at least two imaging deviation indicators from the center offset, target area ratio, boundary cropping ratio, key part visibility ratio, and target area clarity index corresponding to the target area. Step S430: Determine the corresponding sub-item quality score according to each of the imaging deviation indices. The sub-item quality score includes at least two of the following: target centering score, target scale score, boundary integrity score, key part visibility score, and target area sharpness score. Step S440: Determine the evaluation weight corresponding to each of the sub-item quality scores according to the type of the inspection object or the importance of the target area, and determine the coverage quality score corresponding to the target area based on each of the sub-item quality scores and the corresponding evaluation weights; Step S450: Based on the coverage quality score and whether at least one of the imaging deviation indicators does not meet the corresponding constraints, determine whether the acquired inspection image meets the standard shooting constraints.

[0143] The target center allowable range can be used to define the appropriate imaging position of the target area in the acquired inspection image, and can be represented by a preset area based on the image center or an allowable offset range. The target area proportion range is used to define the appropriate imaging scale of the target area in the acquired inspection image. The maximum boundary cropping ratio is used to define the maximum extent to which the target area is allowed to be cropped by the image boundary. The minimum visibility ratio of key parts is used to define the minimum visibility level that key parts in the target area that need to be effectively presented should achieve. The target area sharpness threshold is used to define the minimum sharpness requirement of the corresponding image area of ​​the target area.

[0144] After determining the standard shooting constraints corresponding to the target area, at least two imaging deviation indicators can be determined based on the imaging status of the target area in the acquired inspection images, including center offset, target area ratio, boundary cropping ratio, visible ratio of key parts, and target area clarity index.

[0145] The center offset can be determined based on the positional difference between the imaging center of the target area and the image center of the acquired inspection image; the target area ratio can be determined based on the ratio between the area of ​​the corresponding image area of ​​the target area and the effective image area of ​​the acquired inspection image; the boundary cropping ratio can be determined based on the ratio of the portion of the target area truncated by the image boundary to the overall imaging range of the target area; the visibility ratio of key parts can be determined based on the ratio of the key parts that meet the visibility requirements to all pre-configured key parts.

[0146] The target region sharpness index is used to characterize the sharpness of the image region corresponding to the target region. The target region sharpness index can be determined based on edge variations, grayscale gradients, or high-frequency information of the image region corresponding to the target region. In one implementation, the Laplacian operator can be used to process the image region corresponding to the target region, and the target region sharpness index can be determined based on the variance of the processing result; other image quality evaluation methods that can reflect the sharpness of the image can also be used, and this embodiment does not limit this approach.

[0147] Furthermore, corresponding sub-quality scores can be determined based on each imaging deviation index. The sub-quality scores can include at least two of the following: target centering score, target scale score, boundary integrity score, key part visibility score, and target area sharpness score. Each sub-quality score is used to convert imaging deviation indices of different dimensions or different representations into score data that can be comprehensively evaluated.

[0148] For example, the target centering score can be determined based on the relationship between the center offset and the allowable range of the target center; the greater the center offset, the lower the target centering score can be. The target scale score can be determined based on the deviation between the target area ratio and the target area ratio range. The boundary integrity score can be determined based on the size of the boundary cropping ratio relative to the maximum boundary cropping ratio. The key part visibility score can be determined based on the key part visibility ratio. The target area clarity score can also be determined based on the relationship between the target area clarity index and the target area clarity threshold.

[0149] In the specific implementation process, the scoring rules between each imaging deviation index and the corresponding sub-item quality score can be preset. For example, a segmented scoring method can be used to determine the corresponding score based on the numerical range of the imaging deviation index; or a continuous mapping method can be used to convert the imaging deviation index into a score value within a preset scoring range. This embodiment does not limit the specific calculation form of each sub-item quality score.

[0150] The evaluation weights for each sub-item quality score can be determined based on the type of inspection object or the importance of the target area. Evaluation weights characterize the relative importance of different imaging quality dimensions in the current target area quality evaluation process. For example, for inspection objects where the overall position and integrity of the target are of greater concern, the evaluation weights corresponding to the target centering score or boundary integrity score can be increased; for target areas requiring observation of local structures, the evaluation weights corresponding to the visibility score of key parts can be increased.

[0151] The importance of a target area can be pre-configured in the inspection task, or determined based on the equipment belonging to the target area, the inspection content corresponding to the target area, or the inspection task level. In one implementation, a correspondence between the inspection object type and each evaluation weight can be pre-established; in another implementation, the preset evaluation weights can be adjusted according to the importance of the target area to make the coverage quality evaluation more in line with the actual inspection needs of the current target area.

[0152] Furthermore, the coverage quality score for the target area is determined based on the quality scores of each sub-item and their corresponding evaluation weights. The coverage quality score is used to comprehensively characterize the overall imaging quality of the target area in the acquired inspection images. In one implementation, the quality scores of each sub-item can be weighted according to their corresponding evaluation weights to obtain the coverage quality score; alternatively, other scoring methods that can integrate multiple quality scores can also be used.

[0153] After obtaining the coverage quality score, it can be determined whether the acquired inspection images meet the standard shooting constraints based on the coverage quality score and whether at least one imaging deviation indicator fails to meet the corresponding constraints. Specifically, a quality threshold corresponding to the coverage quality score can be set, and it can be simultaneously determined whether any imaging deviation indicator fails to meet its corresponding constraint. When the coverage quality score meets the quality threshold and there is no imaging deviation indicator that needs to be judged as abnormal, it can be determined that the acquired inspection images meet the standard shooting constraints; when the coverage quality score fails to meet the quality threshold, or at least one imaging deviation indicator fails to meet its corresponding constraint and triggers anomaly judgment, it can be determined that the acquired inspection images do not meet the standard shooting constraints.

[0154] By setting corresponding standard shooting constraints for different types of inspection objects, and determining sub-item quality scores from dimensions such as target location, imaging scale, boundary integrity, visibility of key parts, and clarity of target areas, and then configuring evaluation weights based on the type of inspection object or the importance of the target area, different inspection objects can be evaluated according to the quality focus that matches their actual inspection needs. At the same time, by combining the coverage quality score with the constraint satisfaction of individual imaging deviation indicators to determine whether the image is qualified, the overall score can reduce the masking of obvious individual anomalies, and improve the accuracy and reliability of inspection image quality judgment.

[0155] In one example embodiment of this disclosure, after controlling the drone to acquire supplementary images of the target area according to the supplementary shooting task, the results of this supplementary shooting can be evaluated again, and it can be determined whether to continue automatic supplementary shooting based on the quality improvement of this supplementary shooting compared to the previous image. Specifically, this may include: The system can determine the imaging deviation index of the re-captured image and the previously acquired image corresponding to the target area, and determine the coverage quality score of the re-captured image and the previously acquired image based on the corresponding imaging deviation index. When the re-captured image meets the standard shooting constraints, it is determined as a valid inspection image corresponding to the target area. When the re-captured image does not meet the standard shooting constraints, the improvement of the coverage quality score of the re-captured image relative to the coverage quality score of the previously acquired image is determined. In a series of preset re-captures, if the improvement of the score for each re-capture is lower than the re-capture improvement threshold, automatic re-capture of the target area is stopped, and a manual review prompt is output. When the improvement of the score is not lower than the re-capture improvement threshold, the re-capture correction parameters are re-determined based on the imaging deviation index of the re-captured image, and the next re-capture task is generated.

[0156] The previously acquired image can be either an inspection image obtained during the initial inspection mission or a re-enhanced image obtained during a previous re-enhancing process. Both the re-enhanced image acquired this time and the previously acquired image can be processed using the same imaging deviation index determination method to ensure the comparability of the imaging states corresponding to the two images.

[0157] Furthermore, based on the corresponding imaging deviation indicators, the coverage quality scores for the currently acquired supplementary images and the previously acquired images can be determined separately. The coverage quality score can be determined according to the scoring method described in the aforementioned embodiments, that is, obtaining the corresponding sub-quality scores based on the imaging deviation indicators corresponding to the target area, and combining them with the corresponding evaluation weights to obtain the coverage quality score. By adopting the same scoring rules, the coverage quality score can be used to reflect the changes in the overall imaging quality of the target area under different acquisition cycles.

[0158] If the re-captured images meet the standard shooting constraints, they can be identified as valid inspection images corresponding to the target area. Valid inspection images are those that meet the shooting requirements for the target area and can be used as the inspection results for the current inspection task. After determining that the re-captured images are valid inspection images, the automatic re-capture process for the target area can be terminated.

[0159] When the re-captured image does not meet the standard shooting constraints, the improvement in coverage quality score of the re-captured image compared to the coverage quality score of the previously acquired image can be determined. The improvement in score characterizes the degree of improvement in the image quality of the target area after the re-capture is performed compared to the previous acquisition.

[0160] In one implementation, the improvement in coverage quality can be determined based on the difference between the coverage quality score of the currently acquired supplementary image and the coverage quality score of the previously acquired image. For example, when the current coverage quality score is higher than the previous one, the improvement can be represented by the difference between the two scores; alternatively, it can be represented by the percentage increase in coverage quality.

[0161] Furthermore, the improvement in score can be compared with the reshoot improvement threshold. The reshoot improvement threshold is used to determine whether the current reshoot correction can effectively improve the image quality of the target area, and can be preset according to the inspection task requirements or reshoot control strategy.

[0162] In a series of preset retakes, if the score improvement for each retake is less than the retake improvement threshold, automatic retakes of the target area are stopped, and a manual review prompt is output. The preset number of retakes can be set according to the allowed number of automatic retakes or the requirements of the inspection task. This embodiment does not impose any special limitations on this. Using the score improvement rate of multiple consecutive retakes for judgment can reduce the possibility of automatic retakes being prematurely terminated due to drone attitude fluctuations, environmental interference, or occasional changes in image quality.

[0163] Manual review prompts are used to guide relevant personnel to manually process the image or reshoot process corresponding to the target area. Manual review prompts may include at least one of the following: target area identifier, current number of reshoots, current coverage quality score, or current imaging anomaly attribute, so that reviewers can determine why the target area continues to fail to meet standard shooting constraints.

[0164] When the score improvement is not less than the reshoot improvement threshold, the reshoot correction parameters can be redefined based on the imaging deviation index corresponding to the reshoot image acquired this time, and the next reshoot task can be generated. At this time, although the reshoot has not yet made the reshoot image meet the standard shooting constraints, the current reshoot correction has effectively improved the imaging quality of the target area. Therefore, the reshoot image acquired this time can be used as a new analysis basis, and the reshoot correction parameters can be adjusted again based on its current imaging deviation.

[0165] Specifically, the imaging deviation indicators in the re-captured images that still do not meet the corresponding standard shooting constraints can be reassessed, and the target shooting control degrees of freedom and re-capture correction parameters can be re-determined based on the corresponding imaging anomaly attributes. The next re-capture task can be generated based on the actual shooting state corresponding to the re-captured images and the re-determined re-capture correction parameters, so that subsequent re-capture processes can continue to be adjusted according to the actual imaging changes after the previous re-capture.

[0166] By comparing the coverage quality scores of the current re-enhanced images with those of the previous images, and judging whether the re-enhanced corrections have produced effective improvements based on the improvement in scores, the re-enhanced parameters can be adjusted as the image quality continues to improve. However, if the improvement from multiple consecutive re-enhanced images is insufficient, automatic re-enhancing will be stopped and manual review will be initiated. This reduces invalid repeated re-enhancing and the resulting flight time and power consumption, while also reducing task delays caused by the inability of automatic re-enhancing to converge over a long period of time.

[0167] In one example embodiment of this disclosure, when there are multiple reshooting tasks corresponding to multiple target areas, the execution order of the multiple reshooting tasks can be determined by combining the necessity of each reshooting task with the path cost generated by the UAV executing each reshooting task. Specifically, this may include: The priority of each reshoot task can be determined based on the importance level of each target area, the degree of deviation of the corresponding imaging deviation index from the standard shooting constraints, and the corresponding coverage quality score. The path cost of each reshoot task can be determined based on the distance between the current location of the drone and the reshoot location corresponding to each reshoot task, the transfer distance between different reshoot locations, and the return distance after executing the corresponding reshoot task. Based on the reshoot priority and path cost of each reshoot task, a candidate execution order for multiple reshoot tasks can be determined. According to the candidate execution order, the estimated power consumption required to execute the corresponding reshoot task is accumulated sequentially, and the estimated return power consumption required to return to the return point from the last reshoot location after each accumulated reshoot task is determined. Based on the remaining power of the drone, the estimated power consumption for reshoot, the estimated return power consumption, and the safety margin, the target reshoot task and its target execution order are determined from the candidate execution order. Each target reshoot task is then executed according to the target execution order.

[0168] The reshoot priority value represents the priority of a reshoot task relative to other reshoot tasks. The importance level of a target area can be determined based on the inspection object corresponding to the target area, the importance of the target area in the inspection task, or a pre-configured inspection level. For example, a higher importance level can be configured for target areas requiring focused inspection. The imaging deviation index, representing the degree of deviation from standard shooting constraints, represents the extent to which the current inspection image deviates from the shooting requirements; a greater deviation indicates a greater need for reshooting. The coverage quality score comprehensively reflects the current imaging quality of the target area; a lower coverage quality score indicates a higher necessity for reshooting the corresponding task.

[0169] In the specific implementation process, corresponding evaluation parameters can be set for importance level, deviation degree, and coverage quality score, and the evaluation parameters can be combined to obtain the reshoot priority value. For example, a weighted calculation method can be used to determine the reshoot priority value, or the evaluation parameters can be sorted according to a preset priority rule. This embodiment does not limit the specific calculation method of the reshoot priority value.

[0170] Furthermore, the path cost for each reshooting task can be determined based on the distance between the UAV's current location and the reshooting location corresponding to each reshooting task, the transfer distance between different reshooting locations, and the return distance after executing the corresponding reshooting task. The path cost is used to characterize the flight path cost that the UAV needs to bear to execute the corresponding reshooting task or to execute multiple reshooting tasks in a corresponding order.

[0171] The distance between the drone's current position and the reshoot position reflects the flight distance required for the drone to travel from its current state to the corresponding reshoot position; the transfer distance between different reshoot positions reflects the movement distance between adjacent reshoot positions when the drone is performing multiple reshoot tasks; the return distance reflects the distance required for the drone to return from the corresponding reshoot position to the return point after performing the corresponding reshoot task. These distances can be calculated based on the spatial coordinates of each position, or determined based on the drone's actual executable flight path.

[0172] In one implementation, the path cost can be determined directly based on the cumulative value of the flight distances of each segment; in another implementation, the path cost can be calculated by combining the preset cost coefficients corresponding to different flight stages. This embodiment does not limit the specific representation of the path cost, as long as it reflects the flight path cost corresponding to the execution order of different reshooting tasks.

[0173] Based on the priority value and path cost of each reshoot task, a candidate execution order for multiple reshoot tasks can be determined. The candidate execution order represents the arrangement of reshoot tasks after considering both reshoot priority and flight path cost. Specifically, reshoot tasks with higher priority values ​​can be prioritized, and the order of reshoot tasks with similar priorities can be adjusted based on path cost to reduce unnecessary round trips between different reshoot locations by the UAV.

[0174] In one implementation, a comprehensive ranking parameter can be constructed based on the reshoot priority value and path cost, and the candidate execution order can be determined according to the comprehensive ranking parameter; alternatively, multiple reshoot tasks can be grouped according to the reshoot priority value, and the corresponding execution order can be determined within each group according to the path cost. This embodiment does not limit the specific method for determining the candidate execution order.

[0175] After determining the candidate execution order, the estimated power consumption required to execute the corresponding reshooting tasks can be accumulated sequentially according to the candidate execution order. The estimated power consumption is used to characterize the power consumption required for the drone to execute the accumulated reshooting tasks, and can be estimated based on the flight distance, estimated flight time, and equipment power consumption corresponding to the reshooting action.

[0176] For example, the estimated flight time and average power consumption per unit time required for the drone to execute each segment of the flight path can be used to determine the flight power consumption. This, combined with the estimated power consumption for actions such as gimbal adjustment, hovering, and image acquisition, can determine the estimated power consumption for a single reshooting task. As reshooting tasks are added sequentially to the candidate execution order, the corresponding estimated power consumption can be accumulated for each task.

[0177] For each cumulative step, the estimated return-home battery power required to return from the last reshoot location to the return point after completing the accumulated reshoot tasks is further determined. The return point can be the drone's takeoff location, a preset landing location, or a return location specified in the inspection task. The estimated return-home battery power is determined based on the return path between the last reshoot location and the return point, as well as the drone's corresponding estimated flight power consumption.

[0178] Based on the drone's remaining battery power, estimated battery power for reshooting, estimated battery power for return, and safety margin, the target reshooting task and its execution order can be determined from the candidate execution order. The safety margin is used to reserve additional power consumption due to changes in wind speed, path adjustments, or other operational fluctuations during drone execution, and can be expressed as a fixed battery value or a preset percentage of the drone's remaining battery power.

[0179] Specifically, reshooting tasks can be accumulated item by item according to the candidate execution order, and it can be determined whether the remaining battery power of the drone is sufficient to meet the sum of the currently accumulated expected reshooting battery power, expected return battery power, and safety margin. When the remaining battery power is sufficient to meet the corresponding battery power requirements, the next reshooting task can be accumulated; when the remaining battery power is insufficient to meet the battery power requirements after adding the next reshooting task, the previously accumulated reshooting tasks can be identified as target reshooting tasks, and their order in the candidate execution order can be identified as the target execution order.

[0180] Finally, the reshooting tasks for each target can be executed in the order of target execution. This allows the drone to prioritize reshooting tasks with higher necessity and lower path cost, provided that the return battery power and safety margin requirements are met. This reduces the invalid flight distance during the reshooting task execution and lowers the risk of reshooting tasks being interrupted or unable to return normally due to insufficient remaining battery power.

[0181] By determining the priority of reshooting based on the importance level of the target area, the degree of imaging deviation, and the coverage quality score, and by generating a candidate execution order based on the path cost between reshooting locations, the necessity of the reshooting task and the cost of the UAV's flight path can be balanced, reducing unnecessary round trips between different reshooting locations. Furthermore, by combining the remaining battery power, the estimated battery power for reshooting, the estimated battery power for return, and the safety margin to screen target reshooting tasks, the risk of mission interruption or inability to return normally due to insufficient battery power during the reshooting process can be reduced, thereby improving the completion efficiency and flight safety of reshooting tasks under limited endurance conditions.

[0182] In one optional embodiment of this disclosure, the imaging anomaly attributes, reshoot correction parameters, and score improvement obtained after executing the corresponding reshoot correction parameters for each reshoot can be associated and stored to form a historical reshoot record corresponding to the target area. When the reshoot correction parameters are re-determined, the current imaging anomaly attribute is determined based on the imaging deviation index corresponding to the reshoot image acquired this time, and at least one historical reshoot record matching the current imaging anomaly attribute is determined from the historical reshoot records. The historical reshoot correction parameters and the corresponding score improvement are obtained from at least one historical reshoot record. Based on the score improvement corresponding to each historical reshoot correction parameter, the target historical reshoot correction parameter whose score improvement meets the improvement conditions is determined from the historical reshoot correction parameters. The reshoot correction parameters corresponding to the current reshoot are determined based on the target historical reshoot correction parameters.

[0183] Among them, the historical reshoot record is used to record the actual improvement after reshooting and adjusting for the same target area or similar imaging anomalies, so as to provide a reference when re-determining the reshoot correction parameters in the future.

[0184] Specifically, after each reshoot, the system can record the imaging anomaly attributes before the reshoot, the correction parameters used in the reshoot, and the improvement in score compared to the previous image. This data is stored as a historical reshoot record. Different historical reshoot records can be distinguished by reshoot time, number of reshoots, or task execution order.

[0185] Imaging anomaly attributes characterize the type or feature of the anomaly targeted in this re-enhancing image, and may include at least one of the following: offset direction, offset degree, cropping position, and location of missing key parts. Re-enhancing correction parameters characterize the actual shooting state adjustments made to address the corresponding imaging anomaly. The score improvement reflects the degree of improvement in the coverage quality score of the target area relative to the previous image after implementing the corresponding re-enhancing correction parameters. By storing these three attributes in a correlated manner, a correspondence can be established between imaging anomalies, re-enhancing adjustments, and adjustment results.

[0186] When re-determining the correction parameters for the re-shoot, the current imaging anomaly attribute is determined based on the imaging deviation index corresponding to the re-shoot image acquired this time. The method for determining the current imaging anomaly attribute can refer to the aforementioned embodiment, that is, determining at least one of the following based on the imaging deviation index that does not meet the corresponding standard shooting constraints: offset direction, offset degree, cropping position, or location of missing key parts.

[0187] Furthermore, at least one historical re-enhancing image can be identified from the historical re-enhancing image records that matches the current imaging anomaly attribute. Matching imaging anomaly attributes can mean having the same anomaly type, or, based on having the same anomaly type, further determining whether the anomaly direction, location, or degree meets preset similarity conditions. For example, when the current imaging anomaly attribute includes offset direction and offset degree, historical re-enhancing image records with the same offset direction and offset degree differences within a preset range can be selected from the historical re-enhancing image records.

[0188] In one implementation, matching conditions can be set for different attribute items in the imaging anomaly attributes, and the matching result can be determined based on the number of attribute items that meet the matching conditions; alternatively, corresponding matching weights can be set according to the importance of different attribute items, and historical re-shooting records can be filtered based on the comprehensive matching degree. This embodiment does not limit the specific matching method for historical re-shooting records.

[0189] After identifying at least one matching historical re-enhancing record, the historical re-enhancing correction parameters and corresponding score improvement rates for each record can be obtained. Historical re-enhancing correction parameters refer to the re-enhancing correction parameters actually performed previously for the corresponding imaging anomalies, while the corresponding score improvement rate reflects the actual improvement effect of those historical re-enhancing correction parameters.

[0190] Furthermore, based on the score improvement corresponding to each historical reshoot correction parameter, a target historical reshoot correction parameter whose score improvement meets the improvement conditions can be determined from the historical reshoot correction parameters. The improvement conditions can be preset according to the reshoot improvement needs. For example, the score improvement not being lower than a preset improvement threshold can be used as the improvement condition, or one or more parameters with higher score improvement can be selected from multiple historical reshoot correction parameters as the target historical reshoot correction parameter.

[0191] When multiple historical re-enhancing correction parameters meet the improvement conditions, the historical re-enhancing correction parameter with the largest score improvement can be selected. Alternatively, the target historical re-enhancing correction parameter can be determined by comprehensively considering multiple historical re-enhancing correction parameters that meet the improvement conditions. For example, a historical re-enhancing correction parameter with a high degree of matching with the current imaging anomaly attributes and a large score improvement can be prioritized. This embodiment does not limit the specific method for selecting the target historical re-enhancing correction parameter.

[0192] The correction parameters for the current re-enhancing image can be determined based on the target's historical correction parameters. In one implementation, the target's historical correction parameters can be directly used as the correction parameters for the current re-enhancing image. In another implementation, the target's historical correction parameters can be used as a benchmark, and the parameters can be adjusted by combining the differences between the current imaging anomaly attributes and the corresponding historical imaging anomaly attributes, thereby obtaining the correction parameters for the current re-enhancing image.

[0193] By associating and storing imaging anomaly attributes, reshoot correction parameters, and corresponding score improvement, and by selecting historical reshoot correction parameters that have shown good improvement effects on similar imaging anomalies from historical reshoot records when re-determining reshoot correction parameters, existing reshoot results can be used to guide the determination of current parameters. This reduces the repeated use of reshoot adjustment methods with poor improvement effects, improves the matching degree between reshoot correction parameters and current imaging anomalies, and improves the improvement efficiency and success rate of subsequent automatic reshoots.

[0194] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0195] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described automatic reshooting method for inspection image acquisition is also provided.

[0196] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be embodied in the following forms: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0197] The following reference Figure 5 To describe an electronic device 500 according to such an embodiment of the present disclosure. Figure 5 The electronic device 500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0198] like Figure 5As shown, the electronic device 500 is presented in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0199] The storage unit stores program code that can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 510 can perform actions such as... Figure 1 Step S110, as shown, involves acquiring the target area corresponding to the inspection task, the standard shooting constraints corresponding to the target area, and the acquired inspection images corresponding to the target area, and acquiring the original shooting state when acquiring the acquired inspection images. The original shooting state includes at least one of shooting position, gimbal angle, and shooting distance. Step S120 involves determining the imaging deviation index of the target area in the acquired inspection images, and when at least one of the imaging deviation indices does not meet the corresponding standard shooting constraints, determining the imaging anomaly attribute corresponding to the target area based on the imaging deviation index that does not meet the corresponding standard shooting constraints. The imaging anomaly attribute includes offset direction, offset degree, cropping position, and missing information. At least one of the following is missing: the location of a key part; Step S130: Based on the imaging anomaly attribute, determine the target shooting control degree of freedom from multiple shooting control degrees of freedom, and determine the reshoot correction parameter corresponding to the target shooting control degree of freedom based on the imaging deviation index, wherein the multiple shooting control degrees of freedom include at least two of the following: gimbal rotation degree of freedom, shooting position movement degree of freedom, shooting distance adjustment degree of freedom, and reshoot angle adjustment degree of freedom; Step S140: Determine the target reshoot state based on the original shooting state and the reshoot correction parameter, and generate a reshoot task corresponding to the target area based on the target reshoot state; Step S150: Control the UAV to collect reshoot images of the target area according to the reshoot task.

[0200] Storage unit 520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 521 and / or cache memory (Cache) 522, and may further include read-only memory (ROM) 523.

[0201] Storage unit 520 may also include a program / utility 524 having a set (at least one) program module 525, such program module 525 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0202] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0203] Electronic device 500 can also communicate with one or more external devices 570 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 500, and / or with any device that enables electronic device 500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 550. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 560. As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0204] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0205] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0206] refer to Figure 6As shown, a program product 600 for implementing the above-described automatic re-capture method for inspection image acquisition tasks according to embodiments of the present disclosure is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0207] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0208] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0209] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0210] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0211] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0212] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0213] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This specification is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. It is understood that the specification and embodiments are to be considered exemplary only and are intended to illustrate the true scope and technical concepts of this disclosure.

[0214] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. An automatic retake method for a patrol image collection task, characterized in that, include: The system acquires the target area corresponding to the inspection task, the standard shooting constraints corresponding to the target area, and the collected inspection images corresponding to the target area. It also acquires the original shooting state when the collected inspection images were acquired, and the original shooting state includes at least one of the shooting position, gimbal angle, and shooting distance. The imaging deviation index of the target area in the acquired inspection image is determined, and when at least one of the imaging deviation indices does not meet the corresponding standard shooting constraint, the imaging abnormality attribute corresponding to the target area is determined according to the imaging deviation index that does not meet the corresponding standard shooting constraint. The imaging abnormality attribute includes at least one of offset direction, offset degree, cropping position and missing key part orientation. Based on the imaging anomaly attributes, a target shooting control degree of freedom is determined from multiple shooting control degrees of freedom, and a reshoot correction parameter corresponding to the target shooting control degree of freedom is determined based on the imaging deviation index. The multiple shooting control degrees of freedom include at least two of the following: gimbal rotation degree of freedom, shooting position movement degree of freedom, shooting distance adjustment degree of freedom, and reshoot angle adjustment degree of freedom. The target reshoot state is determined based on the original shooting state and the reshoot correction parameters, and a reshoot task corresponding to the target area is generated based on the target reshoot state. Control the drone to collect supplementary images of the target area according to the supplementary shooting task.

2. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, The imaging deviation index includes the center offset of the target projection frame center of the target area relative to the image center of the acquired inspection image; The step of determining the target shooting control degree of freedom from multiple shooting control degrees of freedom based on the imaging anomaly attributes, and determining the reshoot correction parameters corresponding to the target shooting control degree of freedom based on the imaging deviation index, includes: When determining the gimbal angle adjustment method based on the center offset, the target shooting control degree of freedom is determined to be the gimbal rotation degree of freedom; When the center offset includes a horizontal center offset, the gimbal horizontal angle correction amount is determined based on the ratio of the horizontal center offset to the image width of the acquired inspection image and the horizontal field of view of the camera that acquired the inspection image, and the gimbal horizontal angle is corrected according to the return direction corresponding to the horizontal offset direction of the center offset. When the center offset includes a vertical center offset, the gimbal pitch angle correction is determined based on the ratio of the vertical center offset to the image height of the acquired inspection image and the vertical field of view of the camera, and the gimbal pitch angle is corrected according to the return direction corresponding to the vertical offset direction of the center offset.

3. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, The imaging deviation index includes the percentage of the target area of ​​the target region in the acquired inspection images; The step of determining the target shooting control degree of freedom from multiple shooting control degrees of freedom based on the imaging anomaly attributes, and determining the reshoot correction parameters corresponding to the target shooting control degree of freedom based on the imaging deviation index, includes: Determine the area ratio deviation between the target area ratio and the preset area ratio range corresponding to the standard shooting constraint, and determine the target shooting control degree of freedom as the shooting distance adjustment degree of freedom; When the target area ratio is less than the lower limit of the preset area ratio range, a first shooting distance correction amount for reducing the shooting distance is determined based on the area ratio deviation; When the target area ratio is greater than the upper limit of the preset area ratio range, a second shooting distance correction amount is determined based on the area ratio deviation to increase the shooting distance; The shooting distance in the original shooting state is corrected based on the first shooting distance correction amount or the second shooting distance correction amount.

4. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, The imaging deviation index includes the intersection position between the target segmentation mask of the target area and the image boundary of the acquired inspection image; The step of determining the imaging anomaly attribute corresponding to the target area based on the imaging deviation index that does not meet the corresponding standard shooting constraints includes: Based on the intersection position of the target segmentation mask and the image boundary, the cropping position of the target region is determined, and the cropping position is determined as the imaging anomaly attribute corresponding to the target region; The step of determining the target shooting control degree of freedom from multiple shooting control degrees of freedom based on the imaging anomaly attributes, and determining the reshoot correction parameters corresponding to the target shooting control degree of freedom based on the imaging deviation index, includes: Based on the cropping position, a target compensation direction is determined to make the cropped portion corresponding to the target area enter the range of the reshot image; Based on the target compensation direction, at least one of the gimbal rotation degree of freedom and the shooting position movement degree of freedom is determined as the target shooting control degree of freedom, and based on the cropping position, the reshoot correction parameter for adjusting the gimbal angle or shooting position along the target compensation direction is determined.

5. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, The imaging deviation index includes the visible proportion of key parts corresponding to the target area; the method further includes: When the visible ratio of the critical part is lower than the minimum visible ratio corresponding to the standard shooting constraint, the missing critical part that does not meet the visibility requirement is identified, and the missing orientation of the missing critical part relative to the target area is determined. The original shooting direction when the collected inspection images are acquired is obtained, the degree of freedom of the reshoot angle adjustment is determined to be the target shooting control degree of freedom based on the missing direction, and a lateral reshoot position is generated along the direction corresponding to the missing direction, which is different from the shooting position in the original shooting state. Add a side-facing reshoot angle based on the original shooting direction; At least one reshoot viewpoint is generated based on the lateral reshoot position and the shooting direction after increasing the lateral reshoot angle, and a corresponding reshoot task is generated based on the reshoot viewpoint; When the missing key part is obstructed, at least two reshoot viewpoints located at different lateral positions in the target area are generated, and reshoot tasks are generated based on the at least two reshoot viewpoints respectively.

6. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, Before determining the imaging deviation index of the target area in the acquired inspection images, the method further includes: Target detection is performed on the acquired inspection images to obtain target detection results that characterize whether the target region exists in the acquired inspection images; Based on the shooting position, gimbal angle and shooting distance when the collected inspection images are acquired, as well as the spatial location of the target area, the theoretical coverage area corresponding to the collected inspection images is determined. When the target detection result indicates that the target region exists in the acquired inspection image and the spatial location of the target region is within the theoretical coverage area, the acquired inspection image is determined as a valid candidate image of the target region. When the target detection result does not indicate that the target area exists in the acquired inspection image, and the spatial location of the target area is within the theoretical coverage area, the acquired inspection image is determined as an image to be reviewed for quality. If the target detection result does not indicate that the target area exists in the acquired inspection image, and the spatial location of the target area is not within the theoretical coverage area, the correspondence between the acquired inspection image and the target area is excluded.

7. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, The standard shooting constraints are determined based on the inspection object type corresponding to the target area; the method further includes: Obtain at least two of the following constraints corresponding to the inspection object type: the allowable range of the target center, the range of the target area proportion, the maximum boundary cropping ratio, the minimum visible ratio of the key parts, and the target area clarity threshold; and determine the at least two constraints as the standard shooting constraints corresponding to the target area. For each of the constraints, based on the imaging state of the target area in the acquired inspection images, at least two imaging deviation indicators are determined from the following: center offset, target area ratio, boundary cropping ratio, visible ratio of key parts, and target area clarity index. Based on each of the imaging deviation indices, a corresponding sub-quality score is determined. The sub-quality score includes at least two of the following: target centering score, target scale score, boundary integrity score, key part visibility score, and target area sharpness score. The evaluation weights corresponding to each of the sub-item quality scores are determined based on the type of the inspection object or the importance of the target area, and the coverage quality score corresponding to the target area is determined based on each of the sub-item quality scores and the corresponding evaluation weights. Based on the coverage quality score and whether at least one of the imaging deviation indicators does not meet the corresponding constraints, it is determined whether the acquired inspection images meet the standard shooting constraints.

8. The automatic reshooting method for inspection image acquisition tasks according to claim 1, characterized in that, After controlling the drone to acquire supplementary images of the target area according to the supplementary shooting task, the method further includes: The imaging deviation index of the re-captured image and the image corresponding to the target area obtained in the previous acquisition are determined respectively, and the coverage quality score of the re-captured image and the image obtained in the previous acquisition is determined according to the corresponding imaging deviation index. When the re-shot image acquired in this instance meets the standard shooting constraints, the re-shot image acquired in this instance is determined as a valid inspection image corresponding to the target area; When the re-shot image acquired in this instance does not meet the standard shooting constraints, the improvement rate of the coverage quality score corresponding to the re-shot image acquired in this instance compared to the coverage quality score corresponding to the image acquired in the previous instance is determined. In a series of pre-set reshoots, if the improvement in the score corresponding to each reshoot is less than the reshoot improvement threshold, the automatic reshoot for the target area is stopped, and a manual review prompt is output. When the score improvement is not less than the reshoot improvement threshold, the reshoot correction parameters are re-determined based on the imaging deviation index corresponding to the reshoot image collected this time, and the next reshoot task is generated.

9. The automatic reshooting method for inspection image acquisition tasks according to claim 7, characterized in that, When there are multiple reshooting tasks corresponding to multiple target areas, the method further includes: Based on the importance level of each target area, the degree of deviation of the corresponding imaging deviation index from the standard shooting constraints, and the corresponding coverage quality score, the priority value of each reshoot task is determined. Based on the distance between the current location of the drone and the corresponding shooting location of each shooting task, the transfer distance between different shooting locations, and the return distance after executing the corresponding shooting task, the path cost corresponding to each shooting task is determined. Based on the reshoot priority value and path cost corresponding to each reshoot task, determine the candidate execution order of the multiple reshoot tasks; According to the candidate execution order, the estimated battery power required to execute the corresponding reshooting task is accumulated in sequence, and the estimated return battery power required to return to the return point from the last reshooting position is determined for each accumulated reshooting task. Based on the remaining battery power of the drone, the estimated battery power for reshooting, the estimated battery power for returning to base, and the safety margin, the target reshooting task and the target execution order of the target reshooting task are determined from the candidate execution order; The target reshooting tasks are executed in the order described in the target execution sequence.

10. The automatic reshooting method for inspection image acquisition tasks according to claim 8, characterized in that, The method further includes: The imaging anomaly attributes, reshoot correction parameters, and score improvement obtained after executing the corresponding reshoot correction parameters are associated and stored to form a historical reshoot record for the target area. When the reshoot correction parameters are redefined, the current imaging anomaly attribute is determined based on the imaging deviation index corresponding to the reshoot image acquired this time, and at least one historical reshoot record that matches the current imaging anomaly attribute is determined from the historical reshoot records. The historical reshoot correction parameters and the corresponding score improvement range in the at least one historical reshoot record are then obtained. Based on the score improvement rate corresponding to each of the historical reshoot correction parameters, a target historical reshoot correction parameter whose score improvement rate meets the improvement conditions is determined from the historical reshoot correction parameters. The corresponding reshoot correction parameters are determined based on the target historical reshoot correction parameters.

Images