A dual unmanned aerial vehicle cooperative observation system and method for reducing invalid pixels

Abstract

This invention provides a dual-UAV collaborative observation system and method for invalid pixel reduction, belonging to the field of UAV remote sensing technology. The system includes a flight planning module, a ground control module, a synchronization triggering module, a UAV module, and a data processing module. The method includes calculating the distance between the two UAVs and the angle between their two viewing angles based on observation task parameters, constructing the effective observation range for both viewing angles, generating paired observation strips and segments for the target area, establishing a dual-UAV flight planning model incorporating constraints such as time windows, dual-view observation, turning waiting, and overlap, solving for executable flight paths and timing sequences, synchronously collecting data from both UAVs according to the flight planning results, calculating the comprehensive saturation index for each pixel of the collected data, generating a two-choice mask, and outputting the resulting image and bilateral abnormal pixel markers. This invention can stably reduce oversaturated invalid pixels caused by flares during the dual-UAV data acquisition phase, while simultaneously considering operational efficiency and flight safety.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) remote sensing technology, and in particular to a dual UAV collaborative observation system and method for eliminating invalid pixels. Background Technology

[0002] During UAV remote sensing operations, images of weakly textured and non-Lambertian targets (such as water surfaces, photovoltaic panels, ice surfaces, tidal flats, and salt pans) are prone to flare and overexposure under sunlight, resulting in unusable "invalid pixels." Existing data quality improvement methods mainly rely on post-processing, such as masking to remove overexposed areas, seam optimization and stitching leveling based on multiple overlapping images, and overall color and brightness correction based on radiometric normalization. However, these methods are insufficient to eliminate invalid pixels at the source.

[0003] On the other hand, remote sensing UAV mission planning and actual observation operations need to consider multiple constraints such as time windows (e.g., 10:00–14:00), safety distances, forward / lateral overlap, and energy consumption, while meeting data acquisition requirements. Existing single-unit or multi-unit mission planning models take into account factors such as observation time constraints, energy consumption, and safety distances, but fail to incorporate the requirement for simultaneous dual-unit observation, making it difficult to complete observation tasks efficiently and safely while meeting data acquisition requirements. Summary of the Invention

[0004] This invention provides a dual-UAV collaborative observation system and method for eliminating invalid pixels, which addresses the shortcomings of existing technologies, achieves stable reduction of oversaturated invalid pixels caused by flares during the data acquisition stage, and balances observation efficiency and flight safety.

[0005] In a first aspect, the present invention provides a dual-UAV collaborative observation system for invalid pixel reduction, comprising a flight planning module, a ground control module, a synchronization triggering module, a UAV module, and a data processing module, wherein the modules are connected by wireless or wired communication. The flight planning module determines the set of observation data based on the received mission parameters, establishes a dual-aircraft flight planning model, outputs the missions of each observation segment of the UAV module, and generates a flight mission script. The ground control module receives the flight mission script, sends the tasks of each observation segment to the UAV module, controls the flight of the UAV module, collects the status information of the UAV module, and sends trigger control commands. The synchronization triggering module receives the triggering control command, sends a triggering signal to the UAV module, synchronizes the UAV module's image acquisition and attitude information recording at the same time, and sends triggering event information to the ground control module; The UAV module includes two UAVs, which synchronously execute observation tasks according to the tasks of each observation segment and transmit the collected observation information back to the data processing module. The data processing module completes image geometric registration and pixel alignment based on the observation information, performs optimization of dual-view pixels and bilateral abnormal pixel marking, and obtains a result image in which invalid pixels have been reduced and the corresponding mask file. After stitching, it outputs a result image covering the entire target area and the distribution result of abnormal pixels.

[0006] According to the present invention, a dual-UAV collaborative observation system for invalid pixel reduction is provided, wherein the flight planning module is deployed on a ground control computer, vehicle-mounted terminal or server, and interacts with the ground control module through a task file or network interface; The ground control module is implemented using a handheld remote controller in conjunction with ground station software, or integrated into a vehicle-mounted control terminal or portable ground workstation. It is wirelessly connected to the UAV module and wirelessly or wiredly connected to the flight planning module and the data processing module. The synchronization triggering module is implemented using a hardware trigger line, a wireless trigger signal, or a network time synchronization method, and shares the trigger time tag with the ground control module. The data processing module is deployed in a ground workstation, vehicle-mounted server, or cloud computing platform, and is connected to the ground control module via wired or wireless communication.

[0007] Secondly, the present invention also provides a dual-UAV cooperative observation method for invalid pixel reduction, comprising: Acquire flight mission information for the target area, and determine the flight parameters of the two UAVs and the effective observation range of the two perspectives based on the desired ground resolution; The target area is discretized into paired observation strips, and each paired observation strip is further divided into several observation segments along its length, thus generating a set of discrete observation tasks. A dual-aircraft flight planning model is established based on the discrete observation task set. The optimal flight target, constraints, and decision variables are determined. The dual-aircraft flight planning model is solved to form a dual-UAV escort mission script. The script for the dual-UAV escort mission is sent to the dual UAVs, controlling them to execute the flight mission, synchronously observing the dual UAVs, and acquiring synchronously collected data. Geometric registration is performed on the synchronously acquired data to complete pixel alignment. Pixel saturation index is calculated, and the resulting image of the target area and the distribution results of abnormal pixels are obtained from the pixel saturation index.

[0008] According to the present invention, a dual-UAV cooperative observation method for invalid pixel reduction acquires flight mission information of a target area, determines the dual-UAV flight parameters and the effective observation range of the dual-viewpoint based on the desired ground resolution, including: The observation flight altitude is calculated based on the expected ground resolution of the observed target, the focal length of the observation payload lens, and the pixel size. The single-frame ground swath width is calculated based on the observation flight altitude and the horizontal field of view of the observation payload. The lateral spacing between the two drones is obtained at the center of the ground projection of the optical axis of the two drones. The lateral spacing between the two drones is between the minimum safe spacing between the two drones, the minimum lateral overlap of the paired images, and the upper limit of the spacing between the two drones determined by the width of a single ground frame. The angle between the observation zenith angles of the ground point directly below the midpoint of the line connecting the two UAVs is obtained from the lateral distance between the two UAVs and the observation flight altitude. The geometric overlap area of ​​the paired images from two UAVs is determined as the dual-view observation range. The dual-view observation range is then shrunk by a shrinkage coefficient to obtain the effective dual-view observation range.

[0009] According to the present invention, a dual-UAV cooperative observation method for invalid pixel reduction is provided, wherein the target area is discretely divided into paired observation strips, and each paired observation strip is further divided into several observation segments along its length direction, generating a discrete observation task set, including: Based on the effective observation range of the dual-view system, the target area is discretized into several paired observation strips that are approximately perpendicular to the sun's azimuth. Set the forward overlap and the lateral overlap between paired observation strips, and divide each paired observation strip into several observation segments along its length. Each observation segment corresponds to a continuous ground area along the direction of the paired observation strips, which serves as the basic task unit for simultaneous observation by dual UAVs, forming the discrete observation task set.

[0010] According to the present invention, a dual-UAV cooperative observation method for invalid pixel reduction is provided. Based on the discrete observation task set, a dual-UAV flight planning model is established to determine the optimal flight target, constraints, and decision variables. The dual-UAV flight planning model is then solved to form a dual-UAV escort mission script, including: The objective function is to minimize the total operation time of the drone, the total energy consumption of the drone, or the weighted optimal balance between time and energy consumption. Taking into account flight safety, observation quality, and time requirements, the following constraints were determined: observation time window constraint, longest flight time per sortie constraint, effective observation from dual perspectives constraint, overlap constraint, observation sequence constraint, minimum safe distance and turning constraint, and return-to-base for battery swapping constraint. Using the start and end times of the observation segment and the return-to-base battery swapping node as decision variables, a dual-aircraft flight planning model is established by integrating the objective function, constraints, and decision variables. A centralized heuristic algorithm is used to solve the dual-drone flight planning model to obtain the observation segment execution sequence of the two UAVs and the corresponding take-off time, return time and battery swapping schedule. The script and flight path file for the dual UAV escort mission are generated based on the model solution results.

[0011] According to the present invention, a dual-UAV collaborative observation method for invalid pixel reduction is provided, wherein the dual-UAV escort mission script is sent to the dual UAVs, the dual UAVs are controlled to execute flight missions, and synchronous observation of the dual UAVs is performed to acquire synchronously collected data, including: Based on the dual-UAV escort mission script and flight path file, control the two UAVs to take off and escort observation synchronously at the same observation altitude, the same lateral distance between the two UAVs and the set observation flight direction. Within each observation segment, a trigger signal enables the two UAVs to acquire dual-view images and record attitude information simultaneously. Between adjacent paired observation strips, the two UAVs perform inward and outward turns according to the planned turning and waiting strategy. The UAV on the inside completes the turn with a smaller turning radius and hovers at the waiting position, while the UAV on the outside completes the turn with a larger turning radius. After the two UAVs align at the starting position of the new strip, they enter the next observation segment together. Throughout the process, the distance between the two UAVs is not less than the minimum safe distance between the two UAVs.

[0012] According to the present invention, a dual-UAV collaborative observation method for invalid pixel reduction performs geometric registration on the synchronously acquired data, completes pixel alignment, calculates a pixel saturation index, and obtains the result image of the target area and the distribution results of abnormal pixels from the pixel saturation index, including: Acquire dual-view images and attitude information of two UAVs at the same time, perform radiometric preprocessing and geometric correction on the dual-view images and attitude information, so that the same ground position corresponds to the same grid cell position of the dual-view images, and each grid cell position includes the pixel value of the first UAV and the pixel value of the second UAV. The pixel values ​​of each band radiometrically calibrated in the dual-view image are linearly normalized. Based on the linearly normalized pixel values, the saturation of each pixel in the dual-view image is calculated band by band to generate the saturation raster of each band of the dual UAV. Based on the saturation grid of each band of dual UAVs, the comprehensive saturation index of each grid cell position is calculated, and the abnormal cells on both sides are screened according to the comprehensive saturation threshold to obtain the effective grid cell positions. For all valid raster cell positions, based on the observation geometry and the preset fixed side, pixel-by-pixel optimization is performed to generate a result mask file; Constraints are applied using the resulting mask file, and pixel-level reconstruction is performed to obtain the resulting image after invalid pixel reduction and bilateral abnormal pixel markers. Geometric registration, pixel alignment, calculation of pixel saturation index, and pixel-by-pixel comparison are repeated for all paired data within the target area. The resulting images are then stitched together using a unified coordinate system, and the resulting image of the target area is output. The bilateral abnormal pixel markers are then stitched together synchronously according to the same geometric relationship, and the abnormal pixel distribution result of the target area is output.

[0013] According to the present invention, a dual-UAV collaborative observation method for invalid pixel reduction is provided, which performs pixel-by-pixel optimization for all valid raster pixel positions based on observation geometry and a preset fixed side, and generates a result mask file, including: Obtain the first and second distances of the raster pixel positions from the center line of the dual UAV image, and determine the distance partition threshold; If the absolute value of the difference between the first distance and the second distance is greater than the distance partition threshold, the side with a viewpoint closer to the direct downward viewpoint is selected; otherwise, the selection is made according to the preset fixed side. A two-choice mask is formed for all grid cell positions, the two-choice mask including a first UAV cell, a second UAV cell, and bilateral anomalous cells.

[0014] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the dual UAV cooperative observation method for invalid pixel reduction as described above.

[0015] The present invention provides a dual-UAV collaborative observation system and method for invalid pixel reduction. It achieves stable invalid pixel reduction through synchronous dual-view acquisition. Based on synchronous dual-UAV dual-view acquisition, it provides alternative viewpoint data for the same observation target. Combined with a pixel-by-pixel binary selection rule, it can stably reduce oversaturated invalid pixels caused by flares during the data acquisition phase. Simultaneously, it incorporates constraints such as effective observation range, overlap, time window, turning waiting, and minimum safe distance into the dual-UAV flight planning model. By including quality constraints in flight planning, it effectively balances efficiency and safety. Furthermore, it uses the same mask constraint to achieve consistent pixel selection across multiple bands, avoiding band inconsistencies caused by cross-UAV mixing. It also outputs dual-sided abnormal pixel markers, facilitating quality assessment and subsequent supplementary measurements, thus improving result consistency and usability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the structure of the dual UAV cooperative observation system for invalid pixel reduction provided by the present invention; Figure 2 This is a flowchart illustrating the dual-UAV collaborative observation method for invalid pixel reduction provided by the present invention. Figure 3 This is a schematic diagram of the observation geometry when two UAVs conduct synchronous observations at the same flight altitude, provided by the present invention. Figure 4 This is a schematic diagram of the effective observation range of each group of images provided by the present invention from a dual-view perspective; Figure 5 This is a flowchart illustrating the dual-aircraft flight planning method provided by the present invention; Figure 6 This is a schematic diagram of the path provided by the present invention for the dual-machine system to turn between adjacent observation strips and maintain the minimum safe distance; Figure 7 This is a schematic diagram of the pixel selection process based on the comprehensive saturation index provided by the present invention. Figure 8 This is a schematic diagram of the pixel-by-pixel optimization rule based on the comprehensive saturation index provided by the present invention; Figure 9 This is a schematic diagram of the simulation scene and fusion results provided by the present invention; Figure 10 This is a schematic diagram of the two-choice mask provided by the present invention; Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] To address the various limitations of existing technologies, this invention first proposes a dual-UAV cooperative observation system for invalid pixel reduction, such as... Figure 1As shown, it includes a flight planning module, a ground control module, a synchronization triggering module, an unmanned aerial vehicle (UAV) module, and a data processing module. The modules are connected to each other via wireless or wired communication. The flight planning module determines the set of observation data based on the received mission parameters, establishes a dual-aircraft flight planning model, outputs the missions of each observation segment of the UAV module, and generates a flight mission script. The ground control module receives the flight mission script, sends the tasks of each observation segment to the UAV module, controls the flight of the UAV module, collects the status information of the UAV module, and sends trigger control commands. The synchronization triggering module receives the triggering control command, sends a triggering signal to the UAV module, synchronizes the UAV module's image acquisition and attitude information recording at the same time, and sends triggering event information to the ground control module; The UAV module includes two UAVs, which synchronously execute observation tasks according to the tasks of each observation segment and transmit the collected observation information back to the data processing module. The data processing module completes image geometric registration and pixel alignment based on the observation information, performs optimization of dual-view pixels and bilateral abnormal pixel marking, and obtains a result image in which invalid pixels have been reduced and the corresponding mask file. After stitching, it outputs a result image covering the entire target area and the distribution result of abnormal pixels.

[0020] Specifically, the flight planning module generates flight plans for dual-drone collaborative observation based on mission requirements. This module receives mission parameters such as the observation area boundary, observation time window, ground resolution requirements of the observation target, solar azimuth, UAV and observation payload parameters, calculates the ground swath width of a single image, determines the angle between the two drones' viewing angles and the distance between them, and constructs an effective dual-view observation zone. Based on this, the area to be observed is discretized into several paired observation strips and observation segments within the strips. A dual-drone flight planning model is established, with total time and total energy consumption as objectives and dual-view observation and flight safety as constraints. The model solves for the execution sequence of the observation segments for the two UAVs, as well as the takeoff, landing, and return times for each sortie, generating a flight mission script that can be directly deployed. The flight planning module can be deployed on a ground control computer, a vehicle-mounted terminal, or a server with computing capabilities, interacting with the ground control module through mission files or network interfaces.

[0021] The ground control module is responsible for mission issuance, takeoff and landing control, flight control, and status monitoring. This module obtains observation strips and takeoff and landing sequences from the flight planning module, issues the planned flight paths and takeoff and landing commands to the UAV modules (UAV L and UAV R), and receives real-time status information from both aircraft, including battery level, position, altitude, attitude, and mission progress, for mission monitoring and safety management. The ground control module also receives trigger event information from the synchronization trigger module and records the trigger time and sequence number of each synchronization acquisition. The ground control module can be implemented using a handheld remote controller with ground station software, or integrated into a vehicle-mounted control terminal or portable ground workstation. It communicates with the UAV modules via a wireless data transmission link and interacts with the flight planning module and data processing module via wired or wireless networks.

[0022] The synchronization trigger module outputs a unified trigger signal to the observation payload and positioning unit of the UAV modules (UAV L and UAV R), while simultaneously providing unified trigger event information to the ground control module. Through this module, the two UAVs can synchronously acquire images and attitude information at the same time, thus providing consistent paired images within the effective observation range of both views. This provides a time reference for subsequent pixel-by-pixel optimization and invalid pixel reduction. The synchronization trigger module can be implemented using a hardware trigger line, wireless trigger signal, or network time synchronization, and shares a trigger time stamp with the ground control module.

[0023] The UAV module (UAV L and UAV R) is used to perform dual-UAV collaborative observation tasks. In this embodiment, the two UAVs are identical multi-rotor UAVs, equipped with visible light cameras or multispectral cameras, and configured with positioning and attitude measurement units. Each UAV establishes a data link with the ground control module to receive flight path and control commands, transmit flight status information and payload image data, and simultaneously establishes a trigger interface with the synchronization trigger module to complete observations based on a unified trigger signal. During the mission, UAV L and UAV R fly parallel to each other at the same flight altitude with a preset dual-UAV spacing, employing turning and holding strategies (such as...) between adjacent observation strips. Figure 6 (as shown), and the minimum safe distance between the two aircraft is jointly ensured by the ground control module and the flight planning module.

[0024] The data processing module processes the images, attitude, and time information acquired by the two drones, performing invalid pixel reduction and outputting the results. This module acquires paired images and their corresponding time and attitude information from the drone modules (UAV L and UAV R), performs image preprocessing and geometric registration, calculates the comprehensive saturation index of each pixel on both sides, optimizes dual-view pixels, and obtains the resulting image after invalid pixel reduction, along with bilateral abnormal pixel markers. After stitching, it outputs the resulting image covering the entire target area and the abnormal pixel distribution results. The data processing module can be deployed on a ground workstation, vehicle-mounted server, or cloud computing platform, exchanging data with the ground control module via wired or wireless networks.

[0025] It should be noted that, in this embodiment, apart from the UAV module, other modules can be implemented as independent hardware devices or integrated into the same platform as software functional modules. For example, the flight planning module and the data processing module can be integrated into the same ground workstation, and the synchronization triggering module can also be integrated with the ground control module into a comprehensive ground station system. During deployment, the physical deployment method of the modules can be replaced or combined according to actual needs without affecting the implementation of the technical solution of this invention.

[0026] Based on the above embodiments, this invention also provides a dual-UAV cooperative observation method for invalid pixel reduction, such as... Figure 2 As shown, it includes: Step S1: Obtain flight mission information for the target area, and determine the flight parameters of the two UAVs and the effective observation range of the two perspectives based on the desired ground resolution; Step S2: Discretly divide the target area into paired observation strips, and divide each paired observation strip into several observation segments along its length direction to generate a set of discrete observation tasks; Step S3: Based on the discrete observation task set, establish a dual-aircraft flight planning model, determine the optimal flight target, constraints, and decision variables, solve the dual-aircraft flight planning model, and form a dual-UAV escort mission script; Step S4: Send the dual-UAV escort mission script to the dual UAVs, control the dual UAVs to execute the flight mission, perform synchronous observation of the dual UAVs, and acquire synchronously collected data; Step S5: Perform geometric registration on the synchronously acquired data to complete pixel alignment, calculate the pixel saturation index, and obtain the result image of the target area and the abnormal pixel distribution results from the pixel saturation index.

[0027] Specifically, step S1 involves inputting task parameters, including: Acquire task information such as the observation area boundary, observation time window, solar azimuth and elevation angle, UAV and observation payload parameters, and complete the initialization settings of the observation geometry.

[0028] First, the observation flight altitude is calculated based on the expected ground resolution of the observed target and the focal length and pixel size of the observation payload lens. Based on the observed flight altitude and the horizontal field of view of the observation payload Calculate the width of a single ground section :

[0029] Based on this, such as Figure 3 As shown, the lateral distance between the optical axes of the two UAV sensors at the center of their ground projection is denoted as the lateral distance between the two UAVs. The angle formed by the zenith angle observed from the ground point directly below the midpoint of the line connecting the two cameras is denoted as the double-view angle. In this embodiment, the side-to-side spacing between the two machines... The requirements for flight safety and dual-view overlap quality must be met, and their value ranges are as follows: ,in, To ensure the minimum safe distance between the two machines, The minimum lateral overlap of the paired images is set.

[0030] Determine the side spacing between the two machines Afterwards, the angle between the two perspectives Based on the lateral spacing between the two machines With observation flight altitude H The conversion yields:

[0031] Based on the above parameters, such as Figure 4 As shown, the geometric overlap area of ​​the paired images from the two cameras, i.e., the dual-view observation range. Introducing the shrinkage coefficient By narrowing the field of view, an effective observation range from both perspectives can be established. Ensure 100% dual-view coverage within this area: Among them, the shrinkage coefficient The recommended value range is 0.85-0.95.

[0032] Step S2 generates the set of discrete observation tasks, including: In a flight path approximately perpendicular to the sun's azimuth, based on the width of the effective observation range of the dual-view system in step S1, the target area is discretized into several paired observation strips that are nearly perpendicular to the sun's azimuth, wherein the width of each paired observation strip is consistent with the width of the effective observation range of the dual-view system in step S1.

[0033] Set heading overlap Lateral overlap with paired observation strips Each pair of observation strips is further divided into several observation segments along its length.

[0034] The observation segment is denoted as ,in For the numbering of paired observation strips, This refers to the observation segment number within the corresponding paired observation strip. Each observation segment... A corresponding continuous ground region along the direction of the paired observation strips serves as the basic task unit for dual-aircraft synchronous observation, forming a set of discrete observation tasks. S : ,in, The total number of paired observation strips, For the first Number of observation segments within a paired observation strip.

[0035] Step S3 is the dual-aircraft flight planning, including: Based on the set of observation tasks given in step S2, dual-aircraft flight planning is performed, such as... Figure 5 As shown, step S3 can be further subdivided into sub-steps S301-S305: S301: Mission Objective Setting In this embodiment, the objective function can be selected as the shortest total operation time of the UAV, the lowest total energy consumption of the UAV, or the optimal weighted average of time and energy consumption, depending on the specific task.

[0036] S302: Constraint Analysis Considering requirements for flight safety, observation quality, and time, the constraints involved in dual-aircraft flight observation include, but are not limited to, the following: 1) Observation time window constraints For any observation segment Its starting time End time Must meet: ,in[ , [This represents the task time window.]

[0037] 2) Maximum flight time constraint per sortie Let the longest flight time of a single UAV in this embodiment be . For any given drone sortie, the flight time from takeoff to return cannot exceed [a certain limit]. .

[0038] 3) Dual-view effective observation constraints For each observation segment The requirement is that two drones complete the observation simultaneously, and the dual-view coverage within the effective observation range of the dual view must always remain at 100%.

[0039] 4) Overlap Constraints The data observed by the two drones match the pre-set heading overlap. FO Lateral overlap with paired observation strips .

[0040] 5) Observation order constraints The observation segments in each pair of observation strips must be executed in a pre-set order. The next observation segment can only begin after the previous one is completed. The pair of observation strips should also be completed in a pre-numbered order.

[0041] 6) Minimum safety clearance and turning constraints At any given time, the distance between the two aircraft shall not be less than the minimum safe distance. Especially during the turning phase between paired observation strips, such as Figure 6 As shown, the inner UAV R turns along a path with a smaller radius, while the outer UAV L turns along a path with a larger radius. The UAV that completes the turn first hovers at the waiting position. After both UAVs align at the starting position of the new strip, they enter the next observation segment together. Throughout the entire process, the distance between the two UAVs is maintained at no less than the minimum safe distance. .

[0042] 7) Return-to-base battery swapping constraints Battery swapping upon return is only permitted at the exit of the observation section. (The observation section is defined as follows.) S i,j The exit point is Then any return-to-base battery swap decision only applies to the point set. It happened above.

[0043] S303: Establishment of a Dual-Aircraft Flight Planning Model Based on the mission objectives and constraints, a dual-aircraft flight planning model is established.

[0044] The observation speed, turning speed, and turnaround speed of the UAV to and from the take-off and landing point are usually preset according to the performance and safety requirements of the UAV, and are used as known parameters in this embodiment. In other embodiments, these three speed parameters can also be optimized by enumeration or hierarchical methods, and this invention does not limit this. Based on this, this embodiment uses the start and end times of the observation segment and the return-to-base battery swapping node as decision variables for the dual-aircraft flight planning model. 1) Flight start and end times for each observation segment For each pair of observation strips and its internal observation section Set the observation start time respectively and the end time This indicates that the two drones, L and R, entered the observation segment simultaneously. S i,j The moment of departure and the moment of leaving.

[0045] 2) Return-to-base battery swapping node For each observation segment Set a Boolean variable between segments to indicate whether to return to base for battery swapping. ,when "At" indicates the completion of the observation segment. Afterwards, both drones returned simultaneously to the take-off and landing point for battery swapping, and then flew from the take-off and landing point to the starting point of the next observation segment; when When, it indicates from the observation segment Proceed directly to the next observation segment.

[0046] Based on the objective set in step S301, the objective function of the dual-machine planning model is given. Taking the weighted form as an example, it can be written as: ， in, T total Total time E total Total energy consumption, λ T and λ E These are the weighting coefficients.

[0047] The various constraints in step S302 are incorporated into the dual-aircraft flight planning model in mathematical form, including time windows, single-flight time, turning and holding strategies with a minimum safe distance, path constraints within the effective observation zone of the dual perspectives, heading and lateral overlap constraints, and constraints for returning to base for battery swapping at the exit of the observation segment.

[0048] Combining the above decision variables, objective function, and constraints, a dual-aircraft flight planning model is formed.

[0049] S304: Solving a Dual-Aircraft Flight Planning Model A centralized heuristic algorithm is used to solve the above planning model. The solution algorithm can be particle swarm optimization, simulated annealing, genetic algorithm or equivalent algorithm. This embodiment does not limit the specific solver. Under the premise of satisfying the constraints, the execution sequence of the observation segment of the two UAVs and the corresponding take-off time, return time and battery swapping arrangement are solved and obtained.

[0050] S305: Flight Planning Results Output Based on the solution results, a flight mission script and route file that can be directly issued are generated, including information such as the order of observation segments, waypoints, take-off, landing and return times, turning and holding strategies for each sortie, and the results are provided to the ground control module for dual-aircraft synchronous observation in step S4.

[0051] Step S4 is dual-machine synchronous observation, including: The flight planning results obtained in step S3 are sent to both UAVs. Based on the flight mission script and route file, the two UAVs are controlled to be at the same observation altitude. Predetermined dual-machine side spacing And set the observation course downwards, take off synchronously and conduct escort observations.

[0052] Within each observation segment, a unified trigger signal is sent to the observation payloads and positioning and attitude units of both machines through the synchronous trigger module, so that the left and right machines can acquire dual-view images and record attitude information at the same time. This forms a pair of images with consistent time within the effective observation range of the dual-view, providing a basis for subsequent pixel-by-pixel optimization.

[0053] Between adjacent pairs of observation strips, such as Figure 6 As shown, the two drones executed inward and outward turns according to the planned turning and waiting strategy. The drone on the inside completed the turn with a smaller turning radius and hovered at the waiting position, while the drone on the outside completed the turn with a larger turning radius. After the two drones aligned at the beginning of the new strip, they entered the next observation segment together. Throughout the entire process, the distance between the two drones was maintained at no less than the minimum safe distance. .

[0054] Step S5 involves pixel selection processing and result output, including: Based on the paired image data obtained from the dual-machine synchronous observation in step S4, pixel selection processing and result output are performed, such as... Figure 7 As shown, step S5 can be further subdivided into sub-steps S501-S505: S501: Data Preprocessing and Geometric Registration The paired images acquired simultaneously by the left and right cameras, along with their corresponding attitude, time, and other auxiliary information, are input into the data processing module. Radiometric preprocessing and geometric correction are performed on the images from both cameras. Geometric registration of the paired images is completed based on the synchronization trigger time and attitude information, ensuring that the same ground location corresponds to the same raster pixel position in both images. p For each raster cell position p The pixel values ​​of the left camera (L) and the right camera (R) can be obtained simultaneously, and are denoted as follows: ,in , Indicates band index, This indicates the total number of band indexes.

[0055] S502: Band-by-band saturation calculation For each pair of images from the left and right cameras, the radiometrically calibrated pixel values ​​for each band are... Linear normalization to [0,1] is calculated as follows: ,in, For band index, [ , [Band] The effective range.

[0056] Based on the above formula, the pixel saturation level of the left and right camera images is calculated according to the spectral band, and recorded as saturation indexes respectively. They were then organized into saturation grids for each band of the left and right cameras.

[0057] The calculation method provided in this embodiment is only one implementation of the saturation index for a single band. This invention does not limit its specific expression form, as long as it can reflect the degree of pixel saturation.

[0058] S503: Comprehensive Saturation Index and Validity Judgment Based on the saturation grids of each band of the left and right cameras, for each pixel position p The overall saturation index is calculated separately to determine whether the pixel exhibits flare or overexposure at that viewing angle. The overall saturation index can be a weighted sum of saturation across all bands, the maximum value of saturation across all bands, or other methods; this invention does not limit this approach. Taking the maximum value of saturation across all bands as an example, the overall saturation indices for the left and right cameras are as follows:

[0059]

[0060] Set the overall saturation threshold To determine validity, such as Figure 8As shown.

[0061] when When the pixel is determined to have severe saturation on both the left and right sides, it is recorded as a bilaterally anomalous pixel and is used in a two-choice mask. It is marked as 0 in the middle:

[0062] When the overall saturation on only one side is below the threshold, for example If the left side is considered valid and the right side is considered abnormal, then a two-choice mask is used. The middle pixel is the left-hand pixel:

[0063] Conversely, it is recorded as a right-side pixel:

[0064] When the combined saturation on both the left and right sides is below the threshold, that is If both left and right pixels are considered valid, proceed to the next step of optimization.

[0065] S504: Pixel-by-Pixel Optimization and Mask Generation For the grid cell positions that are valid on both the left and right sides determined in step S503 ,like Figure 8 As shown, this embodiment further optimizes pixel selection according to observation geometry, with the specific rules as follows: set up and These are the positions of the raster cells. Distance from the center line of the drone imagery on the left and right sides. For distance discrimination threshold, when When choosing a viewpoint, prioritize the side that is closer to looking directly downwards.

[0066] when If the observation geometry cannot clearly distinguish the two sides, then the selection will be made according to the preset fixed side (e.g., the left side will be fixed).

[0067] Based on the above rules, the positions of all raster cells are... Forming a two-choice mask This is used to indicate the source side and validity of each pixel, and its value represents a left-side camera pixel, a right-side camera pixel, or a bilaterally anomalous pixel, respectively. ,in, This indicates that the left-side pixel is selected. 0 indicates the selection of right-side pixels, and 0 indicates bilateral abnormal pixels.

[0068] S505: Splicing Process and Result Output Under the same mask constraint obtained in step S504, pixel-level reassembly is performed to ensure that all band data at each pixel location come from the same UAV, avoiding cross-UAV data mixing, and obtaining the resulting image after invalid pixel reduction and the labeling results of bilateral abnormal pixels; steps S501-S504 and the above-mentioned data based on all paired data in the observation area are repeated. The mask selection process is then performed. Based on this, the resulting images are stitched together in a unified coordinate system to output a result image covering the entire target area. The anomalous pixel markers on both sides are then stitched together synchronously according to the same geometric relationship to output the anomalous pixel distribution result of the entire target area.

[0069] To further illustrate the processing flow and technical effects of steps S501-S505 of the present invention, this embodiment constructs a set of simulation data based on dual-view observation geometry for verification and explanation. It should be noted that this embodiment takes a single set of left-right paired images as an example, focusing on demonstrating the comprehensive saturation determination and two-choice mask generation process in steps S501-S504, as well as the resulting image and bilateral anomalous pixel marking results obtained after pixel-level reconstruction under the same mask constraint in step S505. For the multi-set paired image stitching output facing the entire observation area in step S505, the above processing can be repeated on all paired data, and further completed in a unified coordinate system; this embodiment will not elaborate further.

[0070] The simulation scenario in this embodiment corresponds to high-reflectivity, low-texture target areas such as water surfaces, salt fields, tidal flats, or the surfaces of photovoltaic modules. These types of targets are prone to flare and overexposure under solar illumination, resulting in invalid pixels in the image, consistent with the problem scenario addressed by this invention. It should be noted that the images and statistical data in this embodiment are simulation verification results constructed based on the technical solution of this invention, used to illustrate the technical effectiveness of this invention in typical high-reflectivity, low-texture scenarios.

[0071] This embodiment constructs a simulated scene image with a size of 320×480 pixels, sets the ground sampling distance GSD to 0.10m / pixel, and the corresponding scene coverage area is approximately 32m×48m; the image has 3 bands, corresponding to the red, green, and blue bands respectively. First, a basic scene image is constructed, such as... Figure 9The base image, shown in the image, is designed to simulate the idealized brightness distribution of the target area without superimposed flare perturbations from a specific viewing angle. The base image includes a weakly textured background, a gradually varying brightness distribution, and locally high-reflectivity areas. Subsequently, flare / overexposure distributions with slight offsets are superimposed from both left and right viewing angles to create separate imaging results for the left and right cameras. A small number of common high-brightness areas are also included to simulate pixel regions that may fail in both left and right viewing angles, such as... Figure 9 As shown.

[0072] The simulated imaging results on both sides are processed according to the procedures described in steps S501-S505 of this invention. First, the paired images on the left and right sides are preprocessed and geometrically registered so that the same ground location corresponds to the same pixel location in the left and right images. Then, the pixel values ​​of each band are normalized to the interval [0,1] to obtain the band-by-band saturation index. Further, the maximum value of the saturation of each band is used as the comprehensive saturation index, and the comprehensive saturation threshold Tsat=0.93 is set. When the comprehensive saturation of only one side of a pixel location is lower than the threshold, the side with the lower threshold is selected as the result pixel. When the comprehensive saturation of both sides is lower than the threshold, the selection is made according to the observation geometry optimization rule. When the comprehensive saturation of both sides is not lower than the threshold, the pixel is marked as a bilateral anomalous pixel and marked as 0 in the two-choice mask. For positions where both sides contain valid pixels, this embodiment sets a geometric preference threshold delta_px = 6 pixels to define a region with insignificant geometric differences near the image centerline. When a pixel falls into this region, a preset side bias is used for stable selection. Through the above processing, the following is obtained: Figure 10 The two-choice mask shown is used to perform pixel-level reconstruction under the same mask constraint, ensuring that all band data at each pixel location comes from the same side image, avoiding cross-camera image mixing, thus obtaining the resulting image after invalid pixel subtraction and the bilateral abnormal pixel marking results, such as... Figure 9 As shown.

[0073] Based on reproducible simulation data generated under fixed random seed conditions, statistical analyses were performed on the results of single imaging by the left camera, single imaging by the right camera, and the dual-view fusion results of this invention. The statistical results show that the invalid pixel rate of the single imaging result by the left camera is 9.1354%, the invalid pixel rate of the single imaging result by the right camera is 8.6035%, and the invalid pixel rate of the dual-view fusion result of this invention is 5.3034%, corresponding to an effective pixel rate of 94.6966%. Furthermore, compared to the single imaging result by the left camera, the invalid pixel rate of the fusion result of this invention is reduced by approximately 41.9470%; compared to the single imaging result by the right camera, the invalid pixel rate of the fusion result of this invention is reduced by approximately 38.3579%. These results indicate that, under conditions of spatial offset in the flare regions of the left and right views, this invention, by acquiring paired images through simultaneous dual-camera observation and combining comprehensive saturation determination, pixel-by-pixel selection, and same-mask pixel-level reassembly processing in step S505, can effectively reduce the proportion of invalid pixels, improve the effective coverage of the resulting images, and retain bilateral anomalous pixel markers for subsequent quality assessment. Therefore, compared with single-view imaging, the present invention can more stably reduce invalid pixels caused by flares.

[0074] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communications bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other through the communications bus 1140. The processor 1110 can call logic instructions in the memory 1130 to execute a dual-UAV collaborative observation method for invalid pixel reduction. This method includes: acquiring flight mission information for the target area; determining dual-UAV flight parameters and effective observation range based on the desired ground resolution; discretely dividing the target area into paired observation strips, further dividing each paired observation strip into several observation segments along its length to generate a discrete observation task set; establishing a dual-UAV flight planning model based on the discrete observation task set, determining the optimal flight target, constraints, and decision variables, solving the dual-UAV flight planning model, and forming a dual-UAV escort mission script; sending the dual-UAV escort mission script to the dual UAVs, controlling the dual UAVs to execute flight missions, synchronously observing the dual UAVs, and acquiring synchronously collected data; performing geometric registration on the synchronously collected data, completing pixel alignment, calculating pixel saturation indices, and obtaining the resulting image of the target area and the distribution results of abnormal pixels from the pixel saturation indices.

[0075] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0077] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual-UAV cooperative observation system for invalid pixel reduction, characterized in that, It includes a flight planning module, a ground control module, a synchronization triggering module, an unmanned aerial vehicle (UAV) module, and a data processing module. The modules are connected to each other via wireless or wired communication. The flight planning module determines the set of observation data based on the received mission parameters, establishes a dual-aircraft flight planning model, outputs the missions of each observation segment of the UAV module, and generates a flight mission script. The ground control module receives the flight mission script, sends the tasks of each observation segment to the UAV module, controls the flight of the UAV module, collects the status information of the UAV module, and sends trigger control commands. The synchronization triggering module receives the triggering control command, sends a triggering signal to the UAV module, synchronizes the UAV module's image acquisition and attitude information recording at the same time, and sends triggering event information to the ground control module; The UAV module includes two UAVs, which synchronously execute observation tasks according to the tasks of each observation segment and transmit the collected observation information back to the data processing module. The data processing module completes image geometric registration and pixel alignment based on the observation information, performs optimization of dual-view pixels and bilateral abnormal pixel marking, and obtains a result image in which invalid pixels have been reduced and the corresponding mask file. After stitching, it outputs a result image covering the entire target area and the distribution result of abnormal pixels.

2. The dual-UAV cooperative observation system for invalid pixel reduction according to claim 1, characterized in that, The flight planning module is deployed on a ground control computer, vehicle-mounted terminal, or server, and interacts with the ground control module through mission files or network interfaces; The ground control module is implemented using a handheld remote controller in conjunction with ground station software, or integrated into a vehicle-mounted control terminal or portable ground workstation. It is wirelessly connected to the UAV module and wirelessly or wiredly connected to the flight planning module and the data processing module. The synchronization triggering module is implemented using a hardware trigger line, a wireless trigger signal, or a network time synchronization method, and shares the trigger time tag with the ground control module. The data processing module is deployed in a ground workstation, vehicle-mounted server, or cloud computing platform, and is connected to the ground control module via wired or wireless communication.

3. A dual-UAV cooperative observation method for invalid pixel reduction, based on the dual-UAV cooperative observation system for invalid pixel reduction as described in claim 1 or 2, characterized in that, include: Acquire flight mission information for the target area, and determine the flight parameters of the two UAVs and the effective observation range of the two perspectives based on the desired ground resolution; The target area is discretized into paired observation strips, and each paired observation strip is further divided into several observation segments along its length, thus generating a set of discrete observation tasks. A dual-aircraft flight planning model is established based on the discrete observation task set. The optimal flight target, constraints, and decision variables are determined. The dual-aircraft flight planning model is solved to form a dual-UAV escort mission script. The script for the dual-UAV escort mission is sent to the dual UAVs, controlling them to execute the flight mission, synchronously observing the dual UAVs, and acquiring synchronously collected data. Geometric registration is performed on the synchronously acquired data to complete pixel alignment. Pixel saturation index is calculated, and the resulting image of the target area and the distribution results of abnormal pixels are obtained from the pixel saturation index.

4. The dual-UAV cooperative observation method for invalid pixel reduction according to claim 3, characterized in that, Acquire flight mission information for the target area, and determine the flight parameters and effective observation range of the two UAVs based on the desired ground resolution, including: The observation flight altitude is calculated based on the expected ground resolution of the observed target, the focal length of the observation payload lens, and the pixel size. The single-frame ground swath width is calculated based on the observation flight altitude and the horizontal field of view of the observation payload. The lateral spacing between the two drones is obtained at the center of the ground projection of the optical axis of the two drones. The lateral spacing between the two drones is between the minimum safe spacing between the two drones, the minimum lateral overlap of the paired images, and the upper limit of the spacing between the two drones determined by the width of a single ground frame. The angle between the observation zenith angles of the ground point directly below the midpoint of the line connecting the two UAVs is obtained from the lateral distance between the two UAVs and the observation flight altitude. The geometric overlap area of ​​the paired images from two UAVs is determined as the dual-view observation range. The dual-view observation range is then shrunk by a shrinkage coefficient to obtain the effective dual-view observation range.

5. The dual-UAV cooperative observation method for invalid pixel reduction according to claim 3, characterized in that, The target region is discretized into paired observation strips, and each paired observation strip is further divided into several observation segments along its length, generating a set of discrete observation tasks, including: Based on the effective observation range of the dual-view system, the target area is discretized into several paired observation strips that are approximately perpendicular to the sun's azimuth. Set the forward overlap and the lateral overlap between paired observation strips, and divide each paired observation strip into several observation segments along its length. Each observation segment corresponds to a continuous ground area along the direction of the paired observation strips, which serves as the basic task unit for simultaneous observation by dual UAVs, forming the discrete observation task set.

6. The dual-UAV cooperative observation method for invalid pixel reduction according to claim 3, characterized in that, A dual-aircraft flight planning model is established based on the discrete observation task set. The optimal flight objective, constraints, and decision variables are determined. The dual-aircraft flight planning model is solved to form a dual-UAV escort mission script, including: The objective function is to minimize the total operation time of the drone, the total energy consumption of the drone, or the weighted optimal balance between time and energy consumption. Taking into account flight safety, observation quality, and time requirements, the following constraints were determined: observation time window constraint, longest flight time per sortie constraint, effective observation from dual perspectives constraint, overlap constraint, observation sequence constraint, minimum safe distance and turning constraint, and return-to-base for battery swapping constraint. Using the start and end times of the observation segment and the return-to-base battery swapping node as decision variables, a dual-aircraft flight planning model is established by integrating the objective function, constraints, and decision variables. A centralized heuristic algorithm is used to solve the dual-drone flight planning model to obtain the observation segment execution sequence of the two UAVs and the corresponding take-off time, return time and battery swapping schedule. The script and flight path file for the dual UAV escort mission are generated based on the model solution results.

7. The dual-UAV cooperative observation method for invalid pixel reduction according to claim 3, characterized in that, The script for the dual-UAV escort mission is sent to the two UAVs, controlling them to execute the flight mission, synchronously observing the two UAVs, and acquiring synchronously collected data, including: Based on the dual-UAV escort mission script and flight path file, control the two UAVs to take off and escort observation synchronously at the same observation altitude, the same lateral distance between the two UAVs and the set observation flight direction. Within each observation segment, a trigger signal enables the two UAVs to acquire dual-view images and record attitude information simultaneously. Between adjacent paired observation strips, the two UAVs perform inward and outward turns according to the planned turning and waiting strategy. The UAV on the inside completes the turn with a smaller turning radius and hovers at the waiting position, while the UAV on the outside completes the turn with a larger turning radius. After the two UAVs align at the starting position of the new strip, they enter the next observation segment together. Throughout the process, the distance between the two UAVs is not less than the minimum safe distance between the two UAVs.

8. The dual-UAV cooperative observation method for invalid pixel reduction according to claim 3, characterized in that, Geometric registration is performed on the synchronously acquired data to complete pixel alignment. A pixel saturation index is calculated, and the resulting image of the target area and the distribution of abnormal pixels are obtained from the pixel saturation index, including: Acquire dual-view images and attitude information of two UAVs at the same time, perform radiometric preprocessing and geometric correction on the dual-view images and attitude information, so that the same ground position corresponds to the same grid cell position of the dual-view images, and each grid cell position includes the pixel value of the first UAV and the pixel value of the second UAV. The pixel values ​​of each band radiometrically calibrated in the dual-view image are linearly normalized. Based on the linearly normalized pixel values, the saturation of each pixel in the dual-view image is calculated band by band to generate the saturation raster of each band of the dual UAV. Based on the saturation grid of each band of dual UAVs, the comprehensive saturation index of each grid cell position is calculated, and the abnormal cells on both sides are screened according to the comprehensive saturation threshold to obtain the effective grid cell positions. For all valid raster cell positions, based on the observation geometry and the preset fixed side, pixel-by-pixel optimization is performed to generate a result mask file; Constraints are applied using the result mask file, and pixel-level reconstruction is performed to obtain the result image after invalid pixel reduction and bilateral abnormal pixel markers. Geometric registration, pixel alignment, calculation of pixel saturation index, and pixel-by-pixel comparison are repeated for all paired data in the target area. The result images are then stitched together using a unified coordinate system, and the result image of the target area is output. The bilateral abnormal pixel markers are then stitched together synchronously according to the same geometric relationship, and the abnormal pixel distribution result of the target area is output.

9. The dual-UAV cooperative observation method for invalid pixel reduction according to claim 8, characterized in that, For all valid raster cell positions, based on the observation geometry and a preset fixed side, a cell-by-cell optimization is performed, generating a result mask file, including: Obtain the first and second distances of the raster pixel positions from the center line of the dual UAV image, and determine the distance partition threshold; If the absolute value of the difference between the first distance and the second distance is greater than the distance partition threshold, the side with a viewpoint closer to the direct downward viewpoint is selected; otherwise, the selection is made according to the preset fixed side. A two-choice mask is formed for all grid cell positions, the two-choice mask including a first UAV cell, a second UAV cell, and bilateral anomalous cells.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the dual-UAV cooperative observation method for invalid pixel reduction as described in any one of claims 3 to 9.

Images