An implementation method of occluded target optical calculation imaging based on a drone cluster

Abstract

The application discloses an implementation method of occluded target optical calculation imaging based on a UAV cluster, which comprises the following steps: obtaining the offset direction and offset amount of each camera in a camera array based on the calibration parameters of the camera array of the UAV cluster; guiding all the cameras to the same imaging area as a reference camera based on the offset direction and offset amount of each camera; and obtaining the online fine calibration parameters of each camera in the camera array during the movement process, wherein the online fine calibration parameters are composed of a pixel mapping matrix. Based on the calibration parameters and the online fine calibration parameters, the image of the occluded target is obtained. The application can adaptively adjust the camera array carried by the UAV cluster to realize the optical imaging of the occluded or hidden target in the case that the target is occluded.

Description

Technical Field

[0001] This invention relates to the field of computational imaging technology for occluded objects, and in particular to a method for realizing optical computational imaging of occluded targets based on a swarm of unmanned aerial vehicles (UAVs). Background Technology

[0002] Optical imaging technology projects a three-dimensional scene onto a two-dimensional sensor using an optical imaging system, thereby obtaining optical information associated with the three-dimensional scene. With the development of optical design and sensor technology, camera technology has permeated every corner of life. However, real-life scenes are three-dimensional, and due to the rectilinear propagation of light, traditional cameras cannot acquire optical images of hidden objects. To address this, X-ray detection, computational imaging, and photon imaging technologies have been proposed, each capable of imaging hidden objects under specific conditions. However, these methods either utilize the penetrating power of electromagnetic waves, rely on extensive correlation calculations of the captured data, or require extremely high sensitivity from the sensor, making it impossible to efficiently image hidden objects using visible light.

[0003] Computational imaging technology for obstructed objects is frequently used in synthetic aperture radar (SAR) detection. Leveraging the penetrating power of the electromagnetic bands employed by SAR, it provides good imaging results even in scenarios obscured by clouds and fog, finding wide application in air defense early warning, situational awareness, and maritime monitoring. However, research and inventions on imaging technologies for obstructed targets, particularly in the visible light band, are relatively limited. In the field of optical imaging of obstructed targets, optical synthetic aperture technology, represented by light field computation and integrated imaging computation, offers a new technical approach to achieving optical imaging of obstructed targets.

[0004] Light field computation, as a method for processing seven-dimensional light fields, overturns the drawback of lost scene depth in single-lens imaging, enabling the recording and analysis of light field information from the real world. Because light field computation simultaneously records the angular information of light rays, and then uses light field computation algorithms to reconstruct the depth of these rays, it can calculate and synthesize images corresponding to imaging lenses with focal lengths, apertures, or aperture parameters that do not exist in reality. This computational imaging method overcomes the limitation of traditional imaging where aperture and depth of field cannot be simultaneously achieved, enabling ultra-depth-of-field imaging in ultra-large aperture shooting environments. Furthermore, light field imaging technology based on the synthetic aperture principle, due to its large optical equivalent aperture, can "penetrate" sparsely occluded objects and focus on the target object. Although light field computational imaging technology achieves the synthesis of surreal lenses, its fixed camera lens and incompatibility with traditional two-dimensional imaging methods limit its application in flexible UAV target optical imaging scenarios.

[0005] Integrated imaging computation, as a method for acquiring light field information in 3D scenes, has high practical value in fields such as true 3D display, 3D reconstruction, and synthetic aperture camera computational imaging. The information in integrated imaging computation comes from its micro-image array, which is obtained through microlens arrays, camera arrays, and computer rendering. During the acquisition of integrated imaging micro-image arrays, the number of cameras is enormous, and it is difficult to achieve precise physical alignment of the relative spatial positions and optical axes of different cameras. In traditional integrated imaging camera array shooting, calibration boards are needed to correct the images captured by the camera array to overcome shooting errors caused by differences in the poses of different cameras. However, traditional image correction methods for integrated imaging camera arrays are limited by the size of the calibration board, making it impossible to capture large and ultra-large 3D scenes. This severely limits the applicability and practicality of integrated imaging camera array shooting, thus restricting the data source for integrated imaging micro-image arrays and preventing their application in UAV swarm scenarios.

[0006] Given that there is currently no effective solution for imaging optically occluded targets based on flexible motion platforms such as UAV swarms, the challenge of overcoming the limitation that UAV optoelectronic pods cannot image occluded or hidden targets in various environments, enabling them to image occluded targets with flexible imaging pose requirements and simple and accurate computational imaging processes, is a hot topic of widespread interest among those skilled in the art. Summary of the Invention

[0007] To overcome the limitations of current UAV optoelectronic pods in imaging targets that are obscured or hidden in various environments, this invention provides a method for optical computational imaging of obscured targets based on UAV swarms.

[0008] This invention discloses a method for optical computational imaging of occluded targets based on a drone swarm, comprising:

[0009] Step 1: Based on the calibration parameters of the camera array of the drone swarm, obtain the offset direction and offset amount corresponding to each camera in the camera array; wherein, each drone in the drone swarm carries a camera, and all the cameras carried by the drone swarm constitute a camera array.

[0010] Step 2: Based on the offset direction and offset amount corresponding to each camera, guide all cameras to the same imaging area as the reference camera, which is any camera specified in the camera array;

[0011] Step 3: Obtain the online fine calibration parameters of each camera in the camera array during the motion process; wherein, the online fine calibration parameters are composed of a pixel mapping matrix;

[0012] Step 4: Based on the calibration parameters and the online fine calibration parameters, obtain the image of the occluded target.

[0013] Further, step 1 includes:

[0014] Step 11: Calculate the calibration parameters of the camera array; wherein the calibration parameters include imaging extrinsic parameters, imaging intrinsic parameters, and offline coarse mapping parameters;

[0015] Step 12: Determine the spatial relationship between the camera array and the obstruction / target;

[0016] Step 13: Based on the spatial positional relationship, calculate the offset direction and offset amount for each camera.

[0017] Further, step 11 includes:

[0018] The camera array consists of M×N cameras, which capture images of the checkerboard calibration board to obtain M×N corresponding calibration disparity images. The resolution of each calibration disparity image is W. r ×H r Detect the pixel coordinates of checkerboard corner points in the calibrated parallax image;

[0019] Based on the corner coordinates in each calibration board image, the imaging extrinsic and intrinsic parameters of the cameras in the m-th column and n-th row of the camera array are obtained using Zhang's calibration method; wherein, the imaging extrinsic parameters include the camera rotation matrix R. m,n Translation vector t m,n The imaging intrinsic parameters include an intrinsic parameter matrix K composed of the camera's focal length and principal point offset. m,n ;

[0020] Based on the principle of homography transformation, the homography transformation matrix corresponding to each camera in the camera array is calculated and used as the offline coarse mapping parameter between the camera and the reference camera; wherein, the homography transformation matrix is:

[0021]

[0022] Among them, H m,n R0 is the homography transformation matrix, R0 is the rotation matrix in the imaging extrinsic parameters of the reference camera, and m and n are the index values ​​of the cameras in the m-th column and n-th row of the camera array, respectively, where m∈{1,2,3,…,M} and n∈{1,2,3,…,N}.

[0023] Further, step 12 includes:

[0024] The distance ΔD between the occluding target in the target scene area and the camera array is obtained through measurement. hThe spatial distance between any two cameras in the camera array is the same, ΔC; the target scene area includes occlusions and occluding targets.

[0025] Based on the dimensions of the checkerboard calibration board, calculate the size W of the occluded scene captured by the camera array on the plane where the checkerboard calibration board is located. b ×H b .

[0026] Further, the offset direction is denoted as θ. m,n The offset is denoted as S. m,n And respectively satisfy:

[0027] θ m,n =(θ x ,θ y ) m,n

[0028] S m,n =(S x ,S y ) m,n

[0029] Where, θ x θ y The offset directions are θ. m,n In the x-axis and y-axis components, S x S y Offset S m,n Components on the x-axis and y-axis;

[0030] θ x θ y S x S y They respectively satisfy:

[0031]

[0032]

[0033]

[0034]

[0035] Here, round is a function that rounds a numerical value to the nearest integer.

[0036] Further, step 2 includes:

[0037] The system calculates guidance information for the target scene area relative to each camera coordinate system to provide guidance information for the drone operator; wherein, the guidance information includes the azimuth angle Y of the target scene area in the camera coordinate system. tPitch angle P t ;

[0038] Among them, azimuth angle Y t and pitch angle P t They are represented as follows:

[0039] Y t =arctan((Lon t -Lon p )*cos(Lat p ),(Lat t -Lat p ))

[0040]

[0041] Among them, (Lon) p ,Lat p Alt p () represents the geographic coordinates of the UAV platform, derived from the UAV platform's longitude (Lon). p Latitude p and height Alt p Composition, (Lon t ,Lat t Alt t The geographic coordinates of the obscured target are indicated by the longitude of the obscured target. t Latitude t and height Alt t Composition: dist is a function for calculating the distance between two coordinate points.

[0042] Further, step 3 includes:

[0043] The images captured by each camera in the camera array are matched pixel-level with the occlusions between the reference camera and the camera; wherein the pixel-level matching relationship between each camera and the reference camera is represented by a pixel mapping matrix. Let m and n represent the index values ​​of the cameras in the m-th column and n-th row of the camera array, respectively, where m∈{1,2,3,…,M} and n∈{1,2,3,…,N}.

[0044] Furthermore, the pixel-level matching process is as follows:

[0045] The SIFT, SURF, or ORB algorithms are used to extract feature points of the images that require pixel-level matching. The feature points of different images are then matched, and the RANSAC algorithm is used to remove incorrect matching points.

[0046] Further, step 4 includes:

[0047] Step 41: Acquire images of the target scene area using the camera array;

[0048] Step 42: Based on the homography transformation matrix and pixel mapping matrix, calculate the corrected image corresponding to the target scene area image captured by each camera;

[0049] Step 43: Based on the corrected image and the offset direction and offset amount corresponding to each camera, calculate the offset image corresponding to the target scene area image captured by each camera;

[0050] Step 44: Based on all offset images, calculate the image of the occluded target in the target scene region image.

[0051] Furthermore, the corrected image and its corresponding target scene region image captured by the camera satisfy the following:

[0052] I' m,n (x′,y′)=I m,n (x,y)

[0053] in:

[0054]

[0055] Among them, I' m,n (x',y') is the corrected image, I m,n (x,y) represents the target scene region image captured by the camera in the m-th row and n-th column of the camera array, where x and y are the pixel coordinates of the target scene region image, and x' and y' are the pixel coordinates of the corrected image.

[0056] In step 43:

[0057] The offset image is I” m,n (x”,y”),I″ m,n (x″, y″) = I′ m,n (x′,y′),

[0058]

[0059] Where x” and y” are the pixel coordinates of the offset image, respectively, and S x Offset S m,n The direction of the center is θ x The offset, S y Offset S m,n The direction of the center is θ y The offset;

[0060] In step 44:

[0061] The optical image of the obstructed target is:

[0062]

[0063] Where O(x”,y”) is the optical image of the occluded target.

[0064] Because of the adoption of the above technical solution, the present invention has the following advantages:

[0065] (1) It has the ability to image obscured objects. This invention enables drone swarm cameras to "penetrate" obscured objects and focus on the target object, a capability that traditional photoelectric cameras do not possess.

[0066] (2) Flexible camera pose requirements. This invention provides an online calibration method for UAV swarm cameras, which enables UAV swarm cameras to acquire image data of the target scene during dynamic operation. Based on the scene content, it dynamically aligns and corrects the images of each camera, thereby providing online precision calibration parameters for the computational imaging process of optically occluded objects. The flexibility of this invention has significant advantages over light field cameras and integrated imaging camera arrays.

[0067] (3) The computational imaging process is simple and accurate. This invention uses UAV cluster cameras to acquire the offset photoelectric image corresponding to the photoelectric image of the target scene area, and calculates the optical image of the occluded target. This calculation process has no other dependencies and does not need to consider the imaging crosstalk problem in the light field camera, which greatly simplifies the calculation process.

[0068] (4) Adjustable imaging depth range. In this invention, when the distance between the occluding object and the photoelectric camera changes, the same calculation process is repeated to perform computational imaging on the occluded target after the scene changes. Compared with the occluded object imaging method based on integrated imaging calculation, this beneficial effect eliminates the need to set an initial depth plane, and the imaging depth can be set and adjusted as needed.

[0069] (5) This invention is applicable to application scenarios such as dynamic and complex environments, target occlusion and concealment conditions, target search guidance, and photoelectric target recognition and confirmation. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0071] Figure 1 This is a schematic diagram of a scenario for an optical computational imaging method for occluded targets based on a drone swarm, according to the present invention.

[0072] Figure 2 This is a flowchart of an optical computational imaging method for occluded targets based on a drone swarm, according to the present invention.

[0073] Figure 3 This invention describes the online calibration process for drone cluster cameras.

[0074] Figure 4 This invention describes the computational imaging process for optically obstructed objects.

[0075] Figure 5 This is a result diagram of an embodiment of the optically occluded object imaging method of the present invention.

[0076] The reference numerals are as follows: 1 UAV 1-4, 2 field of view of electro-optical cameras 1-4, 3 scene with occluded target, 4 occluded target 1, 5 occluded target 2, 6 electro-optical camera 1, 7 electro-optical camera 2, 8 electro-optical camera 3, 9 electro-optical camera 4, 10 image of electro-optical camera 1, 10 image of electro-optical camera 2, 12 image of electro-optical camera 3, 13 reference electro-optical camera image, 14 optically calculated image of occluded target. Detailed Implementation

[0077] The present invention will be further described in conjunction with the figures and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of the present invention.

[0078] The method proposed in this invention can be described using a scenario consisting of a drone swarm, a drone's electro-optical camera, a scene with obstructed targets, and a scenario composed of both obstructed targets. Figure 1 As shown, a swarm of drones uses mounted electro-optical cameras to perform optical imaging of a target region of interest. However, because the drone swarm and the target region of interest are obstructed by the scene, the target region of interest becomes an obstructed target. The electro-optical cameras mounted on the drones cannot obtain a complete optical image of the obstructed target, making it impossible to obtain detailed information about the target using any single drone's electro-optical camera. In this scenario, the drone swarm is dynamically dispersed on one side of the scene obstructing the target. How to utilize the imaging data from the dispersed drone swarm's electro-optical cameras to achieve computational optical imaging of the obstructed target is the problem scenario addressed by this invention.

[0079] See Figure 2 The embodiments of the present invention include the following processes:

[0080] The process consists of four main steps: offline calibration of UAV swarm cameras, target scene indication and guided imaging, online calibration of UAV swarm cameras, and computational imaging of optically occluded objects.

[0081] The offline calibration process for the UAV swarm camera array first involves calculating the calibration parameters of the UAV swarm camera array. The UAV swarm camera array is then initially adjusted so that the shooting range of each camera covers the space where the checkerboard calibration board is located, and the distance between the UAV swarm camera array and the checkerboard calibration board is ΔD. b In a drone swarm, the number of cameras is M×N. M×N calibration disparity images are obtained by capturing images of a checkerboard calibration board. The resolution of each calibration disparity image is W. r ×H r The pixel coordinates of checkerboard corner points in the calibrated parallax image are detected. Based on Zhang's calibration method, the imaging extrinsic and intrinsic parameters of the UAV camera in the m-th column and n-th row are obtained. Among them, the imaging extrinsic parameters include the camera rotation matrix R. m,n Translation vector t m,n The imaging intrinsic parameters include the intrinsic parameter matrix K, which is composed of the camera focal length and the principal point offset. m,n The rotation matrix in the imaging extrinsic parameters of the reference camera is denoted as R0. The reference camera is any camera arbitrarily selected within the UAV swarm camera array. Based on the homography transformation principle, the homography transformation matrix H corresponding to each camera in the UAV swarm camera array is calculated. m,n H serves as the offline coarse mapping parameter between the drone camera and the benchmark drone camera. m,n Represented as:

[0082]

[0083] Where m and n are the indices of the cameras in the m-th column and n-th row of the camera array, respectively, m∈{1,2,3,…,M}, n∈{1,2,3,…,N}. Then, the spatial relationship between the drone swarm camera array and occluded / hidden objects is determined. The distance ΔD between the hidden objects in the occluded scene and the drone swarm camera array is obtained through measurement. h In a drone swarm camera array, the spatial distance between adjacent cameras in each row and column is the same, ΔC. Figure 3 As shown. Simultaneously, based on the dimensions of the checkerboard calibration board, the size W of the occlusion area captured by the UAV swarm camera array on the plane where the checkerboard calibration board is located is calculated. b ×H b Finally, based on the position of each camera in the drone swarm camera array and the spatial relationship between the occluded scene and the hidden objects, the offset direction and offset amount for each camera are calculated. The offset direction is denoted as θ. m,n The offset is denoted as S. m,n And respectively satisfy:

[0084] θ m,n =(θ x,θ y ) m,n

[0085] S m,n =(S x ,S y ) m,n

[0086] Where, θ x θ y The offset directions are θ. m,n In the x-axis and y-axis components, S x S y Offset S m,n Components along the x and y axes. θ x θ y S x S y They respectively satisfy:

[0087]

[0088]

[0089]

[0090]

[0091] Here, round(*) means rounding the nearest integer to the nearest integer.

[0092] The target scene indication and guidance imaging process aims to guide all cameras to the same imaging area as the reference camera. This process calculates the indication and guidance information of the target scene area relative to the coordinate system of each photoelectric camera. This indication and guidance information includes the azimuth angle γ of the target scene area in the photoelectric camera coordinate system. t Pitch angle P t This guidance information is quantitative and presented visually to provide intuitive guidance to the drone operator. Specifically, the target scene area is located at the azimuth angle Y in the photoelectric camera coordinate system. t and pitch angle P t The slant range D is obtained by combining the geographic coordinates of the target scene area and the photoelectric camera. t The magnitude of this vector is determined. The azimuth angle Y of the target scene region in the coordinate system of the photoelectric pod camera is [missing information]. t and pitch angle P t for:

[0093] Y t =arctan((Lon t -Lon p )*cos(Latp ),(Lat t -Lat p ))

[0094]

[0095] Among them, (Lon) p ,Lat p Alt p () represents the geographic coordinates of the UAV platform, derived from the UAV platform's longitude (Lon). p Latitude p and height Alt p Composition, (Lon t ,Lat t Alt t The geographic coordinates of the target are indicated by the longitude of the target scene area. t Latitude t and height Alt t The function dist(A,B) is used to calculate the distance between points A and B.

[0096] The online calibration process for UAV swarm cameras refers to the process where the UAV swarm cameras acquire image data of the target scene during dynamic operation, dynamically align and correct the images of each camera according to the scene content, and thus provide online fine calibration parameters for the computational imaging process of optically occluded objects. These online fine calibration parameters are derived from the pixel mapping matrix. The process involves several steps. In this process, the scene obscuring the target occupies the vast majority of pixels in the image from the photoelectric camera, forming the foreground of the scene. The obscuring target occupies a very small number of pixels, forming the background. The online calibration process for UAV photoelectric cluster cameras first achieves pixel-level matching of the foreground between images. In this invention, pixel-level matching of target scene area images acquired by different UAV photoelectric cameras is achieved through multi-view image matching technology. Feature points in the images are extracted using SIFT, SURF, or ORB algorithms, and erroneous matching points are removed using the RANSAC algorithm. This allows for the calculation of the matching relationship between each image and the target scene area image acquired by the reference camera. The matching relationship between each UAV camera and the reference camera is represented by a pixel mapping matrix. Let m and n be the index values ​​of the cameras in the m-th column and n-th row of the camera array, respectively, where m∈{1,2,3,…,M} and n∈{1,2,3,…,N}.

[0097] The optically obstructed object computational imaging process first involves acquiring photoelectric images of the target scene area using a cluster of UAV cameras, such as... Figure 4 As shown, the photoelectric image resolution of the target scene area is also W.r ×H r The photoelectric image of the target scene area captured by the camera in column m and row n is I. m,n (x, y), where x and y are the pixel coordinates of the photoelectric image of the target scene region, respectively. Using the corresponding homography transformation matrix H... m,n and pixel mapping matrix The corresponding corrected image I' is calculated. m,n (x',y'), I' m,n (x',y') and I m,n (x,y) satisfies:

[0098] I' m,n (x′,y′)=I m,n (x,y)

[0099] in:

[0100]

[0101] Then, based on the camera's corresponding offset direction θ m,n and offset S m,n Calculate the corresponding offset photoelectric image I” m,n (x”,y”),I” m,n (x”,y”) and I' m,n (x', y') satisfies:

[0102] I″ m,n (x″, y″) = I′ m,n (x′,y′)

[0103] in:

[0104]

[0105] Where x” and y” are the pixel coordinates of the offset image, respectively, and S x It is the offset S m,n The direction of the center is θ x The offset, S y It is the offset S m,n The direction of the center is θ y The offset.

[0106] Preferably, when x” does not satisfy x”∈{1,2,3,…,W r}, or y” does not satisfy y”∈{1,2,3,…,H r When calculating the pixel coordinates, skip the calculation to avoid overflow of the pixel coordinate calculation range. Finally, use the offset photoelectric image I” m,n (x”,y”), calculate the optical image O(x”,y”) of the occluded target:

[0107]

[0108] Where m∈{1,2,3,…,M}, n∈{1,2,3,…,N}, the optical computational image result of the occluded target is as follows: Figure 5 As shown. When the distance ΔD between the obstructing object and the photoelectric camera... h When changes occur, repeating the above process allows for computational imaging of occluded targets after scene changes.

[0109] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1.A method for implementing occluded target optical computational imaging based on a UAV cluster, characterized in that, include: Step 1: Based on the calibration parameters of the camera array of the drone swarm, obtain the offset direction and offset amount corresponding to each camera in the camera array; wherein, each drone in the drone swarm carries a camera, and all the cameras carried by the drone swarm constitute a camera array. Step 2: Based on the offset direction and offset amount corresponding to each camera, guide all cameras to the same imaging area as the reference camera, which is any camera specified in the camera array; Step 3: Obtain the online fine calibration parameters of each camera in the camera array during the motion process; wherein, the online fine calibration parameters are composed of a pixel mapping matrix; Step 4: Based on the calibration parameters and the online fine calibration parameters, obtain the image of the occluded target; Step 1 includes: Step 11: Calculate the calibration parameters of the camera array; wherein the calibration parameters include imaging extrinsic parameters, imaging intrinsic parameters, and offline coarse mapping parameters; Step 12: Determine the spatial relationship between the camera array and the obstruction / target; Step 13: Based on the spatial relationship, calculate the offset direction and offset amount for each camera; Step 11 includes: The camera array has a total of M × N A camera captures images of a checkerboard calibration board to obtain the corresponding... M × N There are calibrated disparity images, and the resolution of the calibrated disparity images is... W r × H r Detect the pixel coordinates of checkerboard corner points in the calibrated parallax image; Based on the corner coordinates in each calibration board image, the Zhang calibration method is used to obtain the first calibration point of the camera array. m Column, No. n The imaging extrinsic and intrinsic parameters of the camera are defined; wherein the imaging extrinsic parameters include the camera's rotation matrix. R m, n Translation vector t m, n The imaging intrinsic parameters include an intrinsic parameter matrix composed of the camera's focal length and principal point offset. K m, n ; Based on the principle of homography transformation, the homography transformation matrix corresponding to each camera in the camera array is calculated and used as the offline coarse mapping parameter between the camera and the reference camera; wherein, the homography transformation matrix is: in, H m, n The homography transformation matrix, R 0 represents the rotation matrix in the imaging extrinsic parameters of the reference camera. m and n The camera array is the first m Column, No. n The index value corresponding to the line camera, m ∈{1, 2, 3, …, M }, n ∈{1, 2, 3,…, N }; Step 12 includes: The distance Δ between the occluding target in the target scene area and the camera array is obtained through measurement. D h The spatial distance between any two cameras in the camera array is the same, which is Δ. C The target scene area includes occlusions and occluding targets. Based on the dimensions of the checkerboard calibration board, calculate the size of the occluded scene captured by the camera array on the plane where the checkerboard calibration board is located. W b × H b ; The offset direction is denoted as θ m, n The offset is denoted as S m, n And respectively satisfy: in, θ x , θ y The offset direction is respectively θ m, n exist x shaft and y Components of the axis, S x , S y Offsets S m, n exist x shaft and y The components of the axis; θ x , θ y , S x , S y They respectively satisfy: in, round This is a function that rounds numerical values ​​to the nearest integer. Step 2 includes: The system calculates guidance information for the target scene area relative to each camera coordinate system to provide guidance information for the drone operator; wherein, the guidance information includes the azimuth angle of the target scene area in the camera coordinate system. Y t Pitch angle P t ; Among them, azimuth angle Y t and pitch angle P t They are represented as follows: in, The geographical coordinates of the drone platform are determined by the longitude of the drone platform. Lon p ,latitude Lat p and height Alt p composition, The geographic coordinates of the obscured target are indicated by the longitude of the obscured target. Lon t ,latitude Lat t and height Alt t Composition: dist is a function for calculating the distance between two coordinate points; Step 3 includes: The images captured by each camera in the camera array are matched pixel-level with the occlusions between the reference camera and the camera; wherein the pixel-level matching relationship between each camera and the reference camera is represented by a pixel mapping matrix. express, m and n The camera array is the first m Column, No. n The index value corresponding to the line camera, m ∈{1, 2, 3, …, M }, n ∈{1,2,3,…, N }; The pixel-level matching process is as follows: The SIFT, SURF, or ORB algorithms are used to extract feature points of the images that need pixel-level matching, and the feature points of different images are matched. The RANSAC algorithm is used to remove incorrect matching points. Step 4 includes: Step 41: Acquire images of the target scene area using the camera array; Step 42: Based on the homography transformation matrix and pixel mapping matrix, calculate the corrected image corresponding to the target scene area image captured by each camera; Step 43: Based on the corrected image and the offset direction and offset amount corresponding to each camera. ， The offset image corresponding to the target scene region image captured by each camera is calculated; Step 44: Based on all offset images, calculate the image of the occluded target in the target scene region image; The corrected image and its corresponding target scene region image captured by the camera satisfy the following: in: in, I' m, n ( x' , y' (To correct the image) I m, n ( x , y ) is the first of the camera arrays m line, number n Images of the target scene area captured by a series of cameras. x and y These are the pixel coordinates of the target scene region image. x' and y' These are the pixel coordinates of the corrected image; In step 43: The offset image is I'' m,n ( x'' , y'' ), , in, x'' and y'' These are the pixel coordinates of the offset image. S x Offset S m, n The direction of the center is θ x The offset, S y Offset S m, n The direction of the center is θ y The offset; In step 44: The optical image of the obstructed target is: in, O ( x'' , y'' () is an optical image of the occluded target.

Images