A dynamic imaging planning method for two-degree-of-freedom aerial area array cameras
By calculating and segmenting the minimum external rectangle of the region of interest and imaging the coverage step by step, the grid center is dynamically corrected, and the seamless coverage and path simplification of aerospace array cameras during imaging is solved, and the approximate 'oxen plow' scanning path is achieved. , improves imaging efficiency and coverage integrity.
Patent Information
- Application Number
- CN202411237240.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-05
AI Technical Summary
When imaging the ground area of interest, existing aerospace array cameras are difficult to achieve seamless coverage and simplify the visual axis movement path, resulting in extended scanning cycles and increased difficulty in servo mechanism control.
By calculating the minimum external rectangle of the area of interest and dividing it into cells, imaging overlay is performed row by row, dynamically correcting the grid center, updating the geometric shape of the uncovered area, and using the golden segmentation numerical iteration method to achieve an approximate 'oxen plow' scan path .
The two-degree-of-freedom aerial surface array camera is used to seamlessly cover the area of interest imaging, which reduces the control difficulty of the servo system, ensures the linearity of the scanning path, and reduces the appearance of visual blind spots and gaps.
Smart Images

Figure CN118762045B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of aerial photoelectric imaging, and in particular relates to the technical field of imaging planning of a mission area by two-degree-of-freedom aerial array scanning. Background Art
[0002] Aerial area array cameras are high-resolution imaging devices that can capture detailed features of the surface. Compared with linear array cameras, area array cameras can capture the entire image at one time, while linear array cameras need to scan multiple times to construct the entire image, which gives area array cameras a clear advantage in imaging speed. In addition, since area array cameras are one-time imaging, data loss or misalignment is not likely to occur, while linear array cameras may cause geometric distortion of the image due to mechanical vibration or other interference during the scanning process.
[0003] In order to improve the flexibility of imaging, aerial array cameras are usually mounted on a mechanical frame with two degrees of freedom, which can realize dynamic imaging under the pitch axis and roll axis. Therefore, when performing staring imaging on a large area of interest on the ground, the aerial array camera can perform block imaging of the area in a certain order (called a "scan"), and finally splice all sub-images into a larger image. The scanning cycle is the update cycle of the large image. The shorter the cycle, the more conducive it is to the detection and extraction of time-sensitive targets. In order to reduce the scanning cycle and take into account the integrity of the coverage (avoiding gaps between sub-images), it is necessary to plan the various imaging angles of the visual axis according to the geometric shape of the area, and at the same time make the scanning motion as linear as possible and simplify the motion of the servo mechanism. This is the problem to be solved in this field. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides a dynamic imaging planning method for a two-degree-of-freedom aerial array camera, the method comprising the following steps:
[0005] S1, calculating the minimum bounding rectangle of the unimaged part P of the region of interest in azimuth, dividing the minimum bounding rectangle into a form represented by a number of cells, and all the cells form a set, called a cell set;
[0006] S2, calculate whether each cell intersects with the region of interest, if so, it is a valid cell;
[0007] S3, starting from a row of cells closest to the area array camera, imaging and covering all valid cells in the cell set row by row, with the rows connected end to end;
[0008] Let the center coordinates of each valid cell be , calculate the pitch frame angle when imaging each effective cell and roll frame angle ; Calculate the center coordinates of the visual axis of the array camera The quadrilateral covering the area when imaging ;renew , if the current valid cell has an area that cannot be covered by the array camera imaging, then this valid cell is considered to be a legacy cell;
[0009] S4, solving the effective center coordinates between the center coordinates of the remaining cell and the line connecting the center coordinates of the subsequent cells so that the remaining cell can be fully covered by the imaging of the area array camera ;
[0010] S5. According to the effective center coordinates The quadrilateral of the area covered by the area array camera when the visual axis points to the imaging area , update P: ;
[0011] And judge: If the current cell is the last valid cell in the cell set, then output the final two-axis frame angle set , used to plan the imaging angle of the area array camera's visual axis; if the current cell is the last valid cell in the current row, return to step S3; otherwise, return to step S1.
[0012] The beneficial effects of the method of the present invention are:
[0013] The present invention implements an imaging planning method for seamlessly covering the region of interest for an aerial array camera with two degrees of freedom. The method evenly divides the region of interest through the minimum coverage area (imaging under vertical conditions) of the array camera, and covers each grid one by one. In order to overcome the problem of irregular projection shape during coverage, the present invention uses the golden section numerical iteration method to dynamically correct the grid center and update the geometric shape of the uncovered region of interest. With each row of cells as the basic scanning unit of the visual axis, a scanning path similar to "ox plowing" is implemented, reducing the control difficulty of the servo system. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 Schematic diagram of an imaging model of an aerial area array camera with two degrees of freedom in an embodiment of the present invention;
[0015] Figure 2 This is a schematic diagram of dividing a minimum circumscribed rectangle into a plurality of cells in an embodiment of the present invention;
[0016] Figure 3 A schematic diagram of an “occupancy matrix” in an embodiment of the present invention;
[0017] Figure 4A schematic diagram of the coverage area when imaging by the area array camera in an embodiment of the present invention;
[0018] Figure 5 A schematic diagram showing a situation where a valid cell is not completely covered given in an embodiment of the present invention;
[0019] Figure 6 is a schematic diagram of effective center coordinates in an embodiment of the present invention;
[0020] Figure 7 The figure is a schematic diagram of the imaging path and imaging area coverage after dynamic imaging of an irregular heptagonal region of interest using the method of the present invention. DETAILED DESCRIPTION
[0021] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.
[0022] Figure 1 The schematic diagram of dynamic imaging of an aerial array camera with two degrees of freedom is described. The array camera is installed in a two-degree-of-freedom frame, and the frame is installed on an airborne platform. The frame uses a dual-axis motor to drive the array camera to perform periodic and rapid scanning of a single area of interest. During each imaging, the servo system will overcome the disturbance of the carrier aircraft and maintain the stability of the line of sight of the array camera by adding a fast reflector, etc., so that the camera can stare and image the area of interest. At the end of each scanning cycle, the image processing system splices the images of each gaze together to form an equivalent image with a large field of view. The difficulties in achieving this goal are: first, how to seamlessly cover the area of interest without visual blind spots, and second, how to simplify the line of sight movement path. The present invention proposes a novel dynamic imaging planning method for aerial array cameras with two degrees of freedom. The method is simple to calculate, and the planning result is a motion path that approximates "ox plowing", which is convenient for the servo system to perform nearly straight-line bidirectional scanning and can seamlessly cover the area of interest on the ground.
[0023] The steps of this method are as follows.
[0024] Step 1: Evenly divide the unimaged part P of the region of interest. P is a polygon consisting of a series of vertices. Determine, where N is the number of vertices.
[0025] Step 1.1: Calculate the minimum circumscribed rectangle B of P in azimuth. The specific method is: suppose the current carrier azimuth is , then the rotation matrix corresponding to the aircraft azimuth is:
[0026] ;
[0027] Multiply the rotation matrix by the coordinates of each vertex of the polygon. , get the rotated vertex set .calculate All vertices in Minimum value of coordinates and maximum value ,as well as Minimum value of coordinates and maximum value Then put the four vertices Left multiplication , and then get the vertex coordinates of B.
[0028] Step 1.2: Calculate the height ( ) and width ( ):
[0029] ;
[0030] ;
[0031] in, and Respectively represent the height and width of the area array camera (in pixels), b represents the pixel size. H represents the relative average height between the array camera and the ground. f Represents the focal length of the area array camera.
[0032] Step 1.3: Calculate the number of cells based on B and C:
[0033] ;
[0034] ;
[0035] in, and Respectively represent the number of rows and columns of the cell. and Represent the height and width of B respectively Cells form a collection.
[0036] like Figure 2 As shown, the irregular convex polygon is P, the circumscribed rectangle of P is B, and the inscribed rectangle is cells.
[0037] Step 2: Calculate whether each cell intersects with P and generate an "occupancy matrix". The diagram of the "occupancy matrix" is as follows: Figure 3 The "occupancy matrix" is a two-dimensional A binary matrix, the matrix element content is "0" or "1", where "0" means that this cell has no intersection with P, and "1" means that this cell has an intersection with P. If the element of the occupied matrix corresponding to a cell is "0", it is called an invalid cell, otherwise it is called a valid cell.
[0038] All valid cells form a valid cell set. The valid cells in the set are express, , where the subscript i Represents the cell index value.
[0039] Step 3: Starting from the row of cells closest to the area array camera, imaging is performed row by row for all valid cells according to the occupancy matrix, with rows connected end to end.
[0040] Step 3.1: Let the center coordinates of each valid cell be , combined with the camera's position in the world coordinate system , calculate the pitch frame angle when each effective cell is imaged and roll frame angle , the calculation formula is as follows:
[0041] , ;
[0042] Specific introduction as well as The solution process:
[0043] Will( ) vector in the coordinate system of the array camera base is set to , , is the position of the area array camera in the world coordinate system, It is the rotation matrix between the world coordinate system and the coordinate system of the array camera base;
[0044] Will After normalization, we get the vector , ,in express middle x The axis corresponds to the vector after the normalization of the coordinates, express middle y The axis corresponds to the vector after the normalization of the coordinates, express middle z The axis corresponds to the vector after the normalization of the coordinates, Indicates that Expressed as a column vector.
[0045] Step 3.2: Calculate the boresight direction of the area array camera The quadrilateral covering the area when imaging like Figure 4 As shown, Determined by four vertices , these vertices are the vertices of the rectangular image sensor of the area array camera ( ) is projected on the ground, and the projection center is the optical center of the area array camera. The projective transformation of is as follows:
[0046] ;
[0047] In the formula, are the vertices of the rectangular image sensor of the area array camera ( ), is the translation vector, is the internal orientation element matrix of the array camera, is the elevation value of the area of interest in the world coordinate system, is the rotation matrix from the world coordinate system to the array camera coordinate system, and the frame angle of the visual axis Sure:
[0048]
[0049] Step 3.3: Update , and calculate the current Not Part of the area covered .if , then it is considered that the current Can be completely covered, i Increment and return to step 3.1, Represents an empty set, that is, the current Does not exist otherwise, it is considered that the current is a legacy cell, proceed to step 4. Figure 5 As shown, given There is no complete coverage, so the is a legacy cell, where the shaded area is , are the center coordinates of the remaining cells, The line of sight of the area array camera The quadrilateral that covers the area when imaging.
[0050] Step 4: The center coordinates of the remaining cells and the cell center coordinates that follow Solve the effective center coordinates between the lines so that the remaining cells can be fully covered by the array camera imaging ;
[0051] Solution The process is as follows:
[0052] Step 1: Order , , ;
[0053] and Indicates the effective center coordinates for solution The intermediate value calculated when Indicates that The coordinate values are assigned to , Indicates that The coordinate values are assigned to , Indicates the coordinates of the effective center of the solution The constant term when ;
[0054] make ;
[0055] make ;
[0056] and Indicates the effective center coordinates for solution The intermediate coordinates calculated when
[0057] Use the procedure in step 3.2 above to calculate and , which are the line of sight pointing coordinates of the array camera and coordinates The quadrilateral that covers the area when imaging;
[0058] Step 2:
[0059] First determine: If , then go to step 4, where Indicates the tolerance error value;
[0060] Further judgment: If , then go to step 3;
[0061] When neither the first judgment nor the second judgment is satisfied: , , ,calculate , return to the first judgment again and start the next round of judgment;
[0062] Step 3: Order , , ,calculate , return to step 2;
[0063] Step 4: If ,but ,if ,but , the solution is completed, .
[0064] and as well as The relationship is as Figure 6 As shown, the center coordinates of the legacy cells are and the cell center coordinates that follow Solve the effective center coordinates between the lines so that the remaining cells can be fully covered by the array camera imaging , Figure 6 middle The line of sight of the area array camera The quadrilateral covering the area when imaging, The line of sight of the area array camera The quadrilateral covering the area when imaging, Indicates a valid cell Not The fully covered portion is shown as the shaded portion in the figure.
[0065] Step 5: According to and the quadrilateral of the area covered when the area array camera's boresight points to it , update P: ; And judge: If the currently accessed cell is the last valid cell in the cell set, then output the final two-axis frame angle set ; If the cell currently being accessed is the last valid cell in the current row, return to step three; otherwise, return to step one.
[0066] The method of the present invention is illustrated by a specific practical application: the focal length of the optical system of the area array camera f =100mm, pixel size is 24μm, number of pixels is 1920×1080, relative altitude is 5000m, an irregular heptagonal region of interest is dynamically imaged, and the planning path is obtained by the present invention as follows Figure 7As shown, it can be seen that this method ensures that the scanning path is mostly collimated straight lines, and can completely cover the target area, and there are no seams between sub-images.
Claims
1. A dynamic imaging planning method for a two-degree-of-freedom aerial array camera, characterized in that: The method comprises the following steps: S1, calculating the minimum bounding rectangle of the unimaged part P of the region of interest in azimuth, dividing the minimum bounding rectangle into a form represented by a number of cells, all of which form a set, called a cell set; S2, calculate whether each cell intersects with the region of interest, if so, it is a valid cell; S3, starting from a row of cells closest to the area array camera, imaging and covering all valid cells in the cell set row by row, with rows connected end to end; Let the center coordinates of each valid cell be , calculate the pitch frame angle when imaging each effective cell and roll frame angle ; Calculate the center coordinates of the visual axis of the array camera The quadrilateral covering the area when imaging ;renew , if the current valid cell has an area that cannot be covered by the array camera imaging, then this valid cell is considered to be a legacy cell; S4, solving the effective center coordinates between the center coordinates of the remaining cell and the line connecting the center coordinates of the subsequent cells so that the remaining cell can be fully covered by the imaging of the area array camera ; S5. According to the effective center coordinates The quadrilateral of the area covered by the area array camera when the visual axis points to the imaging area , update P: ; And judge: If the current cell is the last valid cell in the cell set, then output the final two-axis frame angle set , used to plan the imaging angle of the area array camera's visual axis; If the current cell is the last valid cell in the current row, return to step S3; otherwise, return to step S1.
2. The dynamic imaging planning method of a two-degree-of-freedom aerial array camera according to claim 1, characterized in that: The calculation of the minimum bounding rectangle of the portion P of the region of interest that is not imaged in azimuth is specifically: Assume the current aircraft azimuth is , then the rotation matrix corresponding to the aircraft azimuth is: ; Multiply the rotation matrix by the left rotation matrix for each vertex coordinate of the unimaged part P , get the rotated vertex set ,calculate All vertices in Minimum value of coordinates and maximum value ,as well as Minimum value of coordinates and maximum value , and then the four vertices Left multiplication , and then get the vertex coordinates of the minimum enclosing rectangle.
3. The dynamic imaging planning method of a two-degree-of-freedom aerial array camera according to claim 1, characterized in that: The cell is a rectangle covered by the area array camera when imaging the ground under vertical conditions. He Kuan Specifically: ; ; in, and Respectively represent the height and width of the area array camera, the unit is pixel, b represents the pixel size, H represents the relative average height between the array camera and the ground, f Indicates the focal length of the area array camera.
4. The dynamic imaging planning method of a two-degree-of-freedom aerial array camera according to claim 1, characterized in that: When imaging and covering all valid cells in the cell set row by row, the visual axis of the area array camera points to the center coordinates of each valid cell.
5. The dynamic imaging planning method of a two-degree-of-freedom aerial array camera according to claim 4, characterized in that: Calculate the pitch frame angle when imaging each valid cell and roll frame angle Specifically: , ; Specific introduction as well as The solution process: Will( ) vector in the coordinate system of the array camera base is set to , , is the position of the area array camera in the world coordinate system, It is the rotation matrix between the world coordinate system and the array camera base coordinate system; Will After normalization, we get the vector , ,in express middle x The axis corresponds to the vector after the normalization of the coordinates, express middle y The axis corresponds to the vector after the normalization of the coordinates, express middle z The axis corresponds to the vector after the normalization of the coordinates, Indicates that Expressed as a column vector.
6. The dynamic imaging planning method of a two-degree-of-freedom aerial array camera according to claim 1, characterized in that: Calculate the center coordinates of the area array camera's visual axis The quadrilateral covering the area when imaging Specifically: Determined by four vertices , these vertices are the vertices of the rectangular image sensor of the area array camera The projection on the ground, the projection center is the optical center of the area array camera, solve The projective transformation of is as follows: ; in, is the rotation matrix from the world coordinate system to the area array camera coordinate system, is the translation vector, is the internal orientation element matrix of the array camera, is the elevation value of the area of interest in the world coordinate system.
7. The dynamic imaging planning method of a two-degree-of-freedom aerial array camera according to claim 1, characterized in that: Effective center coordinates The solution is: Step 1: Order , , ; and Indicates the effective center coordinates for solution The intermediate value calculated when Indicates that The coordinate values are assigned to , Indicates that The coordinate values are assigned to , Indicates the effective center coordinates for solution The constant term when ; make ; make ; and Indicates the effective center coordinates for solution The intermediate coordinates calculated when Calculate the line of sight pointing coordinates of the area array camera The area of the quadrilateral covered during imaging , calculate the line of sight pointing coordinates of the area array camera The area of the quadrilateral covered during imaging ; Step 2: First determine: If , then go to step 4, where Indicates the tolerance error value; Further judgment: If , then go to step 3; When neither the first judgment nor the second judgment is satisfied: , , ,calculate , return to the first judgment again and start the next round of judgment; Step 3: Order , , ,calculate , return to step 2; Step 4: If ,but ,if ,but , the solution is completed, .
Citation Information
Patent Citations
A method for optimize regional coverage path of area array staring optical remote sensing satellite
CN109308547A
Control method for continuous scanning imaging of area-array camera
CN112584047A
Method and system for stereoscopic scanning
US20120249739A1
