Image motion compensation method, tilt swing camera and aircraft

Abstract

The invention relates to an image motion compensation method, a tilt pendulum camera and an aircraft. The method comprises the following steps: recording a first attitude rotation angle of the tilt-sweep camera when an aircraft reaches an imaging position through an image motion compensation processor of the tilt-sweep camera, and recording a second attitude rotation angle of the tilt-sweep camera in continuous sweep motion in real time; and calculating a compensation action angle based on the first attitude rotation angle and the second attitude rotation angle, controlling the tilt-sweep camera to perform image motion compensation according to the compensation action angle, and exposing and imaging a corresponding surveying and mapping image by the tilt-sweep camera after the compensation is completed. According to the scheme provided by the invention, the problem that the image motion compensation is not accurate enough under the condition of continuous swing scanning in an inclined wide-width rapid scanning imaging technology in the related technology is solved, and the accurate image motion compensation of the surveying and mapping image is realized.

Description

Technical Field

[0001] The present application relates to the field of aerial mapping technology, and in particular, to an image motion compensation method, an oblique swing-scanning camera, and an aircraft. Background Art

[0002] Oblique photography technology is a cutting-edge technology in the field of aerial mapping. Different from traditional orthophotography that can only perform vertical-angle shooting, it concentrates multiple sensors on the same flight platform and can collect images from multiple angles simultaneously. Compared with traditional methods, oblique photography technology can not only truly reflect the three-dimensional shape of ground objects, but also accurately capture depth information and spatial structure, thereby providing more detailed environmental data to support precise terrain analysis and target recognition.

[0003] Currently, the general oblique photography technology is based on an oblique swing-scanning camera for mapping. During mapping, wide-area imaging is required. During the wide-area imaging process, the oblique swing-scanning camera needs to take multiple pictures for stitching under continuous swing-scanning conditions. Due to the angle offset problem, the quality of a single picture is poor, so that the pictures obtained after wide-area imaging cannot clearly reflect the mapping results. In the related-art oblique wide-area fast-scanning imaging technology, there is a problem that the image motion compensation is not accurate enough under continuous swing-scanning conditions. Therefore, there is a lack of a method for accurately compensating the image motion of pictures with angle offset problems in the related art to meet the requirements. Summary of the Invention

[0004] To solve the problems existing in the related art, the present application provides an image motion compensation method based on an oblique swing-scanning camera, which can perform image motion compensation processing on the mapping images with angle offset problems in the related art.

[0005] The first aspect of the present application provides an image motion compensation method, which is applied to an image motion compensation processor of an oblique swing-scanning camera carried on an aircraft. The method includes:

[0006] When the aircraft reaches the imaging position, determine the first attitude rotation angle of the oblique swing-scanning camera;

[0007] Control the roll plane structure inside the oblique swing-scanning camera to perform continuous swing-scanning motion to obtain a second attitude rotation angle;

[0008] Based on the first attitude rotation angle and the second attitude rotation angle, calculate the compensation action angle;

[0009] According to the compensation action angle, adjust the optical path refraction structure of the oblique swing-scanning camera to obtain an adjusted optical path refraction structure;

[0010] Based on the adjusted optical path refraction structure, perform exposure imaging to obtain a mapping image.

[0011] As an implementation of the first aspect of the present application, before the aircraft reaches the imaging position, the method further includes establishing an inclined swing-scanning camera vector model;

[0012] Among them, the inclined swing-scanning camera vector model is used to represent the camera space coordinate system vector and the attitude rotation angle of the inclined swing-scanning camera.

[0013] As an implementation of the first aspect of the present application, determining the first attitude rotation angle of the inclined swing-scanning camera specifically includes:

[0014] Obtain the flight attitude information collected by the attitude sensor in the inclined swing-scanning camera when the aircraft reaches the imaging position;

[0015] Calculate the first attitude rotation angle based on the flight attitude information.

[0016] As an implementation of the first aspect of the present application, obtaining the second attitude rotation angle specifically includes:

[0017] When the roll plane structure of the inclined swing-scanning camera starts continuous swing-scanning motion, obtain the real-time flight attitude information of the aircraft;

[0018] Calculate the angle offset of each mapping image respectively based on the first attitude rotation angle, the real-time flight attitude information and the real-time rotation angle of the inclined swing-scanning camera;

[0019] Superimpose the angle offset on the basis of the first attitude rotation angle respectively to obtain the second attitude rotation angle corresponding to each mapping image.

[0020] As an implementation of the first aspect of the present application, the first attitude rotation angle includes an initial roll rotation angle and an initial pitch rotation angle;

[0021] The second attitude rotation angle includes an offset roll rotation angle and an offset pitch rotation angle.

[0022] As an implementation of the first aspect of the present application, the compensation action angle corresponding to the mapping image is calculated and obtained through an image motion compensation algorithm. The obtaining process of the compensation action angle specifically includes:

[0023] Obtain the first attitude rotation angle and the second attitude rotation angle corresponding to the mapping image;

[0024] According to the first attitude rotation angle and the second attitude rotation angle, calculate the initial visual axis vector, visual axis vector and equivalent compensation rotation axis vector of the camera when shooting the mapping image;

[0025] Compensate the visual axis vector through the compensation coding angle;

[0026] After compensation, the pointing direction of the visual axis vector is the same as that of the initial visual axis vector. A vector constraint equation is established to convert the compensation problem into an angle-solving problem;

[0027] Solve the vector constraint equation to obtain the compensation action angle.

[0028] As an implementation manner of the first aspect of the present application, during the process of the tilting swing-scanning camera exposing and imaging to obtain multiple mapping images, when the angle offset is small, the corresponding compensation coding angle is also a small quantity, and small-angle approximation simplification can be performed on it to obtain a simplified image motion compensation algorithm.

[0029] The second aspect of the application provides a tilting swing-scanning camera, which at least includes an image motion compensation processor; the image motion compensation processor at least includes: a storage unit, an execution processing unit, and a control unit;

[0030] The storage unit is electrically connected to the execution processing unit and is used to store the first attitude rotation angle and the second attitude rotation angle;

[0031] The execution processing unit is electrically connected to the control unit and is used to calculate the compensation action angle and send a control instruction to the control unit based on the compensation action angle;

[0032] The control unit is used to receive the control instruction sent by the control unit and control the adjustment of the optical path refraction structure according to the control instruction.

[0033] The third aspect of the present application provides an aircraft that can carry the tilting swing-scanning camera for flying imaging of a mapping area.

[0034] The fourth aspect of the present application provides a computer-readable storage medium, on which executable code is stored. When the executable code is executed by the image motion compensation processor of the tilting swing-scanning camera, the image motion compensation processor is made to execute the image motion compensation method.

[0035] The technical solution provided by the present application may include the following beneficial effects: A provided image motion compensation method based on a tilting swing-scanning camera, by recording the first attitude rotation angle of the tilting swing-scanning camera when the aircraft reaches the imaging position, obtaining the corresponding second attitude rotation angle required for each mapping image during the exposure imaging process, then calculating the compensation action angle for each mapping image based on the first attitude rotation angle and the second attitude rotation angle of each mapping image, adjusting the angle of the internal optical path refraction structure of the tilting swing-scanning camera respectively based on the compensation action angle corresponding to each mapping image, and performing exposure imaging after the adjustment is completed, thereby solving the problem of image motion compensation in the continuous swing-scanning condition of the oblique photography technology, and at the same time improving the scanning speed of the swing-scanning camera and enhancing the working efficiency of the imaging system.

[0036] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] By describing the exemplary embodiments of this application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of this application will become more apparent. Among them, in the exemplary embodiments of this application, the same reference numerals generally represent the same components.

[0038] Figure 1 is a schematic flowchart of the image motion compensation method shown in the embodiments of this application;

[0039] Figure 2 is a schematic diagram of the vector model of the tilt swing-scanning camera shown in the embodiments of this application;

[0040] Figure 3 is a schematic flowchart of the process for obtaining the second attitude rotation angle shown in the embodiments of this application;

[0041] Figure 4 is a schematic flowchart of the process for calculating the compensation action angle shown in the embodiments of this application;

[0042] Figure 5 is a schematic diagram of the internal structure of the image motion compensation processor shown in the embodiments of this application;

[0043] Figure 6 is a schematic diagram of the structure of the tilt swing-scanning camera shown in the embodiments of this application;

[0044] Figure 7 is a schematic flowchart of the working process of the tilt swing-scanning camera shown in the embodiments of this application.

[0045] Symbol Description: 1 - Outer frame base; 2 - Roll ring frame; 3 - Roll axis motor; 4 - Pitch mirror; 5 - Pitch axis motor; 6 - Optical path folding mirror; 7 - Compensation mirror; 8 - Compensation axis motor; 9 - Imaging instrument; 10 - Attitude sensor; 11 - Compensation rotation axis; 12 - Roll rotation axis; 13 - Pitch rotation axis. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited by the embodiments set forth herein. Instead, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0047] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0048] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, "a plurality of" means two or more unless otherwise specifically defined.

[0049] An embodiment of this application provides an image motion compensation method based on an oblique swing-scanning camera, which is applied to an image motion compensation processor of an oblique swing-scanning camera carried on an aircraft. The schematic diagram of the image motion compensation method process is as Figure 1 shown and specifically includes:

[0050] S11: When the aircraft reaches the imaging position, determine the first attitude rotation angle of the oblique swing-scanning camera.

[0051] Among them, when reaching the imaging position, the oblique swing-scanning camera does not start exposure imaging. After recording the first attitude rotation angle, the oblique swing-scanning camera immediately starts exposure imaging.

[0052] S12: Control the roll plane structure inside the oblique swing-scanning camera to perform continuous swing-scanning motion to obtain the second attitude rotation angle.

[0053] Among them, during the continuous swing-scanning motion of the oblique swing-scanning camera, the second attitude rotation angle of the oblique swing-scanning camera during the continuous swing-scanning motion is recorded in real time.

[0054] S13: Calculate the compensation action angle based on the first attitude rotation angle and the second attitude rotation angle.

[0055] Among them, when preparing to obtain the mapping image, the image motion compensation processor records the current second attitude rotation angle of the oblique swing-scanning camera for calculating the compensation action angle, and the obtained compensation action angle is the compensation action angle required for the corresponding mapping image during exposure imaging.

[0056] Furthermore, each mapping image has a corresponding compensation action angle.

[0057] S14: Adjust the optical path refraction structure of the tilt swing-scanning camera according to the compensation action angle to obtain the adjusted optical path refraction structure.

[0058] Specifically, before acquiring each surveying and mapping image, the tilt swing-scanning camera adjusts the angle of the optical path refraction structure inside the tilt swing-scanning camera according to the corresponding compensation action angle, so that the tilt swing-scanning camera can clearly image the required surveying and mapping image.

[0059] S15: Perform exposure imaging based on the adjusted optical path refraction structure to obtain the surveying and mapping image.

[0060] In this embodiment, by recording the first attitude rotation angle of the tilt swing-scanning camera when the aircraft reaches the imaging position, and acquiring the corresponding second attitude rotation angle required for each surveying and mapping image, then based on the first attitude rotation angle and the second attitude rotation angle of each surveying and mapping image, calculate the compensation action angle of each surveying and mapping image, and adjust the angle of the optical path refraction structure inside the tilt swing-scanning camera respectively based on the compensation action angle corresponding to each surveying and mapping image. After the adjustment is completed, exposure imaging is performed, thereby solving the problem of image motion compensation in the tilt photography technology under continuous swing-scanning conditions, and at the same time improving the scanning speed of the swing-scanning camera and enhancing the working efficiency of the imaging system.

[0061] In the embodiment of the present application, before the aircraft reaches the imaging position, a tilt swing-scanning camera vector model is established.

[0062] Among them, the tilt swing-scanning camera vector model is stored in the image motion compensation processor, and the schematic diagram of the tilt swing-scanning camera vector model is as Figure 2 shown.

[0063] The tilt swing-scanning camera vector model only needs to be established when the tilt processing camera works for the first time. After being saved in the image motion compensation processor, when the tilt processing camera works each time, the image motion compensation processor can directly call the tilt swing-scanning camera vector model without secondary establishment.

[0064] Among them, the tilt swing-scanning camera vector model is used to represent the camera space coordinate system vector and the attitude rotation angle of the tilt swing-scanning camera.

[0065] Specifically, the parameters that the tilt swing-scanning camera vector model can represent at least include: flight altitude H, flight direction of the aircraft nose Bx, aircraft wing direction By, downward direction of the aircraft bottom Bz, first attitude rotation angle, second attitude rotation angle, visual axis vector equivalent compensation rotation axis vector

[0066] Among them, the visual axis vector points to the Nz direction in the tilt swing-scanning camera vector model, and Nz' is the direction after the visual axis vector rotates when the tilt swing-scanning camera performs continuous swing-scanning motion.

[0067] The equivalent compensation rotation axis vector points in the Nx direction in the vector model of the tilt and swing camera. When the tilt and swing camera performs continuous swing motion, Nx' is the direction after the equivalent compensation rotation axis vector rotates.

[0068] In this embodiment, a vector model of the tilt and swing camera is constructed through various vector parameters, which accurately describes the imaging position offset caused by image motion in the optical system. During the exposure imaging process, the accurate mathematical description required for image motion compensation can be calculated through the vector model, helping the tilt and swing camera quickly find the optimal action compensation angle, thereby reducing errors and unnecessary adjustments during the compensation process. At the same time, the vector model of the tilt and swing camera can provide real-time guidance during the image motion compensation process, further improving the image motion compensation speed.

[0069] In the embodiment of the present application, determining the first attitude rotation angle of the tilt and swing camera specifically includes:

[0070] Obtain the flight attitude information collected by the attitude sensor in the tilt and swing camera when the aircraft reaches the imaging position;

[0071] Calculate the first attitude rotation angle based on the flight attitude information.

[0072] Among them, the flight attitude information is collected in real time by the attitude sensor in the tilt and swing camera.

[0073] After the attitude sensor completes the collection of flight attitude information, it is sent to the image motion compensation processor in real time. When it is determined that the aircraft reaches the imaging position, the image motion compensation processor records the flight attitude information at the time of reaching the imaging position. The image motion compensation processor calculates the first attitude rotation angle based on the recorded flight attitude information.

[0074] The attitude sensor and the image motion compensation processor use a wireless communication method for data transmission. The wireless communication method includes but is not limited to Bluetooth, WiFi, zigbee, etc.

[0075] In this embodiment, the flight attitude information is collected in real time by the attitude sensor and sent to the image motion compensation processor in real time, providing an exact data basis for the image motion compensation processor to calculate the first attitude rotation angle, making the first attitude rotation angle obtained by the image motion compensation processor more accurate and laying a good data foundation for the subsequent calculation of the action compensation angle.

[0076] In the embodiment of the present application, the flow chart for obtaining the second attitude rotation angle is as Figure 3 shown, and specifically includes:

[0077] S21: After the roll plane structure of the tilt and swing camera starts continuous swing motion, obtain the real-time flight attitude information of the aircraft.

[0078] S22: Calculate the angular offset of each mapping image respectively based on the first attitude rotation angle, the real-time flight attitude information, and the real-time rotation angle of the tilt swing camera.

[0079] S23: Superimpose the angular offset respectively on the basis of the first attitude rotation angle to obtain the second attitude rotation angle corresponding to each mapping image.

[0080] Specifically, for the real-time flight attitude information of the image motion compensation processor, when it is necessary to perform exposure imaging to obtain the mapping image, record the flight attitude information, and calculate the second attitude rotation angle in combination with the rotation angle of the tilt swing camera during continuous swing scanning at this time.

[0081] Among them, the second attitude rotation angle is calculated in real time during the continuous swing scanning process of the tilt swing camera. When the image motion compensation processor needs to obtain the mapping image, determine the second attitude rotation angle required for a single mapping image, and each mapping image has its corresponding second attitude rotation angle.

[0082] In this embodiment, by obtaining the real-time flight attitude information of the image motion compensation processor, recording the flight attitude information and the rotation angle of the tilt swing camera required for exposure imaging, calculating the second attitude rotation angle, and adopting the real-time acquisition method, it can quickly respond to the movement of the aircraft or other dynamic changes, capture subtle dynamic changes, improve the accuracy of the obtained second attitude rotation angle, and enable the calculation method to adapt to complex dynamic environments, ensuring that the image motion compensation method is always effective. In addition, it can also reduce unnecessary data storage and improve resource utilization efficiency.

[0083] In the embodiment of the present application, the first attitude rotation angle includes an initial roll rotation angle and an initial pitch rotation angle;

[0084] The second attitude rotation angle includes an offset roll rotation angle and an offset pitch rotation angle.

[0085] Specifically, the initial roll rotation angle is the rotation angle of the roll plane structure in the tilt swing camera at the first moment.

[0086] The initial pitch rotation angle is the rotation angle of the pitch mirror 4 in the tilt swing camera at the first moment.

[0087] The offset roll rotation angle is the angle required to control the rotation of the roll plane structure before imaging the mapping image by the tilt swing camera.

[0088] The offset pitch rotation angle is the angle required to control the rotation of the pitch mirror 4 before imaging the mapping image by the tilt swing camera.

[0089] Among them, each surveying and mapping image has a corresponding offset roll rotation angle and an offset pitch rotation angle.

[0090] In this embodiment, by precisely dividing the first attitude rotation angle and the second attitude rotation angle, the second attitude rotation angle can be precisely adjusted in terms of angle, ensuring that the second attitude rotation angle of each surveying and mapping image is dynamically adjusted according to specific imaging requirements, thereby improving the accuracy and efficiency of surveying and mapping, enabling the oblique push-broom camera to maintain a high-precision imaging effect in a complex flight environment, being particularly suitable for scenarios such as aerial surveying and remote sensing imaging, and improving the flexibility and adaptability of the oblique push-broom camera.

[0091] In the embodiment of the present application, the compensation action angle corresponding to the surveying and mapping image is calculated and obtained through an image motion compensation algorithm. The process of calculating the compensation action angle is as Figure 4 shown, and specifically includes:

[0092] S31: Obtain the first attitude rotation angle and the second attitude rotation angle corresponding to the surveying and mapping image.

[0093] Specifically, when the oblique push-broom camera starts to expose and image, record the initial roll rotation angle as φ start , record the initial pitch rotation angle as θ start . During the exposure of the oblique push-broom camera, after the roll rotation angle rotates by Δφ, the offset roll rotation angle φ start + Δφ is obtained, and at this time the offset pitch rotation angle is θ.

[0094] Among them, Δφ is the angle offset.

[0095] S32: Calculate the initial visual axis vector, visual axis vector, and equivalent compensation rotation axis vector of the camera when shooting the surveying and mapping image according to the first attitude rotation angle and the second attitude rotation angle.

[0096] Among them, the initial visual axis vector is The visual axis vector is The equivalent compensation rotation axis vector is

[0097] Specifically, when starting to expose, the initial visual axis vector is:

[0098]

[0099] As the roll plane structure rotates during the exposure, after the initial roll rotation angle rotates by Δφ and becomes the offset roll rotation angle φ start + Δφ, the pitch mirror synchronously moves to the offset pitch rotation angle θ. At this time, the visual axis vector is:

[0100]

[0101] The equivalent compensation rotation axis vector perpendicular to the line of sight vector is:

[0102]

[0103] S33: Compensate the line of sight vector through the compensation coding angle;

[0104] Specifically, the compensation coding angle is κ, and the line of sight vector is compensated through the compensation coding angle κ. According to the double angle relationship, the actual compensation coding angle is 2κ, and the line of sight vector compensated by the compensation coding angle 2κ The calculation formula is: is:

[0105]

[0106] S34: After compensation, the direction of the line of sight vector is the same as the initial direction of the line of sight vector. Establish a vector constraint equation to convert the compensation problem into a problem of solving for angles.

[0107] Among them, the established vector constraint equation is:

[0108] sinθcos(2κ) = sinθ start

[0109] sin(φ start +Δφ)cosθcos(2κ)+cos(φ start +Δφ)sin(2κ) = cosθ start sinφ start

[0110] cos(φ start +Δφ)cosθcos(2κ)-sin(φ start +Δφ)sin(2κ) = cosθ start cosφ start .

[0111] S35: Solve the vector constraint equation to obtain the compensation action angle.

[0112] Specifically, the obtained compensation action angle is:

[0113]

[0114] In this embodiment, according to the first attitude rotation angle and the second attitude rotation angle, the initial line-of-sight vector, the line-of-sight vector, and the equivalent compensation rotation axis vector are calculated, and a vector constraint equation is established based on the calculation results. By solving the vector constraint equation, the compensation action angle is obtained, and thus the accurate compensation action angle is obtained. Based on the compensation action angle, the accurate adjustment of the optical path refraction angle of the tilt swing-scanning camera is realized, which can make the imaging position of each surveying and mapping image more accurate, thereby improving the accuracy and efficiency of surveying and mapping.

[0115] In the embodiment of the present application, during the process that the tilt swing-scanning camera exposes and images to obtain multiple surveying and mapping images, when the angle offset is small, the corresponding compensation coding angle is also a small quantity, and small-angle approximation simplification can be performed on it to obtain a simplified image motion compensation algorithm.

[0116] Specifically, during the exposure imaging, when the angle offset Δφ is small, the corresponding compensation coding angle 2κ is also a small quantity, and small-angle approximation simplification can be performed on the image motion compensation algorithm. The simplified formula is:

[0117] sin(Δφ)≈Δφ, cos(Δφ)≈1

[0118] sin(2κ)≈2κ, cos(2κ)≈1

[0119] sin(φ start +Δφ)≈sinφ start +Δφcosθ start

[0120] cos(φ start +Δφ)≈cosφ start -Δφsinφ start ;

[0121] Substituting the simplified formula into the image motion compensation algorithm, the formula of the simplified image motion compensation algorithm is obtained by solving:

[0122] θ=θ start

[0123] 2κ=-Δφcosθ start

[0124] Furthermore, during the image motion compensation process, the compensation action angle calculated using the image motion compensation algorithm is an analytical solution, which is an accurate calculation. There are complex trigonometric function operations in the calculation process. When high-precision compensation is required, this calculation method can be used to solve the compensation action angle for compensation.

[0125] At the same time, considering the calculation efficiency and compensation accuracy of the tilt swing-scanning camera comprehensively, the image motion compensation algorithm and the simplified image motion compensation algorithm can be combined to achieve the optimal image motion compensation for the surveying and mapping images.

[0126] In this embodiment, by making a small-angle approximation and simplification of the image motion compensation method when the angle offset is small, a simplified image motion compensation algorithm is obtained, which significantly reduces the computational complexity, enables faster calculation, thereby improving the real-time performance of the image motion compensation of the tilt swing-scanning camera, ensuring that the changes in the flight attitude of the aircraft can be responded to in a timely manner in a dynamic environment, and at the same time can reduce the calculation error and improve the image motion compensation accuracy.

[0127] Since the current tilt swing-scanning cameras are mainly composed of the splicing of multiple array cameras, wide-area imaging of the target ground objects is achieved through the multi-angle placement of four or even more cameras. However, this type of device requires an additional stable platform during imaging, and the overall volume of the device is relatively large. In order to reduce the device weight, the method of rotating the built-in camera for imaging is gradually increasing. However, because the compensation rotation axis under the tilt condition is not coaxial with the roll rotation, continuous swing-scanning motion imaging cannot be compensated, and mostly stop-and-swing imaging is used. During the scanning imaging, the motor needs to be frequently started and stopped, which has a high demand for the motor, and the imaging speed will be limited. Therefore, in the second aspect of this application, a tilt swing-scanning camera is provided to solve the problem that the tilt swing-scanning camera in the related art cannot perform continuous swing-scanning motion imaging.

[0128] In the second aspect of this application, a tilt swing-scanning camera is provided, which at least includes an image motion compensation processor, and the internal structure of the image motion compensation processor is as Figure 5 shown.

[0129] The image motion compensation processor at least includes: a storage unit 14, an execution processing unit 15, and a control unit 16.

[0130] The storage unit 14 is electrically connected to the execution processing unit 15 and is used for storing the first attitude rotation angle and the second attitude rotation angle.

[0131] The execution processing unit 15 is electrically connected to the control unit 16 and is used for calculating the compensation action angle and sending a control instruction to the control unit 16 based on the compensation action angle.

[0132] The control unit 16 is used for receiving the control instruction sent by the control unit 15 and controlling the adjustment of the optical path refraction structure according to the control instruction.

[0133] Wherein, the image motion compensation processor further includes a data receiving unit 17. The data receiving unit 17 is electrically connected to the storage unit 14 and is used for receiving the flight attitude data sent by the attitude sensor and sending the received flight attitude data to the storage unit 14.

[0134] Specifically, when the execution processing unit 15 needs to calculate the first attitude rotation angle and the second attitude rotation angle, it first calls the stored flight attitude data through the storage unit 14. After the calculation is completed, the obtained first attitude rotation angle and the second attitude rotation angle are sent to the storage unit 14 for storage and are called again when the compensation action angle needs to be performed.

[0135] In this embodiment, the image motion compensation processor is designed in a modular manner, optimizing the resource utilization rate and system integration degree of the image motion compensation processor, realizing the real-time processing of flight attitude data, accurate compensation calculation, and real-time adjustment of the optical path refraction structure, thereby improving the imaging quality and the stability of the tilt and swing scanning camera.

[0136] In the embodiment of the present application, the tilt and swing scanning camera specifically includes a camera base 1, a roll plane structure, an optical path refraction structure, an imaging instrument 9, an image motion compensation processor, and an attitude sensor 10. The structure of the tilt and swing scanning camera is as Figure 6 shown.

[0137] The camera base 1 is fixedly installed at the bottom of the aircraft and is used to provide a fixed foundation for the tilt and swing scanning camera.

[0138] The roll plane structure is fixedly installed inside the camera base, with the flight direction of the aircraft determined as the rotation axis. The roll plane structure performs continuous swing scanning motion around the rotation axis.

[0139] Among them, the rotation axis penetrates through the center of the roll plane structure.

[0140] Specifically, the roll plane structure specifically includes a roll ring frame 2, a roll axis motor 3, and a roll axis 12.

[0141] The roll axis 12 is fixedly installed at the center of the camera base 1 along the flight direction of the aircraft and coincides with the rotation axis. The roll ring frame 2 is fixedly installed with the roll axis 12. The roll axis 12 penetrates through the center of the roll ring frame 2. The roll axis motor 3 is installed at one end of the roll axis 12 away from the flight direction of the aircraft. The rotation shaft of the roll axis motor 3 is fixedly connected to the roll axis 12. When performing continuous swing scanning motion, the roll axis motor 3 controls the lateral rotation of the roll axis 12, and the roll axis 12 drives the roll ring frame 2 to rotate laterally.

[0142] The optical path refraction structure is fixedly installed at one end of the roll plane structure facing the ground and is used to refract the scene at the imaging position.

[0143] Among them, the optical path refraction structure specifically includes a pitch mirror 4, a pitch axis motor 5, an optical path turning mirror 6, a compensation mirror 7, a compensation axis motor 8, a compensation rotation axis 11, and a pitch rotation axis 13.

[0144] The pitch mirror 4, the pitch axis motor 5, and the pitch rotation axis 13 are fixedly installed at one end of the roll ring frame 2 close to the flight direction of the aircraft. The pitch rotation axis 13 is fixedly installed on the roll ring frame 2 in a direction perpendicular to the roll axis 12. The pitch axis motor 5 is fixedly installed outside the roll ring frame 2 and is connected to the pitch rotation axis 13 through a rotating shaft. The pitch mirror 4 is fixedly installed with the pitch rotation axis 13, and the pitch rotation axis 13 passes through the center of the pitch mirror 4. When performing image motion compensation, the pitch axis motor 5 controls the rotation of the pitch rotation axis 13, and the pitch rotation axis 13 drives the pitch mirror 4 to adjust the optical path refraction angle.

[0145] The optical path folding mirror 6 is fixedly installed at the central position of the roll ring frame 2, and the optical path refraction angle of the optical path folding mirror 6 is fixed and unchanged.

[0146] The compensation mirror 7, the compensation axis motor 8, and the compensation rotation axis 11 are fixedly installed on one side of the optical path folding mirror 6 in the roll ring frame 2. The compensation rotation axis 11 is fixedly installed on the roll ring frame 2 in a direction perpendicular to the roll axis 12. The compensation axis motor 8 is fixedly installed outside the roll ring frame 2 and is connected to the compensation rotation axis 11 through a rotating shaft. The compensation mirror 7 is fixedly installed with the compensation rotation axis 11, and the compensation rotation axis 11 passes through the center of the compensation mirror 7. When performing image motion compensation, the compensation axis motor 8 controls the rotation of the compensation rotation axis 11, and the compensation rotation axis 11 drives the compensation mirror 7 to adjust the optical path refraction angle.

[0147] The imaging instrument 9 is fixedly installed at one end of the roll plane structure facing the ground, and is used to receive the imaging position scene refracted by the optical path refraction structure and perform an exposure imaging operation on the imaging position.

[0148] The image motion compensation processor is fixedly installed inside the imaging instrument and is used to obtain the mapping image and control the optical path refraction structure to perform image motion compensation.

[0149] The attitude sensor 10 is fixedly installed outside the camera base and is used to transmit the flight attitude information to the image motion compensation processor.

[0150] Among them, the compensation rotation axis 11 is not parallel to the roll rotation axis 12 and there is a certain included angle.

[0151] In this embodiment, by integrating the roll plane structure, optical path refraction structure, and imaging instrument of the tilt sweep camera into the camera base 1, the need for additional stable platforms and multiple cameras required for stitching traditional multi-array cameras is reduced, thereby significantly reducing the overall weight and volume of the device. Moreover, the compact layout of key components such as the roll ring frame 2, roll shaft motor 3, pitch mirror 4, and compensation mirror 7 makes the camera structure more compact and suitable for installation at the bottom of the aircraft. Meanwhile, the roll plane structure uses the flight direction of the aircraft as the rotation axis, and the lateral rotation of the roll shaft 12 is controlled by the roll shaft motor 3 to achieve the continuous sweep motion of the roll ring frame 2. This avoids the need to frequently start and stop the motor in traditional stop-and-image imaging, significantly improving the imaging speed and efficiency. Additionally, the rotation of the pitch mirror 4 and the compensation mirror 7 is controlled by the pitch axis motor 5 and the compensation axis motor 8 respectively to dynamically adjust the optical path refraction angle, ensuring effective image motion compensation during continuous sweep and thus achieving high-quality wide-angle imaging.

[0152] In an embodiment of the present application, a working method of the tilt sweep camera is provided. The working process of the tilt sweep camera is as Figure 7 shown, and specifically includes:

[0153] S41: When the tilt sweep camera starts to work, the attitude sensor 10 starts to collect the aircraft attitude information in real time and sends it to the image motion compensation processor in real time.

[0154] S42: The image motion compensation processor calculates the attitude rotation angle of the tilt sweep camera in real time according to the aircraft attitude information.

[0155] S43: When it is determined that the aircraft reaches the imaging position, the image motion compensation processor configures the moment of reaching the imaging position as the first moment, and the tilt sweep camera records the attitude rotation angle at the first moment as the first attitude rotation angle.

[0156] S44: After reaching the imaging position, the roll shaft motor 3 in the tilt sweep camera drives the roll ring frame 2 to perform continuous sweep motion through the roll shaft 12.

[0157] S45: When preparing to expose and image the survey area, the image motion compensation processor records the attitude rotation angle at this time as the second attitude rotation angle.

[0158] S46: The image motion compensation processor calculates the compensation action angles of the pitch mirror 4 and the compensation mirror 7 based on the image motion compensation algorithm according to the first attitude rotation angle and the second attitude rotation angle.

[0159] S47: The image motion compensation processor controls the pitch mirror 4 and the compensation mirror 7 to rotate to the required optical path refraction angles according to the compensation action angles through the pitch axis motor 5 and the compensation axis motor 8.

[0160] S48: After the pitch mirror 4 and the compensation mirror 7 are rotated to the required optical path refraction angles, the imaging instrument 9 exposes and images the area pointed to by the current optical path to obtain the required surveying and mapping image.

[0161] Among them, when the tilting pushbroom camera performs continuous pushbroom motion, multiple exposure imaging is required. After obtaining the set of surveying and mapping images, the surveying and mapping images are stitched together to obtain a wide-area surveying and mapping image of the surveyed area.

[0162] In this embodiment, by real-time collecting and processing flight attitude information, and combining with the image motion compensation algorithm to dynamically adjust the optical path refraction structure of the tilting pushbroom camera, the imaging quality and efficiency are significantly improved, and the clarity and resolution of the surveying and mapping images are ensured through precise motion compensation and accurate exposure control. At the same time, the adaptability of the tilting pushbroom camera to the dynamic environment is enhanced. At the same time, the continuous pushbroom motion reduces the need for frequent start and stop of the motor, optimizes resource utilization and reduces energy consumption.

[0163] The third aspect of this application provides an aircraft that can carry a tilting pushbroom camera for flight imaging of a surveyed area.

[0164] Among them, the tilting pushbroom camera is installed at the bottom of the aircraft.

[0165] The fourth aspect of this application provides a computer-readable storage medium, on which executable code is stored. When the executable code is executed by the image motion compensation processor of the tilting pushbroom camera, the image motion compensation processor is made to execute the image motion compensation method, and the image motion compensation method at least includes:

[0166] When the aircraft reaches the imaging position, determine the first attitude rotation angle of the tilting pushbroom camera.

[0167] Control the roll plane structure inside the tilting pushbroom camera to perform continuous pushbroom motion to obtain the second attitude rotation angle.

[0168] Based on the first attitude rotation angle and the second attitude rotation angle, calculate the compensation action angle;

[0169] According to the compensation action angle, adjust the optical path refraction structure of the tilting pushbroom camera to obtain an adjusted optical path refraction structure.

[0170] Based on the adjusted optical path refraction structure, perform exposure imaging to obtain a surveying and mapping image.

[0171] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image motion compensation method, characterized in that, An image motion compensation processor applied to an inclined swing-scanning camera carried on an aircraft, the method comprising: When the aircraft reaches the imaging position, determining a first attitude rotation angle of the inclined swing-scanning camera; Controlling a roll plane structure inside the inclined swing-scanning camera to perform continuous swing-scanning motion to obtain a second attitude rotation angle; Calculating a compensation action angle based on the first attitude rotation angle and the second attitude rotation angle; Adjusting an optical path refraction structure of the inclined swing-scanning camera according to the compensation action angle to obtain an adjusted optical path refraction structure; Performing exposure imaging based on the adjusted optical path refraction structure to obtain a mapping image.

2. The image shift compensation method according to claim 1, wherein Before the aircraft reaches the imaging position, the method further comprises establishing a vector model of the inclined swing-scanning camera; Wherein, the vector model of the inclined swing-scanning camera is used to represent the camera space coordinate system vector and the attitude rotation angle of the inclined swing-scanning camera.

3. The image shift compensation method according to claim 2, wherein Determining the first attitude rotation angle of the inclined swing-scanning camera specifically includes: Obtaining flight attitude information collected by an attitude sensor in the inclined swing-scanning camera when the aircraft reaches the imaging position; Calculating a first attitude rotation angle based on the flight attitude information.

4. The image motion compensation method according to claim 1, wherein Obtaining the second attitude rotation angle specifically includes: After the roll plane structure of the inclined swing-scanning camera starts to perform continuous swing-scanning motion, obtaining real-time flight attitude information of the aircraft; Calculating the angle offset of each mapping image respectively based on the first attitude rotation angle, the real-time flight attitude information and the real-time rotation angle of the inclined swing-scanning camera; Respectively, on the basis of the first attitude rotation angle, superimposing the angle offset to obtain a second attitude rotation angle corresponding to each mapping image.

5. The image shift compensation method according to claim 4, characterized in that, The first attitude rotation angle includes an initial roll rotation angle and an initial pitch rotation angle; The second attitude rotation angle includes an offset roll rotation angle and an offset pitch rotation angle.

6. The image shift compensation method according to claim 5, wherein The compensation action angle corresponding to the mapping image is calculated and obtained through an image motion compensation algorithm, and the obtaining process of the compensation action angle specifically includes: Obtaining a first attitude rotation angle and a second attitude rotation angle corresponding to the mapping image; Calculating an initial visual axis vector, a visual axis vector and an equivalent compensation rotation axis vector of the camera when shooting the mapping image according to the first attitude rotation angle and the second attitude rotation angle; Compensating the visual axis vector through a compensation coding angle; After compensation, the direction of the visual axis vector is the same as the direction of the initial visual axis vector, and a vector constraint equation is established to convert the compensation problem into a problem of solving an angle; Solving the vector constraint equation to obtain a compensation action angle.

7. The image motion compensation method according to claim 6, wherein During the process of the inclined swing-scanning camera performing exposure imaging to obtain multiple mapping images, when the angle offset is small, the corresponding compensation coding angle is also a small quantity, and a small angle approximation can be made to it to obtain a simplified image motion compensation algorithm.

8. An inclined swing-scanning camera, characterized in that, At least including an image motion compensation processor; The image motion compensation processor at least includes: a storage unit, an execution processing unit, and a control unit; The storage unit is electrically connected to the execution processing unit and is used to store the first attitude rotation angle and the second attitude rotation angle; The execution processing unit is electrically connected to the control unit, and is configured to calculate a compensation action angle and send a control instruction to the control unit based on the compensation action angle; The control unit is configured to receive the control instruction sent by the control unit and control the optical path refraction structure to make adjustments according to the control instruction.

9. An aircraft, characterized in that, It is capable of carrying the tilt swing scanning camera according to claim 8 and is used for flying imaging of a mapping area.

10. A computer-readable storage medium, on which executable code is stored. When the executable code is executed by an image motion compensation processor of a tilt swing scanning camera, the image motion compensation processor is caused to execute the image motion compensation method according to any one of claims 1-7.

Images