Two-axis turntable image rotation eliminating method based on inertial navigation data

Abstract

The invention provides a two-axis turntable image rotation eliminating method based on inertial navigation data, which comprises the following steps: acquiring an original image in real time by an image acquisition device mounted on a two-axis turntable, and determining parameter information of a video texture map according to the original image; calculating a rotation transformation matrix A1 by using azimuth data and pitch angle data generated by rotation of the rotary table, and calculating a rotation transformation matrix A2 by using inertial navigation data of an inertial navigation system; converting a normal vector of a horizontal plane from a horizontal coordinate system xlylzl to a sensor coordinate system xsyszs, and calculating an image spinor alpha; and carrying out image rotation elimination display on the video texture map by taking the image rotation alpha as a rotation parameter and utilizing a rotation and cutting mechanism of OpenGL (Open Graphics Library). According to the attitude angle of the base of the two-axis rotary table and the axis angle data of the two-axis rotary table, the angle amount of the horizon line projected in the image space is calculated, the angle amount serves as the rotation compensation amount during display, namely, the image spinor alpha, and display is achieved through the rotation and cutting mechanism of OpenGL.

Description

Technical Field

[0001] The present invention belongs to the technical field of software image derotation, and in particular relates to a two-axis turntable image derotation method based on inertial navigation data. Background Art

[0002] Optical observation devices for target search applications typically mount a television or infrared camera on a two- or three-axis turntable. Due to space and cost constraints, many optical observation systems opt for a two-axis turntable. During operation, the turntable rotates along its azimuth and pitch axes, passively adjusting the roll angle of the sensor mounted on the pitch mechanism. Because the two-axis turntable lacks roll freedom, roll compensation cannot be achieved.

[0003] In order to adjust the image angle and restore the image to a stable state, it is necessary to rotate the image, crop the display, and perform other operations to eliminate the rolling phenomenon of the display window image. The invention patent publication with publication number CN105425379B discloses a telescope derotator device and its collimation and adjustment method. In order to allow the telescope to obtain stable, high-quality images, the patent document compensates for the rotation of the field of view. Generally, the electronic derotation method is adopted: by processing the real-time collected image video signal, rotating it according to the required derotation angle, and then controlling the monitor to rotate the image. This method has the following problems. First, electronic derotation reduces the image accuracy to a certain extent. In addition, the effective display field of view must be smaller than the camera field of view, otherwise the picture will be blank when rotating. Due to the limited resolution of the monitor, when the picture rotates along its center, it will cause local distortion of the displayed image. When rotating at high speed, it often causes the picture to flicker, causing inconvenience to observation.

[0004] The use of optical derotation methods can eliminate the drawbacks of the electronic derotation method. In the system optical path, a derotation prism is installed in front of the target imaging device along the optical axis of the device. The rotation of the derotation prism generates the rotation of the target light, thereby achieving the effect of image derotation. The derotation prism adopts a Dove prism or a Pechan prism. According to the calculation of the optical algorithm, the image can be derotated when the rotation speed of the prism is 1 / 2 of the image rotation speed. In order to solve the problem of tracking the rotation of the image field of view of the telescope, the patent document first installs three crosshair targets on the lower part of the precision turntable assembly, the upper part of the derotator box and the horizontal optical path respectively, and uses a micrometer collimator telescope to monitor the center of the crosshair target and adjust it to the rotation axis of the precision turntable. The precision turntable assembly is adjusted so that its rotation axis coincides with the optical axis of the micrometer collimator telescope. Finally, a 45-degree plane reflector is installed. The micrometer collimator telescope is used to monitor the crosshair targets below the precision turntable assembly and in the horizontal optical path. The tilt of the tilt reflector is then iteratively adjusted in two dimensions to align the optical axes of the micrometer collimator telescope, the 45-degree plane reflector, and the precision turntable rotation axis. The alignment and adjustment method for a telescope derotator device described in this patent document is suitable for the optical system of a tracking and aiming telescope and is not applicable to the derotation of an inertial navigation system.

[0005] Most inertial navigation systems utilize a two-axis turntable. However, as the turntable rotates along the azimuth and pitch axes, the roll angle of the sensor mounted on the pitch mechanism is passively adjusted. Because the turntable itself lacks roll freedom, roll compensation cannot be performed. Therefore, the source image must be rotated and cropped to eliminate roll in the display window. Summary of the Invention

[0006] The purpose of the present invention is to solve the shortcomings of the prior art that require image adjustment in the roll direction, and to provide a two-axis turntable derotation method based on inertial navigation data. According to the attitude angle of the two-axis turntable base and the axis angle data of the two-axis turntable itself, the angle of the horizon projection in the image space is calculated, and the angle is used as the rotation compensation amount during display, that is, the image rotation amount α, and the rotation and clipping mechanism of OpenGL is used to achieve derotation display.

[0007] To achieve the above objectives, the technical solutions provided by the present invention are:

[0008] A two-axis turntable derotation method based on inertial navigation data, comprising:

[0009] Step 1: An image acquisition device mounted on a two-axis turntable acquires an original image in real time, and determines parameter information of a video texture map based on the original image;

[0010] Step 2: Calculate the rotation transformation matrix A1 using the azimuth data and pitch angle data generated by the turntable rotation, and calculate the rotation transformation matrix A2 using the inertial navigation data of the inertial navigation system; wherein, the rotation transformation matrix A1 is the turntable coordinate system x p y p z p To the sensor coordinate system x s y s z s The rotation transformation matrix A2 is the horizontal coordinate system x l y l z l To the turntable coordinate system x p y p z p The rotation transformation matrix;

[0011] Step 3: Convert the normal vector of the horizontal plane from the horizontal coordinate system x l y l z l Transform to sensor coordinate system x s y s z s , and calculate the image rotation α; the transformation process is:

[0012]

[0013] In formula (1), Indicates that the horizontal plane B is in the sensor coordinate system x s y s z s The normal vector in , Indicates that the horizontal plane B is in the horizontal coordinate system x l y l z l The normal vector in, n sx express The x-component value, n sy express The y component value, n sz express The z component value of

[0014] The calculation formula of image spin α includes:

[0015] α=arc tg (-n sy / n sz ) Formula (2)

[0016] In formula (2), arc tg represents the inverse tangent function, n sy express The y component value, n sz express The z component value of

[0017] Step 4: Using the image rotation α as a rotation parameter, the rotation and cropping mechanism of OpenGL is used to derotate the video texture map for display.

[0018] As a further limitation of the present invention, the step 1 comprises:

[0019] The image acquisition device is installed on a two-axis turntable, the base of the two-axis turntable is connected to the inertial navigation device, and the azimuth axis of the two-axis turntable is perpendicular to the reference plane of the inertial navigation device;

[0020] Move the image acquisition device so that its optical axis is parallel to the x-axis of the inertial navigation unit. At this time, the azimuth position of the two-axis turntable is the azimuth zero position.

[0021] The image acquisition device acquires the original image in real time, and determines the parameter information of the video texture map according to the size of the original image and the position of the display window; wherein the parameter information includes translation parameters, scale parameters and cropping parameters.

[0022] As a further limitation of the present invention, the step 1 further includes:

[0023] The turntable coordinate system x is established based on the turntable base of the two-axis turntable p y p z p , establish the sensor coordinate system x based on the optical sensor s y s z s , establish the horizontal coordinate system x based on the horizontal plane l y l z l ;Specifically:

[0024] Turntable coordinate system x p y p z p The z-axis is parallel to the turntable azimuth axis, and the turntable coordinate system x p y p z p The x-axis is parallel to the zero-position direction of the turntable, and the turntable coordinate system x p y p z p The y-axis is parallel to the 90° direction of the turntable; the turntable coordinate system x p y p z p Rotate the turntable azimuth around the z-axis Pitch angle θ around the y-axis s Then convert to the sensor coordinate system; turntable azimuth Pitch angle θ s Measured by the shaft angle sensor on the two-axis turntable;

[0025] Sensor coordinate system x s y s z s The center of the image plane is the origin, the top of the image plane is the z-axis, the optical axis of the sensor is the x-axis, and the right of the image plane is the y-axis;

[0026] Horizontal coordinate system x l y l z l x l y l The plane is horizontal, and the horizontal coordinate system x l y l z l Pitch angle θ around the y-axis p , Roll angle γ around the x-axis p Transform to the turntable coordinate system x p y p z p ; Pitch angle θ p represents the turntable plane x p y p Relative level x l y l Pitch angle, roll angle γ p represents the turntable plane x p y p Relative level x l y l Roll angle, pitch angle θ p and roll angle γ p These are the pitch and roll components of the attitude angle data output by the inertial navigation unit.

[0027] As a further limitation of the present invention, the step 2 comprises:

[0028] Step (21) uses the azimuth data and the pitch angle data during the turntable rotation to calculate the rotation transformation matrix A1, which is expressed as:

[0029]

[0030] In formula (3), represents the azimuth data during the turntable rotation, and θ represents the pitch angle data during the turntable rotation; when calculating the rotation transformation matrix A1, the roll amount γ during the turntable rotation is 0;

[0031] Step (22) uses the inertial navigation data of the inertial navigation system to calculate the rotation transformation matrix A2, which is expressed as:

[0032]

[0033] In formula (4), θ p represents the turntable plane xp y p Relative level x l y l The pitch angle, γ p represents the turntable plane x p y p Relative level x l y l The roll angle of the turntable is 0 when calculating the rotation transformation matrix A2.

[0034] As a further limitation of the present invention, the step three comprises:

[0035] In the sensor coordinate system x s y s z s In the plane y s z s As the reference plane C, the equation of the reference plane C in the sensor coordinate system is:

[0036]

[0037] In the horizontal coordinate system x l y l z l Select plane y l z l As the horizontal plane B, in the horizontal coordinate system, the normal vector of the horizontal plane B is [0,0,1] T , the horizontal plane B is in the sensor coordinate system x s y s z s The point-wise equation in is:

[0038] n sx x+n sy y+n sz z=0 Formula (6)

[0039] In formula (6), n sx express The x-component value, n sy express The y component value, n sz express The z component value of

[0040] Calculate n from the following formula sx 、n sy and n sz :

[0041]

[0042] In formula (7), the rotation transformation matrix A1 is expressed as:

[0043]

[0044] In formula (3), represents the azimuth data during the turntable rotation, and θ represents the pitch angle data during the turntable rotation;

[0045] In formula (7), the rotation transformation matrix A2 is expressed as:

[0046]

[0047] In formula (4), θ p represents the turntable plane x p y p Relative level x l y l The pitch angle, γ p represents the turntable plane x p y p Relative level x l y l Roll angle;

[0048] Combining formula (5) and formula (6), we can get the intersection line L of reference plane C and horizontal plane B in sensor coordinate system x s y s z s The following equation is:

[0049]

[0050] In formula (8), x represents the sensor coordinate system x s y s z s The x-axis coordinate of n sy express The y component value of the sensor coordinate system x s y s z s The y-axis coordinate of n sz express The z component value of the sensor coordinate system x s y s z s The z-axis coordinate of the bottom;

[0051] The intersection line L of the reference plane C and the horizontal plane B is in the sensor coordinate system x s y s z s The angle between the image and the y-axis is consistent with the angle between the horizontal line and the bottom edge of the image in the output image of the image acquisition device. Both angles are image rotation α. Specifically, the calculation formula for solving the image rotation α is as follows:

[0052]

[0053] As a further limitation of the present invention, the step 4 includes:

[0054] Based on the image rotation α as the rotation parameter, in the window rendered by OpenGL, the rotation and clipping mechanism of OpenGL is used to perform image rotation elimination.

[0055] As a further limitation of the present invention, the step 4 is specifically as follows:

[0056] The size of the original image in step (41) is w×h, and the video display area is a rectangular area centered at (x, y), with the width and height of the video display area being sw and sh respectively;

[0057] Step (42) draws an inscribed circle O on the image sheet of the original image, and draws an inscribed rectangle R with an aspect ratio of sw / sh within the inscribed circle O, wherein the width and height of the inscribed rectangle R are w′×h′, and the scale factor is f=w′×h′;

[0058] Step (43) The distance between the lower left corner of the inscribed rectangle R and the left side of the window is x0, and the distance between the lower left corner and the bottom of the window is y0. According to the OpenGL clipping mechanism, the clipping is set with the base point (x0, y0) and the width and height (sw, sh) as the clipping parameters;

[0059] Step (44) sets the base point (x0, y0) drawn by OpenGL as the origin, moves the base point to (x, y), performs a scaling transformation with a magnification of f, rotates by an angle of -α, moves the base point (x, y) to (-w / 2, -h / 2), maps the original image to a rectangular area of w×h, and derotates the video texture map.

[0060] The advantages of the present invention are:

[0061] 1. The present invention calculates the angle of the horizon projection in the image space based on the attitude angle of the two-axis turntable base and the axis angle data of the two-axis turntable itself, and uses the angle as the rotation compensation amount during display, namely the image rotation α, and uses the rotation and clipping mechanism of OpenGL to achieve display.

[0062] 2. The present invention uses the angle data generated by the movement of a two-axis turntable to obtain image rotation parameters. The data is reliable and is not affected by the content of the image scene. The present invention directly calculates the image rotation based on an analytical function, which requires little calculation and consumes little resources. In addition, the present invention implements display based on the rotation and cropping mechanisms of OpenGL, fully utilizing the GPU rendering pipeline, and effectively responding to the real-time derotation task of high-resolution video.

[0063] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments with reference to the following drawings, in which:

[0065] Figure 1 : A flow chart of a two-axis turntable derotation method based on inertial navigation data provided by the present invention;

[0066] Figure 2 : A diagram showing the correspondence between the coordinate system and the physical entity of the device in the embodiment provided by the present invention;

[0067] Figure 3 : Illustration of the rotation relationship between the platform coordinate system and the sensor coordinate system provided by the present invention;

[0068] Figure 4 : A diagram illustrating the rotation relationship between the horizontal coordinate system and the platform coordinate system provided by the present invention;

[0069] Figure 5 : Illustration of the spatial relationship between the reference plane, horizontal plane, and image spinor provided by the present invention;

[0070] Figure 6 : The original image of the sensor provided by the present invention is cropped;

[0071] Figure 7 : Flowchart of the method for derotating an image on a two-axis turntable according to an embodiment of the present invention;

[0072] Figure 8 : Illustration of the effects of disabling and enabling de-rotation provided by the present invention. DETAILED DESCRIPTION

[0073] The following describes in detail embodiments of the present invention. The embodiments are exemplary and intended to explain the present invention, but are not to be construed as limiting the present invention.

[0074] For optical observation devices mounted on a two-axis turntable, the turntable's posture adjustment provides two degrees of freedom (Azimuth and Pitch), ensuring free adjustment of the optical axis of the image acquisition device. Furthermore, because the turntable lacks a roll degree of freedom, when the optical axis of the image acquisition device is adjusted, the roll angle of the image is passively adjusted, constrained by the Azimuth and Pitch angles. In other words, the roll motion of the image acquisition device is driven by the Azimuth and Pitch axis movements. Therefore, when operating the turntable to search for a target, the continuous changes in the Azimuth and Pitch axis angles drive the roll motion of the image acquisition device, causing the image output by the image acquisition device to continuously roll. This can easily cause dizziness in the observer, and the resulting disorientation caused by the roll increases the difficulty of operation.

[0075] See also Figure 1 and Figure 7 The present invention discloses a method for derotating a two-axis turntable image based on inertial navigation data. Based on the turntable's base attitude angle and its own axis angle data, the angle of the horizon's projection in image space is calculated. This angle is used as the rotation compensation for display, i.e., the image rotation, and is displayed using OpenGL's rotation and cropping mechanisms. The method includes the following steps:

[0076] Step 1: An image acquisition device installed on a two-axis turntable acquires original images in real time, and determines parameter information of the video texture map based on the original images.

[0077] Specifically, the above step 1 of the embodiment of the present invention includes:

[0078] The image acquisition device is installed on a two-axis turntable, the base of the two-axis turntable is connected to the inertial navigation device, and the azimuth axis of the two-axis turntable is perpendicular to the reference plane of the inertial navigation device;

[0079] Move the image acquisition device so that its optical axis is parallel to the x-axis of the inertial navigation unit. At this time, the azimuth position of the two-axis turntable is the azimuth zero position.

[0080] The image acquisition device acquires the original image in real time and determines the parameter information of the video texture map according to the size of the original image and the position of the display window; wherein the parameter information includes translation parameters, scale parameters and cropping parameters.

[0081] More specifically, see Figure 2 Preferably, the image acquisition device of the embodiment of the present invention is a television camera, and the above step 1 of the embodiment of the present invention further includes: establishing a turntable coordinate system x based on the turntable base of the two-axis turntable p y p z p , establish the sensor coordinate system x based on the optical sensor s y s z s, establish the horizontal coordinate system x based on the horizontal plane l y l z l ;Specifically:

[0082] (1) Turntable coordinate system x p y p z p The z-axis is parallel to the turntable azimuth axis, and the turntable coordinate system x p y p z p The x-axis is parallel to the zero-position direction of the turntable, and the turntable coordinate system x p y p z p The y-axis is parallel to the 90° direction of the turntable; the turntable coordinate system x p y p z p Rotate the turntable azimuth around the z-axis Pitch angle θ around the y-axis s Then convert to the sensor coordinate system; turntable azimuth Pitch angle θ s Measured by the shaft angle sensor on the two-axis turntable; see Figure 3 , the turntable coordinate system x in the embodiment of the present invention p y p z p Around z P Axis rotation turntable azimuth Get the transition coordinate system x'y p z p , transition coordinate system x'y p z p Then go around y s Axis rotation pitch angle θ s Get the sensor coordinate system x s y s z s The embodiment of the present invention establishes the turntable coordinate system x p y p z p When the center of the image plane of the TV camera is taken as the origin, the z-axis is parallel to the azimuth axis of the turntable, the x-axis is parallel to the azimuth zero direction, and the y-axis is parallel to the azimuth 90 degree direction.

[0083] (2) Sensor coordinate system x s y s z s The center of the image plane is taken as the origin, the z-axis is taken as the top of the image plane, the x-axis is taken as the optical axis of the sensor, and the y-axis is taken as the right side of the image plane. In the embodiment of the present invention, the center of the image plane of the TV camera is taken as the origin, and the TV camera is rotated so that the optical axis of the TV camera is parallel to the x-axis of the inertial navigation device. At this time, the azimuth position of the turntable is the azimuth zero position.

[0084] (3) Horizontal coordinate system x l y l z l x l y l The plane is horizontal, and the horizontal coordinate system x l y l z l Pitch angle θ around the y-axis p , Roll angle γ around the x-axis p Transform to the turntable coordinate system x p y p z p ; Pitch angle θ p represents the turntable plane x p y p Relative level x l y l Pitch angle, roll angle γ p represents the turntable plane x p y p Relative level x l y l Roll angle, pitch angle θ p and roll angle γ p The pitch and roll components of the attitude angle data output by the inertial navigation unit. Figure 4 , the horizontal coordinate system x in the embodiment of the present invention l y l z l Around y l Axis rotation pitch angle θ p Get the transition coordinate system x p y l z', transition coordinate system x p y l z' then goes around x p Axis rotation roll angle γ p Transform to the turntable coordinate system x p y p z p .

[0085] The turntable coordinate system x in the embodiment of the present invention p y p z p , sensor coordinate system x s y s z s , horizontal coordinate system x l y l z l The embodiment of the present invention utilizes an inertial navigation device fixedly connected to the base of an optical observation device of a two-axis turntable to measure the attitude angle of the turntable base in real time.

[0086] Step 2: Calculate the rotation transformation matrix A1 using the azimuth data and pitch angle data generated by the turntable rotation, and calculate the rotation transformation matrix A2 using the inertial navigation data of the inertial navigation system; wherein, the rotation transformation matrix A1 is the turntable coordinate system x p y p z p To the sensor coordinate system x s y s z s The rotation transformation matrix A2 is the horizontal coordinate system x l y l z l To the turntable coordinate system x p y p z p The rotation transformation matrix.

[0087] Specifically, the above step 2 of the embodiment of the present invention includes:

[0088] Step (21) uses the azimuth data and the pitch angle data during the turntable rotation to calculate the rotation transformation matrix A1, which is expressed as:

[0089]

[0090] In formula (3), represents the azimuth data during the turntable rotation, and θ represents the pitch angle data during the turntable rotation; when calculating the rotation transformation matrix A1, the roll amount γ during the turntable rotation is 0;

[0091] Step (22) uses the inertial navigation data of the inertial navigation system to calculate the rotation transformation matrix A2, which is expressed as:

[0092]

[0093] In formula (4), θ p represents the turntable plane x p y p Relative level x l y l The pitch angle, γ p represents the turntable plane x p y p Relative level x l y l The roll angle of the turntable is 0 when calculating the rotation transformation matrix A2.

[0094] The embodiment of the present invention utilizes the turntable base attitude angle, combines the turntable azimuth and pitch axis angle data, and calculates the image rotation based on the rotation transformation principle of the coordinate system.

[0095] Step 3: Convert the normal vector of the horizontal plane from the horizontal coordinate system x l yl z l Transform to sensor coordinate system x s y s z s , and calculate the image rotation α; the transformation process is:

[0096]

[0097] In formula (1), Indicates that the horizontal plane B is in the sensor coordinate system x s y s z s The normal vector in , Indicates that the horizontal plane B is in the horizontal coordinate system x l y l z l The normal vector in, n sx express The x-component value, n sy express The y component value, n sz express The z component value of

[0098] The calculation formula of image spin α includes:

[0099] α=arc tg(-n sy / n sz ) Formula (2)

[0100] In formula (2), arc tg represents the inverse tangent function, n sy express The y component value, n sz express The z-component value of .

[0101] Specifically, the above step three of the embodiment of the present invention includes:

[0102] In the sensor coordinate system x s y s z s In the plane y s z s As the reference plane C, the equation of the reference plane C in the sensor coordinate system is:

[0103] x=0Formula (5);

[0104] In the horizontal coordinate system x l y l z l Select plane y l z l As the horizontal plane B, in the horizontal coordinate system, the normal vector of the horizontal plane B is [0,0,1]T , the horizontal plane B is in the sensor coordinate system x s y s z s The point-wise equation in is:

[0105] n sx x+n sy y+n sz z=0 Formula (6)

[0106] In formula (6), n sx express The x-component value, n sy express The y component value, n sz express The z component value of

[0107] Calculate n from the following formula sx 、n sy and n sz :

[0108] [n sx ,n sy ,n sz ] T =A1A2[0,0,1] T Formula (7)

[0109] In formula (7), the rotation transformation matrix A1 is expressed as:

[0110]

[0111] In formula (3), represents the azimuth data during the turntable rotation, and θ represents the pitch angle data during the turntable rotation;

[0112] In formula (7), the rotation transformation matrix A2 is expressed as:

[0113]

[0114] In formula (4), θ p represents the turntable plane x p y p Relative level x l y l The pitch angle, γ p represents the turntable plane x p y p Relative level x l y l Roll angle;

[0115] Combining formula (5) and formula (6), we can get the intersection line L of reference plane C and horizontal plane B in sensor coordinate system xs y s z s The following equation is:

[0116]

[0117] In formula (8), x represents the sensor coordinate system x s y s z s The x-axis coordinate of n sy express The y component value of the sensor coordinate system x s y s z s The y-axis coordinate of n sz express The z component value of the sensor coordinate system x s y s z s The z-axis coordinate of the bottom;

[0118] The intersection line L of the reference plane C and the horizontal plane B is in the sensor coordinate system x s y s z s The angle between the center and the y-axis is consistent with the angle between the horizontal line and the bottom edge of the image in the image acquisition device output image. Both angles are the image rotation α. The calculation formula for solving the image rotation α is as follows:

[0119]

[0120] More specifically, the embodiment of the present invention transforms the point normal equation of the horizontal plane B into the sensor coordinate system x s y s z s In the transformation process, the specific process is: horizontal coordinate system x l y l z l To the turntable coordinate system x p y p z p , turntable coordinate system x p y p z p To the sensor coordinate system x s y s z s Two successive rotation processes, horizontal coordinate system x l y l z l To the turntable coordinate system x p y p z p The rotation process is calculated by referring to the inertial navigation pitch and roll angles, and the turntable coordinate system x p yp z p To the sensor coordinate system x s y s z s The rotation process is calculated by referring to the turntable azimuth and pitch angle, and the intersection line L of the reference plane C and the horizontal plane B is obtained in the sensor coordinate system x s y s z s Since the intersection line L is on the reference plane C, the equation of the intersection line L is a function related only to y and z. In the sensor coordinate system x s y s z s The angle between the intersection line L and the y-axis is calculated on the yz plane, which is the image spin α.

[0121] Step 4: Use the image rotation α as a rotation parameter to use the OpenGL rotation and clipping mechanism to derotate the video texture map.

[0122] Specifically, step 4 of this embodiment of the present invention includes: using the image rotation amount α as a rotation parameter, utilizing OpenGL's rotation and cropping mechanisms to perform derotational display in a window rendered using OpenGL. It should be noted that in this embodiment of the present invention, the window's origin is the window center, with the rightward direction being the positive x-axis direction and the upward direction being the positive y-axis direction.

[0123] More specifically, see Figure 6 , step 4 of the embodiment of the present invention is specifically as follows:

[0124] Step (41) The original image has a size of w×h, the video display area is centered at (x, y), and the width and height of the video display area are a rectangular area of sw and sh respectively; Figure 6 (a) shows that the original image size of the TV camera is 1920×1080. A 1920×1080 window rendered with OpenGL is created. The size of the video display area is 1024×768. The upper edge of the image display area is 50 degrees from the top of the window and is horizontally centered, that is:

[0125] (x, y) = (102, 0), w = 1920, h = 1080, sw = 1024, sh = 768, (x0, y0) = (448,262).

[0126] Step (42) Figure 6 As shown in (a), an inscribed circle O is made for the original image, the diameter of the inscribed circle is 1080, and an inscribed rectangle R with an aspect ratio of sw / sh is made inside the inscribed circle O. The width and height of the inscribed rectangle R are w'×h', and the scale factor is f=w'×h'; preferably The width of the inscribed rectangle R is w′=1080×4÷(42 +3 2 ) 0.5 =1080×0.8=864, the height h′ of the inscribed rectangle R=w′×3÷4=648, the scale factor f=1024÷

[0127] 864=1.1852.

[0128] Step (43) The distance between the lower left corner of the inscribed rectangle R and the left side of the window is x0, and the distance between the lower left corner and the bottom of the window is y0. According to the OpenGL clipping mechanism, the clipping is set with the base point (x0, y0) and the width and height (sw, sh) as the clipping parameters;

[0129] Step (44) Figure 6 As shown in (b), the base point (x0, y0) drawn by OpenGL is set as the origin, the base point is moved to (x, y), a scaling transformation with a magnification of f is performed, and the -α angle is rotated. The base point (x, y) is moved to (-w / 2, -h / 2), and the original image is mapped to a rectangular area of w×h, and the video texture map is derotated and displayed.

[0130] Figure 8 (a) shows the effect of the method of disabling the two-axis turntable derotation in the application example of the embodiment of the present invention, Figure 8 (b) illustrates the effects of using a two-axis turntable derotation method in an example application of an embodiment of the present invention. This method, based on the calculated image rotation, compensates for the reverse rotation of the original image output by the image acquisition device during display, thereby achieving two-axis turntable derotation of inertial navigation data.

[0131] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present invention, and these modifications or replacements should all be included in the scope of protection of the present invention.

Claims

1. A two-axis turntable derotation method based on inertial navigation data, characterized in that: include: Step 1: An image acquisition device mounted on a two-axis turntable acquires an original image in real time, and determines parameter information of a video texture map based on the original image; Step 2: Calculate the rotation transformation matrix A1 using the azimuth data and pitch angle data generated by the turntable rotation, and calculate the rotation transformation matrix A2 using the inertial navigation data of the inertial navigation system; wherein, the rotation transformation matrix A1 is the turntable coordinate system x p y p z p To the sensor coordinate system x s y s z s The rotation transformation matrix A2 is the horizontal coordinate system x l y l z l To the turntable coordinate system x p y p z p The rotation transformation matrix; Step 3: Convert the normal vector of the horizontal plane from the horizontal coordinate system x l y l z l Transform to sensor coordinate system x s y s z s , and calculate the image spinor α; the transformation process is: In formula (1), Indicates that the horizontal plane B is in the sensor coordinate system x s y s z s The normal vector in , Indicates that the horizontal plane B is in the horizontal coordinate system x l y l z l The normal vector in, n sx express The x-component value, n sy express The y component value, n sz express The z component value of The calculation formula of image spin α includes: α=arc tg (-n sy / n sz ) Formula (2) In formula (2), arc tg represents the inverse tangent function, n sy express The y component value, n sz express The z component value of Step 4: Using the image rotation α as a rotation parameter, the rotation and cropping mechanism of OpenGL is used to derotate the video texture map for display.

2. The method for derotating a two-axis turntable based on inertial navigation data according to claim 1, characterized in that: The step one comprises: The image acquisition device is installed on a two-axis turntable, the base of the two-axis turntable is connected to the inertial navigation device, and the azimuth axis of the two-axis turntable is perpendicular to the reference plane of the inertial navigation device; Move the image acquisition device so that its optical axis is parallel to the x-axis of the inertial navigation unit. At this time, the azimuth position of the two-axis turntable is the azimuth zero position. The image acquisition device acquires the original image in real time, and determines the parameter information of the video texture map according to the size of the original image and the position of the display window; wherein the parameter information includes translation parameters, scale parameters and cropping parameters.

3. The method for derotating a two-axis turntable based on inertial navigation data according to claim 1 or 2, characterized in that: The step one further comprises: The turntable coordinate system x is established based on the turntable base of the two-axis turntable p y p z p , establish the sensor coordinate system x based on the optical sensor s y s z s , establish the horizontal coordinate system x based on the horizontal plane l y l z l ;Specifically: Turntable coordinate system x p y p z p The z-axis is parallel to the turntable azimuth axis, and the turntable coordinate system x p y p z p The x-axis is parallel to the zero-position direction of the turntable, and the turntable coordinate system x p y p z p The y-axis is parallel to the 90° direction of the turntable; the turntable coordinate system x p y p z p Rotate the turntable azimuth around the z-axis Pitch angle θ around the y-axis s Then convert to the sensor coordinate system; turntable azimuth Pitch angle θ s Measured by the shaft angle sensor on the two-axis turntable; Sensor coordinate system x s y s z s The center of the image plane is the origin, the top of the image plane is the z-axis, the optical axis of the sensor is the x-axis, and the right of the image plane is the y-axis; Horizontal coordinate system x l y l z l x l y l The plane is horizontal, and the horizontal coordinate system x l y l z l Pitch angle θ around the y-axis p , Roll angle γ around the x-axis p Transform to the turntable coordinate system x p y p z p ; Pitch angle θ p represents the turntable plane x p y p Relative level x l y l Pitch angle, roll angle γ p represents the turntable plane x p y p Relative level x l y l Roll angle, pitch angle θ p and roll angle γ p These are the pitch and roll components of the attitude angle data output by the inertial navigation unit.

4. The method for derotating a two-axis turntable based on inertial navigation data according to claim 1, characterized in that: The second step includes: Step (21) uses the azimuth data and the pitch angle data during the turntable rotation to calculate the rotation transformation matrix A1, which is expressed as: In formula (3), represents the azimuth data during the turntable rotation, and θ represents the pitch angle data during the turntable rotation; when calculating the rotation transformation matrix A1, the roll amount γ during the turntable rotation is 0; Step (22) uses the inertial navigation data of the inertial navigation system to calculate the rotation transformation matrix A2, which is expressed as: In formula (4), θ p represents the turntable plane x p y p Relative level x l y l The pitch angle, γ p represents the turntable plane x p y p Relative level x l y l The roll angle of the turntable is 0 when calculating the rotation transformation matrix A2.

5. The method for derotating a two-axis turntable based on inertial navigation data according to claim 1, characterized in that: The step three includes: In the sensor coordinate system x s y s z s In the plane y s z s As the reference plane C, the equation of the reference plane C in the sensor coordinate system is: x=0 Formula (5); In the horizontal coordinate system x l y l z l Select plane y l z l As the horizontal plane B, in the horizontal coordinate system, the normal vector of the horizontal plane B is [0,0,1] T , the horizontal plane B is in the sensor coordinate system x s y s z s The point-wise equation in is: n sx ·x + n sy ·y + n sz ·z = 0 Equation (6) In formula (6), n sx express The x-component value, n sy express The y component value, n sz express The z component value of Calculate n from the following formula sx 、n sy and n sz : [n sx ,n sy ,n sz ] T =A1A2[0,0,1] T Formula (7) In formula (7), the rotation transformation matrix A1 is expressed as: In formula (3), represents the azimuth data during the turntable rotation, and θ represents the pitch angle data during the turntable rotation; In formula (7), the rotation transformation matrix A2 is expressed as: In formula (4), θ p represents the turntable plane x p y p Relative level x l y l The pitch angle, γ p represents the turntable plane x p y p Relative level x l y l Roll angle; Combining formula (5) and formula (6), we can get the intersection line L of reference plane C and horizontal plane B in sensor coordinate system x s y s z s The following equation is: In formula (8), x represents the sensor coordinate system x s y s z s The x-axis coordinate of n sy express The y component value of the sensor coordinate system x s y s z s The y-axis coordinate of n sz express The z component value of the sensor coordinate system x s y s z s The z-axis coordinate of the bottom; The intersection line L of the reference plane C and the horizontal plane B is in the sensor coordinate system x s y s z s The angle between the center and the y-axis is consistent with the angle between the horizontal line and the bottom edge of the image in the image acquisition device output image. Both angles are the image rotation α. The calculation formula for solving the image rotation α is as follows:

6. The method for derotating a two-axis turntable based on inertial navigation data according to claim 1, characterized in that: The fourth step includes: Based on the image rotation α as the rotation parameter, in the window rendered by OpenGL, the rotation and clipping mechanism of OpenGL is used to perform image rotation elimination.

7. The method for derotating a two-axis turntable based on inertial navigation data according to claim 6, characterized in that: The step 4 is specifically as follows: The size of the original image in step (41) is w×h, and the video display area is a rectangular area centered at (x, y), with the width and height of the video display area being sw and sh respectively; Step (42) draws an inscribed circle O on the image sheet of the original image, and draws an inscribed rectangle R with an aspect ratio of sw / sh within the inscribed circle O, wherein the width and height of the inscribed rectangle R are w′×h′, and the scale factor is f=w′×h′; Step (43) The distance between the lower left corner of the inscribed rectangle R and the left side of the window is x0, and the distance between the lower left corner and the bottom of the window is y0. According to the OpenGL clipping mechanism, the clipping is set with the base point (x0, y0) and the width and height (sw, sh) as the clipping parameters; Step (44) sets the base point (x0, y0) drawn by OpenGL as the origin, moves the base point to (x, y), performs a scaling transformation with a magnification of f, rotates by an angle of -α, moves the base point (x, y) to (-w / 2, -h / 2), maps the original image to a rectangular area of w×h, and derotates the video texture map.

Images