Pose correction method and device, and storage medium

Abstract

The present disclosure relates to a pose correction method, device and storage medium. The pose correction method comprises: acquiring a target image captured by a camera, and performing optical design parameter-based de-distortion on the target image. In the image after the optical design parameter-based de-distortion, an area of interest satisfying a field of view threshold range is extracted. Based on the area of interest, a target angle is determined, the target angle being an included angle between an imaging sensor of the camera and a target when the camera captures the target image. Based on the target angle, a relative pose between the camera and the target is corrected. Through the present disclosure, the hardware environment setting problem of distortion correction in the camera production process is optimized, the error is reduced, and the correction accuracy is improved.

Description

Technical Field

[0001] This disclosure relates to the field of camera technology, and in particular to a pose correction method, device and storage medium. Background Technology

[0002] With the continuous advancement of technology, cameras are being used more and more widely in daily life and in the field of technology, such as in smart terminals, high-end robotics, and autonomous driving.

[0003] Optical distortion correction based on optical design parameters is a common method to ensure the accuracy of subsequent image processing. However, current optical distortion correction based on optical design parameters mainly corrects distortion from the perspective of the image, which still suffers from low accuracy and unsatisfactory results. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a pose correction method, device and storage medium.

[0005] According to a first aspect of the present disclosure, a pose correction method is provided, comprising: acquiring a target image captured by a camera, and performing distortion correction on the target image based on optical design parameters; extracting a region of interest (ROI) that satisfies a field-of-view threshold range in the distortion-corrected image; determining a target angle based on the ROI, the target angle being the angle between the imaging sensor of the camera and the target when the camera captures the target image; and correcting the relative pose between the camera and the target based on the target angle.

[0006] In one embodiment, determining the target angle based on the region of interest includes: mapping the region of interest to a world coordinate system; determining the target angle between the target image and the target coordinate axis in the world coordinate system based on target pixels in the region of interest; the target pixels include pixels in the same row direction in the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixels include pixels in the same column direction in the region of interest, and the target coordinate axis is the y-axis of the world coordinate system; or the target pixels include pixels in the direction of the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system.

[0007] In another embodiment, the target pixel includes pixels in the same row direction of the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixel includes pixels in the same column direction of the region of interest, and the target coordinate axis is the y-axis of the world coordinate system; determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest includes: determining a first distance between a first pixel in the target pixels and the target coordinate axis in the world coordinate system; determining a second distance between a second pixel in the target pixels and the target axis in the world coordinate system; determining a first projection length of the target pixel on the target coordinate axis in the world coordinate system; and performing an arctangent function operation based on the first distance, the second distance, and the first projection length to obtain the target angle.

[0008] In another embodiment, the target pixel includes pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system. Determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest includes: obtaining a second projection length of the target pixel in the world coordinate system; determining the length of the pixels in the region of interest along the line connecting the imaging sensor and the target in the world coordinate system as a third distance; determining the difference between the third distance and the second projection length; determining the ratio between the distance between the imaging sensor and the target and the focal length of the camera to obtain a first ratio; determining the ratio between the difference and the first ratio to obtain a second ratio; and performing an arctangent function operation on the second ratio to obtain the target angle.

[0009] In another embodiment, the step of correcting the relative pose between the camera and the target based on the target angle includes: if there is a target angle greater than or equal to a preset angle threshold, then adjusting the target or the camera in the opposite direction along the target coordinate axis. The target angle includes a first angle between the target image and the x-axis direction of the world coordinate system; or a second angle between the target image and the y-axis direction of the world coordinate system; or a third angle between the target image and the z-axis direction of the world coordinate system.

[0010] In another embodiment, correcting the relative pose between the camera and the target based on the target angle includes: if the target angle is less than a preset angle threshold, generating a rotation matrix based on the target angle, and adjusting the target or the camera based on the rotation matrix. The target angle includes a first angle between the target image and the x-axis of the world coordinate system, a second angle between the target image and the y-axis of the world coordinate system, and a third angle between the target image and the z-axis of the world coordinate system.

[0011] In another embodiment, the method further includes: before extracting the region of interest that satisfies the field of view threshold range in the distortion-free image, determining the field of view threshold range corresponding to when the optical design parameters are less than a set distortion threshold based on the correspondence between the optical design parameters and the field of view.

[0012] According to a second aspect of the present disclosure, a pose correction device is provided, comprising: a processing unit, configured to acquire a target image captured by a camera and to perform distortion correction on the target image based on optical design parameters; an extraction unit, configured to extract a region of interest satisfying a field-of-view threshold range in the distortion-corrected image; a determination unit, configured to determine a target angle in the region of interest, the target angle being the angle between the imaging sensor of the camera and the target when the camera captures the target image; and a correction unit, configured to correct the relative pose between the camera and the target based on the target angle.

[0013] In one embodiment, the determining unit determines the target angle based on the region of interest (ROI) in the following manner: mapping the ROI to a world coordinate system; determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the ROI; wherein the target pixels include pixels in the same row direction in the ROI, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixels include pixels in the same column direction in the ROI, and the target coordinate axis is the y-axis of the world coordinate system; or the target pixels include pixels in the direction of the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system.

[0014] In another embodiment, the target pixel includes pixels in the same row direction of the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixel includes pixels in the same column direction of the region of interest, and the target coordinate axis is the y-axis of the world coordinate system; the determining unit determines the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest in the following manner: determining a first distance between a first pixel in the target pixel and the target coordinate axis in the world coordinate system; determining a second distance between a second pixel in the target pixel and the target axis in the world coordinate system; determining a first projection length of the target pixel on the target coordinate axis in the world coordinate system; and performing an arctangent function operation based on the first distance, the second distance, and the first projection length to obtain the target angle.

[0015] In another embodiment, the target pixel includes pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system. The determining unit determines the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest as follows: obtaining the second projection length of the target pixel in the world coordinate system; determining the length of the pixels in the region of interest along the line connecting the imaging sensor and the target in the world coordinate system as a third distance; determining the difference between the third distance and the second projection length; determining the ratio between the distance between the imaging sensor and the target and the focal length of the camera to obtain a first ratio; determining the ratio between the difference and the first ratio to obtain a second ratio; and performing an arctangent function operation on the second ratio to obtain the target angle.

[0016] In another embodiment, the correction unit corrects the relative pose between the camera and the target based on the target angle in the following manner: if there is a target angle greater than or equal to a preset angle threshold, the target or the camera is adjusted in the opposite direction along the target coordinate axis. The target angle includes a first angle between the target image and the x-axis of the world coordinate system; or a second angle between the target image and the y-axis of the world coordinate system; or a third angle between the target image and the z-axis of the world coordinate system.

[0017] In another embodiment, the correction unit corrects the relative pose between the camera and the target based on the target angle as follows: if the target angle is less than a preset angle threshold, a rotation matrix is ​​generated based on the target angle, and the target or the camera is adjusted based on the rotation matrix. The target angle includes a first angle between the target image and the x-axis of the world coordinate system, a second angle between the target image and the y-axis of the world coordinate system, and a third angle between the target image and the z-axis of the world coordinate system.

[0018] In another embodiment, the apparatus further includes: before extracting the region of interest that satisfies the field of view threshold range in the distorted image, determining the field of view threshold range corresponding to when the optical design parameters are less than a set distortion threshold based on the correspondence between the optical design parameters and the field of view.

[0019] According to a third aspect of the present disclosure, a pose correction apparatus is provided, characterized in that it includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: perform a pose correction method according to the first aspect or any embodiment of the first aspect.

[0020] According to a fourth aspect of the present disclosure, a storage medium is provided, characterized in that the storage medium stores instructions that, when executed by a processor of a terminal, enable the terminal including the processor to perform the pose correction method of the first aspect or any embodiment of the first aspect.

[0021] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: During the pose correction process for the relatively parallel positional relationship between the camera and the target, a target image is acquired. After distortion correction of the target image based on optical design parameters, a region of interest is extracted, and the target angle between the target image and the camera imaging sensor is calculated within the region of interest. Pose correction of the target image is performed based on the target angle, reducing errors and improving correction accuracy.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0024] Figure 1 This is a flowchart illustrating a pose correction method according to an exemplary embodiment.

[0025] Figure 2 This is a flowchart illustrating a method for determining a target angle based on a region of interest, according to an exemplary embodiment.

[0026] Figure 3 This is a flowchart illustrating a method for determining the target angle between a target image and the target coordinate axis in the world coordinate system based on target pixels in a region of interest, according to an exemplary embodiment.

[0027] Figure 4 This is a flowchart illustrating a method for determining the target angle between a target image and the target coordinate axis in the world coordinate system based on target pixels in a region of interest, according to an exemplary embodiment.

[0028] Figure 5 This is a flowchart illustrating a method for correcting the relative pose between a camera and a target based on a target angle, according to an exemplary embodiment.

[0029] Figure 6 This is a flowchart illustrating a method for correcting the relative pose between a camera and a target based on a target angle, according to an exemplary embodiment.

[0030] Figure 7 This is a flowchart illustrating a method for correcting the relative pose between a camera and a target based on a target angle, according to an exemplary embodiment.

[0031] Figure 8 This is a flowchart illustrating the processing of a target image according to an exemplary embodiment.

[0032] Figure 9 This is a schematic diagram illustrating the distortion variation curve with the field of view according to an exemplary embodiment.

[0033] Figure 10 This is a schematic diagram illustrating the selection of a region of interest according to an exemplary embodiment.

[0034] Figure 11 A schematic diagram showing the angle between the target image 2 and the camera imaging sensor in the x-axis direction in an exemplary embodiment of this disclosure is shown.

[0035] Figure 12(a) shows a schematic diagram of the angle between the target image 2 and the camera imaging sensor in the x-axis direction in an exemplary embodiment of the present disclosure.

[0036] Figure 12(b) shows a schematic diagram of the angle between the target image 2 and the camera imaging sensor in the x-axis direction in an exemplary embodiment of this disclosure.

[0037] Figure 13A schematic diagram showing the angle between the target image 2 and the camera imaging sensor in the y-axis direction in an exemplary embodiment of this disclosure is shown.

[0038] Figure 14 A schematic diagram showing the angle between the target image 2 and the camera imaging sensor in the y-axis direction in an exemplary embodiment of this disclosure is shown.

[0039] Figure 15 A schematic diagram is shown showing the angle between the line connecting the target image 2 and the imaging sensor of the camera in an exemplary embodiment of this disclosure and the z-axis direction.

[0040] Figure 16 A schematic diagram is shown showing the angle between the line connecting the target image 2 and the imaging sensor of the camera in an exemplary embodiment of this disclosure and the z-axis direction.

[0041] Figure 17 This illustration shows a schematic diagram of adjusting the target according to the angle between the target image 2 and the sensor of the camera in the x-axis direction, according to an exemplary embodiment of the present disclosure.

[0042] Figure 18 This is a block diagram illustrating a pose correction device according to an exemplary embodiment.

[0043] Figure 19 This is a block diagram illustrating an apparatus for pose correction according to an exemplary embodiment. Detailed Implementation

[0044] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0045] In related technologies, a scheme is used to obtain distortion correction coefficients based on optical design parameters through distortion calibration and then perform distortion correction. However, this scheme cannot truly achieve the hardware environment setting for the relative position between the camera and the target. Camera distortion correction is often based on optical design parameters, without addressing the pose correction between the target and the camera. Furthermore, the hardware environment for distortion correction during camera manufacturing is often poor, resulting in a poor user experience during camera calibration.

[0046] To address the aforementioned issues, this disclosure provides a pose correction method to achieve the desired alignment between the target and the camera, such as ensuring the relative positions of the target and camera are parallel along the x, y, and z axes. It also optimizes the hardware environment settings for distortion correction during camera manufacturing.

[0047] Figure 1This is a flowchart illustrating a pose correction method according to an exemplary embodiment, such as... Figure 1 As shown, the pose correction method includes the following steps.

[0048] In step S11, the target image captured by the camera is acquired, and the target image is distorted based on optical design parameters.

[0049] In this embodiment of the disclosure, a target image captured by a camera is acquired. The target image is an image obtained by the camera from the target. For example, the target may be a drawing to correct camera distortion. The style of the target image may be a checkerboard pattern or a target drawing with straight line features; there is no limitation on the specific style of the target image.

[0050] In this embodiment of the disclosure, distortion is one of the important factors limiting the accuracy of optical measurements, causing image deformation. For an ideal optical system, the magnification is constant on a pair of conjugate object-image planes. However, for practical optical systems, this property only exists when the field of view is small. When the field of view is large or very large, the image magnification varies with the field of view, causing the image to lose its similarity to the object. This imaging defect that distorts the image is called distortion.

[0051] Among them, conjugate object-image planes are a pair of object-image planes matched according to a certain rule, or two object-image planes symmetrical about a certain axis.

[0052] In this embodiment, a target image is obtained by photographing a target drawing that can be in a checkerboard pattern or has straight line features. The obtained target image is then subjected to distortion correction based on optical design parameters. These optical design parameters can be adjusted using the optical design parameters themselves. These parameters can be percentages representing the ratio of the difference between the actual target image's height and the ideal target image's height to the ideal target image's height. One optical distortion correction operation can be performed on the target image based on the optical design parameters.

[0053] In step S12, regions of interest that satisfy the field of view threshold range are extracted from the distorted image.

[0054] In this embodiment of the disclosure, the field of view is the maximum range that the camera can observe.

[0055] In this embodiment, in the target image after optical distortion correction, a suitable field-of-view threshold is selected based on the distortion versus field-of-view curve plotted with the image height of the actual target image as the x-axis and the image height of the ideal target image as the y-axis. The field-of-view threshold range is the threshold for a field of view with minimal optical distortion. Pose correction is then performed within this field of view to reduce errors.

[0056] In this embodiment of the disclosure, the region of interest is the field of view region that satisfies the field of view threshold range.

[0057] In step S13, the target angle is determined based on the region of interest. The target angle is the angle between the camera's imaging sensor and the target when the camera captures the target image.

[0058] In this embodiment of the disclosure, the angle between the camera's imaging sensor and the target is determined within the region of interest of the target image. Specifically, the angle between the camera's imaging sensor and the target along the x, y, and z axes.

[0059] In step S14, the relative pose between the camera and the target is corrected based on the target angle.

[0060] In this embodiment of the present disclosure, based on the angle between the imaging sensor of the camera and the target on the x, y, and z axes, the corresponding angles are adjusted in the opposite direction on the x, y, and z axes to achieve the purpose of correcting the pose between the camera and the target.

[0061] The pose correction method provided in this disclosure makes the relative positions of the camera and the target as parallel as possible, optimizes the hardware environment settings for distortion correction of the camera during the production process, and improves the user experience.

[0062] The following embodiments further explain and illustrate the method for determining the target angle in the above embodiments of this disclosure.

[0063] Figure 2 This is a flowchart illustrating a method for determining a target angle based on a region of interest, according to an exemplary embodiment. Figure 2 As shown, the method for determining the target angle based on the region of interest includes the following steps.

[0064] In step S21, the region of interest is mapped to the world coordinate system.

[0065] In this embodiment, the region of interest (ROI) in the target image and the camera imaging sensor can be mapped to a world coordinate system, which can be arbitrary and not limited thereto. Based on the relative position between the ROI and the imaging sensor in the target image, the angle between them can be obtained.

[0066] In step S22, the target angle between the target image and the target coordinate axis in the world coordinate system is determined based on the target pixels in the region of interest.

[0067] In this embodiment of the disclosure, the target pixels in the region of interest can be pixels in the same row or column, and the target coordinate axis in the world coordinate axis can be the x-axis, y-axis or z-axis, but is not limited to the three axes of x-axis, y-axis and z-axis. For example, the axis of x=3 can also be used as the target coordinate axis.

[0068] In step S23, the target pixel includes pixels in the same row direction in the region of interest, and the target coordinate axis is the x-axis of the world coordinate system.

[0069] In this embodiment of the disclosure, the target pixel can be any pixel in the same row of the region of interest, and the target pixels in the same row can be abstracted as a straight line.

[0070] In step S24, the target pixel includes pixels in the same column direction in the region of interest, and the target coordinate axis is the y-axis of the world coordinate system.

[0071] In this embodiment of the disclosure, the target pixel can also be any pixel in the same column of the region of interest, and the target pixels in the same column can be abstracted as a straight line.

[0072] In step S25, the target pixel includes pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system.

[0073] In this embodiment of the disclosure, the target pixel can also be a pixel on the same row as the line connecting the imaging sensor of the camera and the region of interest on the target, and the target pixels on the same row can be abstracted as a straight line.

[0074] The following embodiments of this disclosure further explain and illustrate the method for determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest described above.

[0075] Figure 3 This is a flowchart illustrating a method for determining the target angle between a target image and the target coordinate axis in the world coordinate system based on target pixels in a region of interest, according to an exemplary embodiment. Figure 3 As shown, the method for determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest includes the following steps.

[0076] In step S31, the first distance between the first pixel in the target pixel and the target coordinate axis in the world coordinate system is determined.

[0077] In this embodiment of the disclosure, the target pixel is a pixel in the same row of the region of interest, and the first pixel can be any pixel on the straight line that the target pixel is abstracted from. A perpendicular line is drawn from the first pixel to the target coordinate axis, and the length of the perpendicular line is a first distance. Wherein, if the target pixel is a pixel in the same row of the region of interest, the target coordinate axis is the x-axis; if the target pixel is a pixel in the same column of the region of interest, the target coordinate axis is the y-axis.

[0078] In step S32, the second distance between the second pixel in the target pixel and the target coordinate axis in the world coordinate system is determined.

[0079] In this embodiment of the disclosure, the second pixel can be a pixel in the region of interest that is in the same row as the first pixel but different from the first pixel. A perpendicular line is drawn from the second pixel to the target coordinate axis, and the length of the perpendicular line is the second distance.

[0080] In step S33, the first projection length of the target pixel on the target coordinate axis of the world coordinate system is determined.

[0081] In this embodiment of the disclosure, the projection length of the target pixel on the target coordinate axis can be the distance between the perpendicular point of the first pixel on the target coordinate axis and the perpendicular point of the second pixel on the target coordinate axis.

[0082] In step S34, the arctangent function is calculated based on the first distance, the second distance, and the first projection length to obtain the target angle.

[0083] In this embodiment of the disclosure, if the first pixel and the second pixel are located on either side of the intersection point of pixels in the same row of the region of interest and the coordinate axis (i.e., the x-axis and y-axis are drawn with the intersection point of pixels in the same row of the first pixel and the second pixel and the coordinate axis as the origin), and the first pixel and the second pixel are respectively in the first quadrant and the third quadrant of the established coordinate system, or respectively in the second quadrant and the fourth quadrant of the established coordinate system, then the ratio between the sum of the first distance and the second distance and the first projection length is determined.

[0084] In this embodiment of the disclosure, if the first pixel and the second pixel are located on the same side of the intersection point of pixels in the same row of the region of interest and the coordinate axis, that is, the x-axis and y-axis are drawn with the intersection point of pixels in the same row of the first pixel and the second pixel and the coordinate axis as the origin, and the first pixel and the second pixel are respectively in the same quadrant of the established coordinate system, then the ratio between the absolute value of the distance difference between the first distance and the second distance and the first projection length is determined.

[0085] In this embodiment, the target angle can be obtained based on the arctangent function and the measured first distance, second distance, and first projected length. Specifically, if the target pixel is a pixel in the same row as the region of interest (ROI), the target angle is the angle between the ROI and the imaging sensor along the x-axis. If the target pixel is a pixel in the same column as the ROI, the target angle is the angle between the ROI and the imaging sensor along the y-axis.

[0086] In this embodiment of the disclosure, taking the intersection of the first pixel and the second pixel on the same row of the region of interest with the coordinate axis as an example, for example, the vertical distance from the first pixel on the same row of the region of interest to the x-axis of the world coordinate system is 1, the vertical distance from the second pixel in the row direction of the region of interest to the x-axis of the world coordinate system is 1, and the projection length on the x-axis of the world coordinate system is 2, the angle between the pixel in the row direction of the region of interest and the x-axis of the world coordinate system can be obtained by the arctangent function as 45 degrees, that is, the target angle is 45 degrees.

[0087] In this embodiment of the disclosure, taking the first pixel and the second pixel located on the same side of the intersection of the pixels in the same row of the region of interest and the coordinate axis as an example, for example, the vertical distance from the first pixel in the column direction of the region of interest to the y-axis of the world coordinate system is 3, the vertical distance from the second pixel in the column direction of the region of interest to the y-axis of the world coordinate system is 1, and the projection length on the y-axis of the world coordinate system is 2, the angle between the pixels in the column direction of the region of interest and the y-axis of the world coordinate system can be obtained according to the arctangent function as 45 degrees, that is, the target angle is 45 degrees.

[0088] The following embodiments of this disclosure further explain and illustrate the method for determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest described above.

[0089] Figure 4 This is a flowchart illustrating a method for determining the target angle between a target image and the target coordinate axis in the world coordinate system based on target pixels in a region of interest, according to an exemplary embodiment. Figure 4 As shown, the method for determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest includes the following steps.

[0090] In step S41, the second projection length of the target pixel in the world coordinate system is obtained.

[0091] In this embodiment of the disclosure, the target pixel can be projected into the world coordinate system, and the projection length can be measured to obtain the second projection length.

[0092] In step S42, the length of the pixels in the region of interest along the line connecting the imaging sensor and the target in the world coordinate system is determined as the third distance.

[0093] In step S43, the difference between the third distance and the second projection length is determined.

[0094] In step S44, the ratio between the distance between the imaging sensor and the target and the focal length of the camera is determined to obtain a first ratio.

[0095] In step S45, the ratio between the difference and the first ratio is determined to obtain the second ratio.

[0096] In step S46, the arctangent function is performed on the second ratio to obtain the target angle.

[0097] In this embodiment of the disclosure, for example, the pixel length of the region of interest along the line connecting the camera's imaging sensor and the target is denoted as 2 (third distance), the projection length in the world coordinate system is denoted as 1 (second projection length), the distance between the target and the camera's imaging sensor is 1, and the camera's focal length is 1. Based on the arctangent function, the angle along the line connecting the camera's imaging sensor and the target in the region of interest is determined to be 45 degrees, i.e., the target angle is 45 degrees.

[0098] The following embodiments further explain and illustrate the method for correcting the relative pose between the camera and the target based on the target angle in the above embodiments of this disclosure.

[0099] Figure 5 This is a flowchart illustrating a method for correcting the relative pose between a camera and a target based on a target angle, according to an exemplary embodiment. Figure 5 As shown, the method for correcting the relative pose between the camera and the target based on the target angle includes the following steps.

[0100] In step S51, if there is a target angle greater than or equal to a preset angle threshold, the target or camera is adjusted in the opposite direction along the target coordinate axis.

[0101] In this embodiment of the disclosure, the angle threshold can be set to 1 degree. When the target angle is greater than or equal to the preset angle threshold of 1 degree, it is considered that the angle between the target and the camera is large, and the angle of the camera or the target can be adjusted in the opposite direction.

[0102] In this embodiment of the disclosure, for example, the target angle formed by the target image and the x-axis of the world coordinate system is 3 degrees, the preset angle threshold is 1 degree, and the target angle is greater than the preset angle threshold. The target image or camera is adjusted 3 degrees in the opposite direction along the x-axis of the world coordinate system. The target angle formed by the target image and the y-axis of the world coordinate system is 5 degrees, the preset angle threshold is 1 degree, and the target angle is greater than the preset angle threshold. The target image is adjusted 5 degrees in the opposite direction along the y-axis of the world coordinate system. The target angle formed by the target image and the z-axis of the world coordinate system is 4 degrees, the preset angle threshold is 1 degree, and the target angle is greater than the preset angle threshold. The target image is adjusted 4 degrees in the opposite direction along the z-axis of the world coordinate system.

[0103] In step S52, the target angle includes a first angle between the target image and the x-axis of the world coordinate system; a second angle between the target image and the y-axis of the world coordinate system; and a third angle between the target image and the z-axis of the world coordinate system.

[0104] In this embodiment of the disclosure, the target angle includes the angle between a pixel in a row direction of the region of interest in the target image and the x-axis direction of the world coordinate system, or the angle between a pixel in a column direction of the region of interest in the target image and the y-axis direction of the world coordinate system, or the angle between the line connecting the region of interest in the target image and the camera imaging sensor and the z-axis direction of the world coordinate system.

[0105] The following embodiments further explain and illustrate the method for correcting the relative pose between the camera and the target based on the target angle in the above embodiments of this disclosure.

[0106] Figure 6 This is a flowchart illustrating a method for correcting the relative pose between a camera and a target based on a target angle, according to an exemplary embodiment. Figure 6 As shown, the method for correcting the relative pose between the camera and the target based on the target angle includes the following steps.

[0107] In step S61, if the target angle is less than a preset angle threshold, a rotation matrix is ​​generated based on the target angle, and the target or camera is adjusted based on the rotation matrix.

[0108] In this embodiment of the disclosure, the angle threshold is a preset angle, such as 1 degree. When the first angle formed by the target image and the x-axis direction of the world coordinate system, the second angle formed by the target image and the y-axis direction of the world coordinate system, and the third angle formed by the target image and the z-axis direction of the world coordinate system are all less than the preset angle of 1 degree, a rotation matrix is ​​generated, and the target is adjusted based on the rotation matrix.

[0109] In this embodiment of the disclosure, the rotation matrix can be calculated in the following way, for example: the rotation matrix generated based on the first angle α formed by the target image and the x-axis direction of the world coordinate system can be a three-row, three-column matrix, wherein the data in the first row of the matrix can be 1, 0, 0 from left to right, the data in the second row of the matrix can be 0, the cosine function of the first angle α (cosα), and the negative value of the sine function of the first angle α (-sinα) from left to right, and the data in the third row of the matrix can be 0, the sine function of the first angle α (sinα), and the cosine function of the first angle α (cosα) from left to right.

[0110] The rotation matrix generated based on the second angle β formed by the target image and the y-axis direction of the world coordinate system can be a three-row, three-column matrix. The data in the first row of the matrix, from left to right, can be the cosine function of the second angle β, cosβ, 0, and the sine function of the second angle β, sinβ. The data in the second row of the matrix, from left to right, can be 0, 1, and 0. The data in the third row of the matrix, from left to right, can be the negative value of the sine function of the second angle β, -sinβ, 0, and the cosine function of the second angle β, cosβ.

[0111] The rotation matrix generated based on the third angle γ formed by the z-axis direction of the target image and the world coordinate system can be a three-row, three-column matrix. The data in the first row of the matrix, from left to right, can be the cosine function of the third angle γ (cosγ), the negative value of the sine function of the third angle γ (-sinγ), and 0. The data in the second row of the matrix, from left to right, can be the sine function of the third angle γ (sinγ), the cosine function of the third angle γ (cosγ), and 0. The data in the third row of the matrix, from left to right, can be 0, 0, and 1.

[0112] In this embodiment of the disclosure, the target is adjusted based on the obtained rotation matrix. The adjustment method can be right-multiplying the rotation matrix generated by the third angle γ by the rotation matrix generated by the second angle β by the rotation matrix generated by the first angle α.

[0113] In step S62, the target angle includes a first angle between the target image and the x-axis of the world coordinate system, a second angle between the target image and the y-axis of the world coordinate system, and a third angle between the target image and the z-axis of the world coordinate system.

[0114] The following embodiments further explain and illustrate the method for determining the field of view threshold range in the above embodiments of this disclosure.

[0115] Figure 7 This is a flowchart illustrating a method for correcting the relative pose between a camera and a target based on a target angle, according to an exemplary embodiment. Figure 7As shown, the method for correcting the relative pose between the camera and the target based on the target angle includes the following steps.

[0116] In step S71, the field of view threshold range corresponding to when the optical design parameters are less than the set distortion threshold is determined based on the correspondence between the optical design parameters and the field of view.

[0117] In this embodiment of the disclosure, a field-of-view threshold range less than the optical design parameter threshold is determined based on the curve of the optical design parameters changing with the field of view. The field-of-view threshold can be selected by using the maximum value of the undistorted curve or the portion with the smallest distortion in the curve of the optical design parameters changing with the field of view. The field-of-view threshold range can be obtained based on the curve of the optical design parameters changing with the field of view.

[0118] For example, consider a circular region with radius R. Within this circular region, there is a smaller circular region with radius r. If the ratio of r to R is 0.1, then the region represented by r is the 0.1 field of view.

[0119] In step S72, the region of interest that meets the field of view threshold range is extracted from the distortion-free image.

[0120] In this embodiment of the disclosure, a field of view with small optical distortion is obtained based on the field of view threshold and set as the region of interest.

[0121] The following embodiments of this disclosure use a checkerboard-patterned target as an example to illustrate the target pose correction method and the method for selecting the region of interest involved in the above embodiments of this disclosure.

[0122] Figure 8 This is a flowchart illustrating the processing of a target image according to an exemplary embodiment. See also... Figure 8 As shown, firstly, the target image captured by the camera is acquired. Based on the ratio of the difference between the actual image height and the ideal image height to the ideal image height, the optical distortion coefficient is obtained. For example, if the actual image height is 100 and the ideal image height is 50, the optical distortion coefficient can be obtained as 50%. Based on the obtained optical distortion coefficient, optical distortion correction is performed on the target image to obtain target image 1.

[0123] In this embodiment of the disclosure, Figure 9 This is a schematic diagram illustrating the distortion variation with field of view according to an exemplary embodiment, see reference. Figure 9 As shown, the maximum value of the straight-line portion of the curve is selected as the field-of-view threshold, and the region of interest is extracted based on this threshold. For example, if the distortion curve as a function of the field of view begins to show distortion at a field of view of 0.8, then the field-of-view threshold can be selected as 0.8. That is, in the target image, the region with a radius equal to 0.8 of the target image's center is selected as the region of interest. (See also...) Figure 10 As shown, Figure 10 This is a schematic diagram illustrating the selection of a region of interest according to an exemplary embodiment.

[0124] In this embodiment of the disclosure, target image 2 is obtained after extracting the region of interest. Based on target image 2, the angle between target image 2 and the camera imaging sensor is calculated.

[0125] in, Figure 11 A schematic diagram of the angle between the target image 2 and the camera imaging sensor in the x-axis direction is shown in an exemplary embodiment of the present disclosure. Figures 12(a) and 12(b) respectively show schematic diagrams of the angle between the target image 2 and the camera imaging sensor in the x-axis direction in an exemplary embodiment of the present disclosure.

[0126] Referring to Figure 12(a), the pixels in the second row of the target image are abstracted as straight lines. A world coordinate system is established with the plane of the camera's imaging sensor as the horizontal direction of the coordinate system. The x-axis of the world coordinate system is abstracted as a dashed line. The angle between the pixels in the second row of the target image and the x-axis of the world coordinate system is calculated. Here, h1 and h2 represent the distances from the pixels in the second row of the target image to the x-axis of the world coordinate system, respectively. H represents the distance from the projection of the pixels in the second row of the target image onto the x-axis of the world coordinate system. Based on h1, h2, H, and the arctangent function formula, the angle α between the pixels in the second row of the target image and the x-axis of the world coordinate system can be obtained, where α = arctan[(h1 + h2) / H].

[0127] Referring to Figure 12(b), the pixels in the second row of the target image are abstracted as straight lines. A world coordinate system is established with the plane of the camera's imaging sensor as the horizontal direction of the coordinate system. The x-axis of the world coordinate system is abstracted as a dashed line. The angle between the pixels in the second row of the target image and the x-axis of the world coordinate system is calculated. Here, h1 and h2 represent the distances from the pixels in the second row of the target image to the x-axis of the world coordinate system, respectively. H represents the distance from the projection of the pixels in the second row of the target image onto the x-axis of the world coordinate system. Based on h1, h2, H, and the arctangent function formula, the angle α between the pixels in the second row of the target image and the x-axis of the world coordinate system can be obtained, where α = arctan[(h1-h2) / H].

[0128] Figure 13 A schematic diagram showing the angle between the target image 2 and the camera imaging sensor in the y-axis direction in an exemplary embodiment of this disclosure is shown. Figure 14 A schematic diagram showing the angle between the target image 2 and the camera imaging sensor in the y-axis direction in an exemplary embodiment of this disclosure is shown.

[0129] See Figure 14 As shown, the pixels in the two columns of the target image are abstracted as straight lines, and the y-axis of the world coordinate system is abstracted as a dashed line. The angle between the pixels in the two columns of the target image and the y-axis of the world coordinate system is calculated. Here, l1 and l2 represent the distances between the pixels in the two columns of the target image and the y-axis of the world coordinate system, respectively. L represents the distance projected from the pixels in the two columns of the target image onto the y-axis of the world coordinate system. Based on l1, l2, L, and the arctangent function formula, the angle β between the pixels in the two columns of the target image and the y-axis of the world coordinate system can be obtained, where β = arctan[(l1+l2) / L].

[0130] Figure 15 A schematic diagram is shown showing the angle between the line connecting the target image 2 and the imaging sensor of the camera in an exemplary embodiment of this disclosure and the z-axis direction. Figure 16 A schematic diagram is shown showing the angle between the line connecting the target image 2 and the imaging sensor of the camera in an exemplary embodiment of this disclosure and the z-axis direction.

[0131] See Figure 16 As shown, pixels along the line connecting target image 2 and the camera's imaging sensor are abstracted as straight lines, and the z-axis of the world coordinate system is abstracted as a dashed line. The angle between pixels along the line connecting target image 2 and the camera's imaging sensor and the z-axis of the world coordinate system is calculated. Here, H1 and H2 represent the distance of pixels projected onto the z-axis of the world coordinate system and the pixel length along the line connecting target image 2 and the camera's imaging sensor, respectively. Dist can be used to represent the distance between target image 2 and the camera's imaging sensor, and f can be used to represent the focal length of the camera. Based on H1, H2, Dist, f, and the arctangent function formula, the angle γ between pixels along the line connecting target image 2 and the camera's imaging sensor and the z-axis of the world coordinate system can be obtained, where γ = arctan[(H2-H1) / (Dist / f)].

[0132] Figure 17 This illustration shows a schematic diagram of adjusting the target according to the angle between the target image 2 and the camera sensor in the x-axis direction, as described in an exemplary embodiment of this disclosure. (See also...) Figure 17 As shown, if the angle between the target image 2 and the camera sensor in the x-axis direction is greater than or equal to a preset angle, the target is adjusted in the opposite direction along the x-axis. If the angle between the target image 2 and the camera sensor in the x-axis direction is less than the preset angle, a rotation matrix is ​​generated based on the angle, and the rotation matrix is ​​applied to the target image 2. Similarly, the target can be adjusted according to the angle between the target image 2 and the camera sensor in the y and z-axis directions.

[0133] Based on the same concept, embodiments of this disclosure also provide a pose correction device.

[0134] It is understood that the pose correction device provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0135] Figure 18 This is a block diagram illustrating a pose correction device according to an exemplary embodiment. (Refer to...) Figure 18 The device includes a processing unit 101, an extraction unit 102, a determination unit 103, and a correction unit 104.

[0136] The processing unit 101 is used to acquire target images captured by the camera and to perform distortion correction on the target images based on optical design parameters.

[0137] The extraction unit 102 is used to extract regions of interest that meet the field of view threshold range in an image after distortion correction based on optical design parameters.

[0138] The determining unit 103 is used to determine the target angle in the region of interest, where the target angle is the angle between the camera's imaging sensor and the target when the camera captures an image of the target.

[0139] The correction unit 104 corrects the relative pose between the camera and the target based on the target angle.

[0140] In one embodiment, the determining unit 103 determines the target angle based on the region of interest in the following manner: mapping the region of interest to the world coordinate system; determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest; the target pixels include pixels in the same row direction in the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixels include pixels in the same column direction in the region of interest, and the target axis is the y-axis of the world coordinate system; or the target pixels include pixels in the direction of the line connecting the imaging sensor and the target, and the target axis is the z-axis of the world coordinate system.

[0141] In another implementation, the target pixel includes pixels in the same row direction within the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixel includes pixels in the same column direction within the region of interest, and the target axis is the y-axis of the world coordinate system. The determining unit 103 determines the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest in the following manner: determining a first distance between a first pixel in the target pixel and the target coordinate axis in the world coordinate system; determining a second distance between a second pixel in the target pixel and the target axis in the world coordinate system; determining a first projection length of the target pixel on the target coordinate axis in the world coordinate system; and performing an arctangent function operation based on the first distance, the second distance, and the first projection length to obtain the target angle.

[0142] In another embodiment, the target pixels include pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system. The determining unit 103 determines the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest as follows: obtaining the second projection length of the target pixels in the world coordinate system; determining the length of the pixels in the region of interest along the line connecting the imaging sensor and the target in the world coordinate system as the third distance; determining the difference between the third distance and the second projection length; determining the ratio between the distance between the imaging sensor and the target and the focal length of the camera to obtain a first ratio; determining the ratio between the difference and the first ratio to obtain a second ratio; and performing an arctangent function operation on the second ratio to obtain the target angle.

[0143] In another embodiment, the correction unit 104 corrects the relative pose between the camera and the target based on the target angle in the following manner: if there is a target angle greater than or equal to a preset angle threshold, the target or camera target angle is adjusted in the opposite direction of the target coordinate axis, including the first angle between the target image and the x-axis direction of the world coordinate system; or the second angle between the target image and the y-axis direction of the world coordinate system; or the third angle between the target image and the z-axis direction of the world coordinate system.

[0144] In another embodiment, the correction unit 104 corrects the relative pose between the camera and the target based on the target angle as follows: if the target angle is less than a preset angle threshold, a rotation matrix is ​​generated based on the target angle, and the target or camera is adjusted based on the rotation matrix. The target angle includes a first angle between the target image and the x-axis direction of the world coordinate system, a second angle between the target image and the y-axis direction of the world coordinate system, and a third angle between the target image and the z-axis direction of the world coordinate system.

[0145] In another embodiment, the apparatus further includes: before extracting the region of interest that satisfies the field of view threshold range in the distorted image, determining the field of view threshold range corresponding to when the optical design parameters are less than a set distortion threshold based on the correspondence between the optical design parameters and the field of view.

[0146] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0147] Figure 19 This is a block diagram illustrating an apparatus 200 for pose correction according to an exemplary embodiment. For example, apparatus 200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0148] Reference Figure 19 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0149] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0150] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0151] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0152] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0153] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0154] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0155] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0156] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0157] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0158] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0159] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0160] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0161] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0162] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0163] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0164] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A pose correction method, characterized in that, include: Acquire target images captured by a camera, and perform distortion correction on the target images based on optical design parameters; In the distortion-corrected image, extract the region of interest that meets the field-of-view threshold range; Based on the region of interest, a target angle is determined, wherein the target angle is the angle between the camera's imaging sensor and the target when the camera captures the image of the target; Determining the target angle based on the region of interest includes: Map the region of interest to the world coordinate system; Based on the target pixels in the region of interest, determine the target angle between the target image and the target coordinate axis in the world coordinate system; The target pixel includes pixels in the same row direction in the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; Alternatively, the target pixel may include pixels in the same column direction within the region of interest, and the target coordinate axis may be the y-axis of the world coordinate system. Alternatively, the target pixel may include pixels along the line connecting the imaging sensor and the target, and the target coordinate axis may be the z-axis of the world coordinate system. Based on the target angle, the relative pose between the camera and the target is corrected.

2. The method according to claim 1, characterized in that, The target pixel includes pixels in the same row direction in the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixel includes pixels in the same column direction in the region of interest, and the target axis is the y-axis of the world coordinate system. Determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest includes: Determine the first distance between the first pixel in the target pixels and the target coordinate axis of the world coordinate system; Determine the second distance between the second pixel in the target pixels and the target coordinate axis of the world coordinate system; Determine the first projection length of the target pixel on the target coordinate axis in the world coordinate system; Based on the first distance, the second distance, and the first projection length, an arctangent function is performed to obtain the target angle.

3. The method according to claim 1, characterized in that, The target pixel includes pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system; Determining the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest includes: Obtain the second projection length of the target pixel in the world coordinate system; The length of the pixels in the region of interest along the line connecting the imaging sensor and the target in the world coordinate system is defined as the third distance. Determine the difference between the third distance and the second projection length; The ratio between the distance between the imaging sensor and the target and the focal length of the camera is determined to obtain a first ratio. Determine the ratio between the difference and the first ratio to obtain the second ratio; The target angle is obtained by performing an arctangent function operation on the second ratio.

4. The method according to any one of claims 1 to 3, characterized in that, The step of correcting the relative pose between the camera and the target based on the target angle includes: If there is a target angle greater than or equal to a preset angle threshold, then the target or the camera is adjusted in the opposite direction along the target coordinate axis. The target angle includes a first angle between the target image and the x-axis of the world coordinate system; or a second angle between the target image and the y-axis of the world coordinate system; or a third angle between the target image and the z-axis of the world coordinate system.

5. The method according to any one of claims 1 to 3, characterized in that, The step of correcting the relative pose between the camera and the target based on the target angle includes: If the target angle is less than a preset angle threshold, a rotation matrix is ​​generated based on the target angle, and the target or the camera is adjusted based on the rotation matrix. The target angle includes a first angle between the target image and the x-axis of the world coordinate system, a second angle between the target image and the y-axis of the world coordinate system, and a third angle between the target image and the z-axis of the world coordinate system.

6. The method according to claim 1, characterized in that, The method further includes: Before extracting the region of interest that meets the field of view threshold range in the distortion-corrected image, the field of view threshold range corresponding to when the optical design parameters are less than the set distortion threshold is determined based on the correspondence between the optical design parameters and the field of view.

7. A posture correction device, characterized in that, include: The processing unit is used to acquire the target image captured by the camera and to perform distortion correction on the target image based on optical design parameters; The extraction unit is used to extract the region of interest that meets the field of view threshold range in the distortion-free image; A determining unit is configured to determine a target angle in the region of interest, wherein the target angle is the angle between the imaging sensor of the camera and the target when the camera captures the image of the target; The determining unit determines the target angle based on the region of interest in the following manner: Map the region of interest to the world coordinate system; Based on the target pixels in the region of interest, determine the target angle between the target image and the target coordinate axis in the world coordinate system; The target pixels include pixels in the same row direction within the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or The target pixels include pixels in the same column direction within the region of interest, and the target coordinate axis is the y-axis of the world coordinate system; or The target pixel includes pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system; The correction unit is used to correct the relative pose between the camera and the target based on the target angle.

8. The apparatus according to claim 7, characterized in that, The target pixel includes pixels in the same row direction in the region of interest, and the target coordinate axis is the x-axis of the world coordinate system; or the target pixel includes pixels in the same column direction in the region of interest, and the target axis is the y-axis of the world coordinate system. The determining unit determines the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest in the following manner: Determine the first distance between the first pixel in the target pixels and the target coordinate axis of the world coordinate system; Determine the second distance between the second pixel in the target pixels and the target coordinate axis of the world coordinate system; Determine the first projection length of the target pixel on the target coordinate axis in the world coordinate system; Based on the first distance, the second distance, and the first projection length, an arctangent function is performed to obtain the target angle.

9. The apparatus according to claim 7, characterized in that, The target pixel includes pixels along the line connecting the imaging sensor and the target, and the target coordinate axis is the z-axis of the world coordinate system; The determining unit determines the target angle between the target image and the target coordinate axis in the world coordinate system based on the target pixels in the region of interest in the following manner: Obtain the second projection length of the target pixel in the world coordinate system; The length of the pixels in the region of interest along the line connecting the imaging sensor and the target in the world coordinate system is defined as the third distance. Determine the difference between the third distance and the second projection length; The ratio between the distance between the imaging sensor and the target and the focal length of the camera is determined to obtain a first ratio. Determine the ratio between the difference and the first ratio to obtain the second ratio; The target angle is obtained by performing an arctangent function operation on the second ratio.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The correction unit corrects the relative pose between the camera and the target based on the target angle in the following manner: If there is a target angle greater than or equal to a preset angle threshold, then the target or the camera is adjusted in the opposite direction along the target coordinate axis. The target angle includes a first angle between the target image and the x-axis of the world coordinate system; or a second angle between the target image and the y-axis of the world coordinate system; or a third angle between the target image and the z-axis of the world coordinate system.

11. The apparatus according to any one of claims 7 to 9, characterized in that, The correction unit corrects the relative pose between the camera and the target based on the target angle in the following manner: If the target angle is less than a preset angle threshold, a rotation matrix is ​​generated based on the target angle, and the target or the camera is adjusted based on the rotation matrix. The target angle includes a first angle between the target image and the x-axis of the world coordinate system, a second angle between the target image and the y-axis of the world coordinate system, and a third angle between the target image and the z-axis of the world coordinate system.

12. The apparatus according to claim 7, characterized in that, The device further includes: Before extracting the region of interest that meets the field of view threshold range in the distortion-corrected image, the field of view threshold range corresponding to when the optical design parameters are less than the set distortion threshold is determined based on the correspondence between the optical design parameters and the field of view.

13. A posture correction device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the pose correction method according to any one of claims 1 to 6.

14. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the processor of the terminal, enable the terminal including the processor to perform the pose correction method according to any one of claims 1 to 6.

Images