Vehicle-mounted image calibration method, device, equipment and storage medium

Abstract

The invention provides a vehicle-mounted image calibration method and device, equipment and a storage medium. The method comprises the steps that first pose information of cameras located on the two opposite sides of a vehicle body under a world coordinate system is acquired; converting the first pose information into second pose information of the camera in a standard coordinate system; and based on the second pose information, determining a homography transformation matrix of the camera, so as to determine a corresponding rendered image of the camera under the target visual angle based on the homography transformation matrix. Thus, the first pose information of the cameras on the two opposite sides of the vehicle body is converted into the second pose information, it is guaranteed that the view angles of the cameras on the two opposite sides of the vehicle body are symmetric, and the symmetric rendering images under the target view angle are finally obtained. Besides, the first pose information is converted into the second pose information, so that the individual difference of the installation angles of the cameras is ignored in a standard coordinate system, independent wheel visual angle parameter adjustment does not need to be carried out on each camera, and the vehicle-mounted image calibration efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle panoramic imaging technology, and specifically to a vehicle-mounted image calibration method, device, equipment and storage medium. Background Art

[0002] With the rapid development of science and technology and domestic automobile companies in recent years, the national vehicle ownership rate has gradually increased, and the accompanying driving safety issues have also become the focus of people's attention. With the continuous improvement of the level of automobile intelligence, digital assisted driving systems have emerged. The wheel perspective is applied to intelligent driving assistance system technology. It is mainly used to obtain the front and rear wheels of the vehicle and the blind spots near the vehicle body through cameras installed around the vehicle body when passing through narrow roads or when there are obstacles around the vehicle body, and timely feedback information to the driver to reduce the incidence of scratches and narrow road accidents.

[0003] At present, in order to facilitate the driver to perceive obstacles around the vehicle body, the following methods are mainly used: bird's-eye view, radar distance detection and wheel perspective. The bird's-eye view transforms the camera image into a top view by homography, and then feeds back the information around the vehicle body to the driver by splicing and fusion. However, this method focuses more on showing the overall obstacle information around the vehicle body. When the surrounding obstacles are close to the vehicle body, there will be more serious stretching distortion, and at the same time, there will be different degrees of misalignment and ghosting due to the different horizontal heights of the obstacles. Radar distance detection often reminds the driver through alarm sounds and simple rendering on the central control screen, such as fan-shaped rendering, which is often not intuitive enough; the wheel perspective can be based on the camera's posture. By performing homography transformation on the camera image, the camera's fisheye image can be converted into a perspective that focuses more on the wheels, such as the front wheel perspective and the rear wheel perspective. At the same time, the camera's posture can also be adjusted to generate a perspective similar to a bird's-eye view.

[0004] Currently, there are two types of wheel perspectives on the market. One is to simply dedistort the camera images on the left and right sides or directly render the original images to the central control screen. This cannot provide the driver with video images that conform to human visual habits. The other is to process the left and right camera images and convert the camera's fisheye image into a perspective that focuses more on the wheels. However, when directly adjusting parameters based on the camera's external parameters, a large number of parameter adjustment experiments are required when applied to different projects or even different vehicles of the same model. The calibration efficiency is low, and it is difficult to ensure that the images presented by the left and right cameras remain symmetrical. Summary of the invention

[0005] The present application provides a vehicle-mounted image calibration method, device, equipment and storage medium, which can improve the efficiency of vehicle-mounted image calibration and ensure that the corresponding rendered images of cameras located on opposite sides of the vehicle body are symmetrical under the target viewing angle.

[0006] The technical solution of this application is implemented as follows:

[0007] An embodiment of the present application provides a vehicle-mounted image calibration method, including: obtaining first pose information of cameras located on opposite sides of a vehicle body in a world coordinate system; converting the first pose information into second pose information of the camera in a standard coordinate system; the standard coordinate system is a coordinate system determined in a state where the pitch angle, yaw angle and roll angle of the camera are all zero; based on the second pose information, determining the homography transformation matrix of the camera, so as to determine the corresponding rendered image of the camera under the target viewing angle based on the homography transformation matrix.

[0008] According to the above technical means, by converting the first pose information of the cameras on the opposite sides of the vehicle body in the world coordinate system into the second pose information in the standard coordinate system, the camera perspectives on the opposite sides of the vehicle body can be guaranteed to be symmetrical, and the parameters of each camera change from the world coordinate system to the target perspective are the same, so that the corresponding rendering images of the cameras on the opposite sides of the vehicle body under the target perspective are symmetrical to each other. In addition, by converting the first pose information of the cameras on the opposite sides of the vehicle body in the world coordinate system into the second pose information in the standard coordinate system, the individual differences in the camera installation angles are ignored in the standard coordinate system, so there is no need to adjust the wheel perspective parameters for each camera separately, thereby improving the efficiency of vehicle-mounted image calibration.

[0009] Furthermore, based on the second posture information, the homography transformation matrix of the camera is determined, including: obtaining a first rotation matrix of the camera in the camera coordinate system; determining a second rotation matrix from the standard coordinate system to the target coordinate system according to the first rotation matrix and the second posture information; the target coordinate system is determined based on a virtual posture corresponding to the camera under the target viewing angle; based on the second rotation matrix, and the posture translation vector of the camera from the camera coordinate system to the target coordinate system, the homography transformation matrix is ​​determined.

[0010] According to the above technical means, the second rotation matrix from the standard coordinate system to the target coordinate system is determined through the first rotation matrix and the second posture information of the camera in the standard coordinate system, and the homography transformation matrix is ​​determined according to the second rotation matrix and the posture translation vector of the camera from the camera coordinate system to the target coordinate system, so as to obtain the matrix of the conversion relationship between the three-dimensional camera coordinates and the two-dimensional image coordinates, so as to facilitate the subsequent determination of the corresponding rendering image of the camera under the target perspective according to the homography transformation matrix.

[0011] Furthermore, the method also includes: obtaining original pose information of the camera in the camera coordinate system and third pose information of the camera in the target coordinate system; based on the original pose information and the third pose information, determining a pose translation vector of the camera from the camera coordinate system to the target coordinate system.

[0012] According to the above technical means, the pose translation vector of the camera from the camera coordinate system to the target coordinate system can be directly determined through the original pose information of the camera in the camera coordinate system and the third pose information of the camera in the target coordinate system, so as to facilitate the subsequent calculation of the homography transformation matrix based on the pose translation vector.

[0013] Furthermore, the camera includes a first camera located on the left side of the vehicle body and a second camera located on the right side of the vehicle body, and the first coordinate axis of the standard coordinate system corresponding to the first camera and the first coordinate axis of the standard coordinate system corresponding to the second camera are parallel or perpendicular to each other.

[0014] According to the above technical means, since the first coordinate axis of the standard coordinate system corresponding to the first camera and the first coordinate axis of the standard coordinate system corresponding to the second camera are parallel or perpendicular to each other, after the first camera and the second camera are subsequently adjusted to the same angle, a left-right symmetrical image can be obtained.

[0015] Furthermore, the method also includes: obtaining the affine transformation matrix of the camera; updating the homography transformation matrix based on the affine transformation matrix to obtain a new homography transformation matrix, so as to determine the corresponding rendering image of the camera under the target viewing angle based on the new homography transformation matrix.

[0016] According to the above technical means, by updating the homography transformation matrix through the affine transformation matrix, invalid areas such as ghosting and black edges caused by directly performing perspective transformation on the image based on the homography transformation matrix can be avoided, thereby improving the quality of the corresponding rendered image obtained by the camera at the target viewing angle.

[0017] Furthermore, the method also includes: based on the homography transformation matrix, performing an inverse perspective transformation on the first pixel coordinates of the image corresponding to the camera at the target viewing angle to obtain the second pixel coordinates of the dedistorted image; performing inverse distortion processing on the second pixel coordinates to obtain the third pixel coordinates of the original image captured by the camera; based on the third pixel coordinates, obtaining the target image content from the original image; and filling the target image content to the corresponding position of the first pixel coordinates to obtain a rendered image.

[0018] According to the above technical means, by performing inverse perspective transformation through the homography transformation matrix, the first pixel coordinates of the image corresponding to the camera at the target viewing angle can be mapped to the second pixel coordinates of the dedistorted image. Furthermore, by performing inverse distortion processing on the second pixel coordinates, the second pixel coordinates can be mapped to the third pixel coordinates in the original image. By filling the target pixel content at the position corresponding to the third pixel coordinate in the original image to the corresponding position of the first pixel coordinate, the rendering of the corresponding image of the camera at the target viewing angle can be obtained, thereby realizing the mapping of the image content in the original image to the image content of the corresponding rendered image of the camera at the target viewing angle.

[0019] Furthermore, the camera includes two cameras located at symmetrical positions on the left and right sides of the vehicle body; the method also includes: outputting the rendered images corresponding to the two cameras at the target perspective to different areas of the human-computer interaction interface for comparative display; if an adjustment instruction for the target perspective of a reference camera in the human-computer interaction interface is received, adjusting the target perspective of the reference camera to obtain a new target perspective; the reference camera is at least one of the two cameras; based on the homography transformation matrix corresponding to the reference camera in the new standard coordinate system, determining the rendered image corresponding to the reference camera at the new target perspective.

[0020] According to the above technical means, by outputting the corresponding rendering images of the two cameras located at symmetrical positions on the left and right sides of the vehicle body under the target viewing angle to different areas of the human-computer interaction interface for comparative display, it is convenient for calibration personnel or other technical personnel to fine-tune the target viewing angle of the camera according to the display effect of the rendered image, so as to obtain symmetrical rendering images on the left and right sides.

[0021] The embodiment of the present application provides a vehicle-mounted image calibration device, characterized by comprising:

[0022] A first acquisition module is used to acquire first position information of cameras located on opposite sides of the vehicle body in the world coordinate system;

[0023] A first conversion module is used to convert the first pose information into second pose information of the camera in a standard coordinate system; the standard coordinate system is a coordinate system determined in a state where the pitch angle, yaw angle and roll angle of the camera are all zero;

[0024] The first determination module is used to determine the homography transformation matrix of the camera based on the second posture information, so as to determine the rendering image corresponding to the camera under the target viewing angle based on the homography transformation matrix.

[0025] The embodiment of the present application provides a vehicle-mounted image calibration device, including:

[0026] A memory for storing computer programs that can be run on the processor;

[0027] The processor is used to execute the vehicle-mounted image calibration method provided in the embodiment of the present application when running a computer program.

[0028] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are configured to execute the above-mentioned vehicle-mounted image calibration method. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a process flow of a vehicle-mounted image calibration method provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of the architecture of a wheel viewing angle calibration system based on vehicle-mounted panoramic images provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of a process of calibrating a wheel viewing angle based on a vehicle-mounted panoramic image provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of a preview window of a human-computer interaction interface provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the effect of the front wheel perspective after synthesizing a rendered image provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of the effect of a vehicle body perspective after a rendering image is synthesized according to an embodiment of the present application;

[0035] Figure 7 A schematic diagram of the effect of a rear wheel perspective after synthesizing a rendered image provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of a rendering image effect of a single perspective of a left vehicle body provided in an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of a rendering image effect of a single perspective of a right vehicle body provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the structure of a vehicle-mounted image calibration device provided in an embodiment of the present application;

[0039] Fig.11 A schematic diagram of the composition structure of a vehicle-mounted image calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0041] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by other ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0042] In the following description, reference is made to “some embodiments\other embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments\other embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by other technicians in the technical field of this application. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0045] Based on the problems existing in the related art, the embodiment of the present application provides a vehicle image calibration method, which can improve the calibration efficiency of vehicle images and ensure that the corresponding rendering images of the cameras located on opposite sides of the vehicle body are symmetrical under the target viewing angle. Figure 1 FIG. 1 is a flow chart of a vehicle image calibration method provided in an embodiment of the present application, and the method comprises the following steps:

[0046] S101, obtaining first position information of cameras located on opposite sides of a vehicle body in a world coordinate system.

[0047] It should be noted that there may be multiple cameras located on opposite sides of the vehicle body, for example, there may be a camera located on the left front side of the vehicle body and a camera located on the right front side of the vehicle body, there may be a camera located on the left rear side of the vehicle body and a camera located on the right rear side of the vehicle body, there may be a camera located on the front side of the vehicle body and a camera located on the rear side of the vehicle body.

[0048] In some embodiments, the first pose information can be represented by the three-dimensional spatial coordinates of the cameras located on opposite sides of the vehicle body in the world coordinate system, for example, it can be represented by (a, b, c), where a represents the x coordinate value of the camera in the world coordinate system, b represents the y coordinate value of the camera in the world coordinate system, and c represents the z coordinate value of the camera in the world coordinate system. Since the world coordinate system is fixed, the corresponding first pose information can be determined according to the position of each camera.

[0049] S102: Convert the first pose information into second pose information of the camera in a standard coordinate system.

[0050] In some embodiments, the standard coordinate system is a coordinate system determined when the pitch angle, yaw angle, and roll angle of the camera are all zero. Each camera corresponds to a standard coordinate system, and the standard coordinate system can be determined based on the position and posture of the camera. The standard coordinate systems corresponding to different cameras may be different. After the pitch angle, yaw angle, and roll angle of each camera are adjusted to 0, the optical center of each camera can be used as the origin, and the straight line where the optical axis of the camera is located corresponds to the direction of the viewing angle as the z-axis, the horizontal right direction as the x-axis, and the vertical downward direction as the y-axis to establish the standard coordinate system corresponding to each camera.

[0051] In some embodiments, the camera includes a first camera located on the left side of the vehicle body and a second camera located on the right side of the vehicle body, and the first coordinate axis of the standard coordinate system corresponding to the first camera is parallel to or perpendicular to the first coordinate axis of the standard coordinate system corresponding to the second camera. The first coordinate axis may be the x-axis, y-axis or z-axis of the standard coordinate system. For example, the viewing direction corresponding to the first camera may be toward the front of the vehicle, and the corresponding z-axis of the standard coordinate system is toward the front of the vehicle. The viewing direction corresponding to the second camera may be horizontally toward the right, and the corresponding z-axis of the standard coordinate system is horizontally toward the right. Therefore, the z-axis of the standard coordinate system corresponding to the first camera and the z-axis of the standard coordinate system corresponding to the second camera are perpendicular to each other. Since the first coordinate axis of the standard coordinate system corresponding to the first camera and the first coordinate axis of the standard coordinate system corresponding to the second camera are parallel to or perpendicular to each other, after the first camera and the second camera are adjusted to the same angle, a left-right symmetrical image can be obtained.

[0052] In some embodiments, after the standard coordinate system is determined, the second pose information of each camera in its corresponding standard coordinate system can be determined. The second pose information can be the three-dimensional space coordinates of the camera in the standard coordinate system, such as the position coordinates of the center of the camera in the standard coordinate system.

[0053] S103: Determine a homography transformation matrix of the camera based on the second posture information, so as to determine a rendered image corresponding to the camera under a target viewing angle based on the homography transformation matrix.

[0054] In some embodiments, after determining the homography transformation matrix corresponding to each camera, the rendered images corresponding to different cameras under the target perspective can be determined based on the homography transformation matrix and the internal parameters of the corresponding camera, where the target perspective can be any perspective selected by the user through the human-computer interaction interface.

[0055] In some embodiments, the original image captured by the camera's current perspective can be mapped to an image of a target perspective through a homography transformation matrix. By performing pixel coordinate transformation based on the homography transformation matrix, the pixel coordinates of the image of the target perspective can be corresponded to the pixel coordinates in the original image to obtain the corresponding pixel content, and the obtained pixel content is filled into the pixel coordinates of the image of the target perspective to obtain a rendered image at the target perspective.

[0056] In an embodiment of the present application, the first pose information of the cameras located on the opposite sides of the vehicle body in the world coordinate system is obtained; the first pose information is converted into the second pose information of the camera in the standard coordinate system; the standard coordinate system is a coordinate system determined by the pitch angle, yaw angle and flip angle of the camera are all zero; based on the second pose information, the homography transformation matrix of the camera is determined to determine the corresponding rendering image of the camera under the target viewing angle based on the homography transformation matrix. In this way, by converting the first pose information of the cameras on the opposite sides of the vehicle body in the world coordinate system into the second pose information in the standard coordinate system, the camera viewing angles on the opposite sides of the vehicle body can be guaranteed to be symmetrical, and when the parameters of each camera change from the world coordinate system to the target viewing angle are the same, the rendering images corresponding to the cameras on the opposite sides of the vehicle body under the target viewing angle can be made symmetrical to each other. In addition, by converting the first pose information of the cameras located on the opposite sides of the vehicle body in the world coordinate system into the second pose information in the standard coordinate system, the individual differences in the camera installation angles are ignored in the standard coordinate system, so there is no need to adjust the wheel viewing angle parameters for each camera separately, thereby improving the calibration efficiency of the vehicle-mounted image.

[0057] In some embodiments of the present application, the first position information of the cameras located on opposite sides of the vehicle body in the world coordinate system is obtained, that is, the above-mentioned step S101 can be implemented by the following steps S1011 to S1012, and each step is described separately below.

[0058] S1011. Obtain the extrinsic parameter matrix of each camera and the original position information of each camera in the camera coordinate system.

[0059] In some embodiments, the extrinsic parameter matrix can be used to represent the conversion relationship between the camera coordinate system and the world coordinate system. The extrinsic parameter matrix can be obtained by pre-calibrating the camera. The extrinsic parameter matrices of the cameras located on opposite sides of the vehicle body are different.

[0060] In some embodiments, the camera coordinate system can be a three-dimensional space coordinate system established with the optical center of the camera as the origin, the straight line where the optical axis is located and perpendicular to the imaging plane as the z-axis, the horizontal right direction of the imaging plane as the x-axis, and the vertical upward direction of the imaging plane as the y-axis. Since the postures of different cameras are different after the vehicle leaves the factory, the camera coordinate systems corresponding to different cameras are different.

[0061] In some embodiments, the original posture information may be the three-dimensional spatial coordinates of the cameras on the vehicle body in their respective corresponding camera coordinate systems after the vehicle leaves the factory. The three-dimensional spatial coordinates may be the position coordinates of the center or center of mass of the camera in the camera coordinate system.

[0062] S1012: Based on the extrinsic parameter matrix, convert the original pose information into the first pose information of each camera in the world coordinate system.

[0063] In some embodiments, after obtaining the extrinsic matrix of each camera, the first position information of each camera in the world coordinate system can be determined based on the extrinsic matrix and the original position information of each camera. In implementation, the extrinsic matrix can be multiplied by the original position information to obtain the corresponding first position information.

[0064] According to the above technical means, the original pose information of each camera is converted into the first pose information of the world coordinate system through the external parameter matrix, and the pose information of each camera can be represented in a unified world coordinate system, which is convenient for the subsequent conversion of the pose information of each camera in the world coordinate system into the pose information in the standard coordinate system to obtain the same viewing angle.

[0065] In some embodiments of the present application, "determining the homography transformation matrix of the camera based on the second pose information" in step S103 can be implemented through the following steps S201 to S203, and each step is described below.

[0066] S201. Obtain a first rotation matrix of the camera in a camera coordinate system.

[0067] In some embodiments, the first rotation matrix can be an orthogonal matrix used to describe the rotation around the coordinate axis of the camera coordinate system. The first rotation matrix can be used to transform the camera coordinate system, for example, convert the camera coordinate system into other coordinate systems. Since the camera coordinate system includes three coordinate axes, the first rotation matrix can include three.

[0068] Exemplarily, the first rotation matrix can be expressed by formula (1):

[0069]

[0070] Among them, R pitch , R pitch and R pitch Respectively represent the rotation matrices corresponding to r degrees of rotation around the x-axis, y-axis, and z-axis of the camera coordinate system.

[0071] S202: Determine a second rotation matrix from the standard coordinate system to the target coordinate system according to the first rotation matrix and the second position and posture information.

[0072] In some embodiments, the target coordinate system is determined based on the virtual pose corresponding to the camera under the target perspective. Different cameras correspond to one target coordinate system, and the angle of transformation or rotation of each camera from the standard coordinate system to the target coordinate system is the same, so that under the target coordinate system, symmetrical images can be obtained under the perspectives of the cameras on opposite sides of the vehicle body.

[0073] In some embodiments, the product of the first rotation matrix and the second pose information can be used as a second rotation matrix from the standard coordinate system to the target coordinate system. For example, the second rotation matrix R camera2target It can be expressed by formula (2):

[0074] R camera2target =R pitch ·R yaw ·R roll ·R std (2);

[0075] Among them, R std Represents the second pose information.

[0076] S203: Determine a homography transformation matrix based on the second rotation matrix and a position translation vector of the camera from the camera coordinate system to the target coordinate system.

[0077] In some embodiments, after determining the second rotation matrix, the homography transformation matrix can be calculated based on the second rotation matrix, the camera's pose translation vector from the camera coordinate system to the standard coordinate system, the normal vector of the plane undergoing perspective transformation in the camera coordinate system, and the distance between the camera and the ground.

[0078] For example, the homography transformation matrix can be expressed by formula (3):

[0079] H camera2target =K(R camera2target +t camera2target / dN T )K -1(3);

[0080] Where N represents the normal vector of the plane undergoing perspective transformation in the camera coordinate system, t camera2target It represents the distance between the camera and the ground, and K represents the intrinsic parameter of the camera.

[0081] According to the above technical means, the second rotation matrix from the standard coordinate system to the target coordinate system is determined through the first rotation matrix and the second posture information of the camera in the standard coordinate system, and the homography transformation matrix is ​​determined according to the second rotation matrix and the posture translation vector of the camera from the camera coordinate system to the target coordinate system, so as to obtain the matrix of the conversion relationship between the three-dimensional camera coordinates and the two-dimensional image coordinates, so as to facilitate the subsequent determination of the corresponding rendering image of the camera under the target perspective according to the homography transformation matrix.

[0082] In some embodiments of the present application, before determining the homography transformation matrix based on the second rotation matrix and the pose translation vector of the camera from the camera coordinate system to the target coordinate system, that is, before step S203, the following steps S301 to S302 may also be executed, and each step is described below.

[0083] S301, obtaining original pose information of the camera in the camera coordinate system and third pose information of the camera in the target coordinate system.

[0084] In some embodiments, the original pose information may be the position coordinates of the camera in the camera coordinate system, and the third pose information may be the position coordinates of the camera in the target coordinate system. The original pose information may be the real position information determined based on the actual position of the camera on the vehicle body after the vehicle leaves the vehicle, and the third pose information may be the virtual position information of the camera in the target coordinate system determined after converting the camera coordinate system of the camera.

[0085] S302: Based on the original posture information and the third posture information, determine a posture translation vector of the camera from the camera coordinate system to the target coordinate system.

[0086] In some embodiments, the posture translation vector can be determined based on the change in the position coordinate value in the original posture information and the corresponding position coordinate value in the third posture information. For example, when the original posture information and the third posture information are both three-dimensional space position coordinates, the posture translation vector can be determined based on the difference between the x-coordinate value in the original posture information and the x-coordinate value in the third posture coordinate system, the difference between the y-coordinate value in the original posture information and the y-coordinate value in the third posture information, and the difference between the z-coordinate value in the original posture information and the y-coordinate value in the third posture information.

[0087] According to the above technical means, the pose translation vector of the camera from the camera coordinate system to the target coordinate system can be directly determined through the original pose information of the camera in the camera coordinate system and the third pose information of the camera in the target coordinate system, so as to facilitate the subsequent calculation of the homography transformation matrix based on the pose translation vector.

[0088] In some embodiments of the present application, after "determining the homography transformation matrix of the camera based on the second pose information" in step S103, the following steps S401 to S402 may also be executed, and each step is described below.

[0089] S401, obtaining the affine transformation matrix of the camera.

[0090] In some embodiments, the affine transformation matrix can translate, rotate, and scale the pixel coordinates of the two-dimensional image. For example, the affine transformation matrix can be expressed by the following formula (4):

[0091]

[0092] Among them, M translate 、M rotation and M scaling They represent the translation matrix, rotation matrix, and scaling matrix of the pixel coordinates of a two-dimensional image respectively.

[0093] S402: Update the homography transformation matrix based on the affine transformation matrix to obtain a new homography transformation matrix, so as to determine the rendering image corresponding to the camera under the target viewing angle based on the new homography transformation matrix.

[0094] In some embodiments, the homography transformation matrix can be fine-tuned based on the affine transformation matrix, for example, by left-multiplying the homography transformation matrix by the perspective transformation matrix to obtain a new homography transformation matrix, and then the corresponding rendering image of the camera under the target perspective can be determined based on the new homography transformation matrix.

[0095] In some embodiments, the updated new homography transformation matrix can be calculated by formula (5):

[0096] H=M translate ·M rotation ·M scaling ·H camera2target (5);

[0097] Among them, H camera2target represents the original homography transformation matrix, and H represents the new homography transformation matrix obtained after updating.

[0098] According to the above technical means, by updating the homography transformation matrix through the affine transformation matrix, invalid areas such as ghosting and black edges caused by directly performing perspective transformation on the image based on the homography transformation matrix can be avoided, thereby improving the quality of the corresponding rendered image obtained by the camera at the target viewing angle.

[0099] In some embodiments of the present application, after “determining the homography transformation matrix of the camera based on the second pose information”, the following steps S501 to S504 may also be executed, and each step is described below.

[0100] S501 . Based on a homography transformation matrix, perform an inverse perspective transformation on first pixel coordinates of an image corresponding to a camera at a target viewing angle to obtain second pixel coordinates of a dedistorted image.

[0101] In some embodiments, the first pixel coordinates may be one or more pixel coordinates of an image corresponding to the camera at the target viewing angle, and the dedistorted image may be an image obtained by dedistorting the original image (fisheye image) actually captured by the camera.

[0102] In some embodiments, the first pixel coordinate can be multiplied by the inverse of the homography transformation matrix to achieve an inverse perspective transformation, and the first pixel coordinate of the image corresponding to the camera at the target viewing angle is mapped to the second pixel coordinate of the dedistorted image. The position of the first pixel coordinate in the image corresponding to the camera at the target viewing angle and the position of the second pixel coordinate in the dedistorted image may be different. For example, if the first pixel coordinate is (0, 0), the second pixel coordinate may be (1, 1). The correspondence between the first pixel coordinate and the second pixel coordinate here is merely an exemplary description and is not limited to this in the present application.

[0103] S502: Perform inverse distortion processing on the second pixel coordinates to obtain third pixel coordinates of the original image captured by the camera.

[0104] In some embodiments, the third pixel coordinate may be the coordinate of a pixel in the original image. In the process of performing inverse distortion processing on the second pixel coordinate, the second pixel coordinate may be inversely distorted according to the distortion coefficient of the camera to determine the third pixel coordinate corresponding to the second pixel coordinate in the original image, wherein the distortion coefficient of the camera is an internal parameter of the camera and can be obtained after calibrating the camera.

[0105] S503: Based on the third pixel coordinate, obtain the target image content from the original image.

[0106] In some embodiments, after the third pixel coordinate is determined, the target image content can be obtained from the original image based on the third pixel coordinate. The target image content can be the RGB value of the pixel. During implementation, the RGB value of the pixel can be obtained from the third pixel coordinate of the original image through texture sampling.

[0107] S504 , filling the target image content into the corresponding position of the first pixel coordinate to obtain a rendered image.

[0108] In some embodiments, after obtaining the target image content, pixel content filling can be performed, that is, the target image content corresponding to the third pixel position in the original image is filled into the corresponding position of the first pixel coordinate of the image corresponding to the camera at the target viewing angle, thereby obtaining the rendered image corresponding to the camera at the target viewing angle.

[0109] According to the above technical means, by performing inverse perspective transformation through the homography transformation matrix, the first pixel coordinates of the image corresponding to the camera at the target viewing angle can be mapped to the second pixel coordinates of the dedistorted image. Furthermore, by performing inverse distortion processing on the second pixel coordinates, the second pixel coordinates can be mapped to the third pixel coordinates in the original image. By filling the target pixel content at the position corresponding to the third pixel coordinate in the original image to the corresponding position of the first pixel coordinate, the rendering of the corresponding image of the camera at the target viewing angle can be obtained, thereby realizing the mapping of the image content in the original image to the image content of the corresponding rendered image of the camera at the target viewing angle.

[0110] In some embodiments of the present application, the following steps S601 to S603 may be further executed after step S103, and each step is described below.

[0111] S601: Outputting the corresponding rendering images of the two cameras under the target viewing angle to different areas of the human-computer interaction interface for comparative display.

[0112] In some embodiments, the rendered image corresponding to the left front camera of the vehicle body at the target perspective and the rendered image corresponding to the right front camera of the vehicle body at the target perspective can be output to different areas of the human-computer interaction interface for comparative display; the rendered image corresponding to the left rear camera of the vehicle body at the target perspective and the rendered image corresponding to the right rear camera of the vehicle body at the target perspective can be output to different areas of the human-computer interaction interface for comparative display; the rendered image corresponding to the front camera of the vehicle body at the target perspective and the rendered image corresponding to the rear camera of the vehicle body at the target perspective can also be output to different areas of the human-computer interaction interface for comparative display.

[0113] S602: If an adjustment instruction for a target viewing angle of a reference camera in a human-computer interaction interface is received, the target viewing angle of the reference camera is adjusted to obtain a new target viewing angle.

[0114] In some embodiments, the reference camera can be two cameras located at symmetrical positions on the left and right sides of the vehicle body, or can be any one of the two cameras. The adjustment instruction can be triggered by the calibration personnel or other technicians adjusting the coordinate axes of the standard coordinate system through the human-computer interaction interface. If the calibration personnel or other technicians observe through the human-computer interaction interface that the corresponding rendered images of the cameras located on opposite sides of the vehicle body are asymmetric under the target perspective, they can operate the controls of the coordinate axes of the coordinate system corresponding to the target perspective of the camera through the human-computer interaction interface, thereby triggering the corresponding adjustment instruction. After receiving the adjustment instruction, the coordinate axes of the coordinate system corresponding to the target perspective can be adjusted according to the adjustment instruction to obtain a new target perspective.

[0115] In some embodiments, the adjustment instruction may be an adjustment of the direction of one, two or three of the three coordinate axes of the coordinate system corresponding to the target viewing angle. For example, if the adjustment instruction is for the x-axis, the direction of the x-axis may be adjusted; if the adjustment instruction is for the x-axis and the z-axis, the directions of the x-axis and the z-axis may be adjusted.

[0116] S603 . Determine a new rendered image corresponding to the reference camera at the new target viewing angle based on the homography change matrix corresponding to the reference camera at the new target viewing angle.

[0117] In some embodiments, after determining the new target viewing angle, the pose information of the reference cameras at their respective corresponding new target viewing angles can be determined, and the homography transformation matrix corresponding to the reference cameras can be determined based on the pose information. The rendered image corresponding to the reference cameras at the new target viewing angle can then be calculated based on the homography transformation matrix.

[0118] In some embodiments, if the new rendered images corresponding to the two cameras located at symmetrical positions on the left and right sides of the vehicle body are asymmetrical under the target viewing angle, the calibration personnel or other technicians can operate the controls of the coordinate axes of the coordinate system corresponding to the reference camera under the new target viewing angle through the human-computer interaction interface to continue adjusting the new target viewing angle until the rendered images on the left and right sides are symmetrical.

[0119] According to the above technical means, by outputting the corresponding rendering images of the two cameras located at symmetrical positions on the left and right sides of the vehicle body under the target viewing angle to different areas of the human-computer interaction interface for comparative display, it is convenient for calibration personnel or other technical personnel to fine-tune the target viewing angle according to the display effect of the rendered image, so as to obtain a rendered image symmetrical on the left and right sides.

[0120] In an embodiment of the present application, the first pose information of the cameras located on the opposite sides of the vehicle body in the world coordinate system is obtained; the first pose information is converted into the second pose information of the camera in the standard coordinate system; the standard coordinate system is a coordinate system determined by the pitch angle, yaw angle and flip angle of the camera are all zero; based on the second pose information, the homography transformation matrix of the camera is determined to determine the corresponding rendering image of the camera under the target viewing angle based on the homography transformation matrix. In this way, by converting the first pose information of the cameras on the opposite sides of the vehicle body in the world coordinate system into the second pose information in the standard coordinate system, the camera viewing angles on the opposite sides of the vehicle body can be guaranteed to be symmetrical, and when the parameters of each camera change from the world coordinate system to the target viewing angle are the same, the rendering images corresponding to the cameras on the opposite sides of the vehicle body under the target viewing angle can be made symmetrical to each other. In addition, by converting the first pose information of the cameras located on the opposite sides of the vehicle body in the world coordinate system into the second pose information in the standard coordinate system, the individual differences in the camera installation angles are ignored in the standard coordinate system, so there is no need to adjust the wheel viewing angle parameters for each camera separately, thereby improving the efficiency of vehicle-mounted image calibration.

[0121] Next, the implementation process of the application embodiment in the actual application scenario is introduced.

[0122] like Figure 2 As shown, the present application provides a schematic diagram of the architecture of a wheel viewing angle calibration system based on vehicle-mounted panoramic images. The wheel viewing angle calibration system 700 based on vehicle-mounted panoramic images includes a human-computer interaction interface 701, an algorithm module 702 and a rendering module 703.

[0123] The human-computer interaction interface 701 provides a convenient parameter adjustment interface for calibration personnel, realizes rapid calibration, and provides the required calibration parameters for the algorithm library. The human-computer interaction interface 701 includes a view switching instruction, a target view camera pose instruction, an affine transformation instruction, a calibration instruction small window preview instruction, etc. The view switching instruction can trigger the algorithm module 702 to update the camera view, the target view camera pose instruction can trigger the algorithm module 702 to update the camera pose, the affine transformation instruction can trigger the algorithm module 702 to update the affine transformation matrix, and the calibration instruction can trigger the algorithm module 702 to generate calibration products, such as a homography transformation matrix; the small window preview instruction can trigger the rendering module 703 to display the generated rendering image in the rendering window.

[0124] The algorithm module 702 adjusts the camera's posture and calculates the homography transformation matrix according to the prior camera intrinsic parameters, distortion coefficients, camera extrinsic parameters and other data and the camera posture adjustment parameters of the human-computer interaction interface 701. The algorithm module 702 converts the camera coordinate system into the world coordinate system through the camera extrinsic parameters, converts the world coordinate system to the standard coordinate system, and calculates the homography transformation matrix according to the camera posture and affine transformation matrix of the target perspective, and sends the homography transformation matrix to the rendering module 703.

[0125] The rendering module 703 performs inverse perspective transformation and inverse dedistortion on the image pixels according to the homography transformation matrix and camera internal parameters calculated by the algorithm module 702 , samples the original image, and renders it to the human-computer interaction interface 701 .

[0126] Figure 3 This application provides a flow chart of a method for wheel viewing angle calibration based on vehicle-mounted panoramic images. The method is applied to Figure 2 The wheel viewing angle calibration system based on vehicle-mounted panoramic image shown in Figure 3 The method is described, and the method comprises:

[0127] S801. Obtain left and right camera intrinsic parameters, distortion coefficients, and camera extrinsic parameters through calibration.

[0128] In some embodiments, the left and right camera intrinsic parameters, distortion coefficients, and camera extrinsic parameters may be pre-calibrated, for example, they may be directly obtained from the parameters of the left and right cameras provided after the vehicle leaves the factory.

[0129] S802: Convert the camera coordinate system to the world coordinate system through the camera external reference.

[0130] In some embodiments, the rotation matrix R from the camera coordinate system to the world coordinate system with the center of the vehicle body as the origin can be calculated based on the external parameters provided by the vehicle factory. camera2world , according to the coordinate system conversion formula and the rotation matrix, the camera is converted from the camera coordinate system to the world coordinate system.

[0131] S803: Convert the world coordinate system into a standard coordinate system through coordinate conversion.

[0132] In some embodiments, the position and posture of the camera can be corrected to obtain a standard state R in which the pitch angle, yaw angle and roll angle of each camera are all 0. std, and the coordinate system where the camera in this state is located is named the standard coordinate system, and the camera pose is named the standard pose. Since this step converts the pose of the original camera into the standard pose, when the angle is adjusted later, it can ensure that the images of the left and right cameras can remain symmetrical under the premise of the same parameters. And because the pose of the original camera is converted to the standard coordinate system through external parameters, the standard pose of the left and right cameras of each car model is consistent, that is to say, when the external parameters of each car are accurate when calibrated at the factory, the individual differences in the camera installation angles will be ignored in the standard coordinate system, so there is no need to adjust the wheel perspective parameters for each camera separately.

[0133] S804, after adjusting the camera posture of the standard coordinate system, obtain the target view camera external parameter matrix.

[0134] In some embodiments, the human-computer interaction interface supports the calibration personnel to quickly adjust the target position of the camera in the human-computer interaction interface, supports the camera posture in three dimensions of yaw angle, pitch angle and roll angle, the position in three dimensions of X-axis, Y-axis and Z-axis, and the translation, rotation and scaling of the final rendered image, so as to obtain the ideal wheel perspective view through multi-dimensional adjustment.

[0135] In some embodiments, the current pose matrix of the target camera and the affine transformation matrix of the rendered image can be calculated according to the instructions transmitted from the human-computer interaction interface to the algorithm module 702. The rotation matrix calculation formula can be expressed by formula (1), and the affine transformation matrix can be expressed by formula (2). Afterwards, as shown in formula (3), the rotation matrix from the standard coordinate system to the target coordinate system, i.e., the target view camera external parameter matrix, can be calculated according to the rotation matrix.

[0136] S805, calculating a homography transformation matrix based on the target viewing angle camera extrinsic parameter matrix.

[0137] In some embodiments, after obtaining the view camera extrinsic parameter matrix (rotation matrix) of the camera posture at each orientation converted to the virtual posture of the target view camera, the translation vector from the original camera posture and the target camera posture can be calculated through the rotation matrix; according to the posture relationship between the two view cameras and the prior information of the plane to be perspective transformed, such as the normal vector of the plane to be perspective transformed in the camera coordinate system and the distance between the camera and the ground, the homography transformation matrix is ​​calculated. The calculation formula of the homography transformation matrix is ​​shown in formula (4).

[0138] S806: Render the target perspective image using the homography transformation matrix and the camera internal parameters.

[0139] In some embodiments, in order to achieve the purpose of real-time rendering, the original image cannot be directly transformed by using the homography transformation matrix. Instead, based on the Open Graphics Library (OpenGL), the texture coordinates of the rendering window are first converted to the pixel coordinates of the rendering window as the input of the fragment shader, and the pixel coordinates of the dedistorted image are obtained by applying the coordinates to the inverse of the homography transformation matrix for inverse perspective transformation; then the pixel coordinates of the dedistorted image are inversely distorted to obtain the coordinates in the original image, and the RGB values ​​of the pixels are obtained from the original image through texture sampling, and rendered to the human-computer interaction interface 701 to provide real-time calibration result display for the calibration personnel. Among them, the inverse distortion of the pixel coordinates can be calculated by formula (6):

[0140] r(θ)=k 0 θ+k 1 θ 3 +k 2 θ 5 +k 3 θ 7 +k 4 θ 9 (6);

[0141] Where θ is the incident angle of the camera light, k 0 =1, k 1 , k 2 , k 3 , k 4 are the distortion coefficients of the camera.

[0142] In addition, in order to further ensure the symmetry of the left and right views, such as Figure 4 As shown in the figure, a small preview window is added to the human-computer interaction interface. The window is divided into a left view and a right view. The target area in the corresponding view is rendered respectively, and the wheel views on both sides are compared and corrected at the same time during calibration. Figure 4 The small preview window is in the lower right corner.

[0143] For example, Figure 5 This is the effect of the front wheel perspective after the rendered image is synthesized. Figure 6 This is the effect of the car body perspective after the rendered image is synthesized. Figure 7 This is the effect of the rear wheel perspective after the rendered image is synthesized. It can be found that no matter the front wheel perspective, the body perspective or the rear wheel perspective, the synthesized rendered image has no traces of splicing or misalignment. Figure 8 This is a rendering of the left vehicle body from a single perspective. Fig. 9 This is a rendering of the right vehicle body from a single perspective. Figure 8 and Fig. 9 It can be found that the images on the left and right sides of the car body are symmetrical.

[0144] The wheel perspective calibration method based on vehicle-mounted panoramic images provided in the embodiment of the present application uses the camera external parameters calibrated in the calibration workshop of the vehicle to send a posture adjustment command using the human-computer interaction interface, first unifies the postures of the left and right cameras in the world coordinate system, and then adjusts the postures of the cameras on both sides to the same extent to ensure the symmetry of the left and right perspectives. Calculate the corresponding homography transformation matrix and apply it to the images of the left and right cameras to obtain the ideal wheel perspective image, and finally obtain the correspondence between the rendered points and the pixels in the fisheye image based on the calculated perspective transformation matrix and inverse distortion, and render the image of the wheel perspective in real time.

[0145] The wheel perspective calibration method based on vehicle-mounted panoramic images provided in the embodiment of the present application performs special processing on the camera image to distinguish it from the wheel perspective that directly uses the fisheye image and the dedistortion image, so as to provide the driver with an image that conforms to human visual habits; the design of the human-machine interface does not require a large amount of parameter adjustment, and a simple adjustment can obtain the ideal perspective, so that fast parameter adjustment can be achieved; when adjusting the camera's posture transformation, instead of directly adjusting the parameters of a single camera separately, the camera postures of the left and right cameras are unified to the standard coordinate system, and the posture of the target perspective camera is adjusted based on the standard coordinate system, which can ensure the symmetry of the left and right perspectives; only one parameter adjustment is required for a vehicle model, and there is no need to adjust the parameters separately for cameras with individual differences in installation positions, thereby improving the calibration efficiency of the camera.

[0146] The present application provides a vehicle-mounted image calibration device. Fig.10 A schematic diagram of the structure of a vehicle-mounted image calibration device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the vehicle-mounted image calibration device 900 includes:

[0147] A first acquisition module 901 is used to acquire first position information of cameras located at opposite sides of the vehicle body in the world coordinate system;

[0148] A first conversion module 902, used to convert the first posture information into second posture information of the camera in a standard coordinate system; the standard coordinate system is a coordinate system determined in a state where the pitch angle, yaw angle and roll angle of the camera are all zero;

[0149] The first determination module 903 is used to determine the homography transformation matrix of the camera based on the second posture information, so as to determine the rendering image corresponding to the camera under the target viewing angle based on the homography transformation matrix.

[0150] In some embodiments, the first determining module 903 includes:

[0151] A first acquisition submodule, used to acquire a first rotation matrix of the camera in a camera coordinate system;

[0152] A first determination submodule is used to determine a second rotation matrix from the standard coordinate system to a target coordinate system according to the first rotation matrix and the second posture information; the target coordinate system is determined based on a virtual posture corresponding to the camera at a target viewing angle;

[0153] The first determination submodule is used to determine the homography transformation matrix based on the second rotation matrix and the position translation vector of the camera from the camera coordinate system to the target coordinate system.

[0154] In some embodiments, the vehicle-mounted image calibration device 900 further includes:

[0155] A second acquisition module is used to acquire original pose information of the camera in the camera coordinate system and third pose information of the camera in the target coordinate system;

[0156] The second determination module is used to determine the posture translation vector of the camera from the camera coordinate system to the target coordinate system based on the original posture information and the third posture information.

[0157] In some embodiments, the camera includes a first camera located on the left side of the vehicle body and a second camera located on the right side of the vehicle body, and the first coordinate axis of the standard coordinate system corresponding to the first camera and the first coordinate axis of the standard coordinate system corresponding to the second camera are parallel or perpendicular to each other.

[0158] In some embodiments, the vehicle-mounted image calibration device 900 further includes:

[0159] A third acquisition module is used to acquire the affine transformation matrix of the camera;

[0160] The first updating module is used to update the homography transformation matrix based on the affine transformation matrix to obtain a new homography transformation matrix, so as to determine the rendering image corresponding to the camera under the target viewing angle based on the new homography transformation matrix.

[0161] In some embodiments, the vehicle-mounted image calibration device 900 further includes:

[0162] A perspective transformation module, configured to perform an inverse perspective transformation on the first pixel coordinates of the image corresponding to the camera at the target viewing angle based on the homography transformation matrix to obtain the second pixel coordinates of the dedistorted image;

[0163] A distortion processing module, used for performing inverse distortion processing on the second pixel coordinates to obtain third pixel coordinates of the original image acquired by the camera;

[0164] A fourth acquisition module, used for acquiring target image content from the original image based on the third pixel coordinate;

[0165] A filling module is used to fill the target image content to the corresponding position of the first pixel coordinate to obtain the rendered image.

[0166] In some embodiments, the camera includes two cameras located at symmetrical positions on the left and right sides of the vehicle body; the vehicle-mounted image calibration device 900 also includes:

[0167] An output module, used for outputting the rendered images corresponding to the two cameras at the target viewing angle to different areas of the human-computer interaction interface for comparative display;

[0168] an adjustment module, configured to adjust the target viewing angle of the reference camera in the human-computer interaction interface to obtain a new target viewing angle upon receiving an adjustment instruction for the target viewing angle of the reference camera in the human-computer interaction interface; the reference camera is at least one of the two cameras;

[0169] The third determination module is used to determine the rendering image corresponding to the reference camera at the new target perspective based on the homography transformation matrix corresponding to the reference camera at the new target perspective.

[0170] It should be noted that the description of the vehicle-mounted image calibration device in the embodiment of the present application is similar to the description of the corresponding vehicle-mounted image calibration embodiment described above, and has similar beneficial effects as the method embodiment, so it is not repeated here. For technical details not disclosed in this embodiment, please refer to the description of the method embodiment of the present application for understanding.

[0171] It should be noted that in the embodiments of the present application, if the above-mentioned vehicle-mounted image calibration method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0172] Accordingly, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the vehicle-mounted image calibration method provided in the above embodiment is implemented.

[0173] Fig.11 A schematic diagram of the structure of a vehicle-mounted image calibration device provided in an embodiment of the present application is as follows: Fig.11 As shown, the vehicle-mounted image calibration device 10 includes: a memory 11, a processor 12, a communication interface 13 and a communication bus 14. The memory 11 is used to store executable vehicle-mounted image calibration instructions; the processor 12 is used to execute the executable vehicle-mounted image calibration instructions stored in the memory 11 to implement the vehicle-mounted image calibration method provided in the above embodiment.

[0174] The description of the above vehicle-mounted image calibration device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the vehicle-mounted device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0175] It should be noted that, in this article, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises at least one ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0176] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0177] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0178] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0179] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, disks or optical disks.

[0180] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a product to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0181] The above are only implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technical object familiar with the technical field can be easily thought of within the technical scope disclosed in the present application. Changes or substitutions should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A vehicle image calibration method, characterized in that: include: Obtain the first position information of the cameras located on opposite sides of the vehicle body in the world coordinate system; Converting the first posture information into second posture information of the camera in a standard coordinate system; the standard coordinate system is a coordinate system determined in a state where the pitch angle, yaw angle and roll angle of the camera are all zero; Based on the second pose information, a homography transformation matrix of the camera is determined to determine a rendered image corresponding to the camera under a target viewing angle based on the homography transformation matrix.

2. The method according to claim 1, characterized in that: The determining, based on the second pose information, a homography transformation matrix of the camera includes: Obtain a first rotation matrix of the camera in a camera coordinate system; Determine a second rotation matrix from the standard coordinate system to a target coordinate system according to the first rotation matrix and the second posture information; the target coordinate system is determined based on a virtual posture corresponding to the camera at a target viewing angle; The homography transformation matrix is ​​determined based on the second rotation matrix and the pose translation vector of the camera from the camera coordinate system to the target coordinate system.

3. The method according to claim 2, characterized in that The method further comprises: Obtaining original pose information of the camera in the camera coordinate system and third pose information of the camera in the target coordinate system; Based on the original pose information and the third pose information, a pose translation vector of the camera from the camera coordinate system to the target coordinate system is determined.

4. The method according to claim 1, characterized in that: The camera includes a first camera located on the left side of the vehicle body and a second camera located on the right side of the vehicle body, and a first coordinate axis of a standard coordinate system corresponding to the first camera and a first coordinate axis of a standard coordinate system corresponding to the second camera are parallel or perpendicular to each other.

5. The method according to claim 1, characterized in that The method further comprises: Obtaining the affine transformation matrix of the camera; The homography transformation matrix is ​​updated based on the affine transformation matrix to obtain a new homography transformation matrix, so as to determine a rendered image corresponding to the camera under a target viewing angle based on the new homography transformation matrix.

6. The method according to claim 1, characterized in that The method further comprises: Based on the homography transformation matrix, an inverse perspective transformation is performed on the first pixel coordinates of the image corresponding to the camera at the target viewing angle to obtain the second pixel coordinates of the dedistorted image; Performing inverse distortion processing on the second pixel coordinates to obtain third pixel coordinates of the original image captured by the camera; Based on the third pixel coordinates, acquiring target image content from the original image; The target image content is filled into the corresponding position of the first pixel coordinate to obtain the rendered image.

7. The method according to any one of claims 1 to 6, characterized in that: The camera includes two cameras located at symmetrical positions on the left and right sides of the vehicle body; the method also includes: Outputting the rendered images corresponding to the two cameras at the target viewing angle to different areas of the human-computer interaction interface for comparative display; If an adjustment instruction for the target viewing angle of a reference camera in the human-computer interaction interface is received, the target viewing angle of the reference camera is adjusted to obtain a new target viewing angle; the reference camera is at least one of the two cameras; Based on the homography transformation matrix corresponding to the reference camera at the new target viewing angle, a rendering image corresponding to the reference camera at the new target viewing angle is determined.

8. A vehicle-mounted image calibration device, characterized in that: include: A first acquisition module is used to acquire first position information of cameras located on opposite sides of the vehicle body in the world coordinate system; A first conversion module, used to convert the first posture information into second posture information of the camera in a standard coordinate system; the standard coordinate system is a coordinate system determined in a state where the pitch angle, yaw angle and roll angle of the camera are all zero; The first determination module is used to determine the homography transformation matrix of the camera based on the second posture information, so as to determine the rendered image corresponding to the camera under the target viewing angle based on the homography transformation matrix.

9. A vehicle-mounted image calibration device, characterized in that: include: processor and memory; wherein, The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the method according to any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores computer program code, and when the computer program code is executed by a computer, the method according to any one of claims 1 to 7 is executed.

Images