Vehicle-mounted camera external parameter calibration method and device, computer equipment and storage medium

Abstract

The invention relates to a vehicle-mounted camera external parameter calibration method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring multiple frames of images acquired by a vehicle-mounted camera; determining target pose information of the vehicle-mounted camera corresponding to each frame of image based on a matching relationship among the feature points in the multiple frames of images; determining a first conversion relation from a world coordinate system to a vehicle body coordinate system and a second conversion relation from a camera coordinate system to the world coordinate system based on the target pose information; and determining the external parameters of the vehicle-mounted camera based on the first conversion relation and the second conversion relation. The vehicle-mounted camera external parameter calibration method provided by the embodiment of the invention is wider in application scene and simpler and more effective in calibration mode. And on the other hand, by obtaining the image in the actual scene for calibration, the calibrated external parameters of the vehicle can be more adaptive to the actual scene. According to the invention, the universality and reliability of vehicle external parameter calibration can be effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of in-vehicle camera calibration, and particularly to an in-vehicle camera extrinsic parameter calibration method, device, computer device, and storage medium. Background Art

[0002] When a vehicle is first delivered, or after vehicle repairs involving camera replacement and position changes, the in-vehicle camera needs to be automatically calibrated before the perception algorithms of the in-vehicle camera can be started and used normally to assist driving. Automatic calibration of the in-vehicle camera can simplify the production line calibration process, reduce production line costs, and improve production line efficiency at the production end, and can also effectively reduce the operation and maintenance costs of returning to the factory and 4S stores at the after-sales end.

[0003] However, the in-vehicle camera calibration methods in traditional technologies are usually offline calibration methods, that is, a checkerboard is set on the vehicle production line to calibrate the extrinsic parameters of the in-vehicle camera. However, on the one hand, due to the single vehicle production environment, the extrinsic parameters of the vehicle calibrated offline are difficult to meet complex road environments; on the other hand, after vehicle repairs involving camera replacement and position changes, it is still necessary to return to the factory for re-calibration. Therefore, the calibration methods in traditional technologies have poor versatility and low reliability. Summary of the Invention

[0004] Based on this, it is necessary to provide an in-vehicle camera extrinsic parameter calibration method, device, computer device, and storage medium that can improve the versatility and reliability of vehicle extrinsic parameter calibration for the above technical problems.

[0005] In a first aspect, this application provides an in-vehicle camera extrinsic parameter calibration method. The method includes:

[0006] Obtain multiple frames of images collected by the in-vehicle camera;

[0007] Based on the matching relationship between the feature points in the multiple frames of images, determine the target pose information of the in-vehicle camera corresponding to each frame of image;

[0008] Based on the target pose information, determine the first conversion relationship from the world coordinate system to the vehicle body coordinate system and the second conversion relationship from the camera coordinate system to the world coordinate system;

[0009] Based on the first conversion relationship and the second conversion relationship, determine the in-vehicle camera extrinsic parameters.

[0010] In one embodiment, the determining the target pose information of the in-vehicle camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of images includes:

[0011] Based on the number of matching feature points, determine an initial image in the multiple frames of images, where the matching feature points include the projection points of the same three-dimensional point on different frames of images;

[0012] Determine the initial pose information based on the matching feature points of the initial image, and determine the three-dimensional point cloud information based on the initial pose information;

[0013] Determine the target pose information based on the initial pose information and the three-dimensional point cloud information.

[0014] In one embodiment, the target pose information includes position information, and the determining the first conversion relationship from the vehicle body coordinate system to the world coordinate system based on the target pose information includes:

[0015] Determine the first vector coordinates from the vehicle body coordinate system to the world coordinate system based on the position information;

[0016] Determine the first conversion relationship based on the first vector coordinates and the second vector coordinates in the vehicle body coordinate system.

[0017] In one embodiment, the first vector coordinates include the first x-axis vector coordinates, the first y-axis vector coordinates, and the first z-axis vector coordinates, and the second vector coordinates include the second x-axis vector coordinates;

[0018] The determining the first vector coordinates from the vehicle body coordinate system to the world coordinate system based on the position information includes:

[0019] Determine the first x-axis vector coordinates based on the position information and the second x-axis vector coordinates;

[0020] Determine the first z-axis vector coordinates based on the matching feature points;

[0021] Determine the first y-axis vector coordinates based on the first x-axis vector coordinates and the first z-axis vector coordinates.

[0022] In one embodiment, the second vector coordinates include the second z-axis vector coordinates, and the determining the first z-axis vector coordinates based on the matching feature points includes:

[0023] Determine the first z-axis vector coordinates based on the ground feature points and the second z-axis vector coordinates;

[0024] Or;

[0025] Perform plane fitting on the three-dimensional point cloud information to determine the first z-axis vector coordinates.

[0026] In one embodiment, the second vector coordinates include the second y-axis vector coordinates, and the determining the first conversion relationship based on the first vector coordinates and the second vector coordinates in the vehicle body coordinate system includes:

[0027] Perform singular value decomposition on the first vector coordinates and the second vector coordinates to determine the first conversion relationship, where the second vector coordinates include the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates in the vehicle body coordinate system.

[0028] In one embodiment, the target pose information includes attitude information, and determining the second conversion relationship from the camera coordinate system to the world coordinate system based on the target pose information includes:

[0029] Determine the second conversion relationship based on the average matrix of the attitude information.

[0030] In one embodiment, determining the extrinsic parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship includes:

[0031] Determine the extrinsic parameters of the vehicle-mounted camera based on the product of the first conversion relationship and the second conversion relationship.

[0032] In one embodiment, determining the target pose information based on the initial pose information and the three-dimensional point cloud information includes:

[0033] Determine the reconstructed image in the multiple frames of images based on the number of matching feature points;

[0034] Determine the current frame pose information based on the matching feature points of the reconstructed image, and update the three-dimensional point cloud information based on the current frame pose information to determine the current three-dimensional point cloud information;

[0035] Jointly update the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function to determine the target pose information.

[0036] In one embodiment, jointly updating the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function includes:

[0037] Jointly update the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function and the matching feature points of the co-visible frame images of the reconstructed image to determine the first pose information and the first three-dimensional point cloud information, where the co-visible frame images are determined based on the number of matching feature points of the reconstructed image;

[0038] Repeat jointly updating the current frame pose information and the current three-dimensional point cloud information until the preset update condition is met, and then determine the second pose information and the second three-dimensional point cloud information;

[0039] Based on the preset optimization function and the matching feature points of all the reconstructed images, jointly update the second pose information and the second 3D point cloud information to determine the target pose information.

[0040] In one embodiment, the preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the 3D point cloud information.

[0041] In a second aspect, the present application also provides a device for calibrating the external parameters of a vehicle-mounted camera. The device includes:

[0042] An image acquisition module, configured to acquire multiple frames of images collected by the vehicle-mounted camera;

[0043] A target pose information determination module, configured to determine the target pose information of the vehicle-mounted camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of images;

[0044] A conversion relationship determination module, configured to determine a first conversion relationship from the vehicle body coordinate system to the world coordinate system and a second conversion relationship from the camera coordinate system to the world coordinate system based on the target pose information;

[0045] A vehicle-mounted camera external parameter determination module, configured to determine the external parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

[0046] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any one of the vehicle-mounted camera external parameter calibration methods in the first aspect are implemented.

[0047] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the vehicle-mounted camera external parameter calibration methods in the first aspect are implemented.

[0048] For the above vehicle-mounted camera external parameter calibration method, device, computer device, and storage medium, by acquiring multiple frames of images collected by the vehicle-mounted camera, determining the matching relationship between the feature points in the multiple frames of images, and determining the target pose information of the vehicle-mounted camera corresponding to each frame of image, the external parameters of the vehicle-mounted camera are calibrated through the target pose information and the coordinate system conversion relationship. After the vehicle is first delivered, or after the vehicle-mounted camera is replaced or its position changes, the external parameters of the vehicle-mounted camera can be calibrated in a conventional use scenario without having to return to the factory for recalibration, with a wider applicable scenario and a simpler and more effective calibration method. On the other hand, by acquiring images in an actual scenario for calibration, the calibrated vehicle external parameters can also be better adapted to the actual scenario. The present application can effectively improve the versatility and reliability of vehicle external parameter calibration.

[0049] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments and descriptions thereof are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0051] Figure 1 It is an application environment diagram of the external parameter calibration method for an in-vehicle camera in one embodiment;

[0052] Figure 2 It is a schematic flowchart of the external parameter calibration method for an in-vehicle camera in one embodiment;

[0053] Figure 3 It is an image acquired by an infrared camera in one embodiment;

[0054] Figure 4 It is an image acquired by a color camera in one embodiment;

[0055] Figure 5 It is multiple frames of images acquired by a near-infrared camera in one embodiment;

[0056] Figure 6 It is a schematic diagram of a co-visible frame image of a reconstructed image in one embodiment;

[0057] Figure 7 It is a schematic diagram of a vehicle body coordinate system in one embodiment;

[0058] Figure 8 It is a schematic diagram of the current frame pose information and the current three-dimensional point cloud information in one embodiment;

[0059] Figure 9 It is a schematic diagram of the current frame pose information in one embodiment;

[0060] Figure 10 It is a schematic diagram of ground point cloud information in one embodiment;

[0061] Figure 11 It is a schematic flowchart of the external parameter calibration method for an in-vehicle camera in a specific embodiment;

[0062] Figure 12 It is a structural block diagram of an external parameter calibration device for an in-vehicle camera in one embodiment;

[0063] Figure 13 It is an internal structure diagram of a computer device in one embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] In order to make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] As used hereinafter, terms such as "module" and "unit" are combinations of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in hardware, implementation in software, or a combination of software and hardware is also possible and contemplated.

[0066] The method for calibrating the extrinsic parameters of an in-vehicle camera provided by an embodiment of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers. The terminal 102 acquires multiple frames of images collected by the in-vehicle camera and sends them to the server 104. The server 104 determines the target pose information of the in-vehicle camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of images; determines the first conversion relationship from the world coordinate system to the vehicle body coordinate system and the second conversion relationship from the camera coordinate system to the world coordinate system based on the target pose information; determines the extrinsic parameters of the in-vehicle camera based on the first conversion relationship and the second conversion relationship, and sends them to the terminal 102. In other embodiments, the steps of the method for calibrating the extrinsic parameters of the in-vehicle camera in the above embodiments can also be executed only by the terminal 102. Among them, the terminal 102 can include in-vehicle devices such as in-vehicle cameras. The server 104 can include a vehicle cloud server, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0067] In one embodiment, as Figure 2 shown, a method for calibrating the extrinsic parameters of an in-vehicle camera is provided. Taking the application scenario in Figure 1 as an example, the method includes the following steps:

[0068] S201: Acquire multiple frames of images collected by the in-vehicle camera.

[0069] In the embodiments of the present application, the vehicle-mounted camera may include an imaging device disposed on the vehicle. Specifically, the vehicle-mounted camera may include a fisheye camera, a near and far infrared camera, etc. The installation methods of the vehicle-mounted camera on the vehicle may include panoramic setting, surround view setting, front view setting, etc. The multiple frames of images collected by the vehicle-mounted camera may include continuously collecting a preset number of frames of images by the vehicle-mounted camera, or may include multiple frames of images at intervals of a preset number of frames in the continuous frame images. It can be understood that the type of the multiple frames of images matches the type of the vehicle-mounted camera. For example, if the vehicle-mounted camera is an infrared camera, the multiple frames of images are infrared images, as Figure 3 shown; if the vehicle-mounted camera is a color camera, the multiple frames of images are RGB images, as Figure 4 shown.

[0070] In some specific embodiments, if the vehicle-mounted camera includes a near-infrared camera, the multiple frames of images obtained are as Figure 5 shown. Figure 5 The multiple frames of images obtained by the near-infrared camera are shown in . The number of the multiple frames of images is 18, and the multiple frames of images are respectively image_20_01.png - image_20_18.png.

[0071] In the embodiments of the present application, the vehicle-mounted camera collecting multiple frames of images may include multiple frames of images obtained when the vehicle travels a preset distance on the road. Specifically, to further improve the quality of the collected multiple frames of images, the vehicle can be controlled to travel straight at a preset speed for a preset distance on a flat road surface and collect multiple consecutive frames of images. Among them, both sides of the flat road surface may include environmental objects, such as buildings, trees, etc., and there is no interference from dynamic objects in the shooting scene. The flat road surface may include feature markings, such as double dotted lines, zebra crossings, arrows or numbers, etc. The preset speed may include a preset speed range or a preset speed value. The preset speed range may be 20 km / h - 50 km / h, and the preset speed value may be any value within the preset speed range.

[0072] S203: Based on the matching relationship between the feature points in the multiple frames of images, determine the target pose information of the vehicle-mounted camera corresponding to each frame of image.

[0073] In the embodiments of the present application, the feature points in the multiple frames of images may include points where the image gray value changes drastically in the multiple frames of images or points with a large curvature on the image edge, such as corner points, etc. The feature points in the multiple frames of images can be extracted based on an AI algorithm, which can effectively improve the effectiveness and accuracy of the feature points in the multiple frames of images obtained by different types of vehicle-mounted cameras.

[0074] In the embodiments of the present application, based on the matching relationship between the feature points of different frame images, the target pose information of the vehicle-mounted camera corresponding to each frame image can be determined through an optimization algorithm according to the relationship between the spatial coordinates and the image coordinates of the feature points. In some other embodiments, based on the matching relationship between the feature points, the movement of the feature points in different frame images can also be tracked, and the target pose information of the vehicle-mounted camera corresponding to each frame image can be determined through a filter (such as a particle filter, etc.) or an optimization algorithm. In other embodiments, the target pose information of the vehicle-mounted camera corresponding to each frame image can also be determined based on SFM (Structure From Motion). Among them, the matching relationship between the feature points of different frame images can be determined through algorithms such as AI algorithms or KNN (K-Nearest Neighbors) algorithms. Specifically, the 3D matching feature points can be further determined after the 2D matching feature points are determined.

[0075] S205: Determine a first conversion relationship from the world coordinate system to the vehicle body coordinate system and a second conversion relationship from the camera coordinate system to the world coordinate system based on the target pose information.

[0076] S207: Determine the extrinsic parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

[0077] In the embodiments of the present application, the conversion relationship from the vehicle body coordinate system to the world coordinate system can be determined through a line fitting algorithm based on the target pose information, and then the first conversion relationship from the world coordinate system to the vehicle body coordinate system can be determined according to the orthogonal relationship of the coordinate axes. In some other embodiments, the environmental features can also be obtained through the camera and matched with the known high-precision map to determine the conversion relationship from the vehicle-mounted camera coordinate system to the world coordinate system, and then the first conversion relationship from the world coordinate system to the vehicle body coordinate system can be determined based on the target pose information of the vehicle-mounted camera. In other embodiments, the target pose information of the vehicle-mounted camera can also be directly regressed or classified based on a deep learning model (such as a neural network model), and the model can be trained through the labeled data in the training set, so that after the target pose information is input, the first conversion relationship from the world coordinate system to the vehicle body coordinate system can be output through the deep learning module.

[0078] In the embodiments of the present application, the second conversion relationship from the camera coordinate system to the world coordinate system can be determined by taking the average value of the matrix of the target pose information. In other embodiments, the target pose information of the vehicle-mounted camera can also be directly regressed or classified based on a deep learning model (such as a neural network model), and the model can be trained through the labeled data in the training set, so that after the target pose information is input, the second conversion relationship from the camera coordinate system to the world coordinate system can be output through the deep learning module.

[0079] In the embodiments of the present application, after determining the first conversion relationship from the world coordinate system to the vehicle body coordinate system and the second conversion relationship from the camera coordinate system to the world coordinate system, multiplying the first conversion relationship by the second conversion relationship can determine the extrinsic parameters of the vehicle-mounted camera. Of course, the extrinsic parameters of the vehicle-mounted camera can also be determined by training a deep learning model and inputting the first conversion relationship and the second conversion relationship into the model for output.

[0080] The method for calibrating the extrinsic parameters of a vehicle-mounted camera provided by the embodiments of the present application obtains multiple frames of images collected by the vehicle-mounted camera, determines the matching relationship between the feature points in the multiple frames of images, and after determining the target pose information of the vehicle-mounted camera corresponding to each frame of image, calibrates the extrinsic parameters of the vehicle-mounted camera through the target pose information and the coordinate system conversion relationship. After the vehicle is first delivered, or after the vehicle-mounted camera is replaced or its position changes, the extrinsic parameters of the vehicle-mounted camera can be calibrated in a conventional use scenario without having to return to the factory for re-calibration, with a wider applicable scenario and a simpler and more effective calibration method. On the other hand, by obtaining images in an actual scenario for calibration, the calibrated extrinsic parameters of the vehicle can also be better adapted to the actual scenario. The present application can effectively improve the generality and reliability of vehicle extrinsic parameter calibration.

[0081] The following illustrates a method for determining target pose information through embodiments of the present application. In some embodiments, determining the target pose information of the vehicle-mounted camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of images includes:

[0082] S301: Based on the number of matching feature points, determine an initial image among the multiple frames of images, where the matching feature points include the projection points of the same three-dimensional point on different frames of images.

[0083] S303: Based on the matching feature points of the initial image, determine the initial pose information, and determine the three-dimensional point cloud information based on the initial pose information.

[0084] S305: Based on the initial pose information and the three-dimensional point cloud information, determine the target pose information.

[0085] In the embodiments of the present application, the matching feature points include the projection points of the same three-dimensional point in space on different frames of images. Based on the number of matching feature points, an initial image can be determined among the multiple frames of images. The initial image can include two frames of images among the multiple frames of images whose number of matching feature points meets the preset requirements. In some specific embodiments, two frames of images with the largest number of matching feature points among the multiple frames of images can be determined as the initial image. In other embodiments, a quantity threshold can also be set, and two frames of images with the number of matching feature points greater than the quantity threshold and the highest image quality among the multiple frames of images can be determined as the initial image.

[0086] Based on the matching feature points of the initial image, the initial pose information can be determined by constructing and solving the essential matrix. In some embodiments, let the matching point pairs of the initial image be [p1, p2], the camera internal parameter be K, the fundamental matrix be F, and the essential matrix be E. The initial pose information [Ri, Ti] can be determined by solving the essential matrix E. Then, based on the initial pose information, the 3D point cloud information can be determined through triangulation calculation, and the 3D point cloud includes a sparse 3D point cloud. The specific methods for solving the essential matrix and triangulation calculation can refer to traditional techniques, which will not be elaborated in this application. Based on the initial pose information and the 3D point cloud information, the target pose information of the vehicle-mounted camera corresponding to each frame of image can be determined.

[0087] In some embodiments, the determining the target pose information based on the initial pose information and the 3D point cloud information includes:

[0088] S401: Based on the number of the matching feature points, determine a reconstruction image from the multiple frames of images.

[0089] S403: Based on the matching feature points of the reconstruction image, determine the current frame pose information, and update the 3D point cloud information based on the current frame pose information to determine the current 3D point cloud information.

[0090] S405: Based on a preset optimization function, jointly update the current frame pose information and the current 3D point cloud information to determine the target pose information.

[0091] In the embodiments of the present application, based on the number of the matching feature points, the next frame of image is searched from the multiple frames of images as the reconstruction image. In some embodiments, an image that satisfies the co-visible region condition with the initial image is determined from the multiple frames of images as the reconstruction image. Specifically, an image with the largest number of matching feature points with any one of the images in the initial image can be determined as the reconstruction image. An image with the largest ratio of the distribution region of the matching feature points covering all the images to any one of the images in the initial image can also be determined as the reconstruction image. The above judgment conditions can also be combined. For example, sort the images according to the number of the matching feature points from large to small, and select the image with the largest coverage ratio in the order from large to small of the number of the matching feature points as the reconstruction image.

[0092] Based on the matching feature points of the reconstructed image, determine the pose information of the current frame, and based on the pose information of the current frame, determine the three-dimensional point cloud information corresponding to the reconstructed image, and update the three-dimensional point cloud information based on the three-dimensional point cloud information corresponding to the reconstructed image to determine the current three-dimensional point cloud information. Among them, the method of determining the pose information of the current frame and determining the three-dimensional point cloud information corresponding to the reconstructed image based on the pose information of the current frame can refer to the method of determining the initial pose information in the above embodiments and determining the three-dimensional point cloud information based on the initial pose information, which will not be elaborated here. Based on a preset optimization function, jointly update the pose information of the current frame and the current three-dimensional point cloud information to determine the target pose information.

[0093] In some specific embodiments, the pose information of the current frame of the reconstructed image and the current three-dimensional point cloud information are as Figure 8 shown. Figure 8 Among them, cam_pose represents the pose information of the current frame, and 3D points represent the current three-dimensional point cloud information. After magnifying the pose information of the current frame, as Figure 9 shown, each frustum schematically shows the position (yellow circle position) and orientation (green arrow) of the vehicle-mounted camera.

[0094] In some embodiments, the jointly updating the pose information of the current frame and the current three-dimensional point cloud information based on a preset optimization function includes:

[0095] S501: Based on a preset optimization function and the matching feature points of the co-visible frame images of the reconstructed image, jointly update the pose information of the current frame and the current three-dimensional point cloud information to determine the first pose information and the first three-dimensional point cloud information, where the co-visible frame images are determined based on the number of matching feature points of the reconstructed image.

[0096] S503: Repeatedly jointly update the pose information of the current frame and the current three-dimensional point cloud information until after meeting the preset update conditions, determine the second pose information and the second three-dimensional point cloud information.

[0097] S505: Based on the preset optimization function and the matching feature points of all the reconstructed images, jointly update the second pose information and the second three-dimensional point cloud information to determine the target pose information.

[0098] In the embodiments of the present application, the co-visible frames include images with the number of matching feature points with the reconstructed image greater than a preset number threshold, that is, images having a co-visible area with the reconstructed image. Taking Figure 6 as an example, if Figure 6If the newly added frame image shown in the figure is a reconstructed image, then the mapping of the common three-dimensional points between the reconstructed image and the adjacent images 1, 2, and 3, that is, the number of matching feature points between each of the images 1, 2, and 3 and the reconstructed image is greater than the preset number threshold, then it is determined that the images 1, 2, and 3 are the co-visual frame images of the reconstructed image.

[0099] In some embodiments, the preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the three-dimensional point cloud information. Specifically, the preset optimization function may include the cost function E shown in Equation (1):

[0100]

[0101] In Equation (1), P i represents the projection matrix (internal and external camera parameters) of the i-th vehicle-mounted camera, X k represents the k-th three-dimensional point in the current three-dimensional point cloud information, π represents the projection mapping, that is, projecting the three-dimensional point onto the plane of the i-th vehicle-mounted camera, and x ik represents the two-dimensional feature point corresponding to the three-dimensional point on the plane of the i-th vehicle-mounted camera, that is, the matching feature point of the co-visual frame image of the reconstructed image, and ρ ik represents the weight.

[0102] Based on the preset optimization function and the matching feature points of the co-visual frame image of the reconstructed image, the current frame pose information and the current three-dimensional point cloud information are jointly updated by local BA (Bundle Adjustment) optimization to determine the first pose information and the first three-dimensional point cloud information. Specifically, the current frame pose information and the current three-dimensional point cloud information can be jointly and non-linearly optimized, and the optimization goal is to minimize the three-dimensional point cloud reprojection error. The purpose of local BA optimization is to make the determined first pose information and the first three-dimensional point cloud information more accurate, and to improve the constraint ability with the addition of co-visual frame images. The specific method of local BA optimization can refer to traditional techniques, and this application will not elaborate on it.

[0103] Repeat the joint update of the current frame pose information and the current three-dimensional point cloud information until the preset update condition is met, and then determine the second pose information and the second three-dimensional point cloud information. In some embodiments, after determining the corresponding co-visual frame images for each reconstructed image and performing the update described in the above embodiments, it is determined that the preset update condition is met. In other embodiments, a preset number of image frames can also be set. For example, when the number of co-visual frame images of all reconstructed images is greater than the preset number of image frames, it is determined that the preset update condition is met. Before the preset update condition is met, continuously update the current three-dimensional point cloud information according to the method described in the above embodiments to determine the second pose information and the second three-dimensional point cloud information.

[0104] Based on the preset optimization function and the matching feature points of all the reconstructed images, the second pose information and the second 3D point cloud information are jointly updated to determine the target pose information. In some embodiments, after meeting the preset update conditions, a global BA optimization is performed again, that is, all the reconstructed images participate in the update process. The specific preset optimization function and the update process can refer to the above formula (1) and the update process described in the above embodiments. The difference is that due to the different images participating in the optimization, the number of vehicle-mounted cameras and the number of 3D points participating in the summation in the preset optimization function are different. After jointly updating the second pose information and the second 3D point cloud information, the target pose information is determined.

[0105] The following illustrates the specific method for determining the extrinsic parameters of the vehicle-mounted camera through the embodiments of the present application. In some embodiments, the target pose information includes position information, and the determination of the first conversion relationship from the vehicle body coordinate system to the world coordinate system based on the target pose information includes:

[0106] S601: Determine the first vector coordinate from the vehicle body coordinate system to the world coordinate system based on the position information.

[0107] S603: Determine the first conversion relationship based on the first vector coordinate and the second vector coordinate in the vehicle body coordinate system.

[0108] In the embodiments of the present application, the world coordinate system is denoted as W, the vehicle body coordinate system is denoted as V, and the camera coordinate system is denoted as C. Among them, the world coordinate system W includes the coordinate system where the target pose information and the 3D point cloud information are located. The x-axis, y-axis, and z-axis of the vehicle body coordinate system V are as Figure 7 shown. The target pose information Pose = {R i , T i} i∈1～N includes the position information Ti, where N represents the number of multiple frames of images. Based on the position information, the first vector coordinate from the vehicle body coordinate system to the world coordinate system can be determined.

[0109] In some embodiments, the first vector coordinate includes the vector coordinate of the axis vector in the vehicle body coordinate system in the world coordinate system. If the axis vector represents the vehicle body, the first vector coordinate represents the spatial position and orientation of the vehicle body in the world coordinate system. The first vector coordinate can be used to determine the relative position between the vehicle body and the external environment subsequently, providing a basis for the calibration of the extrinsic parameters of the vehicle-mounted camera. In other embodiments, the first vector coordinate can also represent the coordinates of other objects or vectors in the vehicle body coordinate system in the world coordinate system. The present application does not make specific limitations on this, as long as the vector coordinate of the object or target vector in the vehicle body coordinate system in the world coordinate system can be represented by the first vector coordinate.

[0110] The first vector coordinates from the vehicle body coordinate system to the world coordinate system include the first x-axis vector coordinate, the first y-axis vector coordinate, and the first z-axis vector coordinate. In some specific embodiments, the first x-axis vector coordinate, the first y-axis vector coordinate, and the first z-axis vector coordinate respectively represent the x-axis unit vector coordinate, the y-axis unit vector coordinate, and the z-axis unit vector coordinate of the vehicle body coordinate system in the world coordinate system.

[0111] In some embodiments, the second vector coordinates include the vector coordinates of the position and direction of the central axis vector in the vehicle body coordinate system. If the axis vector represents the vehicle body, the second vector coordinates represent the vehicle body coordinates in the vehicle body coordinate system. Through the second vector coordinates of the axis vector in the vehicle body coordinate system and the first vector coordinates of the axis vector in the world coordinate system, the conversion relationship between the vehicle body coordinate system and the world coordinate system can be determined, and thus the conversion from the vehicle body coordinate system to the world coordinate system can be realized. In other embodiments, the second vector coordinates can also represent the vector coordinates of other objects or vectors in the vehicle body coordinate system, and the present application does not make specific limitations thereon.

[0112] The second vector coordinates in the vehicle body coordinate system include the second x-axis vector coordinate, the second y-axis vector coordinate, and the second z-axis vector coordinate. In some specific embodiments, the second x-axis vector coordinate, the second y-axis vector coordinate, and the second z-axis vector coordinate respectively represent the x-axis unit vector coordinate, the y-axis unit vector coordinate, and the z-axis unit vector coordinate in the vehicle body coordinate system.

[0113] First, the method for determining the first x-axis vector coordinate will be described.

[0114] In some embodiments, the first vector coordinates include the first x-axis vector coordinate, the first y-axis vector coordinate, and the first z-axis vector coordinate, and the second vector coordinates include the second x-axis vector coordinate; the determining of the first vector coordinates from the vehicle body coordinate system to the world coordinate system based on the position information includes:

[0115] S701: Determine the first x-axis vector coordinate based on the position information and the second x-axis vector coordinate.

[0116] S703: Determine the first z-axis vector coordinate based on the matching feature points.

[0117] S705: Determine the first y-axis vector coordinate based on the first x-axis vector coordinate and the first z-axis vector coordinate.

[0118] In the embodiments of the present application, the position information {T i} i∈1～NPerform a straight-line fitting to remove outliers. The outliers include position information not on the line. Select any two position information (Ti, Tj) that fall on the line, and then the coordinates of the second x-axis vector x_axis_V[1,0,0] in the vehicle body coordinate system in the world coordinate system can be determined as x_axis_W=(T ix -T jx ,T iy -T jy ,T iz -T jz ).

[0119] The following illustrates the method for determining the coordinates of the first z-axis vector and the method for determining the coordinates of the first y-axis vector through embodiments of the present application. In some embodiments, the second vector coordinates include the coordinates of the second z-axis vector. The determination of the coordinates of the first z-axis vector based on the matching feature points includes:

[0120] S801: Determine the coordinates of the first z-axis vector based on the ground feature points and the coordinates of the second z-axis vector; or; perform a plane fitting on the three-dimensional point cloud information to determine the coordinates of the first z-axis vector.

[0121] In the embodiments of the present application, by obtaining the ground two-dimensional feature points and using the reprojection error to fit the plane, the coordinates of the first z-axis vector z_axis_W in the world coordinate system can be determined, which is the coordinates of the second z-axis vector z_axis_V[0,0,1] in the vehicle body coordinate system. In other embodiments, directly extract the ground point cloud information from the three-dimensional point cloud information, and perform a plane fitting on the ground point cloud information based on the ransac algorithm to determine the coordinates of the first z-axis vector z_axis_W in the world coordinate system. If the plane equation of the ground point cloud information is Ax + By + Cz + D = 0, then the coordinates of the first z-axis vector are z_axis_w=(A,B,C). Specifically, the ground point cloud information is as Figure 10 shown.

[0122] Based on the coordinates of the first x-axis vector and the coordinates of the first z-axis vector, according to the three-axis orthogonality relationship of the coordinate axes, the coordinates of the first y-axis vector y_axis_W of the second y-axis vector y_axis_V[0,1,0] in the vehicle body coordinate system in the world coordinate system can be determined as y_axis_W = cross(z_axis_W, x_axis_V), where cross represents the cross product.

[0123] After determining the coordinates of the first vector, the first transformation relationship from the world coordinate system to the vehicle body coordinate system can be determined based on the coordinates of the first vector and the coordinates of the second vector in the vehicle body coordinate system.

[0124] In some embodiments, the second vector coordinates include second y-axis vector coordinates. Based on the first vector coordinates and the second vector coordinates in the vehicle body coordinate system, determining the first conversion relationship includes:

[0125] S901: Perform singular value decomposition on the first vector coordinates and the second vector coordinates to determine the first conversion relationship. The second vector coordinates include the second x-axis vector coordinate, the second y-axis vector coordinate, and the second z-axis vector coordinate in the vehicle body coordinate system.

[0126] In the embodiments of the present application, the first conversion relationship Rv2w from the world coordinate system to the vehicle body coordinate system is determined as Rv2w = SVD([x_axis_W, y_axis_W, z_axis_W], [x_axis_V, y_axis_V, z_axis_V]). Here, x_axis_W, y_axis_W, and z_axis_W respectively represent the first x-axis vector coordinate, the first y-axis vector coordinate, and the first z-axis vector coordinate in the world coordinate system. x_axis_V, y_axis_V, and z_axis_V respectively represent the second x-axis vector coordinate, the second y-axis vector coordinate, and the second z-axis vector coordinate in the vehicle body coordinate system. SVD represents singular value decomposition.

[0127] The following illustrates the method for determining the second conversion relationship through the embodiments of the present application. In some embodiments, the target pose information includes attitude information. Based on the target pose information, determining the second conversion relationship from the camera coordinate system to the world coordinate system includes determining the second conversion relationship based on the average matrix of the attitude information. In the embodiments of the present application, the target pose information Pose = {R i , T i} i∈1～N also includes the attitude information Ri. The second conversion relationship Rw2c from the camera coordinate system to the world coordinate system is Rw2c = average(Ri), where average represents solving the average matrix.

[0128] In some embodiments, based on the first conversion relationship and the second conversion relationship, determining the extrinsic parameters of the vehicle-mounted camera includes. Determining the extrinsic parameters of the vehicle-mounted camera based on the product of the first conversion relationship and the second conversion relationship. Specifically, the extrinsic parameters of the vehicle-mounted camera may include the extrinsic parameters from the camera coordinate system to the vehicle body coordinate system. The extrinsic parameters of the vehicle-mounted camera Rv2c = Rv2w * Rw2c, where Rv2w represents the first conversion relationship from the world coordinate system to the vehicle body coordinate system, and Rw2c represents the second conversion relationship from the camera coordinate system to the world coordinate system.

[0129] In some specific embodiments, the method for calibrating the extrinsic parameters of the vehicle-mounted camera is as Figure 11As shown in the figure. In the vehicle straight - driving state, multiple frames of images are collected. Feature points and descriptors of each frame of image are extracted based on the deep - learning method and frame - to - frame matching is performed. According to the frame - to - frame matching relationship and through BA optimization, the pose information of each frame of image and the three - dimensional point cloud information (3D point information) are solved. Then, the external - parameter rotation vector R of the vehicle - mounted camera is determined according to the ground feature points and the pose information.

[0130] In some specific embodiments, the test data for the calibration of the external parameters of the vehicle - mounted camera is shown in Table 1.

[0131] Table 1

[0132]

[0133] In Table 1, Roll represents the barrel - roll angle, that is, the rotation angle around the x - axis. Pitch represents the pitch angle, that is, the rotation angle around the y - axis. Yaw represents the yaw angle, that is, the rotation angle around the z - axis.

[0134] It should be understood that although the steps in the flowcharts involved in the above - mentioned embodiments are shown in sequence according to the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above - mentioned embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiments of the present application also provide a vehicle - mounted camera external - parameter calibration device 1300 for implementing the vehicle - mounted camera external - parameter calibration method involved above. The solution provided by this device to solve the problem is similar to the solution recorded in the above - mentioned method. Therefore, the specific limitations in one or more embodiments of the vehicle - mounted camera external - parameter calibration device 1300 provided below can refer to the limitations on the vehicle - mounted camera external - parameter calibration method in the above text, and will not be repeated here.

[0136] In one embodiment, as Figure 12 shown, a vehicle - mounted camera external - parameter calibration device 1300 is provided, including:

[0137] An image acquisition module 1301, configured to acquire multiple frames of images collected by the vehicle - mounted camera;

[0138] The target pose information determination module 1302 is configured to determine the target pose information of the vehicle-mounted camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of images;

[0139] The conversion relationship determination module 1303 is configured to determine a first conversion relationship from the vehicle body coordinate system to the world coordinate system and a second conversion relationship from the camera coordinate system to the world coordinate system based on the target pose information;

[0140] The external parameters determination module 1304 of the vehicle-mounted camera is configured to determine the external parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

[0141] In some embodiments, the target pose information determination module 1302 is further configured to determine an initial image from the multiple frames of images based on the number of matching feature points, where the matching feature points include the projection points of the same three-dimensional point on different frames of images; determine the initial pose information based on the matching feature points of the initial image, and determine the three-dimensional point cloud information based on the initial pose information; and determine the target pose information based on the initial pose information and the three-dimensional point cloud information.

[0142] In some embodiments, the target pose information includes position information, and the conversion relationship determination module 1303 is further configured to determine a first vector coordinate from the vehicle body coordinate system to the world coordinate system based on the position information; and determine the first conversion relationship based on the first vector coordinate and a second vector coordinate in the vehicle body coordinate system.

[0143] In some embodiments, the first vector coordinate includes a first x-axis vector coordinate, a first y-axis vector coordinate, and a first z-axis vector coordinate, and the second vector coordinate includes a second x-axis vector coordinate; the conversion relationship determination module 1303 is further configured to determine the first x-axis vector coordinate based on the position information and the second x-axis vector coordinate; determine the first z-axis vector coordinate based on the matching feature points; and determine the first y-axis vector coordinate based on the first x-axis vector coordinate and the first z-axis vector coordinate.

[0144] In some embodiments, the second vector coordinate includes a second z-axis vector coordinate, and the conversion relationship determination module 1303 is further configured to determine the first z-axis vector coordinate based on the ground feature points and the second z-axis vector coordinate; or perform plane fitting on the three-dimensional point cloud information to determine the first z-axis vector coordinate.

[0145] In some embodiments, the second vector coordinates include second y-axis vector coordinates, and the conversion relationship determination module 1303 is further configured to perform singular value decomposition on the first vector coordinates and the second vector coordinates to determine the first conversion relationship, where the second vector coordinates include the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates in the vehicle body coordinate system.

[0146] In some embodiments, the target pose information includes attitude information, and the conversion relationship determination module 1303 is further configured to determine the second conversion relationship based on the average matrix of the attitude information.

[0147] In some embodiments, the on-vehicle camera extrinsic parameter determination module 1304 is further configured to determine the on-vehicle camera extrinsic parameters based on the product of the first conversion relationship and the second conversion relationship.

[0148] In some embodiments, the target pose information determination module 1302 is further configured to determine a reconstructed image in the multiple frames of images based on the number of matched feature points; determine the current frame pose information based on the matched feature points of the reconstructed image, and update the three-dimensional point cloud information based on the current frame pose information to determine the current three-dimensional point cloud information; jointly update the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function to determine the target pose information.

[0149] In some embodiments, the target pose information determination module 1302 is further configured to jointly update the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function and the matched feature points of the co-visual frame images of the reconstructed image to determine the first pose information and the first three-dimensional point cloud information, where the co-visual frame images are determined based on the number of matched feature points of the reconstructed image; repeatedly jointly update the current frame pose information and the current three-dimensional point cloud information until a preset update condition is satisfied, and then determine the second pose information and the second three-dimensional point cloud information; jointly update the second pose information and the second three-dimensional point cloud information based on the preset optimization function and the matched feature points of all the reconstructed images to determine the target pose information.

[0150] In some embodiments, the preset optimization function is determined based on the projection matrix of the on-vehicle camera and the three-dimensional point cloud information.

[0151] Each module in the above on-vehicle camera extrinsic parameter calibration device 1300 can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0152] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structural diagram may be as shown in Figure 13 . The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for calibrating the external parameters of an in-vehicle camera.

[0153] Those skilled in the art can understand that Figure 13 the structure shown in

[0154] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0155] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the method for calibrating the external parameters of an in-vehicle camera described in any of the above embodiments.

[0156] Unless otherwise defined, technical terms or scientific terms involved in this application shall have the ordinary meanings understood by those with ordinary skills in the technical field to which this application belongs. In this application, words such as "a", "an", "one kind", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The words such as "connect", "be connected", "couple" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0158] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0159] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0160] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for calibrating extrinsic parameters of a vehicle-mounted camera, characterized in that: The method comprises: Get multiple frames of images captured by the vehicle-mounted camera; Determine the target position information of the vehicle-mounted camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of image; Determine a first conversion relationship from a world coordinate system to a vehicle coordinate system and a second conversion relationship from a camera coordinate system to the world coordinate system based on the target posture information; Based on the first conversion relationship and the second conversion relationship, an external parameter of the vehicle-mounted camera is determined.

2. The method according to claim 1, characterized in that The determining, based on the matching relationship between the feature points in the multiple frames of images, the target position information of the vehicle-mounted camera corresponding to each frame of image comprises: Determining an initial image in the multiple frames of images based on the number of matching feature points, wherein the matching feature points include projection points of the same three-dimensional point on different frames of images; Determining initial pose information based on the matching feature points of the initial image, and determining three-dimensional point cloud information based on the initial pose information; The target pose information is determined based on the initial pose information and the three-dimensional point cloud information.

3. The method according to claim 2, characterized in that The target posture information includes position information, and determining a first conversion relationship from a vehicle body coordinate system to a world coordinate system based on the target posture information includes: Determine a first vector coordinate from the vehicle body coordinate system to the world coordinate system based on the position information; The first conversion relationship is determined based on the first vector coordinates and the second vector coordinates in the vehicle body coordinate system.

4. The method according to claim 3, characterized in that The first vector coordinates include a first x-axis vector coordinate, a first y-axis vector coordinate, and a first z-axis vector coordinate, and the second vector coordinates include a second x-axis vector coordinate; The determining of the first vector coordinates from the vehicle body coordinate system to the world coordinate system based on the position information comprises: Determine the first x-axis vector coordinate based on the position information and the second x-axis vector coordinate; Determine the first z-axis vector coordinate based on the matching feature points; The first y-axis vector coordinate is determined based on the first x-axis vector coordinate and the first z-axis vector coordinate.

5. The method according to claim 4, characterized in that The second vector coordinates include a second z-axis vector coordinate, and determining the first z-axis vector coordinates based on the matching feature points includes: Determine the first z-axis vector coordinate based on the ground feature point and the second z-axis vector coordinate; or; Perform plane fitting on the three-dimensional point cloud information to determine the first z-axis vector coordinate.

6. The method according to claim 3, characterized in that The second vector coordinate includes a second y-axis vector coordinate, and determining the first conversion relationship based on the first vector coordinate and the second vector coordinate in the vehicle body coordinate system includes: The first vector coordinates and the second vector coordinates are subjected to singular value decomposition to determine the first transformation relationship, wherein the second vector coordinates include a second x-axis vector coordinate, a second y-axis vector coordinate, and a second z-axis vector coordinate in the vehicle body coordinate system.

7. The method according to claim 1, characterized in that The target posture information includes posture information, and determining the second transformation relationship from the camera coordinate system to the world coordinate system based on the target posture information includes: The second conversion relationship is determined based on the average matrix of the posture information.

8. The method according to claim 1, characterized in that The determining the vehicle-mounted camera external parameter based on the first conversion relationship and the second conversion relationship includes: The vehicle-mounted camera extrinsic parameter is determined based on the product of the first conversion relationship and the second conversion relationship.

9. The method according to claim 2, characterized in that: The determining the target pose information based on the initial pose information and the three-dimensional point cloud information comprises: Determining a reconstructed image in the multiple frames of images based on the number of matching feature points; Determine current frame pose information based on the matching feature points of the reconstructed image, and update the three-dimensional point cloud information based on the current frame pose information to determine current three-dimensional point cloud information; Based on a preset optimization function, the current frame pose information and the current three-dimensional point cloud information are jointly updated to determine the target pose information.

10. The method according to claim 9, characterized in that The jointly updating the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function includes: Based on a preset optimization function and matching feature points of the common view frame image of the reconstructed image, jointly updating the current frame pose information and the current three-dimensional point cloud information, determining the first pose information and the first three-dimensional point cloud information, wherein the common view frame image is determined based on the number of matching feature points of the reconstructed image; Repeatedly jointly updating the current frame pose information and the current three-dimensional point cloud information until a preset update condition is met, and then determining second pose information and second three-dimensional point cloud information; Based on the preset optimization function and the matching feature points of all reconstructed images, the second pose information and the second three-dimensional point cloud information are jointly updated to determine the target pose information.

11. The method according to claim 9 or 10, characterized in that: The preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the three-dimensional point cloud information.

12. A vehicle-mounted camera extrinsic calibration device, characterized in that: The device comprises: An image acquisition module is used to acquire multiple frames of images captured by a vehicle-mounted camera; A target posture information determination module, used to determine the target posture information of the vehicle-mounted camera corresponding to each frame of image based on the matching relationship between the feature points in the multiple frames of image; A conversion relationship determination module, used to determine a first conversion relationship from a vehicle body coordinate system to a world coordinate system and a second conversion relationship from a camera coordinate system to the world coordinate system based on the target posture information; The vehicle-mounted camera extrinsic parameter determination module is used to determine the vehicle-mounted camera extrinsic parameter based on the first conversion relationship and the second conversion relationship.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images