Method and apparatus for calibrating extrinsic parameters of in-vehicle camera, and computer device and storage medium

Abstract

A method and apparatus for calibrating extrinsic parameters of an in-vehicle camera, and a computer device and a storage medium. The method comprises: acquiring a plurality of image frames collected by means of an in-vehicle camera; on the basis of a matching relationship between feature points in the plurality of image frames, determining target pose information of the in-vehicle camera that corresponds to each image frame; on the basis of the target pose information, determining a first transformation relationship between a world coordinate system and a vehicle coordinate system and a second transformation relationship between a camera coordinate system and the world coordinate system; and on the basis of the first transformation relationship and the second transformation relationship, determining extrinsic parameters of the in-vehicle camera.

Description

Methods, devices, computer equipment, and storage media for calibrating external parameters of vehicle-mounted cameras

[0001] Related applications

[0002] This application claims priority to Chinese Patent Application No. 202411907436.9, filed on December 23, 2024, entitled “Method, Apparatus, Computer Equipment and Storage Medium for Calibration of External Parameters of Vehicle Camera”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of vehicle-mounted camera calibration technology, and in particular to a method, apparatus, computer equipment, and storage medium for calibrating the external parameters of a vehicle-mounted camera. Background Technology

[0004] When a vehicle is first delivered, or after vehicle repairs involving camera replacement or repositioning, automatic calibration of the onboard camera is required to activate and properly utilize its perception and other algorithms to assist driving. Automatic calibration of onboard cameras simplifies the production line calibration process, reduces production line costs, and improves production line efficiency. In the after-sales service sector, it can effectively reduce return-to-factory and 4S store maintenance costs.

[0005] However, the calibration methods for vehicle-mounted cameras in related technologies are typically offline, meaning that the extrinsic parameters of the vehicle-mounted cameras are calibrated using a checkerboard pattern on the vehicle production line. But on the one hand, due to the uniformity of vehicle production environments, offline calibration of vehicle extrinsic parameters is difficult to meet the demands of complex road environments; on the other hand, after vehicle repairs involving camera replacement or repositioning, recalibration at the factory is still required. Therefore, the calibration methods in related technologies have poor versatility and low reliability. Summary of the Invention

[0006] According to various embodiments of this application, a method, apparatus, computer equipment, and storage medium for calibrating the extrinsic parameters of an on-board camera are provided, which can improve the versatility and reliability of vehicle extrinsic parameter calibration.

[0007] Firstly, this application provides a method for calibrating the extrinsic parameters of an onboard camera. The method includes:

[0008] Acquire multiple frames of images captured by the vehicle-mounted camera;

[0009] Based on the matching relationship between feature points in the multi-frame images, the target pose information of the vehicle-mounted camera corresponding to each frame of the multi-frame images is determined.

[0010] Based on the target pose information, a first transformation relationship from the world coordinate system to the vehicle coordinate system and a second transformation relationship from the camera coordinate system to the world coordinate system are determined;

[0011] Based on the first conversion relationship and the second conversion relationship, the extrinsic parameters of the vehicle-mounted camera are determined.

[0012] In one embodiment, determining the target pose information of the vehicle-mounted camera corresponding to each frame of the multi-frame images based on the matching relationship between feature points in the multi-frame images includes:

[0013] Based on the number of matching feature points, an initial image is determined in the multiple frames of images, wherein the matching feature points include the projection points of the same three-dimensional point on different frames of images;

[0014] Based on the matching feature points of the initial image, the initial pose information is determined, and the three-dimensional point cloud information is determined based on the initial pose information.

[0015] The target pose information is determined based on the initial pose information and the three-dimensional point cloud information.

[0016] In one embodiment, the target pose information includes position information, and determining the first transformation relationship from the world coordinate system to the vehicle coordinate system based on the target pose information includes:

[0017] Based on the location information, determine the first vector coordinates from the vehicle coordinate system to the world coordinate system;

[0018] The first transformation relationship is determined based on the first vector coordinates and the second vector coordinates in the vehicle coordinate system.

[0019] In one embodiment, the first vector coordinates include a first x-axis vector coordinate, a first y-axis vector coordinate, and a first z-axis vector coordinate, and the second vector coordinates include a second x-axis vector coordinate;

[0020] The step of determining the first vector coordinates from the vehicle coordinate system to the world coordinate system based on the location information includes:

[0021] Based on the location information and the second x-axis vector coordinates, the first x-axis vector coordinates are determined;

[0022] The coordinates of the first z-axis vector are determined based on the matched feature points;

[0023] The first y-axis vector coordinate is determined based on the first x-axis vector coordinate and the first z-axis vector coordinate.

[0024] In one embodiment, the second vector coordinates include a second z-axis vector coordinate, and determining the first z-axis vector coordinates based on the matching feature points includes:

[0025] The coordinates of the first z-axis vector are determined based on ground feature points and the coordinates of the second z-axis vector.

[0026] or;

[0027] The coordinates of the first z-axis vector are determined by performing planar fitting on the three-dimensional point cloud information.

[0028] In one embodiment, the second vector coordinate includes a second y-axis vector coordinate, and determining the first transformation relationship based on the first vector coordinate and the second vector coordinate in the vehicle coordinate system includes:

[0029] Singular value decomposition is performed on the first vector coordinates and the second vector coordinates to determine the first transformation relationship. The second vector coordinates include the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates in the vehicle coordinate system.

[0030] In one embodiment, the target pose information includes attitude information, and determining the second transformation relationship from the camera coordinate system to the world coordinate system based on the target pose information includes:

[0031] The second transformation relationship is determined based on the average matrix of the attitude information.

[0032] In one embodiment, determining the extrinsic parameters of the vehicle-mounted camera based on the first transformation relationship and the second transformation relationship includes:

[0033] The extrinsic parameters of the vehicle-mounted camera are determined based on the product of the first transformation relationship and the second transformation relationship.

[0034] In one embodiment, determining the target pose information based on the initial pose information and the 3D point cloud information includes:

[0035] Based on the number of matching feature points, at least one reconstructed image is determined from the multiple frames of images;

[0036] For each of the at least one reconstructed images, the pose information of the current frame is determined based on the matching feature points of the reconstructed image, and the three-dimensional point cloud information is updated based on the pose information of the current frame to determine the current three-dimensional point cloud information.

[0037] Based on a preset optimization function, the current frame pose information and the current 3D point cloud information are jointly updated to determine the target pose information.

[0038] In one embodiment, the step of jointly updating the current frame pose information and the current 3D point cloud information based on the preset optimization function includes:

[0039] Based on the preset optimization function and the matching feature points of the co-view frame image of the reconstructed image, the pose information of the current frame and the current three-dimensional point cloud information are jointly updated to determine the first pose information and the first three-dimensional point cloud information. The co-view frame image is determined based on the number of matching feature points of the reconstructed image.

[0040] The current frame pose information and the current 3D point cloud information are repeatedly updated together until the preset update conditions are met, and then the second pose information and the second 3D point cloud information are determined.

[0041] Based on the preset optimization function and the matching feature points of all at least one reconstructed image, the second pose information and the second three-dimensional point cloud information are jointly updated to determine the target pose information.

[0042] In one embodiment, the preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the three-dimensional point cloud information.

[0043] Secondly, this application also provides a vehicle-mounted camera extrinsic parameter calibration device. The device includes:

[0044] The image acquisition module is used to acquire multiple frames of images captured by the vehicle-mounted camera;

[0045] The target pose information determination module is used to determine the target pose information of the vehicle camera corresponding to each frame of the multi-frame images based on the matching relationship between feature points in the multi-frame images;

[0046] The transformation relationship determination module is used to determine a first transformation relationship from the world coordinate system to the vehicle coordinate system and a second transformation relationship from the camera coordinate system to the world coordinate system based on the target pose information.

[0047] The vehicle-mounted camera extrinsic parameter determination module is used to determine the extrinsic parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

[0048] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the vehicle-mounted camera extrinsic parameter calibration methods described in the first aspect above.

[0049] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the vehicle-mounted camera extrinsic parameter calibration method described in any embodiment of the first aspect above.

[0050] The aforementioned vehicle-mounted camera extrinsic parameter calibration method, apparatus, computer equipment, and storage medium acquire multiple frames of images captured by the vehicle-mounted camera, determine the matching relationship between feature points in the multiple frames, and determine the target pose information of the vehicle-mounted camera corresponding to each frame. Then, the extrinsic parameters of the vehicle-mounted camera are calibrated using the target pose information and coordinate system transformation relationships. This allows for extrinsic parameter calibration of the vehicle-mounted camera in typical usage scenarios after initial vehicle delivery, camera replacement, or camera position change, eliminating the need for factory recalibration. This broadens the applicability of the calibration process and makes it simpler and more effective. Furthermore, calibrating by acquiring images in real-world scenarios ensures that the calibrated vehicle extrinsic parameters are more adaptable to actual conditions. This application effectively improves the versatility and reliability of vehicle extrinsic parameter calibration.

[0051] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. The drawings described below are merely embodiments of this application; those skilled in the art can obtain other drawings based on the disclosed drawings without creative effort. In the drawings:

[0053] Figure 1 shows the application environment of the vehicle-mounted camera extrinsic parameter calibration method in one embodiment.

[0054] Figure 2 is a flowchart illustrating a method for calibrating the extrinsic parameters of a vehicle-mounted camera in one embodiment.

[0055] Figure 3 shows an image acquired by an infrared camera in one embodiment.

[0056] Figure 4 shows an image acquired by a color camera in one embodiment.

[0057] Figure 5 shows multiple frames of images acquired by a near-infrared camera in one embodiment.

[0058] Figure 6 is a schematic diagram of the common-view frame image of the reconstructed image in one embodiment.

[0059] Figure 7 is a schematic diagram of the vehicle coordinate system in one embodiment.

[0060] Figure 8 is a schematic diagram of the current frame pose information and the current 3D point cloud information in one embodiment;

[0061] Figure 9 is a schematic diagram of the current frame pose information in one embodiment.

[0062] Figure 10 is a schematic diagram of ground point cloud information in one embodiment.

[0063] Figure 11 is a flowchart illustrating a method for calibrating the extrinsic parameters of a vehicle-mounted camera in a specific embodiment.

[0064] Figure 12 is a structural block diagram of an on-board camera extrinsic parameter calibration device in one embodiment.

[0065] Figure 13 is an internal structure diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0067] The terms “module”, “unit”, etc., used below refer to a combination of software and / or hardware that can perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in hardware, implementation in software, or a combination of software and hardware, is also possible and contemplated.

[0068] The vehicle-mounted camera extrinsic parameter calibration method provided in this application embodiment can be applied to the application environment shown in Figure 1. The terminal 102 communicates with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104 or placed in the cloud or on another network server. The terminal 102 acquires multiple frames of images captured by the vehicle-mounted camera and sends them to the server 104. Based on the matching relationship between feature points in the multiple frames, the server 104 determines the target pose information of the vehicle-mounted camera corresponding to each frame; based on the target pose information, it determines a first transformation relationship from the world coordinate system to the vehicle coordinate system and a second transformation relationship from the camera coordinate system to the world coordinate system; based on the first and second transformation relationships, it determines the extrinsic parameters of the vehicle-mounted camera and sends them to the terminal 102. In other embodiments, the steps of the vehicle-mounted camera extrinsic parameter calibration method in the above embodiments can also be performed solely by the terminal 102. The terminal 102 may include vehicle-mounted devices, such as vehicle-mounted cameras. The server 104 may include a vehicle cloud server, and the server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0069] In one embodiment, as shown in Figure 2, a method for calibrating the extrinsic parameters of a vehicle-mounted camera is provided. Taking the application scenario in Figure 1 as an example, the method includes the following steps:

[0070] S201: Acquire multiple frames of images captured by the vehicle-mounted camera.

[0071] In this embodiment, the vehicle-mounted camera may include a camera device installed on the vehicle. Specifically, the vehicle-mounted camera may include a fisheye camera, a near-infrared camera, etc. The installation method of the vehicle-mounted camera on the vehicle may include panoramic view, surround view, forward view, etc. The multi-frame images captured by the vehicle-mounted camera may include consecutive frame images of a preset number of frames captured by the vehicle-mounted camera, or multiple frame images at preset intervals within consecutive frame images. It is understood that the type of the multi-frame images matches the type of the vehicle-mounted camera. For example, if the vehicle-mounted camera is an infrared camera, then the multi-frame images are infrared images, as shown in Figure 3; if the vehicle-mounted camera is a color camera, then the multi-frame images are RGB images, as shown in Figure 4.

[0072] In some specific embodiments, if the vehicle-mounted camera includes a near-infrared camera, the acquired multi-frame images are shown in Figure 5. Figure 5 shows the multi-frame images acquired by the near-infrared camera. The number of multi-frame images is 18, and the multi-frame images are image_20_01.png, image_20_02.png, image_20_03.png, image_20_04.png, image_20_05.png, image_20_06.png, image_20_07.png, image_20_08.png, image_20_09.png, image_20_10.png, image_20_11.png, image_20_12.png, image_20_13.png, image_20_14.png, image_20_15.png, image_20_16.png, image_20_17.png, and image_20_18.png.

[0073] In this embodiment, the multi-frame images captured by the vehicle-mounted camera can include multiple frames acquired during the vehicle's journey of a preset distance on a road. Specifically, to further improve the quality of the captured multi-frame images, the vehicle can be controlled to travel a preset distance at a preset speed on a smooth road surface and capture consecutive multi-frame images. The sides of the smooth road surface can include environmental objects such as buildings and trees, and there should be no moving objects interfering with the shooting scene. The smooth road surface can include feature markings such as double dashed lines, zebra crossings, arrows, or numbers. The preset speed can include a preset speed range or a preset speed value. The preset speed range can be 20km / h-50km / h, and the preset speed value can be any value within the preset speed range.

[0074] S203: Based on the matching relationship between feature points in the multi-frame images, determine the target pose information of the vehicle camera corresponding to each frame image.

[0075] In this embodiment, feature points in multi-frame images may include points where the grayscale value of the images changes drastically or points with large curvature at the image edges, such as corner points. Feature points in multi-frame images can be extracted based on AI algorithms, which can effectively improve the effectiveness and accuracy of feature points in multi-frame images acquired by different types of vehicle-mounted cameras.

[0076] In this embodiment, based on the matching relationship between feature points in different frames, the target pose information of the vehicle camera corresponding to each frame can be determined by an optimization algorithm according to the relationship between the spatial coordinates of the feature points and the image coordinates. In other embodiments, the motion of feature points in different frames can be tracked based on the matching relationship between feature points, and the target pose information of the vehicle camera corresponding to each frame can be determined by a filter (such as a particle filter) or an optimization algorithm. In other embodiments, the target pose information of the vehicle camera corresponding to each frame can be determined based on SFM (Structure From Motion). The matching relationship between feature points in different frames can be determined by AI algorithms or KNN (K-Nearest Neighbors) algorithms. Specifically, 3D matching feature points can be further determined after 2D matching feature points are determined.

[0077] S205: Based on the target pose information, determine the first transformation relationship from the world coordinate system to the vehicle coordinate system and the second transformation relationship from the camera coordinate system to the world coordinate system.

[0078] S207: Determine the extrinsic parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

[0079] In this embodiment, the transformation relationship from the vehicle coordinate system to the world coordinate system can be determined based on the target pose information using a straight-line fitting algorithm. Then, based on the orthogonality of the coordinate axes, a first transformation relationship from the world coordinate system to the vehicle coordinate system can be determined. In other embodiments, environmental features can be acquired through a camera, and these features can be matched with a known high-precision map to determine the transformation relationship from the vehicle camera coordinate system to the world coordinate system. Then, based on the target pose information of the vehicle camera, a first transformation relationship from the world coordinate system to the vehicle coordinate system can be determined. In other embodiments, the target pose information of the vehicle camera can be directly regressed or classified using a deep learning model (such as a neural network model). The model is trained using labeled data in a training set, and after inputting the target pose information, the deep learning module can output the first transformation relationship from the world coordinate system to the vehicle coordinate system.

[0080] In this embodiment, the second transformation relationship from the camera coordinate system to the world coordinate system can be determined by averaging the matrix of target pose information. In other embodiments, the target pose information of the vehicle camera can be directly regressed or classified based on a deep learning model (such as a neural network model). The model is trained using labeled data in the training set, and then, after inputting the target pose information, the deep learning module can output the second transformation relationship from the camera coordinate system to the world coordinate system.

[0081] In this embodiment, after determining the first transformation relationship from the world coordinate system to the vehicle coordinate system and the second transformation relationship from the camera coordinate system to the world coordinate system, multiplying the first and second transformation relationships determines the extrinsic parameters of the vehicle-mounted camera. Alternatively, a deep learning model can be trained, and the first and second transformation relationships can be input into the model to output the extrinsic parameters of the vehicle-mounted camera.

[0082] The vehicle-mounted camera extrinsic parameter calibration method provided in this application acquires multiple frames of images captured by the vehicle-mounted camera, determines the matching relationship between feature points in the multiple frames, and determines the target pose information of the vehicle-mounted camera corresponding to each frame. Then, it calibrates the extrinsic parameters of the vehicle-mounted camera using the target pose information and coordinate system transformation relationships. This method can achieve extrinsic parameter calibration in normal usage scenarios after the vehicle's initial delivery, camera replacement, or camera position change, without requiring recalibration at the factory. It has a wider range of applications and a simpler and more effective calibration method. Furthermore, by acquiring images in actual scenarios for calibration, the calibrated vehicle extrinsic parameters can be more adapted to real-world conditions. This application effectively improves the versatility and reliability of vehicle extrinsic parameter calibration.

[0083] The following describes a method for determining target pose information through embodiments of this application. In some embodiments, determining the target pose information of the vehicle-mounted camera corresponding to each frame image based on the matching relationship between feature points in the multi-frame images includes:

[0084] S301: Based on the number of matching feature points, determine an initial image in the multiple frames of images, wherein the matching feature points include the projection points of the same three-dimensional point on different frames of images.

[0085] S303: Based on the matching feature points of the initial image, determine the initial pose information, and based on the initial pose information, determine the three-dimensional point cloud information.

[0086] S305: Determine the target pose information based on the initial pose information and the three-dimensional point cloud information.

[0087] In this embodiment, the matching feature points include the projection points of the same three-dimensional point in space onto different frame images. Based on the number of matching feature points, an initial image can be determined from multiple frames. The initial image may include two frames from the multiple frames whose number of matching feature points meets a preset requirement. In some specific embodiments, the two frames with the most matching feature points from the multiple frames can be determined as the initial images. In other embodiments, a quantity threshold can be set, and the two frames with a number of matching feature points greater than the quantity threshold and the highest image quality from the multiple frames can be determined as the initial images.

[0088] Based on the matching feature points of the initial image, the initial pose information can be determined by constructing and solving the essential matrix. In some embodiments, let the matching point pair of the initial image be [p1, p2], the camera intrinsic parameter be K, the fundamental matrix be F, and the essential matrix be E. The initial pose information [Ri, Ti] can be determined by solving the essential matrix E. Then, based on the initial pose information, the three-dimensional point cloud information can be determined through triangulation calculation, wherein the three-dimensional point cloud includes a sparse three-dimensional point cloud. For the specific methods of solving the essential matrix and triangulation calculation, please refer to relevant technologies, which will not be elaborated here. Based on the initial pose information and the three-dimensional point cloud information, the target pose information of the vehicle-mounted camera corresponding to each frame of the image can be determined.

[0089] In some embodiments, determining the target pose information based on the initial pose information and the three-dimensional point cloud information includes:

[0090] S401: Based on the number of matching feature points, determine at least one reconstructed image in the multi-frame images.

[0091] S403: For each of the at least one reconstructed images, determine the current frame pose information based on the matching feature points of the reconstructed image, and update the three-dimensional point cloud information based on the current frame pose information to determine the current three-dimensional point cloud information.

[0092] S405: Based on a preset optimization function, jointly update the current frame pose information and the current 3D point cloud information to determine the target pose information.

[0093] In this embodiment, the next frame image is selected from the multiple frames of images as the reconstructed image based on the number of matching feature points. In some embodiments, the image that satisfies the co-viewing region condition with the initial image is determined as the reconstructed image from the multiple frames of images. Specifically, the image with the largest number of matching feature points among any one of the initial images can be selected as the reconstructed image. Alternatively, the image whose feature point distribution area covers the largest proportion of the entire image among any one of the initial images can be selected as the reconstructed image. Furthermore, the above judgment conditions can be combined, such as sorting by the number of matching feature points from largest to smallest, and then selecting the image with the largest coverage proportion in descending order of the number of matching feature points as the reconstructed image.

[0094] Based on the matching feature points of the reconstructed image, the pose information of the current frame is determined, and the 3D point cloud information corresponding to the reconstructed image is determined based on the pose information of the current frame. The 3D point cloud information is then updated based on the 3D point cloud information corresponding to the reconstructed image to determine the current 3D point cloud information. The method for determining the pose information of the current frame and the method for determining the 3D point cloud information corresponding to the reconstructed image based on the pose information of the current frame can refer to the method for determining the initial pose information and the method for determining the 3D point cloud information based on the initial pose information in the above embodiment, which will not be repeated here. Based on a preset optimization function, the pose information of the current frame and the current 3D point cloud information are jointly updated to determine the target pose information.

[0095] In some specific embodiments, the current frame pose information and the current 3D point cloud information of the reconstructed image are shown in Figure 8. In Figure 8, cam_pose represents the current frame pose information, and 3D points represent the current 3D point cloud information. The current frame pose information is magnified as shown in Figure 9, where each view frustum indicates the position (yellow circle) and orientation (green arrow) of the vehicle-mounted camera.

[0096] In some embodiments, jointly updating the current frame pose information and the current 3D point cloud information based on a preset optimization function includes:

[0097] S501: Based on a preset optimization function and the matching feature points of the co-view frame image of the reconstructed image, jointly update the current frame pose information and the current three-dimensional point cloud information to determine the first pose information and the first three-dimensional point cloud information. The co-view frame image is determined based on the number of matching feature points of the reconstructed image.

[0098] S503: Repeatedly update the current frame pose information and the current three-dimensional point cloud information together until the preset update conditions are met, and then determine the second pose information and the second three-dimensional point cloud information.

[0099] S505: Based on the preset optimization function and the matching feature points of all at least one reconstructed image, jointly update the second pose information and the second three-dimensional point cloud information to determine the target pose information.

[0100] In this embodiment, the shared-view frame image includes images whose number of matching feature points with the reconstructed image is greater than a preset threshold, that is, images that share a common viewing area with the reconstructed image. Taking Figure 6 as an example, if the newly added frame image shown in Figure 6 is a reconstructed image, then the mapping between images 1, 2, and 3 adjacent to the reconstructed image and the reconstructed image contains common three-dimensional points, that is, the number of matching feature points between each of images 1, 2, and 3 and the reconstructed image is greater than a preset threshold, then images 1, 2, and 3 are determined to be shared-view frame images of the reconstructed image.

[0101] In some embodiments, the preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the 3D point cloud information. Specifically, the preset optimization function may include the cost function E shown in equation (1):

[0102] In equation (1), Pi represents the projection matrix (camera intrinsic and extrinsic parameters) of the i-th vehicle-mounted camera, Xk represents the k-th 3D point in the current 3D point cloud information, π represents projection mapping, that is, projecting the 3D point onto the i-th vehicle-mounted camera plane, xik represents the 2D feature point corresponding to the 3D point on the i-th vehicle-mounted camera plane, that is, the matching feature point of the co-view frame image of the reconstructed image, and ρik represents the weight.

[0103] Based on a preset optimization function and matching feature points of the co-view frame images of the reconstructed image, local Bundle Adjustment (BA) optimization is used to jointly update the current frame pose information and the current 3D point cloud information to determine the first pose information and the first 3D point cloud information. Specifically, joint nonlinear optimization can be performed on the current frame pose information and the current 3D point cloud information, with the optimization objective being to minimize the 3D point cloud reprojection error. The purpose of local BA optimization is to make the determined first pose information and first 3D point cloud information more accurate, and to improve the constraint capability with the addition of co-view frame images. Specific methods of local BA optimization can be found in related technologies, and will not be elaborated upon here.

[0104] The current frame pose information and the current 3D point cloud information are repeatedly updated together until a preset update condition is met, at which point the second pose information and the second 3D point cloud information are determined. In some embodiments, for each reconstructed image, a corresponding common-view frame image is determined and updated as described in the above embodiments, after which the preset update condition is determined to be met. In other embodiments, a preset number of image frames can also be set; for example, if the number of common-view frames for all reconstructed images is greater than the preset number of image frames, the preset update condition is determined to be met. Before the preset update condition is met, the current 3D point cloud information is continuously updated according to the method described in the above embodiments to determine the second pose information and the second 3D point cloud information.

[0105] Based on a preset optimization function and matching feature points of all reconstructed images, the second pose information and the second 3D point cloud information are jointly updated to determine the target pose information. In some embodiments, after satisfying the preset update conditions, a global BA optimization is performed again, i.e., all reconstructed images participate in the update process. The specific preset optimization function and update process can refer to the above equation (1) and the update process described in the above embodiments. The difference is that due to the different images involved in the optimization, the number of vehicle cameras involved in the summation and the number of 3D points in the preset optimization function are different. After jointly updating the second pose information and the second 3D point cloud information, the target pose information is determined.

[0106] The following describes the specific method for determining the extrinsic parameters of an onboard camera through embodiments of this application. In some embodiments, the target pose information includes position information, and determining the first transformation relationship from the world coordinate system to the vehicle coordinate system based on the target pose information includes:

[0107] S601: Determine the first vector coordinates from the vehicle coordinate system to the world coordinate system based on the location information.

[0108] S603: Determine the first transformation relationship based on the first vector coordinates and the second vector coordinates in the vehicle coordinate system.

[0109] In this embodiment, the world coordinate system is denoted as W, the vehicle coordinate system as V, and the camera coordinate system as C. The world coordinate system W includes the coordinate system containing the target pose information and the 3D point cloud information. The x-axis, y-axis, and z-axis of the vehicle coordinate system V are shown in Figure 7. Target pose information Pose = {R...} i ,T i} i∈1～N This includes position information Ti, where N represents the number of frames. Based on the position information, the first vector coordinates from the vehicle coordinate system to the world coordinate system can be determined.

[0110] In some embodiments, the first vector coordinates include the vector coordinates of the axis vector in the vehicle coordinate system in the world coordinate system. If the axis vector represents the vehicle body, then the first vector coordinates represent the spatial position and orientation of the vehicle body in the world coordinate system. The first vector coordinates can be used to subsequently determine the relative position of the vehicle body and the external environment, providing a basis for the subsequent calibration of the extrinsic parameters of the vehicle-mounted camera. In other embodiments, the first vector coordinates may also represent the coordinates of other objects or vectors in the vehicle coordinate system in the world coordinate system. This application does not impose specific limitations on this, as long as the first vector coordinates can represent the vector coordinates of objects or target vectors in the vehicle coordinate system in the world coordinate system.

[0111] The first vector coordinates from the vehicle coordinate system to the world coordinate system include the first x-axis vector coordinates, the first y-axis vector coordinates, and the first z-axis vector coordinates. In some specific embodiments, the first x-axis vector coordinates, the first y-axis vector coordinates, and the first z-axis vector coordinates represent the unit x-axis vector coordinates, the unit y-axis vector coordinates, and the unit z-axis vector coordinates of the vehicle coordinate system in the world coordinate system, respectively.

[0112] In some embodiments, the second vector coordinates include the spatial position and direction vector coordinates of the axis vector in the vehicle coordinate system. If the axis vector represents the vehicle body, then the second vector coordinates represent the vehicle body coordinates in the vehicle coordinate system. By using the second vector coordinates of the axis vector in the vehicle coordinate system and the first vector coordinates of the axis vector in the world coordinate system, the transformation relationship from the vehicle coordinate system to the world coordinate system can be determined, thereby realizing the transformation from the vehicle coordinate system to the world coordinate system. In other embodiments, the second vector coordinates may also represent the vector coordinates of other objects or vectors in the vehicle coordinate system; this application does not impose specific limitations on this.

[0113] The second vector coordinates in the vehicle coordinate system include the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates. In some specific embodiments, the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates respectively represent the unit vector coordinates of the x-axis, y-axis, and z-axis in the vehicle coordinate system.

[0114] First, let's explain how to determine the coordinates of the first x-axis vector.

[0115] In some embodiments, the first vector coordinates include a first x-axis vector coordinate, a first y-axis vector coordinate, and a first z-axis vector coordinate, and the second vector coordinates include a second x-axis vector coordinate; determining the first vector coordinates from the vehicle coordinate system to the world coordinate system based on the position information includes:

[0116] S701: Determine the first x-axis vector coordinates based on the location information and the second x-axis vector coordinates.

[0117] S703: Determine the coordinates of the first z-axis vector based on the matched feature points.

[0118] S705: Determine the first y-axis vector coordinate based on the first x-axis vector coordinate and the first z-axis vector coordinate.

[0119] In this embodiment of the application, the location information {T} can be processed based on the ransac (random sample consensus) algorithm. i} i∈1～N Line fitting is performed to eliminate outliers, which include positions not on the line. Any two positions (Ti, Tj) falling on the line are selected. Then, the coordinates of the second x-axis vector x_axis_V[1,0,0] in the vehicle coordinate system and its coordinates in the world coordinate system can be determined: x_axis_W = (T...). ix -T jx ,T iy -T jy ,T iz -T jz ).

[0120] The following describes the methods for determining the first z-axis vector coordinates and the first y-axis vector coordinates through embodiments of this application. In some embodiments, the second vector coordinates include the second z-axis vector coordinates, and determining the first z-axis vector coordinates based on the matching feature points includes:

[0121] S801: Determine the first z-axis vector coordinates based on ground feature points and the second z-axis vector coordinates; or: Perform plane fitting on the three-dimensional point cloud information to determine the first z-axis vector coordinates.

[0122] In this embodiment, two-dimensional feature points on the ground are obtained, and the first z-axis vector coordinate z_axis_W in the world coordinate system can be determined by fitting the plane with the reprojection error. This first z-axis vector coordinate z_axis_W is the same as the second z-axis vector coordinate z_axis_V[0,0,1] in the vehicle coordinate system. In other embodiments, ground point cloud information is directly extracted from the three-dimensional point cloud information, and the first z-axis vector coordinate z_axis_W in the world coordinate system is determined by plane fitting of the ground point cloud information based on the RANSAC algorithm. If the plane equation of the ground point cloud information is Ax+By+CZ+D=0, then the first z-axis vector coordinate is z_axis_w=(A,B,C). Specifically, the ground point cloud information is shown in Figure 10.

[0123] Based on the first x-axis vector coordinates and the first z-axis vector coordinates, and according to the orthogonality of the three axes, the first y-axis vector coordinates of the second y-axis vector y_axis_V[0,1,0] in the vehicle coordinate system can be determined as y_axis_W = cross(z_axis_W, x_axis_V) in the world coordinate system, where cross represents the cross product.

[0124] After determining the first vector coordinates, the first transformation relationship from the world coordinate system to the vehicle coordinate system can be determined based on the first vector coordinates and the second vector coordinates in the vehicle coordinate system.

[0125] In some embodiments, the second vector coordinates include a second y-axis vector coordinate, and determining the first transformation relationship based on the first vector coordinates and the second vector coordinates in the vehicle coordinate system includes:

[0126] S901: Perform singular value decomposition on the first vector coordinates and the second vector coordinates to determine the first transformation relationship. The second vector coordinates include the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates in the vehicle coordinate system.

[0127] In this embodiment, a first transformation relationship from the world coordinate system to the vehicle coordinate system is determined as Rv2w = SVD([x_axis_W,y_axis_W,z_axis_W], [x_axis_V,y_axis_V,z_axis_V]). Here, x_axis_W, y_axis_W, and z_axis_W represent the first x-axis vector coordinate, the first y-axis vector coordinate, and the first z-axis vector coordinate in the world coordinate system, respectively. x_axis_V, y_axis_V, and z_axis_V represent the second x-axis vector coordinate, the second y-axis vector coordinate, and the second z-axis vector coordinate in the vehicle coordinate system, respectively. SVD represents singular value decomposition.

[0128] The following describes the method for determining the second transformation relationship through embodiments of this application. In some embodiments, the target pose information includes posture information, and determining the second transformation relationship from the camera coordinate system to the world coordinate system based on the target pose information includes: determining the second transformation relationship based on the average matrix of the posture information. In this embodiment, the target pose information Pose = {R i ,T i} i∈1～N It also includes attitude information Ri. The second transformation relationship from the camera coordinate system to the world coordinate system is expressed as Rw2c = average(Ri), where average represents the solution of the average matrix.

[0129] In some embodiments, determining the extrinsic parameters of the vehicle-mounted camera based on the first transformation relationship and the second transformation relationship includes: determining the extrinsic parameters of the vehicle-mounted camera based on the product of the first transformation relationship and the second transformation relationship. Specifically, the extrinsic parameters of the vehicle-mounted camera may include the extrinsic parameters from the camera coordinate system to the vehicle coordinate system. The extrinsic parameters of the vehicle-mounted camera are represented as Rv2c = Rv2w * Rw2c, where Rv2w represents the first transformation relationship from the world coordinate system to the vehicle coordinate system, and Rw2c represents the second transformation relationship from the camera coordinate system to the world coordinate system.

[0130] In some specific embodiments, the extrinsic parameter calibration method for the vehicle-mounted camera is shown in Figure 11. Multiple frames of images are acquired while the vehicle is traveling straight. Feature points and descriptors for each frame are extracted using deep learning methods, and inter-frame matching is performed. Based on the inter-frame matching relationship, pose information and 3D point cloud information (3D point information) for each frame are solved using BA optimization. Then, the rotation vector R in the extrinsic parameters of the vehicle-mounted camera is determined based on ground feature points and pose information.

[0131] In some specific embodiments, the test data for the calibration of the vehicle-mounted camera's extrinsic parameters are shown in Table 1.

[0132] Table 1

[0133] In Table 1, Roll represents the barrel roll angle, which is the rotation angle around the x-axis. Pitch represents the pitch angle, which is the rotation angle around the y-axis. Yaw represents the yaw angle, which is the rotation angle around the z-axis.

[0134] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0135] Based on the same inventive concept, this application also provides a vehicle camera extrinsic parameter calibration device 1300 for implementing the above-described vehicle camera extrinsic parameter calibration method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the vehicle camera extrinsic parameter calibration device 1300 provided below can be found in the limitations of the vehicle camera extrinsic parameter calibration method described above, and will not be repeated here.

[0136] In one embodiment, as shown in FIG12, a vehicle-mounted camera extrinsic parameter calibration device 1300 is provided, comprising:

[0137] Image acquisition module 1301 is used to acquire multiple frames of images captured by the vehicle-mounted camera;

[0138] The target pose information determination module 1302 is used to determine the target pose information of the vehicle camera corresponding to each frame of the multi-frame images based on the matching relationship between feature points in the multi-frame images.

[0139] The transformation relationship determination module 1303 is used to determine a first transformation relationship from the world coordinate system to the vehicle coordinate system and a second transformation relationship from the camera coordinate system to the world coordinate system based on the target pose information.

[0140] The vehicle-mounted camera extrinsic parameter determination module 1304 is used to determine the extrinsic parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

[0141] In some embodiments, the target pose information determination module 1302 is further configured to determine an initial image in the multiple frames of images based on the number of matching feature points, wherein the matching feature points include projection points of the same three-dimensional point on different frames of images; determine initial pose information based on the matching feature points of the initial image, and determine three-dimensional point cloud information based on the initial pose information; and determine the target pose information based on the initial pose information and the three-dimensional point cloud information.

[0142] In some embodiments, the target pose information includes position information, and the transformation relationship determination module 1303 is further configured to determine a first vector coordinate from the vehicle coordinate system to the world coordinate system based on the position information; and determine the first transformation relationship based on the first vector coordinate and a second vector coordinate in the vehicle coordinate system.

[0143] In some embodiments, the first vector coordinates include a first x-axis vector coordinate, a first y-axis vector coordinate, and a first z-axis vector coordinate, and the second vector coordinates include a second x-axis vector coordinate; the transformation relationship determination module 1303 is further configured to determine the first x-axis vector coordinate based on the position information and the second x-axis vector coordinate; determine the first z-axis vector coordinate based on the matching feature points; and determine the first y-axis vector coordinate based on the first x-axis vector coordinate and the first z-axis vector coordinate.

[0144] In some embodiments, the second vector coordinates include the second z-axis vector coordinates, and the transformation relationship determination module 1303 is further configured to determine the first z-axis vector coordinates based on ground feature points and the second z-axis vector coordinates; or; perform planar fitting on the three-dimensional point cloud information to determine the first z-axis vector coordinates.

[0145] In some embodiments, the second vector coordinates include the second y-axis vector coordinates, and the transformation relationship determination module 1303 is further configured to perform singular value decomposition on the first vector coordinates and the second vector coordinates to determine the first transformation relationship, wherein the second vector coordinates include the second x-axis vector coordinates, the second y-axis vector coordinates and the second z-axis vector coordinates in the vehicle coordinate system.

[0146] In some embodiments, the target pose information includes pose information, and the transformation relationship determination module 1303 is further configured to determine the second transformation relationship based on the average matrix of the pose information.

[0147] In some embodiments, the vehicle-mounted camera extrinsic parameter determination module 1304 is further configured to determine the extrinsic parameters of the vehicle-mounted camera based on the product of the first transformation relationship and the second transformation relationship.

[0148] In some embodiments, the target pose information determination module 1302 is further configured to determine at least one reconstructed image in the multiple frames of images based on the number of matching feature points; for each reconstructed image in the at least one reconstructed image, determine the current frame pose information based on the matching feature points of the reconstructed image, and update the three-dimensional point cloud information based on the current frame pose information to determine the current three-dimensional point cloud information; and jointly update the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function to determine the target pose information.

[0149] In some embodiments, the target pose information determination module 1302 is further configured to jointly update the current frame pose information and the current three-dimensional point cloud information based on a preset optimization function and matching feature points of the co-view frame image of the reconstructed image, to determine the first pose information and the first three-dimensional point cloud information, wherein the co-view frame image is determined based on the number of matching feature points of the reconstructed image; repeatedly jointly update the current frame pose information and the current three-dimensional point cloud information until a preset update condition is met, and then determine the second pose information and the second three-dimensional point cloud information; and jointly update the second pose information and the second three-dimensional point cloud information based on the preset optimization function and matching feature points of all at least one reconstructed image to determine the target pose information.

[0150] In some embodiments, the preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the three-dimensional point cloud information.

[0151] Each module in the aforementioned vehicle-mounted camera extrinsic calibration device 1300 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call them to execute the corresponding operations.

[0152] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram is shown in Figure 13. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface includes an I / O interface and a communication interface for communicating with external terminals via a network. When the computer program is executed by the processor, it can implement the aforementioned vehicle-mounted camera extrinsic parameter calibration method.

[0153] Those skilled in the art will understand that the structure shown in Figure 13 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle-mounted camera extrinsic parameter calibration method described in any of the above embodiments.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the vehicle-mounted camera extrinsic parameter calibration method described in any of the above embodiments.

[0156] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0158] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for calibrating extrinsic parameters of a vehicle-mounted camera, characterized in that, The method includes: Acquire multiple frames of images captured by the vehicle-mounted camera; Based on the matching relationship between feature points in the multi-frame images, the target pose information of the vehicle-mounted camera corresponding to each frame of the multi-frame images is determined. Based on the target pose information, a first transformation relationship from the world coordinate system to the vehicle coordinate system and a second transformation relationship from the camera coordinate system to the world coordinate system are determined; Based on the first conversion relationship and the second conversion relationship, the extrinsic parameters of the vehicle-mounted camera are determined.

2. The method of claim 1, wherein, The step of determining the target pose information of the vehicle-mounted camera corresponding to each frame of the multi-frame images based on the matching relationship between feature points in the multi-frame images includes: Based on the number of matching feature points, an initial image is determined in the multiple frames of images, wherein the matching feature points include the projection points of the same three-dimensional point on different frames of images; Based on the matching feature points of the initial image, the initial pose information is determined, and the three-dimensional point cloud information is determined based on the initial pose information. The target pose information is determined based on the initial pose information and the three-dimensional point cloud information.

3. The method of claim 2, wherein, The target pose information includes position information, and determining the first transformation relationship from the world coordinate system to the vehicle coordinate system based on the target pose information includes: Based on the location information, determine the first vector coordinates from the vehicle coordinate system to the world coordinate system; The first transformation relationship is determined based on the first vector coordinates and the second vector coordinates in the vehicle coordinate system.

4. The method of claim 3, wherein, The first vector coordinates include a first x-axis vector coordinate, a first y-axis vector coordinate, and a first z-axis vector coordinate; the second vector coordinates include a second x-axis vector coordinate. The step of determining the first vector coordinates from the vehicle coordinate system to the world coordinate system based on the location information includes: Based on the location information and the second x-axis vector coordinates, the first x-axis vector coordinates are determined; The coordinates of the first z-axis vector are determined based on the matched feature points; The first y-axis vector coordinate is determined based on the first x-axis vector coordinate and the first z-axis vector coordinate.

5. The method of claim 4, wherein, The second vector coordinates include the second z-axis vector coordinates, and determining the first z-axis vector coordinates based on the matching feature points includes: The coordinates of the first z-axis vector are determined based on ground feature points and the coordinates of the second z-axis vector. or; The coordinates of the first z-axis vector are determined by performing planar fitting on the three-dimensional point cloud information.

6. The method of claim 3, wherein, The second vector coordinates include the second y-axis vector coordinates. Determining the first transformation relationship based on the first vector coordinates and the second vector coordinates in the vehicle coordinate system includes: Singular value decomposition is performed on the first vector coordinates and the second vector coordinates to determine the first transformation relationship. The second vector coordinates include the second x-axis vector coordinates, the second y-axis vector coordinates, and the second z-axis vector coordinates in the vehicle coordinate system.

7. The method of claim 1, wherein, The target pose information includes attitude information, and determining the second transformation relationship from the camera coordinate system to the world coordinate system based on the target pose information includes: The second transformation relationship is determined based on the average matrix of the attitude information.

8. The method of claim 1, wherein, The determination of the extrinsic parameters of the vehicle-mounted camera based on the first transformation relationship and the second transformation relationship includes: The extrinsic parameters of the vehicle-mounted camera are determined based on the product of the first transformation relationship and the second transformation relationship.

9. The method of claim 2, wherein, Determining the target pose information based on the initial pose information and the 3D point cloud information includes: Based on the number of matching feature points, at least one reconstructed image is determined from the multiple frames of images; For each of the at least one reconstructed images, the pose information of the current frame is determined based on the matching feature points of the reconstructed image, and the three-dimensional point cloud information is updated based on the pose information of the current frame to determine the current three-dimensional point cloud information. Based on a preset optimization function, the current frame pose information and the current 3D point cloud information are jointly updated to determine the target pose information.

10. The method of claim 9, wherein, The step of jointly updating the current frame pose information and the current 3D point cloud information based on the preset optimization function includes: Based on the preset optimization function and the matching feature points of the co-view frame image of the reconstructed image, the pose information of the current frame and the current three-dimensional point cloud information are jointly updated to determine the first pose information and the first three-dimensional point cloud information. The co-view frame image is determined based on the number of matching feature points of the reconstructed image. The current frame pose information and the current 3D point cloud information are repeatedly updated together until the preset update conditions are met, and then the second pose information and the second 3D point cloud information are determined. Based on the preset optimization function and the matching feature points of all at least one reconstructed image, the second pose information and the second three-dimensional point cloud information are jointly updated to determine the target pose information.

11. The method of claim 9 or 10, wherein, The preset optimization function is determined based on the projection matrix of the vehicle-mounted camera and the three-dimensional point cloud information.

12. A vehicle-mounted camera extrinsic parameter calibration device, characterized in that, The device includes: The image acquisition module is used to acquire multiple frames of images captured by the vehicle-mounted camera; The target pose information determination module is used to determine the target pose information of the vehicle camera corresponding to each frame of the multi-frame images based on the matching relationship between feature points in the multi-frame images; The transformation relationship determination module is used to determine a first transformation relationship from the world coordinate system to the vehicle coordinate system and a second transformation relationship from the camera coordinate system to the world coordinate system based on the target pose information. The vehicle-mounted camera extrinsic parameter determination module is used to determine the extrinsic parameters of the vehicle-mounted camera based on the first conversion relationship and the second conversion relationship.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images