Method, device, equipment, medium and program product for determining camera posture of vehicle

Abstract

The present application relates to a method, device, computer equipment, computer-readable storage medium and computer program product for determining the camera posture of a vehicle. The method comprises: obtaining a calibration image, performing image adjustment on the calibration image according to image adjustment parameters, performing image detection on the adjusted image to obtain image coordinates of calibration feature points; clustering the calibration feature points according to the image coordinates of the calibration feature points, obtaining multiple cluster centers and calibration feature points belonging to each cluster center, eliminating calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining a position mapping matrix for the remaining calibration feature points according to the corresponding image coordinates and world coordinates; filtering out calibration feature points with similar position mapping matrices from the remaining calibration feature points, and calculating the camera posture of the camera according to the image coordinates and world coordinates of the filtered calibration feature points, which can improve the calibration accuracy and calibration efficiency of the camera posture under the condition of poor image quality.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for determining a camera posture of a vehicle. Background Art

[0002] Camera pose refers to the position and orientation of the camera in the world coordinate system. Camera pose is one of the key parameters to ensure the accuracy and reliability of the system, and an important part of the vehicle's autonomous driving and other tasks. Camera pose not only determines the visual perception effect of the system, but also directly affects the distance estimation of targets such as motor vehicles, non-motor vehicles and pedestrians on the road. With the help of the positional relationship between the vehicle's camera pose (such as the camera pose of the vehicle's electronic rearview mirror) and the target, the system can accurately calculate the lateral and longitudinal distances of the target, thereby providing reliable distance measurement results for the vehicle's driving assistance and safety protection.

[0003] At present, the determination of the camera pose of a vehicle depends on the accurate detection of feature points on the calibration plate during the camera calibration process. This detection process is extremely sensitive to external environmental conditions, especially in the case of drastic changes in lighting, reflection interference or complex background. The stability and accuracy of feature point detection will be significantly reduced, thus affecting the calibration accuracy of the vehicle's camera pose. For this reason, the calibration personnel have to adjust the placement of the calibration plate and the lighting conditions many times, resulting in low calibration efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for determining the camera pose of a vehicle in response to the above-mentioned technical problems, which can improve the calibration accuracy and efficiency of the camera pose of the vehicle when the external environmental conditions are relatively harsh resulting in poor image quality.

[0005] In a first aspect, the present application provides a method for determining a camera pose of a vehicle, comprising:

[0006] Acquire a calibration image, where the calibration image is obtained by the camera of the vehicle capturing an image of a calibration plate, and when the image is captured, the calibration plate is placed at a set position in a world coordinate system;

[0007] Performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image;

[0008] Performing image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image;

[0009] Clustering the calibration feature points according to the image coordinates of each of the calibration feature points to obtain a plurality of cluster centers and calibration feature points belonging to each cluster center, removing calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining the remaining calibration feature points;

[0010] For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system;

[0011] The calibration feature points with similar corresponding position mapping matrices are selected from the remaining calibration feature points, and the camera pose of the camera is calculated according to the image coordinates and world coordinates of the selected calibration feature points.

[0012] In one of the embodiments, obtaining the calibration image includes: acquiring an image of a calibration plate placed at a set position in a world coordinate system through a camera of the vehicle to obtain a captured image; acquiring a calibration plate detection model, calling the calibration plate detection model to perform calibration plate detection on the captured image, and identifying an area where the calibration plate is located; and extracting a calibration image from the captured image according to the area where the calibration plate is located.

[0013] In one of the embodiments, performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image includes: performing image adjustment on the calibration image according to multiple different preset image adjustment parameters to obtain multiple adjusted images; wherein the multiple different preset image adjustment parameters are used to adjust image attributes in the calibration image, and the image attributes include at least one of exposure, chroma and contrast.

[0014] In one of the embodiments, performing image detection on the adjusted image to obtain image coordinates of calibrated feature points in the calibrated image includes: acquiring a feature point detection model, calling the feature point detection model to perform feature point detection on the adjusted image, and identifying the image coordinates of the calibrated feature points in the adjusted image.

[0015] In one embodiment, the calibration feature points are clustered according to the image coordinates of each of the calibration feature points to obtain multiple cluster centers and calibration feature points belonging to each cluster center, and the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold are eliminated to obtain the remaining calibration feature points, including: obtaining the array size of the calibration plate and obtaining a first number according to the array size; according to the first number, the calibration feature points are clustered according to the image coordinates of each of the calibration feature points to obtain the first number of cluster clusters, wherein each of the cluster clusters includes a cluster center with image coordinates and calibration feature points belonging to each cluster cluster; for each cluster cluster, the distance from the calibration feature point to the cluster center is calculated according to the image coordinates; the calibration feature points whose distance is greater than the preset threshold are regarded as abnormal feature points, and the abnormal feature points are eliminated to obtain the remaining calibration feature points.

[0016] In one of the embodiments, the step of selecting calibration feature points having similar corresponding position mapping matrices from the remaining calibration feature points and calculating the camera pose of the camera according to the image coordinates and world coordinates of the selected calibration feature points comprises: calculating the similarity between each of the multiple position mapping matrices based on the multiple position mapping matrices calculated for the remaining calibration feature points; selecting and selecting a second number of calibration feature points according to the similarity; and calculating the camera pose according to the image coordinates and world coordinates of the selected calibration feature points.

[0017] In one of the embodiments, calculating the camera pose based on the image coordinates and world coordinates of the filtered calibration feature points includes: for the filtered calibration feature points, determining the calibration feature points corresponding to the same world coordinates; for the calibration feature points corresponding to the same world coordinates, calculating the mean based on the image coordinates of each of the calibration feature points, and using the mean as the fused image coordinates corresponding to the same world coordinates; calculating the camera pose based on the fused image coordinates and world coordinates of the calibration feature points.

[0018] In a second aspect, the present application also provides a device for determining a camera position and posture of a vehicle, comprising:

[0019] An acquisition module, used for acquiring a calibration image, wherein the calibration image is obtained by the camera of the vehicle acquiring an image of a calibration plate, and the calibration plate is placed at a set position in a world coordinate system during image acquisition;

[0020] An image adjustment module, used to perform image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image;

[0021] A feature point detection module, used to perform image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image;

[0022] An outlier filtering module is used to cluster the calibration feature points according to the image coordinates of each of the calibration feature points, obtain multiple cluster centers and calibration feature points belonging to each cluster center, and remove the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold to obtain the remaining calibration feature points;

[0023] The feature point screening module obtains a position mapping matrix for the remaining calibrated feature points according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system; and screens out calibrated feature points with similar corresponding position mapping matrices from the remaining calibrated feature points;

[0024] The camera pose calculation module is used to calculate the camera pose of the camera according to the image coordinates and world coordinates of the filtered calibration feature points.

[0025] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] Acquire a calibration image, where the calibration image is obtained by the camera of the vehicle capturing an image of a calibration plate, and when the image is captured, the calibration plate is placed at a set position in a world coordinate system;

[0027] Performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image;

[0028] Performing image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image;

[0029] Clustering the calibration feature points according to the image coordinates of each of the calibration feature points to obtain a plurality of cluster centers and calibration feature points belonging to each cluster center, removing calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining the remaining calibration feature points;

[0030] For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system;

[0031] The calibration feature points with similar corresponding position mapping matrices are selected from the remaining calibration feature points, and the camera pose of the camera is calculated according to the image coordinates and world coordinates of the selected calibration feature points.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0033] Acquire a calibration image, where the calibration image is obtained by the camera of the vehicle capturing an image of a calibration plate, and when the image is captured, the calibration plate is placed at a set position in a world coordinate system;

[0034] Performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image;

[0035] Performing image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image;

[0036] Clustering the calibration feature points according to the image coordinates of each of the calibration feature points to obtain a plurality of cluster centers and calibration feature points belonging to each cluster center, removing calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining the remaining calibration feature points;

[0037] For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system;

[0038] The calibration feature points with similar corresponding position mapping matrices are selected from the remaining calibration feature points, and the camera pose of the camera is calculated according to the image coordinates and world coordinates of the selected calibration feature points.

[0039] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0040] Acquire a calibration image, where the calibration image is obtained by the camera of the vehicle capturing an image of a calibration plate, and when the image is captured, the calibration plate is placed at a set position in a world coordinate system;

[0041] Performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image;

[0042] Performing image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image;

[0043] Clustering the calibration feature points according to the image coordinates of each of the calibration feature points to obtain a plurality of cluster centers and calibration feature points belonging to each cluster center, removing calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining the remaining calibration feature points;

[0044] For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system;

[0045] The calibration feature points with similar corresponding position mapping matrices are selected from the remaining calibration feature points, and the camera pose of the camera is calculated according to the image coordinates and world coordinates of the selected calibration feature points.

[0046] The camera pose determination method, device, computer equipment, computer-readable storage medium and computer program product of the above-mentioned vehicle, the calibration image is obtained by the vehicle's camera performing image acquisition on the calibration plate, and the calibration plate is placed at a set position in the world coordinate system during image acquisition. After obtaining the calibration image, the calibration image is adjusted according to preset image adjustment parameters to obtain an adjusted image; image detection is performed on the adjusted image to obtain image coordinates of calibration feature points in the calibration image; the calibration feature points are clustered according to the image coordinates of each calibration feature point to obtain multiple cluster centers and calibration feature points belonging to each cluster center, and the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold are eliminated to obtain the remaining calibration feature points; for the remaining calibration feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, and the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system; calibration feature points with similar corresponding position mapping matrices are screened out from the remaining calibration feature points, and the camera pose of the camera is calculated according to the image coordinates and world coordinates of the screened calibration feature points.

[0047] On the one hand, the calibration image is processed with preset image adjustment parameters before feature point detection. After adjustment, the calibration feature points in the overexposed or dark areas of the image become clearer, which can significantly improve the integrity, accuracy and stability of feature point detection, be more robust to environmental changes, and enhance the overall performance of determining camera pose. On the other hand, by clustering the calibration feature points according to the image coordinates of each calibration feature point, the abnormal feature points can be eliminated according to the clustering results, which helps to further improve the accuracy of feature point detection. On the other hand, for the remaining calibration feature points, the position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, and the calibration feature points with similar corresponding position mapping matrices are selected from the remaining calibration feature points, and the calibration feature points with unreasonable detection are further filtered out. Finally, the camera pose of the camera is calculated according to the image coordinates and world coordinates of the selected calibration feature points, which can greatly improve the accuracy of the determined camera pose. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a schematic diagram of a calibration plate provided in some embodiments;

[0050] Figure 2 Schematic diagram of a calibration plate provided in some other embodiments;

[0051] Figure 3 is a schematic diagram of feature points detected from a calibration image in some embodiments;

[0052] Figure 4 A diagram of an application environment of a method for determining a camera pose of a vehicle in some embodiments;

[0053] Figure 5 is a schematic flow chart of a method for determining a camera pose of a vehicle in some embodiments;

[0054] Figure 6 A schematic diagram showing comparison of detection results before and after calibration image adjustment in some embodiments;

[0055] Figure 7 Schematic diagrams showing comparisons of detection results before and after calibration image adjustment in other embodiments;

[0056] Figure 8 is a structural block diagram of a camera pose determination device for a vehicle in some embodiments;

[0057] Fig. 9 is an internal structure diagram of a computer device in some embodiments;

[0058] Fig.10 1 is a diagram of the internal structure of a computer device in some other embodiments. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] At present, the cameras (or cameras) installed on vehicles have many practical functions and uses. For example, the electronic rearview mirror is an indirect vision device that obtains a specified field of view through a system composed of a camera and a monitor. It includes electronic equipment such as a high-definition camera, a digital visual processing system, a safety system, and a liquid crystal display. It is a new type of rearview mirror that can replace the traditional optical rearview mirror. For another example, a roof camera is usually installed on the top of a car or other means of transportation to provide a field of view of the vehicle's surroundings.

[0061] In some cases, in order to establish a geometric model of camera imaging and obtain accurate field of view information, so as to accurately judge objects and distances, it is necessary to calibrate some cameras on the vehicle, that is, to determine the camera pose of the vehicle. The determination of the camera pose of the vehicle depends on the accurate detection of feature points on the calibration plate during the camera calibration process. Figure 1 FIG. 2 is a schematic diagram of a calibration plate provided in some embodiments. Figure 1 , the calibration board is a checkerboard calibration board, in which the calibration feature points used to implement camera calibration are the intersection points of the checkerboard. Figure 2 FIG. 2 is a schematic diagram of a calibration plate provided in some embodiments. Figure 2 , the calibration plate is a circular calibration plate, in which the calibration feature point used to implement camera calibration is the center of the circle. Of course, in some embodiments, the calibration plate can also adopt other patterns, which is not limited in the embodiment of the present application.

[0062] The detection process of feature points on the calibration plate is sensitive to external environmental conditions, especially in the case of drastic changes in lighting, reflection interference or complex background, which often leads to inconsistent clarity of the calibration plate area in the calibration image, and the stability and accuracy of feature point detection will be significantly reduced, thus affecting the calibration accuracy of the vehicle's camera pose. For example, the calibration image may be overexposed (reflective), or it may be too dark and unclear, resulting in missing or inaccurate feature points. The missing or wrong feature points directly affect the accuracy of the camera pose solution, and thus have a negative impact on the subsequent application of the camera data.

[0063] like Figure 3 Part (A) is a schematic diagram of the feature points detected by taking a calibration image in a good environment. Figure 3 In part (A), the calibration feature point detection is accurate and complete. Figure 3 Part (B) of the diagram is a schematic diagram of the feature points detected by taking a calibration image under dim conditions, referring to Figure 3 In part (B), the feature points detected in the lower left corner of the calibration image are missing and the detection positions of some feature points are inaccurate. Figure 3Part (C) of FIG. 1 shows a schematic diagram of feature points detected after taking a calibration image under overexposure conditions. Figure 3 In part (C), the feature points detected in the upper right corner of the calibration image are missing and the detection positions of some feature points are inaccurate.

[0064] Therefore, in order to improve the calibration effect under the above-mentioned conditions of drastic lighting changes, reflection interference or complex background, the calibrator has to adjust the placement of the calibration plate and the lighting conditions many times, resulting in low calibration efficiency.

[0065] In view of this, the camera pose determination method of the vehicle provided in the embodiment of the present application, on the one hand, processes the calibration image in combination with the preset image adjustment parameters and then performs feature point detection. After the adjustment, the calibration feature points in the originally overexposed area or too dark area in the image become clearer, which can significantly improve the integrity, accuracy and stability of feature point detection, be more robust to environmental changes, and enhance the overall performance of determining the camera pose. On the other hand, by clustering the calibration feature points according to the image coordinates of each calibration feature point, the abnormal feature points can be eliminated according to the clustering results, which helps to further improve the accuracy of feature point detection. On the other hand, for the remaining calibration feature points, the position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, and the calibration feature points with similar corresponding position mapping matrices are screened out from the remaining calibration feature points, and the calibration feature points with unreasonable detection are further filtered out. Finally, the camera pose of the camera is calculated according to the image coordinates and world coordinates of the screened calibration feature points, which can greatly improve the accuracy of the determined camera pose.

[0066] The vehicle camera posture determination method provided in the embodiment of the present application can be applied to Figure 4 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0067] The calibration plate is placed at a set position in the world coordinate system, and the set position belongs to the visual range of the vehicle's camera. The vehicle's camera collects images of the calibration plate placed at the set position. In some embodiments, the terminal 102 can obtain a calibration image, and the terminal 102 can also obtain the world coordinates of the calibration feature points in the calibration plate placed at the set position in the world coordinate system. Then, the terminal 102 can perform image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image; the terminal 102 can call a feature point detection model from the server 104, perform image detection on the adjusted image, and obtain image coordinates of the calibration feature points in the calibration image; cluster the calibration feature points according to the image coordinates of each calibration feature point, obtain multiple cluster centers and calibration feature points belonging to each cluster center, remove the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtain the remaining calibration feature points; for the remaining calibration feature points, obtain a position mapping matrix according to the corresponding image coordinates and world coordinates, and the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system; select calibration feature points with similar corresponding position mapping matrices from the remaining calibration feature points, and calculate the camera pose of the camera according to the image coordinates and world coordinates of the selected calibration feature points.

[0068] Of course, in some implementations, the terminal 102 can send the calibration image to the server, and the server executes the method provided in the embodiment of the present application, that is, based on the calibration image and the world coordinates of each calibration feature point on the calibration plate in the world coordinate system, to determine the camera pose of the camera on the vehicle.

[0069] In an exemplary embodiment, Figure 5 As shown, a method for determining the camera position of a vehicle is provided, and the method is applied to Figure 4 The computer device (terminal 102 or server 104) in the example is used for explanation, and the steps include the following steps 502 to 512. Among them:

[0070] Step 502 , obtaining a calibration image. The calibration image is obtained by the vehicle's camera performing image acquisition on the calibration plate. When performing image acquisition, the calibration plate is placed at a set position in the world coordinate system.

[0071] Among them, the calibration image is obtained by the camera of the vehicle acquiring the image of the calibration plate. When acquiring the image, the calibration plate is placed at a set position in the world coordinate system. The set position is a position within the visible range of the camera of the vehicle. The world coordinate system can be a three-dimensional coordinate system with the upper left corner of the calibration plate as the origin, the z-axis perpendicular to the calibration plate, and the xoy plane coinciding with the calibration plate. The vehicle can be an autonomous driving vehicle or an assisted driving vehicle. The camera of the vehicle is set on the vehicle, and the camera of the vehicle can be, for example, an electronic rearview mirror camera of the vehicle. It can be understood that the calibration plate is placed at a set position in the world coordinate system, and the world coordinates of each calibration feature point on the calibration plate in the world coordinate system are known.

[0072] In some embodiments, the vehicle can control the camera to shoot the scene within the field of view, and the calibration plate is placed within the field of view, and the camera of the vehicle can collect an image including the calibration plate. Optionally, the icon can be obtained by an electronic device on the vehicle, and the image can be sent to a computer device, and the computer device can obtain a calibration image.

[0073] In some embodiments, obtaining a calibration image includes: acquiring an image of a calibration plate placed at a set position in a world coordinate system through a vehicle's camera to obtain a captured image; acquiring a calibration plate detection model, calling the calibration plate detection model to perform calibration plate detection on the captured image, and identifying an area where the calibration plate is located; and based on the area where the calibration plate is located, extracting the calibration image from the captured image.

[0074] The calibration plate detection model is a computer vision model based on convolutional neural networks. The computer device calls the calibration plate detection model to perform calibration plate detection on the captured image and identifies the area where the calibration plate is located. The area is the smallest rectangular area that can include the calibration plate. The computer device can intercept the original captured image based on the area to obtain a calibration image.

[0075] In this embodiment, by performing calibration plate detection on the original captured image, the calibration plate area that needs to be focused on for subsequent image adjustment and feature point detection is identified from the captured image, and a calibration image is obtained, which can improve the accuracy of subsequent feature point detection.

[0076] Step 504: perform image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image.

[0077] The preset image adjustment parameters are used to adjust the image attributes of the calibration image. The image attributes to be adjusted by the preset image adjustment parameters include at least one of exposure, chroma and contrast. In some embodiments, the image attributes may also include image brightness, saturation, etc.

[0078] Specifically, the computer device performs image adjustment on the calibration image according to preset image adjustment parameters. It can be that the calibration image is adjusted according to a preset image adjustment parameter to obtain an adjusted image, or it can be that the calibration image is adjusted in sequence according to multiple preset image adjustment parameters to obtain an adjusted image, or it can be that the calibration image is adjusted separately according to multiple preset image adjustment parameters to obtain multiple different adjusted images, or it can be that the calibration image is adjusted according to the preset image adjustment parameters to obtain an adjusted image, the feature point detection mentioned below is performed on the adjusted image to obtain the image coordinates of each calibration feature point, and then the image adjustment is continued to be performed on the adjusted image, and subsequent feature point detection is performed to obtain the image coordinates of each calibration feature point. It can be seen that multi-order adjustment and multi-order detection are performed here, and there may be multiple different image coordinates of the calibration feature points corresponding to the same world coordinates.

[0079] In some embodiments, image adjustment is performed on the calibration image according to preset image adjustment parameters to obtain an adjusted image, including: performing image adjustment on the calibration image according to multiple different preset image adjustment parameters to obtain multiple adjusted images; wherein the multiple different preset image adjustment parameters are used to adjust image attributes in the calibration image, and the image attributes include at least one of exposure, chroma and contrast.

[0080] In this embodiment, the calibration image is processed in combination with preset image adjustment parameters before feature point detection. After the adjustment, the calibration feature points in the originally overexposed or dark areas of the image become clearer, which can significantly improve the integrity, accuracy and stability of feature point detection, make it more robust to environmental changes, and enhance the overall performance of determining the camera posture.

[0081] Step 506: perform image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image.

[0082] Among them, image detection is the process of detecting the image coordinates of the calibrated feature points in the calibration image. It can be understood that the world coordinates represent the real coordinates of the object in the three-dimensional world coordinate system. The world coordinates are represented by (X, Y, Z). The image coordinates refer to the image pixel coordinates. The origin of the image pixel coordinate system is in the upper left corner, and the unit is pixel. The image coordinates can be recorded as (u, v), where u represents the horizontal coordinate and v represents the vertical coordinate. The range of image coordinates is usually from (0, 0) to (1, 1), with the upper left corner being (0, 0) and the lower right corner being (1, 1).

[0083] In some embodiments, image detection is performed on the adjusted image to obtain image coordinates of calibrated feature points in the calibrated image, including: acquiring a feature point detection model, calling the feature point detection model to perform feature point detection on the adjusted image, and identifying the image coordinates of the calibrated feature points in the adjusted image.

[0084] The feature point detection model is a computer vision model based on a convolutional neural network. The computer device calls the feature point detection model to perform feature point detection on the adjusted image, and identifies the position information of each calibrated feature point in the adjusted image, that is, detects the image coordinates of the calibrated feature point.

[0085] like Figure 6 FIG. 1 is a schematic diagram showing a comparison of detection results before and after calibration image adjustment in an embodiment. Figure 6 , the first row, from left to right, is the calibration image P1 acquired under dim conditions, the adjusted image P2 obtained by adjusting the calibration image P1 according to the preset image adjustment parameter s1, and the adjusted image P3 obtained by adjusting the calibration image P1 according to the preset image adjustment parameter s2. The second row, from left to right, is the detection results corresponding to the calibration image P1, the adjusted image P2, and the adjusted image P3. It can be seen that there are 5 feature points in the calibration image P1 that are not accurately detected due to unclear images, there is 1 feature point in the adjusted image P2 that is not accurately detected due to unclear images, and there is no feature point in the adjusted image P3 that is not accurately detected due to unclear images.

[0086] like Figure 7 FIG. 1 is a schematic diagram showing a comparison of detection results before and after calibration image adjustment in another embodiment. Figure 7 , the first row, from left to right, is the calibration image P4 acquired under overexposure conditions, the adjusted image P5 obtained by adjusting the calibration image according to the preset image adjustment parameter s3, and the adjusted image P6 obtained by adjusting the calibration image according to the preset image adjustment parameter s4. The second row, from left to right, is the detection results corresponding to the calibration image P4, the adjusted image P5, and the adjusted image P6. It can be seen that there are 3 feature points in the calibration image P4 that are inaccurately detected due to image overexposure, there is 1 feature point in the adjusted image P5 that is inaccurately detected due to image overexposure, and there is no feature point in the adjusted image P6 that is inaccurately detected due to image overexposure.

[0087] based on Figure 6 , Figure 7 It can be seen that after parameter adjustment, the feature points that were originally missed are detected, and the detection results of the feature points that were originally inaccurately detected are more accurate.

[0088] Step 508, clustering the calibrated feature points according to their image coordinates, obtaining multiple cluster centers and calibrated feature points belonging to each cluster center, removing calibrated feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining the remaining calibrated feature points.

[0089] In an embodiment of the present application, some constraints are used to filter the detected calibrated feature points, eliminate abnormal feature points that are obviously inaccurately detected, and retain feature points that meet the constraints. That is, the calibrated feature points are clustered according to the image coordinates of each calibrated feature point, and abnormal feature points can be eliminated based on the clustering results, which helps to further improve the accuracy of feature point detection.

[0090] The computer device can cluster the calibration feature points according to the image coordinates of each calibration feature point, obtain multiple cluster centers and calibration feature points belonging to each cluster center, and then remove the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold according to the image coordinates of the calibration feature points and the image coordinates of the cluster centers to which they belong, to obtain the remaining calibration feature points. In this way, feature points that are obviously not satisfied with the position constraint relationship or distance constraint relationship of the feature points on the calibration plate can be removed. The image coordinates of these feature points are obviously offset, and removal can help further improve the accuracy of feature point detection. Among them, the embodiment of the present application does not limit the clustering method. For example, a K-mean clustering algorithm, a density-based non-parametric clustering algorithm, and the like can be used.

[0091] In some embodiments, the calibration feature points are clustered according to the image coordinates of each calibration feature point to obtain multiple cluster centers and calibration feature points belonging to each cluster center, and the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold are eliminated to obtain the remaining calibration feature points, including: acquiring the array size of the calibration plate and obtaining a first number according to the array size; according to the first number, the calibration feature points are clustered according to the image coordinates of each calibration feature point to obtain a first number of cluster clusters, wherein each cluster cluster includes a cluster center with image coordinates and calibration feature points belonging to each cluster cluster; for each cluster cluster, the distance from the calibration feature point to the cluster center is calculated according to the image coordinates; the calibration feature points whose distance is greater than the preset threshold are regarded as abnormal feature points, and the abnormal feature points are eliminated to obtain the remaining calibration feature points.

[0092] For example, assuming that the array size of the checkerboard calibration plate is m×n, the number of cluster centers is m×n. The calibration feature points are clustered according to the image coordinates of each calibration feature point according to the m×n number to obtain m×n clusters. The specific clustering process is to first initialize the coordinates of the m×n cluster centers, calculate the distance from the corresponding image coordinates to each cluster center for each calibration feature point, and divide the calibration feature points into the clusters with the nearest cluster center. After all the calibration feature points are divided, for each cluster, the mean is calculated according to the image coordinates of all the calibration feature points included, and the mean is used as the new cluster center of the cluster to update the cluster center. Then, continue a new round of iterations, and for each calibrated feature point, calculate the distance from the corresponding image coordinates to each updated cluster center, and divide the calibrated feature points into the clusters with the nearest cluster center. After all the calibrated feature points are divided, for each cluster, calculate the mean according to the image coordinates of all the calibrated feature points included, and use the mean as the new cluster center of the cluster. Repeat this process until the number of iterations reaches the set value, or until the difference between the currently calculated mean and the mean calculated in the previous round for each cluster is less than the set threshold, the clustering stops, and m×n clusters can be obtained, each with a cluster center. Then, for each cluster, calculate the distance from the calibrated feature point to the cluster center according to the image coordinates, and use the calibrated feature points with a distance greater than the preset threshold as abnormal feature points. Remove the abnormal feature points to obtain the remaining calibrated feature points.

[0093] Step 510: For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates. The position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system.

[0094] Furthermore, after removing abnormal calibration feature points, for the remaining calibration feature points, the computer device can obtain a position mapping matrix with the corresponding image coordinates and world coordinates. The position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system. The position mapping matrix can be a homography matrix. For the remaining calibration feature points, assuming that the corresponding image coordinates are accurately detected, the multiple position mapping matrices solved will be completely consistent. Based on this, there are multiple calibration feature points remaining. For each calibration feature point, the position mapping matrix is ​​solved according to the corresponding image coordinates and world coordinates to obtain multiple position mapping matrices. In fact, because the image coordinates may not be accurate enough, there will be differences in the multiple position mapping matrices.

[0095] For example, the calibration plate is a checkerboard calibration plate, and the length of the checkerboard grid is known, so the world coordinates (X, Y, Z) of each calibration feature point on the calibration plate in the world coordinate system can be known, where Z=0. The position mapping matrix can be used to map a certain world coordinate (X, Y, 0) to the image coordinate (u, v). A position mapping matrix can be calculated based on the world coordinates and the image coordinates.

[0096] Step 512: Filter out the calibration feature points with similar corresponding position mapping matrices from the remaining calibration feature points, and calculate the camera pose of the camera according to the image coordinates and world coordinates of the filtered calibration feature points.

[0097] For the multiple solved position mapping matrices, in fact, because the image coordinates may not be accurate enough, there will be differences in the multiple position mapping matrices. The computer device can perform similarity calculation on the position mapping matrices to obtain the similarity between the position mapping matrices, and based on the similarity, screen out the calibration feature points with similar corresponding position mapping matrices, and then calculate the camera pose of the camera based on the image coordinates and world coordinates of the screened calibration feature points. In this way, the accuracy of the image coordinates can be inferred based on the similarity of the position mapping matrices, and the remaining calibration feature points can be further screened, and the calibration feature points with more accurate corresponding image coordinates can be selected from them. Based on the image coordinates and world coordinates of the calibration feature points finally screened out, the camera pose of the camera is calculated, and the accuracy of the obtained camera pose can be greatly improved.

[0098] In some embodiments, the calibration feature points with similar corresponding position mapping matrices are selected from the remaining calibration feature points, and the camera pose of the camera is calculated according to the image coordinates and world coordinates of the selected calibration feature points, including: calculating the similarity between the multiple position mapping matrices based on the multiple position mapping matrices calculated for the remaining calibration feature points; selecting and selecting a second number of calibration feature points according to the similarity; and calculating the camera pose according to the image coordinates and world coordinates of the selected calibration feature points. For example, the computer device can select the most similar N calibration feature points according to the similarity.

[0099] In some embodiments, the computer device can use the PnP (Perspective-n-Point) algorithm to solve the camera pose for the filtered calibration feature points according to the corresponding image coordinates and world coordinates. The camera pose, that is, the rotation matrix and displacement vector of the camera, can be used to link the world coordinate system and the camera coordinate system.

[0100] In some embodiments, calculating the camera pose based on the image coordinates and world coordinates of the filtered calibration feature points includes: for the filtered calibration feature points, determining the calibration feature points corresponding to the same world coordinates; for the calibration feature points corresponding to the same world coordinates, calculating the mean based on the image coordinates of each calibration feature point, and using the mean as the fused image coordinates corresponding to the same world coordinates; calculating the camera pose based on the fused image coordinates and world coordinates of the calibration feature points.

[0101] As mentioned above, the calibration image can be adjusted according to a plurality of different preset image adjustment parameters to obtain a plurality of adjusted images. In the plurality of adjusted images, there may be a plurality of calibration feature points corresponding to the same world coordinate. Therefore, for the selected calibration feature points, there may be calibration feature points corresponding to the same world coordinate. For the calibration feature points corresponding to the same world coordinate, the computer device can calculate the mean value according to the image coordinates of each calibration feature point, and use the mean value as the fused image coordinate corresponding to the same world coordinate. The accuracy of the image coordinates used to calculate the camera pose can be further improved by using the mean value as the fused image coordinate corresponding to the same world coordinate. Finally, the camera pose is calculated according to the fused image coordinates and world coordinates of the calibration feature points. The accuracy is greatly improved, and the calibration result is more robust.

[0102] The camera pose determination method of the above-mentioned vehicle, on the one hand, processes the calibration image in combination with the preset image adjustment parameters before performing feature point detection, and the calibration feature points in the originally overexposed or dark areas of the adjusted image become clearer, which can significantly improve the integrity, accuracy and stability of feature point detection, be more robust to environmental changes, and enhance the overall performance of determining the camera pose. On the other hand, by clustering the calibration feature points according to the image coordinates of each calibration feature point, the abnormal feature points can be eliminated according to the clustering results, which helps to further improve the accuracy of feature point detection. On the other hand, for the remaining calibration feature points, the position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, and the calibration feature points with similar corresponding position mapping matrices are screened out from the remaining calibration feature points, and the calibration feature points with unreasonable detection are further filtered out. Finally, the camera pose of the camera is calculated according to the image coordinates and world coordinates of the screened calibration feature points, which can greatly improve the accuracy of the determined camera pose.

[0103] In a specific embodiment, the method for determining the camera pose of a vehicle is performed by a computer device and may include the following steps:

[0104] 1. Use the vehicle's camera to collect images of the calibration plate placed at a set position in the world coordinate system to obtain a collected image;

[0105] 2. Obtain the calibration plate detection model, call the calibration plate detection model to perform calibration plate detection on the acquired image, and identify the area where the calibration plate is located;

[0106] 3. According to the area where the calibration plate is located, the calibration image is cut out from the acquired image;

[0107] 4. According to a plurality of different preset image adjustment parameters, the calibration images are respectively adjusted to obtain a plurality of adjusted images; wherein the plurality of different preset image adjustment parameters are used to adjust image attributes in the calibration images, and the image attributes include at least one of exposure, chroma and contrast;

[0108] 5. Obtain a feature point detection model, call the feature point detection model to perform feature point detection on the adjusted image, and identify the image coordinates of the calibrated feature points in the adjusted image;

[0109] 6. Obtain an array size of the calibration plate, and obtain a first quantity according to the array size;

[0110] 7. Clustering the calibrated feature points according to the first quantity and the image coordinates of each calibrated feature point to obtain a first number of clusters, wherein each cluster includes a cluster center having image coordinates and calibrated feature points belonging to each cluster;

[0111] 8. For each cluster, calculate the distance from the calibrated feature point to the cluster center according to the image coordinates;

[0112] 9. The calibrated feature points whose distance is greater than the preset threshold are regarded as abnormal feature points, and the abnormal feature points are removed to obtain the remaining calibrated feature points;

[0113] 10. For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates. The position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system;

[0114] 11. Based on the multiple position mapping matrices calculated for the remaining calibration feature points, calculate the similarity between the multiple position mapping matrices;

[0115] 12. Select and filter out a second number of calibration feature points based on the similarity;

[0116] 13. For the selected calibration feature points, determine the calibration feature points corresponding to the same world coordinates;

[0117] 14. For the calibrated feature points corresponding to the same world coordinates, the mean is calculated according to the image coordinates of each calibrated feature point, and the mean is used as the fused image coordinate corresponding to the same world coordinates;

[0118] 15. Calculate the camera pose based on the fused image coordinates and world coordinates of the calibrated feature points.

[0119] The camera posture determination method of the vehicle provided in this embodiment uses the calibration plate detection model to detect the area where the calibration plate is located, thereby eliminating the background area irrelevant to the feature point detection. The feature detection model only performs feature point detection on the area where the calibration plate is located to improve the accuracy and stability of feature point detection; the calibration image is adjusted multiple times for feature point detection. By adjusting the calibration image multiple times and performing multiple feature point detections respectively, the requirements for the calibration environment are reduced, thereby improving the integrity and accuracy of feature point detection; feature point filtering and fusion: according to the results of multiple feature point detections, abnormal feature points are filtered and fused. This process helps to further improve the overall accuracy of feature point detection.

[0120] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiment of the present application also provides a vehicle camera pose determination device for implementing the above-mentioned vehicle camera pose determination method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of the vehicle camera pose determination device provided below can refer to the limitations of the vehicle camera pose determination method above, and will not be repeated here.

[0122] In an exemplary embodiment, Figure 8 As shown, a device 800 for determining a camera pose of a vehicle is provided, comprising: an acquisition module 802, an image adjustment module 804, a feature point detection module 806, an abnormal point filtering module 808, a feature point screening module 810 and a camera pose calculation module 812, wherein:

[0123] An acquisition module 802 is used to acquire a calibration image. The calibration image is obtained by the vehicle's camera acquiring an image of the calibration plate. When acquiring the image, the calibration plate is placed at a set position in the world coordinate system.

[0124] An image adjustment module 804 is used to adjust the calibration image according to preset image adjustment parameters to obtain an adjusted image;

[0125] The feature point detection module 806 is used to perform image detection on the adjusted image to obtain the image coordinates of the calibration feature points in the calibration image;

[0126] The outlier filtering module 808 is used to cluster the calibration feature points according to the image coordinates of each calibration feature point, obtain multiple cluster centers and calibration feature points belonging to each cluster center, and remove the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold to obtain the remaining calibration feature points;

[0127] The feature point screening module 810 obtains a position mapping matrix for the remaining calibrated feature points according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system; and screens out calibrated feature points with similar corresponding position mapping matrices from the remaining calibrated feature points;

[0128] The camera pose calculation module 812 is used to calculate the camera pose of the camera according to the image coordinates and world coordinates of the filtered calibration feature points.

[0129] In an exemplary embodiment, the acquisition module 802 is used to acquire an image of a calibration plate placed at a set position in a world coordinate system through a vehicle's camera to obtain a captured image; acquire a calibration plate detection model, call the calibration plate detection model to perform calibration plate detection on the captured image, and identify the area where the calibration plate is located; and extract the calibration image from the captured image according to the area where the calibration plate is located.

[0130] In an exemplary embodiment, the image adjustment module 804 is used to perform image adjustments on the calibration image according to multiple different preset image adjustment parameters to obtain multiple adjusted images; wherein the multiple different preset image adjustment parameters are used to adjust image properties in the calibration image, and the image properties include at least one of exposure, chroma and contrast.

[0131] In an exemplary embodiment, the feature point detection module 80 is used to obtain a feature point detection model, call the feature point detection model to perform feature point detection on the adjusted image, and identify the image coordinates of the calibrated feature points in the adjusted image.

[0132] In an exemplary embodiment, the abnormal point filtering module 808 is used to obtain the array size of the calibration plate, and obtain a first number according to the array size; according to the first number, the calibration feature points are clustered according to the image coordinates of each calibration feature point to obtain a first number of cluster clusters, wherein each cluster cluster includes a cluster center with image coordinates and calibration feature points belonging to each cluster cluster; for each cluster cluster, the distance from the calibration feature point to the cluster center is calculated according to the image coordinates; the calibration feature points whose distance is greater than a preset threshold are regarded as abnormal feature points, and the abnormal feature points are eliminated to obtain the remaining calibration feature points.

[0133] In an exemplary embodiment, the camera pose calculation module 812 is used to calculate the similarity between multiple position mapping matrices based on the multiple position mapping matrices calculated for the remaining calibration feature points; select and filter out a second number of calibration feature points based on the similarity; and calculate the camera pose based on the image coordinates and world coordinates of the filtered calibration feature points.

[0134] In an exemplary embodiment, the camera pose calculation module 812 is used to determine the calibration feature points corresponding to the same world coordinates from the screened calibration feature points; for the calibration feature points corresponding to the same world coordinates, calculate the mean value based on the image coordinates of each calibration feature point, and use the mean value as the fused image coordinates corresponding to the same world coordinates; calculate the camera pose based on the fused image coordinates and world coordinates of the calibration feature points.

[0135] The camera pose determination device 800 of the above-mentioned vehicle, on the one hand, processes the calibration image in combination with the preset image adjustment parameters and then performs feature point detection. After the adjustment, the calibration feature points in the originally overexposed area or too dark area in the image become clearer, which can significantly improve the integrity, accuracy and stability of feature point detection, be more robust to environmental changes, and enhance the overall performance of determining the camera pose. On the other hand, by clustering the calibration feature points according to the image coordinates of each calibration feature point, the abnormal feature points can be eliminated according to the clustering results, which helps to further improve the accuracy of feature point detection. On the other hand, for the remaining calibration feature points, the position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, and the calibration feature points with similar corresponding position mapping matrices are screened out from the remaining calibration feature points, and the calibration feature points with unreasonable detection are further filtered out. Finally, the camera pose of the camera is calculated according to the image coordinates and world coordinates of the screened calibration feature points, which can greatly improve the accuracy of the determined camera pose.

[0136] Each module in the above-mentioned vehicle camera pose determination device 800 can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0137] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for determining the camera posture of a vehicle is implemented.

[0138] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for determining the camera posture of a vehicle is implemented.

[0139] Those skilled in the art will understand that Fig. 9 , Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method for determining the camera posture of a vehicle provided in an embodiment of the present application are implemented.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for determining the camera pose of a vehicle provided in an embodiment of the present application are implemented.

[0142] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method for determining the camera pose of a vehicle provided in an embodiment of the present application.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining a camera position of a vehicle, characterized in that: The method comprises: Calling the calibration plate detection model to perform calibration plate detection on the captured image, identifying the area where the calibration plate is located, and intercepting a calibration image from the captured image according to the area where the calibration plate is located, wherein the captured image is obtained by the camera of the vehicle performing image acquisition on the calibration plate, and the calibration plate is placed at a set position in the world coordinate system during image acquisition; Performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image; Performing image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image; Clustering the calibration feature points according to the image coordinates of each of the calibration feature points to obtain a plurality of cluster centers and calibration feature points belonging to each cluster center, removing calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold, and obtaining the remaining calibration feature points; For the remaining calibrated feature points, a position mapping matrix is ​​obtained according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system; Based on multiple position mapping matrices calculated for the remaining calibration feature points, calculate the similarity between the multiple position mapping matrices, filter out a second number of calibration feature points based on the similarity, determine the calibration feature points corresponding to the same world coordinates from the filtered calibration feature points, calculate the mean value based on the image coordinates of each of the calibration feature points corresponding to the same world coordinates, use the mean value as the fused image coordinates corresponding to the same world coordinates, and calculate the camera pose of the camera based on the fused image coordinates and world coordinates of the calibration feature points.

2. The method according to claim 1, characterized in that The method further comprises: The camera of the vehicle collects an image of a calibration plate placed at a set position in a world coordinate system to obtain the collected image.

3. The method according to claim 1, characterized in that The step of performing image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image includes: According to a plurality of different preset image adjustment parameters, the calibration images are respectively adjusted to obtain a plurality of adjusted images; The plurality of different preset image adjustment parameters are used to adjust image attributes in the calibration image, and the image attributes include at least one of exposure, chroma and contrast.

4. The method according to claim 1, characterized in that: The performing image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image includes: A feature point detection model is obtained, and the feature point detection model is called to perform feature point detection on the adjusted image, and the image coordinates of the calibrated feature points in the adjusted image are identified.

5. The method according to claim 1, characterized in that: The step of clustering the calibrated feature points according to the image coordinates of each of the calibrated feature points to obtain a plurality of cluster centers and calibrated feature points belonging to each cluster center, and removing calibrated feature points whose distance to the corresponding cluster center is greater than a preset threshold to obtain the remaining calibrated feature points includes: Acquire an array size of the calibration plate, and obtain a first quantity according to the array size; According to the first number, clustering the calibrated feature points according to the image coordinates of each of the calibrated feature points to obtain the first number of clusters, wherein each of the clusters includes a cluster center with image coordinates and calibrated feature points belonging to each cluster; For each cluster, the distance from the calibrated feature point to the cluster center is calculated according to the image coordinates; The calibrated feature points whose distance is greater than a preset threshold are regarded as abnormal feature points, and the abnormal feature points are removed to obtain the remaining calibrated feature points.

6. A camera posture determination device for a vehicle, characterized in that: The device comprises: An acquisition module is used to call a calibration plate detection model to perform calibration plate detection on a captured image, identify an area where the calibration plate is located, and extract a calibration image from the captured image according to the area where the calibration plate is located. The captured image is obtained by the camera of the vehicle performing image acquisition on the calibration plate. When performing image acquisition, the calibration plate is placed at a set position in the world coordinate system; An image adjustment module, used to perform image adjustment on the calibration image according to preset image adjustment parameters to obtain an adjusted image; A feature point detection module, used to perform image detection on the adjusted image to obtain image coordinates of calibration feature points in the calibration image; An outlier filtering module is used to cluster the calibration feature points according to the image coordinates of each of the calibration feature points, obtain multiple cluster centers and calibration feature points belonging to each cluster center, and remove the calibration feature points whose distance to the corresponding cluster center is greater than a preset threshold to obtain the remaining calibration feature points; A feature point screening module, for the remaining calibration feature points, obtains a position mapping matrix according to the corresponding image coordinates and world coordinates, wherein the position mapping matrix represents the position mapping relationship between the world coordinate system and the pixel coordinate system; based on the multiple position mapping matrices calculated for the remaining calibration feature points, calculates the similarity between the multiple position mapping matrices, and screens out a second number of calibration feature points according to the similarity; The camera pose calculation module is used to determine the calibration feature points corresponding to the same world coordinates from the screened calibration feature points, calculate the mean of the calibration feature points corresponding to the same world coordinates according to the image coordinates of each of the calibration feature points, use the mean as the fused image coordinates corresponding to the same world coordinates, and calculate the camera pose of the camera according to the fused image coordinates and world coordinates of the calibration feature points.

7. The device according to claim 6, characterized in that The image adjustment module is further used to perform image adjustment on the calibration image according to multiple different preset image adjustment parameters to obtain multiple adjusted images; wherein the multiple different preset image adjustment parameters are used to adjust image attributes in the calibration image, and the image attributes include at least one of exposure, chroma and contrast.

8. The device according to claim 6, characterized in that The feature point detection module is further used to obtain a feature point detection model, call the feature point detection model to perform feature point detection on the adjusted image, and identify the image coordinates of the calibrated feature points in the adjusted image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images