Camera calibration method, device, equipment, medium and product

Abstract

The invention discloses a camera calibration method, an electronic device, a control device, electronic equipment, a computer readable storage medium and a computer program product, and the method comprises the steps: obtaining first position information, determining a world coordinate system according to the first position information, and determining a second position information of a calibration object in the world coordinate system according to second position information of the calibration object in the world coordinate system; and determining camera parameters of the vehicle camera to complete calibration of the vehicle camera. Therefore, the construction of the world coordinate system can be completed by means of the light emitted by the light emitter, the construction difficulty and the construction cost are reduced, the construction process is simpler and more convenient, the construction efficiency of the world coordinate system and the calibration efficiency of the vehicle camera are improved, and the calibration accuracy of the vehicle camera is improved. The world coordinate system is constructed according to the first position information of the light relative to the vehicle, so that the reliability of the world coordinate system can be guaranteed, and the calibration precision of camera parameters can also be guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of camera calibration, and in particular to a camera calibration method, an electronic device, a control device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] The on-board camera is one of the important components for the vehicle to perceive the external environment. It can be used to collect images of the vehicle's surroundings. The various targets in the image, such as pedestrians, lane signs and other information, can guide the vehicle to change its driving status and remind the driver to make corresponding driving operations.

[0003] However, in the images captured by the vehicle's camera, the position information of various targets is described based on camera coordinates or pixel coordinates, which makes it impossible for the position information to directly reflect the position of the target in the real world, and therefore cannot accurately express the relative position relationship between the target and the vehicle, ultimately leading to deviations in the vehicle's perception of the external environment, thereby affecting the vehicle's driving status and the user's driving experience. Summary of the invention

[0004] The present application provides a camera calibration method, an electronic device, a control device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] The present application provides a camera calibration method, including:

[0006] Acquire first position information, wherein the first position information is used to indicate a position of the light emitted by the light emitter relative to the vehicle;

[0007] Determine a world coordinate system according to the first position information;

[0008] According to the second position information of the calibration object in the world coordinate system, the camera parameters of the vehicle camera are determined to complete the calibration of the vehicle camera, wherein the camera parameters are used to indicate the mapping of the object in the image captured by the vehicle camera in the world coordinate system.

[0009] In this way, in the implementation mode of the present application, the construction of the world coordinate system can be completed with the help of the light emitted by the light emitter. Compared with the method of constructing an additional hardware platform to construct the world coordinate system, the construction difficulty and construction cost are reduced, and the construction process is simpler and more convenient, thereby improving the construction efficiency of the world coordinate system and the calibration efficiency of the vehicle camera. In addition, since the world coordinate system is constructed based on the first position information of the light relative to the vehicle, the reliability of the world coordinate system can be guaranteed to a certain extent, and the calibration accuracy of the camera parameters can also be guaranteed.

[0010] In some embodiments of the present application, the light includes a first light, a second light, a third light, and a fourth light located in the same plane.

[0011] In certain embodiments of the present application, the first light ray is tangent to the first front wheel and the first rear wheel, the second light ray is tangent to the second front wheel and the second rear wheel, the third light ray is perpendicular to the first light ray and the second light ray, the fourth light ray is perpendicular to the first light ray and the second light ray, the first front wheel and the first rear wheel are located on one side of the vehicle, and the second front wheel and the second rear wheel are located on the other side of the vehicle.

[0012] In certain embodiments of the present application, the origin of the world coordinate system is determined based on the position of the axis of the vehicle support shaft, the first coordinate axis of the world coordinate system is parallel to the first light ray and the second light ray, and the second coordinate axis of the world coordinate system is parallel to the third light ray and the fourth light ray.

[0013] In some embodiments of the present application, the light rays include at least two light rays having an intersection, the first position information includes intersection position information of the intersection, and the method further includes:

[0014] The second position information is determined according to the intersection position information and third position information of the grounding point of the calibration object relative to the intersection.

[0015] Thus, in the embodiment of the present application, the second position information of the calibration object in the world coordinate system can be determined according to the intersection position information and the third position information of the grounding point of the calibration object relative to the intersection.

[0016] In certain embodiments of the present application, the calibration object includes a plurality of feature points set with preset parameters, the second position information includes feature point coordinates, and determining the second position information according to the intersection position information and third position information of the grounding point of the calibration object relative to the intersection includes:

[0017] Determine the grounding point coordinates of the grounding point in the world coordinate system according to the third position information and the intersection position information;

[0018] The feature point coordinates of the feature point in the world coordinate system are determined according to the ground point coordinates and the preset parameters.

[0019] In this way, in the implementation mode of the present application, the grounding point coordinates of the grounding point in the world coordinate system can be determined according to the third position information and the intersection position information, and the feature point coordinates of the feature point in the world coordinate system can be determined according to the grounding point coordinates and preset parameters, so that the calibration of the camera parameters can be determined based on the feature point coordinates of each feature point, thereby ensuring the reliability of the camera parameter calibration results.

[0020] In some embodiments of the present application, the grounding point includes a first grounding point and a second grounding point, the third position information includes a first coordinate of the first grounding point and a second coordinate of the second grounding point, and the method further includes:

[0021] The first coordinate and the second coordinate are updated according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object.

[0022] Thus, in the implementation mode of the present application, the first coordinate and the second coordinate can be updated according to the coordinate distance between the first coordinate and the second coordinate, and the size parameters of the calibration object, thereby improving the accuracy of the first coordinate, the second coordinate and the feature point coordinate.

[0023] In some embodiments of the present application, updating the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object includes:

[0024] When the difference between the coordinate distance and the size parameter is greater than a preset threshold, the first coordinate and the second coordinate are updated.

[0025] Thus, in the embodiment of the present application, the first coordinate and the second coordinate may be updated when the difference between the coordinate distance and the size parameter is greater than a preset threshold, thereby ensuring effective updating of the first coordinate and the second coordinate.

[0026] In some embodiments of the present application, the first coordinate includes a first horizontal coordinate and a first vertical coordinate value, the second coordinate includes a second horizontal coordinate and a second vertical coordinate value, and when the difference between the coordinate distance and the size parameter is greater than a preset threshold, updating the first coordinate and the second coordinate includes:

[0027] When the difference is greater than the preset threshold, adjusting the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value according to the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value;

[0028] According to the adjustment result, the first coordinate and the second coordinate are updated.

[0029] In this way, in an embodiment of the present application, the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value can be adjusted according to the first horizontal coordinate, the first vertical coordinate value of the first coordinate, and the second horizontal coordinate and the second vertical coordinate value of the second coordinate, thereby completing the update of the first coordinate and the second coordinate.

[0030] In some embodiments of the present application, the adjustment result includes an adjusted first horizontal coordinate value, an adjusted first vertical coordinate value, an adjusted second horizontal coordinate value, and an adjusted second vertical coordinate value, and updating the first coordinate and the second coordinate according to the adjustment result includes:

[0031] The first coordinate and the second coordinate are updated according to the adjusted first coordinate value and the adjusted second coordinate value, wherein the first coordinate value is the coordinate value with the smallest coordinate adjustment amount among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value, and the second coordinate value is the other coordinate value among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value except the first coordinate value.

[0032] Thus, in the embodiment of the present application, the first coordinate and the second coordinate may be updated according to the adjusted first coordinate value and the adjusted second coordinate value.

[0033] In certain embodiments of the present application, determining the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera includes:

[0034] The camera parameters are determined according to the second position information and fourth position information of the calibration object in the camera coordinate system.

[0035] Thus, in the embodiment of the present application, the camera parameters may be determined according to the second position information of the calibration object in the world coordinate system and the fourth position information of the calibration object in the camera coordinate system.

[0036] In certain embodiments of the present application, the method further comprises:

[0037] Acquire a calibration object image obtained by photographing the calibration object with a vehicle camera;

[0038] The fourth position information is determined according to the calibration object image.

[0039] In this way, in the embodiment of the present application, a calibration object image obtained by photographing the calibration object with a vehicle camera can be obtained, and the fourth position information of the calibration object in the camera coordinate system can be determined based on the calibration object image.

[0040] In some embodiments of the present application, the calibration object includes a first number of feature points, and determining the fourth position information according to the calibration object image includes:

[0041] Determining a second number of feature points in the calibration object image according to the calibration object image;

[0042] In a case where the second number matches the first number, the fourth position information is determined according to the image coordinates of each feature point in the calibration object image.

[0043] Thus, in an embodiment of the present application, the second number of feature points in the calibration object image can be determined based on the calibration object image, and when the second number matches the first number, the fourth position information can be determined based on the image coordinates of each feature point in the calibration object image.

[0044] In some embodiments of the present application, determining the second number of feature points in the calibration object image according to the calibration object image includes:

[0045] According to a preset binarization threshold, the calibration object image is binarized to determine a processed image;

[0046] The second number of the feature points in the processed image is determined according to the processed image.

[0047] Thus, in the embodiment of the present application, the calibration object image is binarized according to a preset binarization threshold to determine a processed image, and the second number of feature points in the processed image is determined according to the processed image.

[0048] In certain embodiments of the present application, the binarization processing of the calibration object image according to a preset binarization threshold value to determine the processed image includes:

[0049] In the case where the second number is smaller than the first number, the binarization threshold is increased to determine a new binarization threshold;

[0050] The binarization process is performed on the calibration object image according to the new binarization threshold to determine the processed image.

[0051] Thus, in an embodiment of the present application, when the second number is less than the first number of matches, the binarization threshold is increased to determine a new binarization threshold, and the calibrated object image is binarized according to the new binarization threshold to determine the processed image until the second number of feature points can be detected.

[0052] In certain embodiments of the present application, the binarization processing of the calibration object image according to a preset binarization threshold value to determine the processed image includes:

[0053] According to the binarization threshold, the calibration object image is binarized to determine a binarized image;

[0054] Performing morphological processing on the binarized image to determine the processed image, wherein the morphological processing includes dilation and / or erosion.

[0055] Thus, in the embodiment of the present application, the calibration object image may be binarized according to the binarization threshold to determine a binarized image, and the binarized image may be morphologically processed to determine a processed image.

[0056] In some embodiments of the present application, the second position information includes feature point coordinates of feature points in the calibration object, the camera parameters include internal parameters and distortion parameters, and determining the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera includes:

[0057] The intrinsic parameter and the distortion parameter are determined according to the feature point coordinates and the coordinates of the feature point in the camera coordinate system, wherein the coordinates in the camera coordinate system are determined according to the calibration object image obtained by photographing the calibration object by the vehicle camera.

[0058] Thus, in the embodiment of the present application, the intrinsic parameters and distortion parameters can be determined according to the coordinates of the feature points in the world coordinate system and the coordinates of the feature points in the camera coordinate system.

[0059] In some embodiments of the present application, the camera parameters include external parameters, and determining the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera includes:

[0060] The external parameter is determined according to the internal parameter, the feature point coordinates and the coordinates of the feature point in a pixel coordinate system, wherein the coordinates in the pixel coordinate system are determined according to the calibration object image.

[0061] Thus, in the embodiment of the present application, the camera extrinsic parameters can be determined according to the intrinsic parameters, the coordinates of the feature points, and the coordinates of the feature points in the pixel coordinate system.

[0062] In certain embodiments of the present application, the method further comprises:

[0063] A first calibration error of the camera parameters is determined based on a projection of the fifth position information of the calibration object on the world coordinate system, wherein the fifth position information is used to indicate the position of the calibration object in a pixel coordinate system, and the pixel coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with the vehicle camera.

[0064] Thus, in the implementation manner of the present application, the first calibration error of the camera parameters can be determined according to the projection of the fifth position information of the calibration object in the world coordinate system, thereby determining the calibration accuracy of the camera parameters.

[0065] In certain embodiments of the present application, the method further comprises:

[0066] A second calibration error of the camera parameters is determined based on a projection of the sixth position information of the calibration object in a pixel coordinate system, wherein the sixth position information is used to indicate the position of the calibration object in the world coordinate system, and the pixel coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with the vehicle camera.

[0067] Thus, in the embodiment of the present application, the second calibration error of the camera parameters may be determined according to the projection of the sixth position information of the calibration object in the pixel coordinate system.

[0068] In certain embodiments of the present application, the method further comprises:

[0069] A third calibration error of the camera parameters is determined by mapping the world coordinate system to the image coordinate system, wherein the image coordinate system is determined based on a calibration object image obtained by photographing the calibration object with the vehicle camera.

[0070] Thus, in the implementation manner of the present application, the third calibration error of the camera parameters can be determined according to the mapping of the world coordinate system to the image coordinate system.

[0071] The embodiment of the present application provides a control device, including a transceiver unit and a processing unit;

[0072] The transceiver unit is configured to obtain first position information of the vehicle, wherein the first position information is used to indicate the position of the light emitted by the light emitter relative to the vehicle;

[0073] The processing unit is configured to determine a world coordinate system according to the first position information, and determine camera parameters of the vehicle camera according to second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera.

[0074] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned camera calibration method is implemented.

[0075] An embodiment of the present application provides an electronic device, including the above-mentioned control device or the above-mentioned electronic device.

[0076] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by one or more processors, the above-mentioned camera calibration method is implemented.

[0077] An embodiment of the present application provides a computer program product, including a computer program / instruction, which implements the above-mentioned camera calibration method when executed by a processor.

[0078] The electronic device, control device, electronic device, computer-readable storage medium and computer program product provided in the embodiments of the present application enable the construction of a world coordinate system to be completed with the help of light emitted by a light emitter. Compared with the method of constructing an additional hardware platform to construct a world coordinate system, the construction difficulty and construction cost are reduced, and the construction process is simpler and more convenient, thereby improving the construction efficiency of the world coordinate system and the calibration efficiency of the vehicle camera. In addition, since the world coordinate system is constructed based on the first position information of the light relative to the vehicle, the reliability of the world coordinate system can be guaranteed to a certain extent, and the calibration accuracy of the camera parameters can also be guaranteed.

[0079] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0081] Figure 1 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0082] Figure 2 A schematic diagram of application scenarios in certain embodiments of the present application;

[0083] Figure 3 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0084] Figure 4 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0085] Figure 5 A schematic diagram of application scenarios in certain embodiments of the present application;

[0086] Figure 6 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0087] Figure 7 A schematic diagram of a calibration object in certain embodiments of the present application;

[0088] Figure 8 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0089] Fig. 9 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0090] Fig.10 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0091] Fig.11 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0092] Fig.12 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0093] Fig.13 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0094] Fig.14 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0095] Fig.15 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0096] Fig.16 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0097] Fig.17 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0098] Fig.18 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0099] Fig.19 A schematic diagram of application scenarios in certain embodiments of the present application;

[0100] Fig. 20 A schematic diagram of a camera calibration method in some embodiments of the present application;

[0101] Fig.21 This is a schematic diagram of an interface of a computer program product in certain embodiments of the present application;

[0102] Fig. 22 This is a schematic diagram of an interface of a computer program product in certain embodiments of the present application;

[0103] Fig.23 A schematic diagram of an application scenario in certain embodiments of the present application. DETAILED DESCRIPTION

[0104] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0105] In related technologies, vehicle cameras are one of the important components for vehicles to perceive the external environment. They can be used to collect images of the vehicle's surroundings, so that the vehicle can analyze the images to obtain target information in the external environment, such as pedestrian positions, lane markings, etc., and change its own state based on these target information, and can also remind the driver to adjust the driving behavior. However, since the various targets in the images taken by the vehicle camera are described in the camera coordinate system, the image analysis results are also based on the camera coordinate system representation. Therefore, the image analysis results cannot accurately express the relative position relationship between the target and the vehicle in the real world, resulting in inaccurate perception of the vehicle's external environment.

[0106] Furthermore, in order to enable the vehicle to accurately perceive the external environment, the vehicle camera is usually calibrated in related technologies to determine the spatial relationship between the camera and the vehicle, so as to obtain the spatial relationship between the target and the vehicle in the image, thereby achieving more accurate environmental perception. Therefore, on-board camera calibration is a basic and important process for the vehicle to perceive the external environment.

[0107] The essence of camera calibration is to obtain the conversion relationship from the image coordinate system to the vehicle coordinate system, so as to convert the target in the image to the vehicle coordinate system, and thus obtain the spatial relationship between the target and the vehicle in the real world. In order to accurately obtain this conversion relationship, related technologies usually use targets with known features to assist in camera calibration, such as camera calibration based on the corner features of a checkerboard target, camera calibration based on the circular features provided by a ring target, etc.

[0108] Furthermore, in the actual calibration process, the coordinate values ​​of multiple sets of target features in the image coordinate system and the vehicle coordinate system can be used to solve the transformation relationship from the image coordinate system to the vehicle coordinate system using optimization methods such as the least squares method to achieve camera calibration. In this process, the accuracy of obtaining the coordinate values ​​of the target features will greatly affect the accuracy of solving the above coordinate transformation relationship and affect the accuracy of calibration. In order to accurately obtain the coordinate values ​​of the target features, the relevant technology proposes a variety of different solutions to accurately measure the feature coordinates of the target, thereby achieving fast and accurate calibration.

[0109] For example, in a scheme for measuring a target, a checkerboard calibration plate can be used as a calibration target to assist camera calibration. After placing the calibration plate in the field of view of the camera, an image of the calibration plate is collected, and the coordinates of the corner points of the calibration plate in the image are obtained by image processing, and then the conversion relationship from the image coordinate system to the calibration plate coordinate system is calculated. Then, the three-dimensional coordinates of n points on the calibration plate in the theodolite coordinates are measured using a dual-theodolite three-dimensional coordinate measurement system, and the pose relationship between the calibration plate coordinate system and the theodolite coordinates is solved, thereby obtaining the pose relationship between the two calibration plates. Finally, the pose relationship between the two cameras is calculated using the pose relationship between the camera and the calibration plate measured by the camera, and the pose relationship between the two calibration plates calibrated by the theodolite, and the camera calibration is completed. However, due to the high price of the dual-theodolite three-dimensional coordinate measurement system, the implementation cost of this scheme is relatively high, and this scheme requires the calibration plate coordinate system to be used as an intermediate coordinate system to participate in the calibration process, so that three coordinate system transformations are required during the calibration process, resulting in complicated calculations and low efficiency.

[0110] For another example, in another scheme for measuring the target, the calibration of the vehicle camera can be carried out by building a calibration system including a test platform, a calibration device and a positioning device. Specifically, first, the position of the origin of the vehicle coordinate system is determined based on the positioning device, and the conversion relationship between the vehicle coordinate system and the test platform coordinate system is determined. Then, the calibration device moves the calibration target to a preset position in the test platform, and reads the three-dimensional coordinates of the calibration plate at the preset position in the vehicle coordinate system. Then, the vehicle camera is controlled to capture the image of the calibration target, and the image coordinates of the calibration target features in the image are obtained through image processing. Finally, the conversion relationship between the image coordinate system and the vehicle coordinate system is calculated through the vehicle coordinates and image coordinates of the target features, and the calibration of the vehicle camera is completed. However, this scheme requires the pre-construction of a calibration system, so there are situations where the execution is complex and the construction cost is high, and the calibration system itself cannot be moved, so it is difficult to flexibly complete the task of calibrating the vehicle camera.

[0111] For another example, in another scheme for measuring targets, the vehicle coordinate system is first determined by fitting with a high-precision positioning device, and then the three-dimensional coordinates of the calibration plate in the vehicle coordinate system are determined by the device. The image of the calibration plate is then captured by the vehicle camera, and the image coordinates of the calibration plate are obtained after image processing. Finally, the vehicle three-dimensional coordinates and image coordinates of the calibration plate are used to solve the conversion relationship from the image coordinate system to the vehicle coordinate system, and the calibration of the vehicle camera is completed. However, this scheme requires expensive high-precision positioning equipment on the one hand, and also requires the road where the vehicle is parked to have a certain slope on the other hand, which restricts the practical application of the scheme.

[0112] Based on the above problems you may encounter, please refer to Figure 1 , the present application embodiment provides a camera calibration method, including:

[0113] 01: Acquire first position information, wherein the first position information is used to indicate the position of the light emitted by the light emitter relative to the vehicle;

[0114] 02: Determine the world coordinate system according to the first position information;

[0115] 03: According to the second position information of the calibration object in the world coordinate system, determine the camera parameters of the vehicle camera to complete the calibration of the vehicle camera, wherein the camera parameters are used to indicate the mapping of the object in the image captured by the vehicle camera in the world coordinate system.

[0116] The embodiment of the present application provides a camera calibration device. The camera calibration method of the embodiment of the present application can be implemented by the camera calibration device of the embodiment of the present application. Specifically, the camera calibration device includes a transceiver unit and a processing unit. The transceiver unit is used to obtain first position information, and the first position information is used to indicate the position of the light emitted by the light emitter relative to the vehicle. The processing unit is used to determine the world coordinate system according to the first position information, and to determine the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera, and the camera parameters are used to indicate the mapping of the object in the image captured by the vehicle camera in the world coordinate system.

[0117] The embodiment of the present application also provides an electronic device, which includes a memory and a processor. The camera calibration method of the embodiment of the present application can be implemented by the electronic device of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain first position information, and to determine a world coordinate system based on the first position information, and to determine the camera parameters of the vehicle camera based on the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera, the first position information is used to indicate the position of the light emitted by the light emitter relative to the vehicle, and the camera parameters are used to indicate the mapping of the object in the image captured by the vehicle camera in the world coordinate system.

[0118] Specifically, in the embodiment of the present application, after one or more light emitters are arranged near the vehicle, the electronic device can obtain the first position information of the light emitted by the light emitter relative to the vehicle. Then, the electronic device constructs a world coordinate system based on the first position information. Finally, the electronic device calibrates the internal parameters, external parameters and other parameters of the vehicle camera based on the position of the calibration object in the world coordinate system.

[0119] In this way, in the implementation mode of the present application, the construction of the world coordinate system can be completed with the help of the light emitted by the light emitter. Compared with the method of constructing an additional hardware platform to construct the world coordinate system, the construction difficulty and construction cost are reduced, and the construction process is simpler and more convenient, thereby improving the construction efficiency of the world coordinate system and the calibration efficiency of the vehicle camera. In addition, since the world coordinate system is constructed based on the first position information of the light relative to the vehicle, the reliability of the world coordinate system can be guaranteed to a certain extent, and the calibration accuracy of the camera parameters can also be guaranteed.

[0120] In one example, the electronic device may refer to a device that can be connected to the vehicle and is independent of the vehicle, such as a tablet computer.

[0121] In one example, the electronic device is one of the devices integrated inside the vehicle, such as a domain controller

[0122] In one example, the light emitter is a cross laser capable of emitting laser light in a "cross" pattern.

[0123] In one example, the light emitters include multiple, such as four.

[0124] In one example, the light emitters are four cross lasers, and the four cross lasers can emit lasers together to present a rectangular frame that can wrap around the vehicle on the ground where the vehicle is located. Figure 2 , Figure 2 This is a schematic diagram of application scenarios in certain embodiments of the present application. It should be noted that: Figure 2 The dashed line in the figure represents the laser.

[0125] In one example, the light emitted by the laser emitter includes a first light, a second light, a third light, and a fourth light located in the same plane. Figure 2 The 4 dotted lines in .

[0126] Furthermore, in one example, the first light ray is tangent to the first front wheel and the first rear wheel, the second light ray is tangent to the second front wheel and the second rear wheel, the third light ray is perpendicular to the first light ray and the second light ray, the fourth light ray is perpendicular to the first light ray and the second light ray, the first front wheel and the first rear wheel are located on one side of the vehicle, and the second front wheel and the second rear wheel are located on the other side of the vehicle. Specifically, as Figure 2 As shown in FIG. 1 , one of the two dotted lines, “the dotted line located on the right side of the vehicle and including O2 and O3” and “the dotted line located on the left side of the vehicle and including O1 and O4”, is the first light ray mentioned above, and the other is the second light ray. Figure 2 Among them, one of the two dotted lines, "the dotted line located on the front side of the vehicle and including O4 and O3 at the same time" and "the dotted line located on the left side of the vehicle and including O1 and O2 at the same time", is the third light ray mentioned above, and the other is the fourth light ray mentioned above.

[0127] In one example, the origin of the world coordinate system is determined according to the position of the axis of the vehicle support shaft, the first coordinate axis of the world coordinate system is parallel to the first light ray and the second light ray, and the second coordinate axis of the world coordinate system is parallel to the third light ray and the fourth light ray. Figure 2 As shown, the origin of the world coordinate system is located at the axis of the rear axle of the vehicle, that is, point O. In addition, the first coordinate axis of the world coordinate system is the X-axis, and the X-axis is parallel to any one of the two dotted lines (i.e., the first light ray and the second light ray) of "the dotted line located on the right side of the vehicle and containing O2 and O3 at the same time" and "the dotted line located on the left side of the vehicle and containing O1 and O4 at the same time". In addition, the second coordinate axis of the world coordinate system is the Y-axis, and the Y-axis is parallel to any one of the two dotted lines (i.e., the third light ray and the fourth light ray) of "the dotted line located on the front side of the vehicle and containing O4 and O3 at the same time" and "the dotted line located on the left side of the vehicle and containing O1 and O2 at the same time".

[0128] For an example, see Figure 2 and Figure 3 , Figure 3 : is a flowchart of the camera calibration method in the implementation mode of the present application. That is, in the implementation mode of the present application, the process of constructing the world coordinate system may include:

[0129] like Figure 3 As shown in steps 21 and 22 in the figure, four tripods are placed at the four corners of the vehicle, and a cross laser is placed on each tripod, for a total of four cross lasers. Then, for each of the four cross lasers, the laser emission angle of the cross laser is adjusted so that one of the rays emitted by the cross laser is tangent to the front and rear wheels of the vehicle at the same time, and the other ray is perpendicular to the previous ray.

[0130] Furthermore, after the emission of the four cross lasers is adjusted, the two rays emitted by each cross laser satisfy "one ray is tangent to the front and rear wheels of the vehicle at the same time, and the other ray is perpendicular to the previous ray", so as to present the following Figure 2 The four rays shown (i.e. Figure 2 ), a laser rangefinder can be used to measure the distance of a fixed point in the vehicle relative to the four light rays, thereby obtaining the first position information. Figure 2 In the example shown, the distances from the axis of the rear axle of the vehicle relative to the four light rays can be measured by a laser rangefinder, so that in the subsequent process, corresponding calibration steps can be performed according to the four distance values ​​a, b, c, and d.

[0131] It is worth noting that in the process of obtaining the first position information of the light relative to the vehicle, the light emitter may shake unexpectedly, resulting in the need to readjust the placement angle of the light emitter or the light emission angle. Therefore, in the implementation mode of the present application, after the light emitter is placed near the vehicle and the adjustment of the light emitter is completed, a thin wire or other object can be used to place the light along the cross laser, and the thin wire can be fixed after overlapping with each light beam, and the "position of the light relative to the vehicle" can be determined by the "position of the thin wire relative to the vehicle".

[0132] Therefore, if Figure 3 As shown in step 23 of , four cross lasers can be placed at the four corners of the vehicle, and when the two light rays emitted by each cross laser satisfy "one light ray is tangent to the front and rear wheels of the vehicle at the same time, and the other light ray is perpendicular to the previous light ray", a thin wire can be used to place and fix along the light rays of the above cross lasers.

[0133] Finally, if Figure 3 As shown in step 24, the horizontal distance and vertical distance from the thin line parallel and perpendicular to the rear axle of the vehicle to the axis of the rear axle of the vehicle are measured by the laser rangefinder, and four distances a, b, c, and d can be obtained. Finally, the corresponding calibration steps can be performed according to the four distance values ​​a, b, c, and d, such as measuring the position of the calibration object relative to the vehicle in the world coordinate system through the four distance values ​​a, b, c, and d.

[0134] It is understandable that based on Figure 4 As shown in the example, the embodiments of the present application can utilize a cross laser, a laser rangefinder, and a calibration object to calibrate multiple vehicle-mounted cameras on an ordinary open road, thereby quickly constructing a vehicle coordinate system and a reference coordinate system, and realizing direct measurement of the three-dimensional coordinates of the calibration object in the vehicle coordinate system, thereby reducing the cost of camera calibration and the requirements for the site for camera calibration, thereby realizing low-cost and highly flexible calibration of vehicle cameras.

[0135] See also Figure 4 In some embodiments of the present application, the light rays include at least two light rays having an intersection, the first position information includes intersection position information of the intersection, and further, the camera calibration method further includes:

[0136] 04: Determine the second position information according to the intersection position information and the third position information of the grounding point of the calibration object relative to the intersection.

[0137] The processing unit of the embodiment of the present application is configured to determine the second position information based on the intersection position information and the third position information of the grounding point of the calibration object relative to the intersection.

[0138] The processor of the embodiment of the present application is also used to determine the second position information based on the intersection position information and the third position information of the grounding point of the calibration object relative to the intersection.

[0139] Specifically, in the implementation manner of the present application, the position of the calibration object in the world coordinate system can be determined according to the position of the grounding point of the calibration object relative to the known point in the world coordinate system (ie, the intersection position information).

[0140] In one example, the second position information of the calibration object in the world coordinate system may be determined by two grounding points that are farthest from each other among all grounding points of the calibration object.

[0141] In an example, please refer to Figure 2 and Figure 5 , Figure 5 Schematic diagram of application scenarios in certain embodiments of the present application. That is, in the embodiments of the present application, two grounding points of the calibration object (let the two grounding points be p1 and p2) can be measured by a laser rangefinder relative to O1 ('O1' is not Figure 5 The lateral distance and longitudinal distance of the two grounding points (marked in), that is, x1, x1, y1 and y1, are obtained to obtain the third position information of the two grounding points relative to O1, that is, p1(x1,y1) and p2(x2,y2).

[0142] Furthermore, since the position of O1 is known, that is, (b, -a), the positions of p1 and p2 in the world coordinate system can be determined according to the position of O1 and the positions of p1 and p2 relative to O1 (x1, y1) and (x2, y2), that is, (x1+b, y1-a) and (x2+b, y2-a), thereby characterizing the position of the calibration object in the world coordinate system.

[0143] Thus, in the embodiment of the present application, the second position information of the calibration object in the world coordinate system can be determined according to the intersection position information and the third position information of the grounding point of the calibration object relative to the intersection.

[0144] See also Figure 6 In some embodiments of the present application, the calibration object includes a plurality of feature points set with preset parameters, the second position information includes the coordinates of the feature points, and step 04 includes:

[0145] 040: Determine the grounding point coordinates of the grounding point in the world coordinate system according to the third position information and the intersection position information;

[0146] 041: Determine the feature point coordinates in the world coordinate system based on the ground point coordinates and preset parameters.

[0147] The processing unit of the embodiment of the present application is configured to determine the grounding point coordinates of the grounding point in the world coordinate system based on the third position information and the intersection position information, and to determine the feature point coordinates of the feature point in the world coordinate system based on the grounding point coordinates and preset parameters.

[0148] The processor of the implementation mode of the present application is also used to determine the grounding point coordinates of the grounding point in the world coordinate system based on the third position information and the intersection position information, and to determine the feature point coordinates of the feature point in the world coordinate system based on the grounding point coordinates and preset parameters.

[0149] Specifically, in order to ensure reliable calibration of camera parameters, in the implementation of the present application, the feature point coordinates of multiple feature points on the calibration object in the world coordinate system can be determined, and the calibration of the vehicle camera can be completed based on the feature point coordinates of each feature point.

[0150] In one example, the calibration plate and the object in the calibration plate can refer to Figure 7 , Figure 7 Schematic diagram of the calibration object in some embodiments of the present application. That is, in the embodiments of the present application, the calibration object is a calibration plate with a size of 1.2m×1.2m and circular targets (i.e., feature points). The circular targets on the calibration plate are arranged in 5 rows and 6 columns, i.e., 30 in total. The distance from the center point of the circular target in the first column to the left edge of the calibration plate is 0.1m, the distance from the center point of the circular target in the sixth column to the right edge of the calibration plate is 0.1m, the distance from the center point of the circular target in the first row to the upper edge of the calibration plate is 0.2m, the distance from the center point of the circular target in the fifth row to the lower edge of the calibration plate is 0.2m, and the horizontal and vertical distance between the center points of any two adjacent circular targets is 0.2m.

[0151] In one example, there are two grounding points, and the position of each circular target point in the world coordinate system can be determined by the following formula, namely:

[0152]

[0153] In the formula, point i,jrepresents the three-dimensional coordinates of the circular target located in the i-th row and j-th column in the world coordinate system, x1 o and y1 o Respectively represent the horizontal and vertical coordinates of a ground point in the world coordinate system, x2 o and y2 o Respectively represent the horizontal and vertical coordinates of another ground point in the world coordinate system. z Indicates the height of the vehicle's rear axle.

[0154] In this way, in the implementation mode of the present application, the grounding point coordinates of the grounding point in the world coordinate system can be determined according to the third position information and the intersection position information, and the feature point coordinates of the feature point in the world coordinate system can be determined according to the grounding point coordinates and preset parameters, so that the calibration of the camera parameters can be determined based on the feature point coordinates of each feature point, thereby ensuring the reliability of the camera parameter calibration results.

[0155] See also Figure 8 In some embodiments of the present application, the grounding point includes a first grounding point and a second grounding point, and the third position information includes a first coordinate of the first grounding point and a second coordinate of the second grounding point. Furthermore, the camera calibration method further includes:

[0156] 05: Update the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object.

[0157] The processing unit of the embodiment of the present application is configured to update the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object.

[0158] The processor of the embodiment of the present application is also used to update the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate, and the size parameter of the calibration object.

[0159] Specifically, in order to ensure the validity and reliability of the third position information, in the embodiment of the present application, the coordinate distance between the first coordinate and the second coordinate can be calculated based on the first coordinate of the first grounding point and the second coordinate of the second grounding point in the third position information. Then, according to the coordinate distance and the predetermined size parameters of the calibration object, the first coordinate and the second coordinate are updated, thereby ensuring the reliability of the first coordinate and the second coordinate. Furthermore, the coordinates of the feature point can be determined based on the updated first coordinate and the second coordinate, thereby ensuring the reliability of the coordinates of the feature point.

[0160] Thus, in the implementation mode of the present application, the first coordinate and the second coordinate can be updated according to the coordinate distance between the first coordinate and the second coordinate, and the size parameters of the calibration object, thereby improving the accuracy of the first coordinate, the second coordinate and the feature point coordinate.

[0161] See also Fig. 9 In certain embodiments of the present application, step 05 includes:

[0162] 050: When the difference between the coordinate distance and the size parameter is greater than a preset threshold, the first coordinate and the second coordinate are updated.

[0163] The processing unit of the embodiment of the present application is configured to update the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object.

[0164] The processor of the embodiment of the present application is also used to update the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object.

[0165] Specifically, in the implementation of the present application, the "coordinate difference (or distance) between the first coordinate and the second coordinate" can be used to represent the length and other dimensions of the calibration object. However, due to the existence of errors, the "coordinate difference (or distance) between the first coordinate and the second coordinate" may not match the actual length of the calibration object.

[0166] Therefore, in order to ensure the reliability of the first coordinate and the second coordinate, the electronic device can update the first coordinate and the second coordinate respectively when the difference between the "coordinate difference (or distance) between the first coordinate and the second coordinate" and the "length of the calibrated object (ie, size parameter)" is greater than a preset threshold (such as 0.02m), and determine the coordinates of the feature point based on the updated first coordinate and the second coordinate.

[0167] In one example, whether the first coordinate and the second coordinate can be updated respectively can be determined by the error value error, and the error value error can be determined by the following formula, that is:

[0168]

[0169] Wherein, x1 and y1 are the horizontal and vertical coordinates of the first coordinate, x2 and y2 are the horizontal and vertical coordinates of the second coordinate, and “1.2” is the length of the calibration object.

[0170] Thus, in the embodiment of the present application, the first coordinate and the second coordinate may be updated when the difference between the coordinate distance and the size parameter is greater than a preset threshold, thereby ensuring effective updating of the first coordinate and the second coordinate.

[0171] See also Fig.10 In some embodiments of the present application, the first coordinate includes a first abscissa and a first ordinate value, the second coordinate includes a second abscissa and a second ordinate value, and step 050 includes:

[0172] 0500: when the difference is greater than a preset threshold, adjusting the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value according to the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value;

[0173] 0501: Update the first coordinate and the second coordinate according to the adjustment result.

[0174] The processing unit of the embodiment of the present application is configured to adjust the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value according to the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value when the difference is greater than a preset threshold, and update the first coordinate and the second coordinate according to the adjustment result.

[0175] The processor of the embodiment of the present application is also used to adjust the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value according to the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value when the difference is greater than a preset threshold, and update the first coordinate and the second coordinate according to the adjustment result.

[0176] Specifically, in an embodiment of the present application, the electronic device may adjust the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate value, and the second vertical coordinate value of the first coordinate according to the first horizontal coordinate and the first vertical coordinate value of the first coordinate, and the second horizontal coordinate and the second vertical coordinate value of the second coordinate when the difference between the "coordinate difference (or distance) between the first coordinate and the second coordinate" and the "length of the calibrated object (i.e., size parameter)" is greater than a preset threshold value (such as 0.02m).

[0177] In one example, the adjustment of the second ordinate value can be expressed by the following two formulas, namely:

[0178]

[0179] Wherein, y2' is the second ordinate value after adjustment, x1 and y1 are the first abscissa value and the first ordinate value before adjustment, x2 and y2 are the second abscissa value and the second ordinate value before adjustment, and "1.2" is the length of the calibration object.

[0180] It can be understood that the first horizontal coordinate, the second horizontal coordinate and the update of the first horizontal coordinate can be calculated by similar formulas, and to avoid repetition, they are not described in detail here.

[0181] In this way, in an embodiment of the present application, the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value can be adjusted according to the first horizontal coordinate, the first vertical coordinate value of the first coordinate, and the second horizontal coordinate and the second vertical coordinate value of the second coordinate, thereby completing the update of the first coordinate and the second coordinate.

[0182] In some embodiments of the present application, the adjustment result includes an adjusted first horizontal coordinate value, an adjusted first vertical coordinate value, an adjusted second horizontal coordinate value, and an adjusted second vertical coordinate value. Therefore, step 0501 includes:

[0183] The first coordinate and the second coordinate are updated according to the adjusted first coordinate value and the adjusted second coordinate value, wherein the first coordinate value is the coordinate value with the smallest coordinate adjustment amount among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value, and the second vertical coordinate value, and the second coordinate value is the coordinate value other than the first coordinate value among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value, and the second vertical coordinate value.

[0184] The processing unit of the embodiment of the present application is configured to update the first coordinate and the second coordinate according to the adjusted first coordinate value and the adjusted second coordinate value, wherein the first coordinate value is the coordinate value with the smallest coordinate value adjustment amount among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value, and the second coordinate value is the coordinate value other than the first coordinate value among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value.

[0185] The processor of the embodiment of the present application is also used to update the first coordinate and the second coordinate according to the adjusted first coordinate value and the adjusted second coordinate value, wherein the first coordinate value is the coordinate value with the smallest coordinate adjustment amount among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value, and the second vertical coordinate value, and the second coordinate value is the coordinate value other than the first coordinate value among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value, and the second vertical coordinate value.

[0186] Specifically, after completing the adjustment of the four coordinate values ​​of the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value, the implementation mode of the present application can determine the updated first coordinate and second coordinate based on the first coordinate value with the smallest adjustment amplitude among the four coordinate values ​​and the other three coordinate values ​​among the four coordinate values ​​other than the first coordinate value.

[0187] For example, when the adjusted first horizontal coordinate value, first vertical coordinate value, second horizontal coordinate value and second vertical coordinate value are represented by x1, y1, x2, y2 respectively, and the adjusted first horizontal coordinate value, first vertical coordinate value, second horizontal coordinate value and second vertical coordinate value are represented by x1', y1', x2', y2' respectively, if |y2'-y2| is the smallest in the set {|x1'-x1|,|y1'-y1|,|x2'-x2|,|y2'-y2|}, y2' will be replaced by the original y2, so that the updated first coordinate is (x1, y1), and the updated first coordinate is (x2, y2').

[0188] Thus, in the embodiment of the present application, the first coordinate and the second coordinate may be updated according to the adjusted first coordinate value and the adjusted second coordinate value.

[0189] To more clearly explain the process of determining the feature point coordinates in the implementation mode of this application, please refer to Figure 2 , 5 , 7 and 11, Fig.11 : is a flowchart of a camera calibration method in some embodiments of the present application, that is, the process of determining the coordinates of feature points in the embodiments of the present application may include:

[0190] like Fig.11 As shown in step 31, Figure 5 The calibration plate shown is placed within the field of view of the vehicle camera. The calibration plate is a 1.2m×1.2m calibration plate with circular targets (i.e., feature points). There are 30 circular targets in 5 rows and 6 columns on the calibration plate. The distance from the center point of the circular target in the first column to the left edge of the calibration plate is 0.1m, the distance from the center point of the circular target in the sixth column to the right edge of the calibration plate is 0.1m, the distance from the center point of the circular target in the first row to the upper edge of the calibration plate is 0.2m, the distance from the center point of the circular target in the fifth row to the lower edge of the calibration plate is 0.2m, and the horizontal and vertical distance between the center points of any two adjacent circular targets is 0.2m.

[0191] like Fig.11 As shown in step 32, a laser rangefinder can be used to measure the point where the calibration plate touches the ground (grounding point) to the intersection point O1 of the reference line segment (O1 is not in Figure 5 The distance is shown in the figure), thereby measuring the coordinates of the lower left corner point p1 of the calibration plate as p1(x1,y1), and measuring the coordinates of the lower right corner point p2 as (x2,y2).

[0192] like Fig.11As shown in step 33, since the length of the calibration plate is known to be 1.2m, the coordinate difference between p1 and p2 can be calculated, thereby calculating whether the coordinate difference between p1 and p2 can normally point to the length of the calibration plate. Specifically, the measurement error error can be calculated by the coordinate difference between p1 and p2 and the length of the calibration plate. The measurement error error can be determined by the following formula, namely:

[0193]

[0194] like Fig.11 As shown in step 34, if the measurement error error is less than or equal to 0.002m, it can be considered that the measurement error error is small, and step 35 is executed. On the contrary, if the measurement error error is greater than 0.002m, the remaining coordinate value can be adjusted according to the three coordinate values ​​of x1, y1, x2 and y2 that have been measured. Taking y2 as an example, the adjustment process of y2 can be expressed by the following two formulas, namely:

[0195]

[0196] Wherein, y2' is the adjusted y2, x1 and y1 are the first abscissa value and the first ordinate value before adjustment, x2 and y2 are the second abscissa value and the second ordinate value before adjustment, and "1.2" is the length of the calibration object.

[0197] Furthermore, if |y2'-y2| is the smallest in the set {|x1'-x1|,|y1'-y1|,|x2'-x2|,|y2'-y2|}, y2' will be replaced with the original y2, so that the updated p1 is (x1, y1) and the updated p2 is (x2, y2').

[0198] like Fig.11 As shown in step 35, the three-dimensional coordinates of the feature points of the calibration plate in the vehicle coordinate system are determined according to the updated p1(x1,y1) and p2(x2,y2'). Specifically, firstly, each coordinate value in p1(x1,y1) and p2(x2,y2') is converted to the vehicle coordinate system (i.e., the world coordinate system) by the following formula, namely:

[0199] xo=xa

[0200] yo=y+b

[0201] In the formula, x o Represents the horizontal coordinate in the vehicle coordinate system (i.e., the world coordinate system), y o Represents the vertical coordinate in the vehicle coordinate system (ie, the world coordinate system).

[0202] Next, according to x1 in the vehicle coordinate system o,y1 o , x2 o ,y2 o , and the following formula, we can get the feature point point in the i-th row and j-th column of the calibration plate i,j The three-dimensional coordinates in the vehicle coordinate system are:

[0203]

[0204] Where x1 o and y1 o Respectively represent the horizontal and vertical coordinates of p1 in the world coordinate system, x2 o and y2 o Respectively represent the horizontal and vertical coordinates of p2 in the world coordinate system. z Indicates the height of the vehicle's rear axle.

[0205] It is understandable that based on Fig.11 As shown in the example, in the embodiments of the present application, by utilizing a laser rangefinder, a reference coordinate system, and the physical properties (dimensional parameters) of the calibration object, rapid calculation of the three-dimensional coordinates of feature points on the calibration object in the vehicle coordinate system can be achieved.

[0206] In certain embodiments of the present application, step 03 includes:

[0207] The camera parameters are determined according to the second position information and the fourth position information of the calibration object in the camera coordinate system.

[0208] The processing unit of the embodiment of the present application is configured to determine the camera parameters according to the second position information and the fourth position information of the calibration object in the camera coordinate system.

[0209] The processor of the embodiment of the present application is also used to determine the camera parameters according to the second position information and the fourth position information of the calibration object in the camera coordinate system.

[0210] Specifically, in the implementation mode of the present application, the conversion relationship from the world coordinate system to the image coordinate system in the image captured by the vehicle camera, that is, the camera parameters, can be determined according to the position of the calibration object in the world coordinate system and the position of the calibration object in the image coordinate system.

[0211] Thus, in the embodiment of the present application, the camera parameters may be determined according to the second position information of the calibration object in the world coordinate system and the fourth position information of the calibration object in the camera coordinate system.

[0212] It can be understood that camera calibration is a process of estimating camera parameters. The purpose is to determine the intrinsic and extrinsic parameters and distortion parameters of the camera to determine the mapping relationship between the three-dimensional coordinate points in the real world and their corresponding two-dimensional projection points in the image. The mapping relationship can be expressed by the following formula, namely:

[0213]

[0214] Where f is the focal length of the camera, R is the 3x3 rotation matrix, T is the 3×1 translation matrix, d x and d y They represent the physical size of each pixel on the horizontal axis x and the vertical axis y, the coordinates of the image center points u0 and v0 in pixel coordinates, the first term of the formula on the right side of the equal sign is the camera intrinsic parameter matrix, and the second term is the camera extrinsic parameter matrix.

[0215] For a clearer explanation of the camera calibration process, see Fig.12 , Fig.12 1 is a flow chart of a camera calibration method in certain embodiments of the present application. Fig.12 As shown in the figure, firstly, the point is projected from the world coordinates to the camera coordinates through the extrinsic matrix. This transformation is a rigid body transformation, and the object will not be deformed. It only needs to be rotated and translated to achieve it. Then, the mapping process from the camera coordinates to the image coordinates can be achieved using the triangle similarity principle. Finally, the mapping from the image coordinates to the pixel coordinates is mainly to convert the millimeter (mm) unit of the image coordinates into the pixel unit. The image coordinate system and the pixel coordinate system are in a translation relationship, so the transformation from the image coordinates to the pixel coordinates can be obtained by translation.

[0216] See also Fig.13 In some embodiments of the present application, the camera calibration method further includes:

[0217] 06: Obtain the calibration object image obtained by photographing the calibration object with the vehicle camera;

[0218] 07: Determine fourth position information according to the calibration object image.

[0219] The transceiver unit of the embodiment of the present application is configured to obtain a calibration object image obtained by photographing the calibration object by the vehicle camera. The +6+2 processing unit of the embodiment of the present application is configured to determine the fourth position information according to the calibration object image.

[0220] The processor of the embodiment of the present application is also used to obtain a calibration object image obtained by photographing the calibration object with a vehicle camera, and determine fourth position information based on the calibration object image.

[0221] Specifically, in the implementation manner of the present application, the electronic device can obtain or control the vehicle camera to shoot the calibration object to obtain an image, that is, the calibration object image. And the electronic device can determine the position of the calibration object in the calibration object image in the camera coordinate system and the image coordinate system, that is, the fourth position information, based on the calibration object image.

[0222] In this way, in the embodiment of the present application, a calibration object image obtained by photographing the calibration object with a vehicle camera can be obtained, and the fourth position information of the calibration object in the camera coordinate system can be determined based on the calibration object image.

[0223] See also Fig.14 In some embodiments of the present application, the calibration object includes a first number of feature points, and then step 07 includes:

[0224] 070: Determine a second number of feature points in the calibration object image according to the calibration object image;

[0225] 071: When the second number matches the first number, determine fourth position information according to the image coordinates of each feature point in the calibration object image.

[0226] The processing unit of the embodiment of the present application is configured to determine a second number of feature points in the calibration object image based on the calibration object image, and when the second number matches the first number, determine fourth position information based on the image coordinates of each feature point in the calibration object image.

[0227] The processor of the embodiment of the present application is also used to determine a second number of feature points in the calibration object image based on the calibration object image, and when the second number matches the first number, determine fourth position information based on the image coordinates of each feature point in the calibration object image.

[0228] Specifically, taking into account the impact of lighting factors on calibration, in an embodiment of the present application, after the vehicle captures an image of the calibration object and the electronic device detects the number of feature points in the image of the calibration object, it is determined whether the detected number of feature points (i.e., the second number) is the same as the actual number of feature points (i.e., the first number).

[0229] It can be understood that if the second number is the same as the first number, it indicates that the lighting conditions of the current environment are good, and thus the fourth position information of the marked object in the camera coordinate system can be determined according to the image coordinates of each feature point in the calibration object image.

[0230] Thus, in an embodiment of the present application, the second number of feature points in the calibration object image can be determined based on the calibration object image, and when the second number matches the first number, the fourth position information can be determined based on the image coordinates of each feature point in the calibration object image.

[0231] See also Fig.15 In some embodiments of the present application, step 070 includes:

[0232] 0700: According to the preset binarization threshold, the calibration object image is binarized to determine the processed image;

[0233] 0701: Determine a second number of feature points in the processed image according to the processed image.

[0234] The processing unit of the embodiment of the present application is configured to perform binarization processing on the calibration object image according to a preset binarization threshold, determine the processed image, and determine the second number of feature points in the processed image according to the processed image.

[0235] The processor of the embodiment of the present application is also used to perform binarization processing on the calibration object image according to a preset binarization threshold, determine the processed image, and determine the second number of feature points in the processed image according to the processed image.

[0236] Specifically, in the implementation mode of the present application, according to the preset binarization threshold, the pixels in the calibration object image whose grayscale values ​​are greater than or equal to the binarization threshold are set to white (or bright color), and the pixels whose grayscale values ​​are less than the binarization threshold are set to black (or dark color), so that each pixel in the image is mapped to black or white, thereby obtaining a processed image. Then, the processed image can be detected for feature points to detect the feature points of the processed image. For example, the feature points in the calibration object are as follows Figure 5 When the circular feature shown in the figure is detected, the center of the circular feature in the processed image can be detected.

[0237] It can be understood that compared with the method of directly detecting the number of feature points in the original image (i.e., the calibrated object image before binarization processing), the feature points in the processed image based on binarization processing are easier to detect, thereby improving the efficiency of feature point quantity detection.

[0238] Thus, in the embodiment of the present application, the calibration object image is binarized according to a preset binarization threshold to determine a processed image, and the second number of feature points in the processed image is determined according to the processed image.

[0239] See also Fig.16 In some embodiments of the present application, step 0700 includes:

[0240] 07000: when the second number is smaller than the first number, the binarization threshold is increased to determine a new binarization threshold;

[0241] 07001: According to the new binarization threshold, the calibration object image is binarized to determine the processed image.

[0242] The processing unit of the embodiment of the present application is configured to increase the binarization threshold when the second number is less than the first number, determine a new binarization threshold, and binarize the calibration object image according to the new binarization threshold to determine the processed image.

[0243] The processor of the embodiment of the present application is also used to increase the binarization threshold when the second number is less than the first number, determine a new binarization threshold, and binarize the calibration object image according to the new binarization threshold to determine the processed image.

[0244] Specifically, in an embodiment of the present application, if the electronic device can detect all feature points on the calibrated object, subsequent calibration steps can be performed based on the image coordinates of each detected feature point. Otherwise, the binarization threshold is increased (such as adding 1), and the calibrated object image is binarized according to the increased binarization threshold to obtain a new processed image, and the feature points in the processed image are detected again.

[0245] Thus, in an embodiment of the present application, when the second number is less than the first number of matches, the binarization threshold is increased to determine a new binarization threshold, and the calibrated object image is binarized according to the new binarization threshold to determine the processed image until the second number of feature points can be detected.

[0246] See also Fig.17 In some embodiments of the present application, step 0700 includes:

[0247] Step 07002: Binarize the calibration object image according to the binarization threshold value to determine the binarized image;

[0248] Step 07003: Perform morphological processing on the binarized image to determine the processed image, wherein the morphological processing includes dilation and / or erosion.

[0249] The processing unit of the embodiment of the present application is configured to perform binarization processing on the calibration object image according to a binarization threshold value to determine the binarized image, and perform morphological processing on the binarized image to determine the processed image, wherein the morphological processing includes dilation and / or erosion.

[0250] The processor of the embodiment of the present application is also used to binarize the calibration object image according to the binarization threshold to determine the binarized image, and perform morphological processing on the binarized image to determine the processed image, wherein the morphological processing includes dilation and / or erosion.

[0251] Specifically, in the embodiment of the present application, the electronic device can obtain the original image, that is, the calibration object image, and perform binarization processing on the calibration object image according to the binarization threshold to determine the binarized image. Then, the binarized image is subjected to dilation and erosion morphological processing to close the empty points in the image and disconnect the wrongly connected points in the image, thereby obtaining the processed image.

[0252] Thus, in the embodiment of the present application, the calibration object image may be binarized according to the binarization threshold to determine a binarized image, and the binarized image may be morphologically processed to determine a processed image.

[0253] In certain embodiments of this application, please refer to Figure 7 and Fig.18 , Fig.18 This is a flow chart of a vehicle calibration method in certain embodiments of the present application. Specifically,

[0254] like Fig.18 As shown in step 41, Figure 7 The calibration plate (ie, the calibration object) shown is placed within the field of view of the vehicle camera to be calibrated, and the electronic device can obtain an image of the calibration plate obtained by photographing the calibration plate with the vehicle camera.

[0255] like Fig.18 As shown in step 42, Gaussian blur is performed on the calibration plate image to remove noise in the image.

[0256] like Fig.18 As shown in step 43, the denoised calibration plate image is binarized.

[0257] like Fig.18 As shown in step 44, morphological operations such as corrosion and dilation are performed on the image to close the empty points in the binary image and disconnect the incorrectly connected points in the binary image.

[0258] like Fig.18 As shown in step 45, the binary image after the morphological operation is subjected to detection of the center of the circle.

[0259] like Fig.18 As shown in step 46, if it can be detected that Figure 7 If all the circle centers on the calibration plate shown (i.e., 30) are detected, the process ends; otherwise, the binarization threshold is increased by 1, and the process returns to step 43 until all the circle centers on the calibration plate can be detected.

[0260] It is understandable that based on Fig.18 In the example shown, the embodiment of the present application can use image processing operations such as low-pass filtering (i.e., Gaussian noise reduction), binarization, and morphological operations to reduce the impact of illumination changes on the accuracy of feature detection of the calibration plate image, thereby being able to detect feature points of the calibration plate image with a higher accuracy. Moreover, compared to the situation in the related art where the ambient illumination needs to be stable, the embodiment of the present application can achieve accurate automatic feature extraction of the calibration plate image under different illumination conditions, thereby being able to output the image coordinates corresponding to the feature points in the image, with better robustness.

[0261] In some embodiments of the present application, the second position information includes the coordinates of the feature points of the calibrated object, and the camera parameters include the internal parameters. Then, step 03 includes:

[0262] The intrinsic parameters and the distortion parameters are determined according to the coordinates of the feature points and the coordinates of the feature points in the camera coordinate system, wherein the coordinates in the camera coordinate system are determined according to the calibration object image obtained by photographing the calibration object with the vehicle camera.

[0263] The processing unit of the embodiment of the present application is also configured to determine the internal parameters and distortion parameters based on the feature point coordinates and the coordinates of the feature points in the camera coordinate system, wherein the coordinates in the camera coordinate system are determined based on the calibration object image obtained by photographing the calibration object with the vehicle camera.

[0264] The processor of the embodiment of the present application is also used to determine the internal parameters and distortion parameters based on the feature point coordinates and the coordinates of the feature points in the camera coordinate system, wherein the coordinates in the camera coordinate system are determined based on the calibration object image obtained by photographing the calibration object with the vehicle camera.

[0265] Specifically, in an embodiment of the present application, multiple images (such as 10 to 15 images) obtained by multiple shots of the calibration object by the vehicle camera can be obtained, and the feature points of the calibration plate in the image can be detected to obtain the coordinates of the feature points in the camera coordinate system.

[0266] Next, you can call a preset camera calibration interface such as cv2.calibrateCamera in opencv to determine the internal parameters. It can be understood that cv2.calibrateCamera uses a pinhole camera model for camera calibration.

[0267] It can also be understood that during the intrinsic parameter calibration process, the relative positions of the calibration plate feature points can be used to define the calibration plate feature points in the vehicle coordinate system, and the camera intrinsic parameters and distortion coefficients can be estimated by the least squares method to minimize the reprojection error between the image points and the object points.

[0268] In one example, in order to ensure the accuracy of the camera's intrinsic parameters, the calibration plate is required to cover the entire field of view of the camera as much as possible with various angles, and the calibration plate can be rotated and tilted to increase the diversity of feature points.

[0269] Thus, in the embodiment of the present application, the intrinsic parameters and distortion parameters can be determined according to the coordinates of the feature points in the world coordinate system and the coordinates of the feature points in the camera coordinate system.

[0270] In some embodiments of the present application, the camera parameters include external parameters, and step 03 includes:

[0271] The external parameters are determined according to the internal parameters, the coordinates of the feature points and the coordinates of the feature points in the pixel coordinate system, wherein the coordinates in the pixel coordinate system are determined according to the calibration object image.

[0272] The processing unit of the embodiment of the present application is configured to determine the external parameters based on the internal parameters, the coordinates of the feature points and the coordinates of the feature points in the pixel coordinate system, wherein the coordinates in the pixel coordinate system are determined based on the calibration object image.

[0273] The processor of the embodiment of the present application is also used to determine the external parameters based on the internal parameters, the coordinates of the feature points and the coordinates of the feature points in the pixel coordinate system, wherein the coordinates in the pixel coordinate system are determined based on the calibration object image.

[0274] Specifically, in the implementation of the present application, the absolute position relationship between the camera coordinate system and the world coordinate system, i.e., the absolute translation vector T and the rotation matrix R, can be solved based on at least three calibration object images captured by the vehicle camera and the coordinates of the feature points in each calibration object image in the world coordinate system, as well as the pixel coordinates of the feature points in each calibration object image in the pixel coordinate system, combined with the calibrated camera intrinsic parameters, and using the coordinates of multiple control points in the three-dimensional scene and their perspective projection coordinates in the image. Among them, the control point refers to the coordinates of the feature point in the pixel coordinate system and the world coordinate system.

[0275] It is understandable that the method of solving the absolute translation vector T and the rotation matrix R based on the control points can be called the PNP (Perspective-N-Point, N-point perspective pose solution) problem. And, for perspective projection, at least three sets of control points are required to make the PNP problem solvable.

[0276] At the same time, in order to make the PNP solution more accurate, the calibration objects can be distributed in the left, middle and right positions of the vehicle camera field of view to take three images, and ensure that the calibration plate is perpendicular to the ground to facilitate the generation of three-dimensional coordinates. In an example, the cv2.solvePnP interface implemented by OpenCV can be used to solve the camera external parameters.

[0277] Thus, in the embodiment of the present application, the camera extrinsic parameters can be determined according to the intrinsic parameters, the coordinates of the feature points, and the coordinates of the feature points in the pixel coordinate system.

[0278] In certain embodiments of the present application, the vehicle calibration method further includes:

[0279] A first calibration error of the camera parameters is determined based on a projection of the fifth position information of the calibration object in the world coordinate system, wherein the fifth position information is used to indicate the position of the calibration object in the pixel coordinate system, and the pixel coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with a vehicle camera.

[0280] The processing unit of the embodiment of the present application is configured to determine a first calibration error of the camera parameters based on the projection of the fifth position information of the calibration object in the world coordinate system, wherein the fifth position information is used to indicate the position of the calibration object in the pixel coordinate system, and the pixel coordinate system is determined based on the calibration object image obtained by photographing the calibration object by the vehicle camera.

[0281] The processor of the embodiment of the present application is also used to determine the first calibration error of the camera parameters based on the projection of the fifth position information of the calibration object in the world coordinate system, wherein the fifth position information is used to indicate the position of the calibration object in the pixel coordinate system, and the pixel coordinate system is determined based on the calibration object image obtained by photographing the calibration object by the vehicle camera.

[0282] Specifically, in an embodiment of the present application, the pixel coordinates of the calibrated object feature points can be projected into the world coordinates according to the calibrated camera intrinsic parameters and camera extrinsic parameters to obtain the projection coordinates points, and the average value of the L2 norm between the projection coordinates points and the real coordinates obj_points is calculated to obtain the reprojection error pro_error, which is the first calibration error.

[0283] In an example, the reprojection error pro_error can be calculated by the following formula:

[0284]

[0285] In the formula, points i,j Represents the projection coordinates of the circular target point located at the i-th row and j-th column, obj_point i,j represents the three-dimensional coordinates of the circular target located at the i-th row and j-th column in the pixel coordinate system, and N represents the number of feature points.

[0286] Thus, in the implementation manner of the present application, the first calibration error of the camera parameters can be determined according to the projection of the fifth position information of the calibration object in the world coordinate system, thereby determining the calibration accuracy of the camera parameters.

[0287] In certain embodiments of the present application, the vehicle calibration method further includes:

[0288] A second calibration error of the camera parameters is determined based on a projection of the sixth position information of the calibration object in the pixel coordinate system, wherein the sixth position information is used to indicate the position of the calibration object in the world coordinate system, and the pixel coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with a vehicle camera.

[0289] The processing unit of the embodiment of the present application is configured to determine the second calibration error of the camera parameters based on the projection of the sixth position information of the calibration object in the pixel coordinate system, wherein the sixth position information is used to indicate the position of the calibration object in the world coordinate system, and the pixel coordinate system is determined based on the calibration object image obtained by photographing the calibration object by the vehicle camera.

[0290] The processor of the embodiment of the present application is also used to determine the second calibration error of the camera parameters based on the projection of the sixth position information of the calibration object in the pixel coordinate system, wherein the sixth position information is used to indicate the position of the calibration object in the world coordinate system, and the pixel coordinate system is determined based on the calibration object image obtained by photographing the calibration object by the vehicle camera.

[0291] Specifically, in the implementation mode of the present application, the world coordinates of the feature points of the calibration object can be projected to the pixel coordinates according to the calibrated internal and external parameters to confirm whether they are projected onto the corresponding feature points of the calibration plate, thereby obtaining the projection error, that is, the second calibration error.

[0292] In one example, the projection points of the world coordinates of the feature points of the calibration object in the pixel coordinate system may be visualized.

[0293] Thus, in the embodiment of the present application, the second calibration error of the camera parameters may be determined according to the projection of the sixth position information of the calibration object in the pixel coordinate system.

[0294] In certain embodiments of the present application, the vehicle calibration method further includes:

[0295] A third calibration error of the camera parameters is determined by mapping the world coordinate system to the image coordinate system, wherein the image coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with a vehicle camera.

[0296] The processing unit of the embodiment of the present application is configured to determine the third calibration error of the camera parameters by mapping the world coordinate system to the image coordinate system, wherein the image coordinate system is determined based on the calibration object image obtained by photographing the calibration object with the vehicle camera.

[0297] The processor of the embodiment of the present application is also used to determine the third calibration error of the camera parameters by mapping the world coordinate system to the image coordinate system, wherein the image coordinate system is determined based on the calibration object image obtained by photographing the calibration object with the vehicle camera.

[0298] Specifically, in the implementation manner of the present application, the world (vehicle) coordinate system can be converted into the camera coordinate system, and the difference between the converted world (vehicle) coordinate system and the camera coordinate system is detected to obtain the third calibration error.

[0299] In one example, the process of converting the world coordinate system to the camera coordinate system can be determined by the following formula, namely:

[0300]

[0301] In the formula, the left side of the equal sign is the camera coordinate system, the first term on the right side of the equal sign is the external parameter, and the second term on the right side of the equal sign is the world coordinate system.

[0302] In one example, both the camera coordinate system and the transformed world (vehicle) coordinate system can be visualized, as shown in FIG. Fig.19 As shown, Fig.19 A schematic diagram of an application scenario in certain embodiments of the present application.

[0303] Thus, in the implementation manner of the present application, the third calibration error of the camera parameters can be determined according to the mapping of the world coordinate system to the image coordinate system.

[0304] In an example, see Fig. 20 , Fig. 20 FIG. 1 is a flow chart of a vehicle calibration method in certain embodiments of the present application. Specifically, Fig. 20 As shown in steps 41-47 in the embodiment of the present application, the electronic device may calculate the reprojection error (first calibration error), the projection error (second calibration error) and the like. Fig.19 The third calibration error shown in FIG. 1 may be greater than 0.002 in the case of a reprojection error, or a significant projection deviation, or Fig.19 The third calibration error shown is significantly offset, so it can be considered that the error of the calibration result is relatively large.

[0305] Furthermore, if the error is large, the calibration object (i.e., the coordinates of the feature points in the world coordinate system, corresponding to Fig.11 ) is wrong, or check the calibration plate feature point detection (i.e. the image coordinate detection result of the feature point, corresponding to Fig.18 ) is correct, or check whether the data corresponds when solving PnP, so as to solve the problem of large calibration error.

[0306] An embodiment of the present application further provides an electronic device, including the above-mentioned control device or including the above-mentioned electronic device.

[0307] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned camera calibration method is implemented.

[0308] The embodiment of the present application also provides a computer program product, including a computer program / instruction, which implements the above-mentioned camera calibration method when executed by a processor.

[0309] In an example, see Fig.21 and Fig. 22 , Fig.21 and Fig. 22 All of them are interface diagrams of computer program products in certain embodiments of the present application. Fig.21 is the main interface of the computer program product, Fig. 22 It is the help interface of a computer program product, used to display help documents.

[0310] In one example, the computer program product can automatically calculate the three-dimensional coordinates of the feature points of the calibration plate in the world coordinate system according to the measurement data in response to the user operation, and automatically detect the pixel coordinates of the feature points of the calibration plate in the captured image, and use the detected pixel coordinates of the feature points of the calibration plate to calibrate the camera intrinsic parameters, and can use the three-dimensional coordinates and pixel coordinates of the feature points of the calibration plate to calibrate the camera extrinsic parameters, and calculate the reprojection error, the visualization projection result and the visualization calibration extrinsic parameters at the same time. In addition, the computer program product can also export the calibrated intrinsic and extrinsic parameters and save them in a txt format file.

[0311] To more clearly illustrate the use of the computer program product in the embodiments of the present application, please refer to Fig.23 , Fig.23 Schematic diagram of application scenarios in certain embodiments of the present application. Specifically, Fig.23 As shown, the user first clicks the Select File or Select Image button to select the image folder or single image with the calibration plate. After the system loads the image, it will automatically adjust the binarization threshold, detect the feature points of the calibration plate in the image, and draw the detected feature points on the image, which will be displayed in a list on the right interface.

[0312] Next, click the Intrinsic Calibration button to perform internal parameter calibration and display the calibrated internal parameter matrix and distortion coefficients.

[0313] Next, click on the image in the list on the right to select it, fill in the corresponding calibration plate measurement value in the corresponding position, and then click Save. x1 and y1 are the measurement data of the lower left corner of the calibration plate in the captured image relative to the center point of the vehicle, x2 and y2 are the measurement data of the lower right corner of the calibration plate in the captured image relative to the center point of the vehicle, and z is the height of the center point of the rear axle of the vehicle.

[0314] Then, repeat the previous step until all measurement data are entered and saved.

[0315] Then, click the internal parameter calibration button to perform internal parameter calibration, and the calibration results and reprojection error will be displayed in the interface.

[0316] Finally, after calibrating the camera's internal and external parameters, you can click the Visualize External Parameters button to display the relative position of the camera to the vehicle in three-dimensional space. You can also click the Export Calibration Results button, select a save address, and save the calibration results as a txt file. This completes the calibration.

[0317] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0318] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0319] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A camera calibration method, characterized in that: include: Acquiring first position information, wherein the first position information is used to indicate a position of the light emitted by the light emitter relative to the vehicle; Determine a world coordinate system according to the first position information; According to the second position information of the calibration object in the world coordinate system, the camera parameters of the vehicle camera are determined to complete the calibration of the vehicle camera, wherein the camera parameters are used to indicate the mapping of the object in the image captured by the vehicle camera in the world coordinate system.

2. The method according to claim 1, characterized in that The light rays include a first light ray, a second light ray, a third light ray and a fourth light ray located in the same plane.

3. The method according to claim 2, characterized in that The first light ray is tangent to the first front wheel and the first rear wheel, the second light ray is tangent to the second front wheel and the second rear wheel, the third light ray is perpendicular to the first light ray and the second light ray, the fourth light ray is perpendicular to the first light ray and the second light ray, the first front wheel and the first rear wheel are located on one side of the vehicle, and the second front wheel and the second rear wheel are located on the other side of the vehicle.

4. The method according to claim 3, characterized in that The origin of the world coordinate system is determined according to the position of the axis of the vehicle support shaft, the first coordinate axis of the world coordinate system is parallel to the first light ray and the second light ray, and the second coordinate axis of the world coordinate system is parallel to the third light ray and the fourth light ray.

5. The method according to claim 1, characterized in that The light rays include at least two light rays having an intersection, the first position information includes intersection position information of the intersection, and the method further includes: The second position information is determined according to the intersection position information and third position information of the grounding point of the calibration object relative to the intersection.

6. The method according to claim 5, characterized in that The calibration object includes a plurality of feature points set with preset parameters, the second position information includes coordinates of the feature points, and determining the second position information according to the intersection position information and third position information of the grounding point of the calibration object relative to the intersection includes: Determine the grounding point coordinates of the grounding point in the world coordinate system according to the third position information and the intersection position information; The feature point coordinates of the feature point in the world coordinate system are determined according to the ground point coordinates and the preset parameters.

7. The method according to claim 5, characterized in that The grounding point includes a first grounding point and a second grounding point, the third position information includes a first coordinate of the first grounding point and a second coordinate of the second grounding point, and the method further includes: The first coordinate and the second coordinate are updated according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object.

8. The method according to claim 7, characterized in that The updating of the first coordinate and the second coordinate according to the coordinate distance between the first coordinate and the second coordinate and the size parameter of the calibration object includes: When the difference between the coordinate distance and the size parameter is greater than a preset threshold, the first coordinate and the second coordinate are updated.

9. The method according to claim 8, characterized in that The first coordinate includes a first abscissa and a first ordinate value, the second coordinate includes a second abscissa and a second ordinate value, and when the difference between the coordinate distance and the size parameter is greater than a preset threshold, updating the first coordinate and the second coordinate includes: When the difference is greater than the preset threshold, adjusting the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value according to the first horizontal coordinate, the first vertical coordinate value, the second horizontal coordinate and the second vertical coordinate value; According to the adjustment result, the first coordinate and the second coordinate are updated.

10. The method according to claim 9, characterized in that The adjustment result includes an adjusted first horizontal coordinate value, an adjusted first vertical coordinate value, an adjusted second horizontal coordinate value, and an adjusted second vertical coordinate value, and updating the first coordinate and the second coordinate according to the adjustment result includes: The first coordinate and the second coordinate are updated according to the adjusted first coordinate value and the adjusted second coordinate value, wherein the first coordinate value is the coordinate value with the smallest coordinate adjustment amount among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value, and the second coordinate value is the other coordinate value among the first horizontal coordinate value, the first vertical coordinate value, the second horizontal coordinate value and the second vertical coordinate value except the first coordinate value.

11. The method according to claim 1, characterized in that The determining the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera includes: The camera parameters are determined according to the second position information and fourth position information of the calibration object in the camera coordinate system.

12. The method according to claim 11, characterized in that The method further comprises: Acquire a calibration object image obtained by photographing the calibration object with a vehicle camera; The fourth position information is determined according to the calibration object image.

13. The method according to claim 12, characterized in that The calibration object includes a first number of feature points, and determining the fourth position information according to the calibration object image includes: Determining a second number of feature points in the calibration object image according to the calibration object image; In a case where the second number matches the first number, the fourth position information is determined according to the image coordinates of each feature point in the calibration object image.

14. The method according to claim 13, characterized in that The determining, according to the calibration object image, a second number of the feature points in the calibration object image comprises: According to a preset binarization threshold, the calibration object image is binarized to determine a processed image; The second number of the feature points in the processed image is determined according to the processed image.

15. The method according to claim 14, characterized in that The step of performing binarization processing on the calibration object image according to a preset binarization threshold value to determine a processed image includes: In the case where the second number is smaller than the first number, the binarization threshold is increased to determine a new binarization threshold; The binarization process is performed on the calibration object image according to the new binarization threshold to determine the processed image.

16. The method according to claim 14, characterized in that The step of performing binarization processing on the calibration object image according to a preset binarization threshold value to determine a processed image includes: According to the binarization threshold, the calibration object image is binarized to determine a binarized image; Performing morphological processing on the binarized image to determine the processed image, wherein the morphological processing includes dilation and / or erosion.

17. The method according to claim 1, characterized in that The second position information includes feature point coordinates of feature points in the calibration object, the camera parameters include internal parameters and distortion parameters, and determining the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera includes: The intrinsic parameter and the distortion parameter are determined according to the feature point coordinates and the coordinates of the feature point in the camera coordinate system, wherein the coordinates in the camera coordinate system are determined according to the calibration object image obtained by photographing the calibration object by the vehicle camera.

18. The method according to claim 17, characterized in that The camera parameters include external parameters, and determining the camera parameters of the vehicle camera according to the second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera includes: The external parameter is determined according to the internal parameter, the feature point coordinates and the coordinates of the feature point in a pixel coordinate system, wherein the coordinates in the pixel coordinate system are determined according to the calibration object image.

19. The method according to claim 1, characterized in that The method further comprises: A first calibration error of the camera parameters is determined based on a projection of the fifth position information of the calibration object on the world coordinate system, wherein the fifth position information is used to indicate the position of the calibration object in a pixel coordinate system, and the pixel coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with the vehicle camera.

20. The method according to claim 1, characterized in that The method further comprises: A second calibration error of the camera parameters is determined based on a projection of the sixth position information of the calibration object in a pixel coordinate system, wherein the sixth position information is used to indicate the position of the calibration object in the world coordinate system, and the pixel coordinate system is determined based on an image of the calibration object obtained by photographing the calibration object with the vehicle camera.

21. The method according to claim 1, characterized in that The method further comprises: A third calibration error of the camera parameters is determined by mapping the world coordinate system to the image coordinate system, wherein the image coordinate system is determined based on a calibration object image obtained by photographing the calibration object with the vehicle camera.

22. A control device, characterized in that: including a transceiver unit and a processing unit; The transceiver unit is configured to obtain first position information of the vehicle, wherein the first position information is used to indicate the position of the light emitted by the light emitter relative to the vehicle; The processing unit is configured to determine a world coordinate system according to the first position information, and determine camera parameters of the vehicle camera according to second position information of the calibration object in the world coordinate system to complete the calibration of the vehicle camera.

23. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 21 is implemented.

24. An electronic device, characterized in that: Comprising the device of claim 22 or 23.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 21 is implemented.

26. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 21 is implemented.

Images