Vehicle blind spot camera calibration method, device, terminal and storage medium

Abstract

The present application provides a vehicle body blind spot camera calibration method, device, terminal and storage medium. The method includes: obtaining the original image taken by the target blind spot fisheye camera of the vehicle; taking the grounding point of the target blind spot fisheye camera installation position as the center point, determining the world coordinates of the center point in the vehicle world coordinate system, and selecting at least one reference point corresponding to the center point; converting the world coordinates into image coordinates under multiple preset installation angles; for each preset installation angle, using the reference point and the image coordinates of the center point under the preset installation angle, drawing the corresponding image curve on the original image; selecting the image curve tangent to the outer grounding point of the wheel of the vehicle closest to the target blind spot fisheye camera as the target image curve, and using the preset installation angle of the target image curve as the installation angle of the target blind spot fisheye camera. The present application can improve the accuracy of the installation angle, shorten the installation calibration time, and reduce the installation cost.

Description

Technical Field

[0001] The present application relates to the technical field of automobile blind spot calibration, and in particular to a vehicle body blind spot camera calibration method, device, terminal and storage medium. Background Art

[0002] The intelligent domain control product for commercial vehicles integrates safety functions such as the Autonomous Emergency Braking (AEB), the Blind Spot Detection (BSD), and the Dealer-Management-System (DMS). It contains 9 cameras, 6 of which are used for the perception of the entire vehicle's blind spots. Accurate calibration is the guarantee for the subsequent high accuracy of target distance measurement and speed measurement. In most cases, the camera angle is calibrated by placing specific calibration objects and establishing a mapping relationship between the actual coordinates of the marker and the image coordinates. However, this method is time-consuming, costly, and complex to operate. The calibration site must be flat and open. When multiple blind spot cameras are to be calibrated, it is necessary to develop a convenient and efficient camera calibration method. Summary of the invention

[0003] The present application provides a vehicle body blind spot camera calibration method, device, terminal and storage medium to solve the problem that the prior art vehicle blind spot camera calibration operation is complicated and time-consuming.

[0004] In a first aspect, the present application provides a vehicle body blind spot camera calibration method, comprising:

[0005] Obtain the original image taken by the fisheye camera of the target blind spot of the vehicle;

[0006] Taking the grounding point of the installation position of the target blind spot fisheye camera as the center point, determining the world coordinates of the center point in the world coordinate system of the vehicle, and selecting at least one reference point corresponding to the center point, the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point;

[0007] At multiple preset installation angles of the target blind spot fisheye camera, converting the world coordinates of at least one reference point and the center point into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera;

[0008] For each preset installation angle, using the image coordinates of the reference point and the center point at the preset installation angle, an image curve at the preset installation angle is drawn on the original image;

[0009] An image curve tangent to the outer ground contact point of a target wheel is selected from all image curves on the original image as a target image curve, and a preset installation angle corresponding to the target image curve is used as an installation angle of the target blind spot fisheye camera. The target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera.

[0010] In a second aspect, the present application provides a vehicle body blind spot camera calibration device, comprising:

[0011] An image acquisition module, used to acquire the original image taken by the fisheye camera of the target blind spot of the vehicle;

[0012] a reference point determination module, configured to determine the world coordinates of the center point in the world coordinate system of the vehicle by taking the grounding point of the installation position of the target blind spot fisheye camera as the center point, and select at least one reference point corresponding to the center point, wherein the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point;

[0013] An image coordinate determination module, used for converting the world coordinates of at least one reference point and the center point into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera at multiple preset installation angles of the target blind spot fisheye camera;

[0014] An image curve drawing module is used to draw, for each preset installation angle, an image curve at the preset installation angle on the original image using the image coordinates of the reference point and the center point at the preset installation angle;

[0015] The installation angle determination module is used to select an image curve that is tangent to the outer ground contact point of the target wheel from all image curves on the original image as the target image curve, and use a preset installation angle corresponding to the target image curve as the installation angle of the target blind spot fisheye camera. The target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera.

[0016] In a third aspect, the present application provides a terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0018] The present application provides a vehicle body blind spot camera calibration method, device, terminal and storage medium, which obtain the original image taken by the target blind spot fisheye camera of the vehicle; take the grounding point of the installation position of the target blind spot fisheye camera as the center point, determine the world coordinates of the center point in the world coordinate system of the vehicle, and select at least one reference point corresponding to the center point, the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point; at multiple preset installation angles of the target blind spot fisheye camera, convert the world coordinates of at least one reference point and the center point into image coordinates in the pixel plane coordinate system corresponding to the target blind spot fisheye camera; for each preset installation angle, use the image coordinates of the reference point and the center point at the preset installation angle to draw the image curve at the preset installation angle on the original image; select the image curve tangent to the grounding point on the outer side of the target wheel from all the image curves on the original image as the target image curve, and use the preset installation angle corresponding to the target image curve as the installation angle of the target blind spot fisheye camera, and the target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera. The present application converts the world coordinates of at least one reference point and a center point into image coordinates, draws an image curve corresponding to each preset installation angle on the original image, and selects the preset installation angle of the image curve tangent to the outer ground contact point of the target wheel as the installation angle of the target blind spot fisheye camera. The calculation is simple and there are no special requirements for the calibration site. The calibration can be completed in a few minutes. Moreover, under the premise of ensuring the accuracy of the installation angle of the blind spot camera, the installation and calibration time of the blind spot camera is shortened, and the installation cost of the blind spot camera is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 is a flow chart of the implementation of the vehicle body blind spot camera calibration method provided in the embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a vehicle-based world coordinate system provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of an image curve provided in an embodiment of the present application on an original image;

[0023] Figure 4 It is a structural schematic diagram of a vehicle body blind spot camera calibration device provided in an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0027] Figure 1 The implementation flow chart of the vehicle body blind spot camera calibration method provided in the embodiment of the present application is detailed as follows:

[0028] In step 101, an original image captured by a fisheye camera of a target blind spot of a vehicle is obtained.

[0029] In the embodiment of the present application, an intelligent domain controller is used to collect images in real time as the original images of the target blind area fisheye camera. The original images of the target blind area fisheye camera are used to provide a carrier for the subsequent drawing of image curves.

[0030] Among them, the intelligence and controller can connect multiple blind spot fisheye cameras.

[0031] In step 102, the grounding point of the installation position of the target blind spot fisheye camera is used as the center point, the world coordinates of the center point in the world coordinate system of the vehicle are determined, and at least one reference point corresponding to the center point is selected, the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point.

[0032] In an embodiment of the present application, the grounding point of the installation position of the target blind spot fisheye camera is taken as the center point, the world coordinates of the center point in the world coordinate system are determined, and then at least one reference point with the same world horizontal coordinate and different world vertical coordinate as the center point is selected.

[0033] The world coordinate system is established by taking the front of the vehicle corresponding to the target blind spot fisheye camera as the horizontal coordinate axis of the world coordinate system and taking the center grounding point of the front of the vehicle as the world coordinate origin.

[0034] For example, refer to Figure 2 , the center grounding point of the vehicle's front is used as the world coordinate origin The connecting line between the center grounding point of the vehicle head and the outer grounding point of the target wheel is used as the world horizontal axis , establish the world coordinate system .

[0035] Accordingly, Figure 2 The camera installation position in is the installation position of the target blind spot fisheye camera, and the grounding point of the installation position is the center point.

[0036] In a possible implementation, selecting at least one reference point corresponding to the center point may include:

[0037] In the world coordinate system, at least one reference point is selected, which has the same world horizontal coordinate as the center point and a different world vertical coordinate.

[0038] There can be one reference point, which can be connected with the center point to draw a curve; there can also be more than one reference point, which can be connected with the center point to draw a curve.

[0039] Optionally, in the world coordinate system, the world horizontal coordinate of the center point is used as the world horizontal coordinate, and at least one reference point having different world vertical coordinates is selected.

[0040] For example, refer to Figure 2 , it is known that the outer wheelbase of the target wheel is m, then the world coordinate of the center point is P0(m / 2, y0, z), and 5 reference points are selected at the same interval on the world longitudinal axis 5m to the left and right of the target blind spot fisheye camera, for a total of 10 reference points, and the corresponding world coordinates are: P1(m / 2, y1, z), P2(m / 2, y2, z), P3(m / 2, y3, z), P4(m / 2, y4, z), P5(m / 2, y5, z), P6(m / 2, y6, z), P7(m / 2, y7, z), P8(m / 2, y8, z), P9(m / 2, y9, z), P10(m / 2, y10, z).

[0041] Among them, the above example selects 10 reference points with equal intervals and uniform distribution on the left and right of the center point. The present application can also select reference points with different intervals, or select unequally distributed reference points, or also select only one reference point, and the specific selection depends on actual needs.

[0042] This application obtains reference points and center points with the purpose of drawing image curves from the reference points and center points, providing a basis for determining the installation angle of the fisheye camera in the target blind spot later.

[0043] In step 103, at multiple preset installation angles of the target blind spot fisheye camera, the world coordinates of at least one reference point and the center point are converted into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera.

[0044] In the embodiment of the present application, each preset installation angle of the target blind spot fisheye camera is used to convert the world coordinates of each reference point and the center point into the image coordinates in the pixel plane coordinate system corresponding to the target blind spot fisheye camera, wherein the pixel plane coordinate system is a two-dimensional coordinate system.

[0045] In a possible implementation, at multiple preset installation angles of the target blind spot fisheye camera, converting the world coordinates of at least one reference point and the center point into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera may include:

[0046] Obtain the external parameters corresponding to each preset installation angle of the target blind spot fisheye camera;

[0047] Using the external parameters corresponding to each preset installation angle, respectively convert the world coordinates of at least one reference point and the center point at the corresponding preset installation angle from the world coordinate system to the camera coordinate system corresponding to the target blind spot fisheye camera, to obtain the first coordinates of at least one reference point and the center point;

[0048] Using the distortion parameters, respectively performing an inverse distortion transformation on the first coordinates of at least one reference point and the center point;

[0049] The dedistorted first coordinates of at least one reference point and a center point at each preset installation angle are respectively converted into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera.

[0050] Optionally, each preset installation angle of the target blind spot fisheye camera is used to determine the external parameters corresponding to the preset installation angle. Then, the external parameters of each preset installation angle are used to convert the world coordinates of each reference point and center point under the preset installation angle to the camera coordinate system corresponding to the target blind spot fisheye camera, and obtain the first coordinates of the corresponding reference point and center point. According to the distortion parameters, the first coordinates of each reference point and center point are respectively subjected to anti-distortion transformation. For example, the first coordinates of a reference point are transformed into The anti-distortion transformation is Finally, the first coordinates after dedistortion of each reference point and the center point at each preset installation angle are converted into image coordinates in the pixel plane coordinate system corresponding to the target blind area fisheye camera.

[0051] In a possible implementation, the external parameter may include a rotation matrix, and the preset installation angle may include a pitch angle, a yaw angle, and a roll angle.

[0052] Among them, extrinsic parameters (EP) are parameters that describe the position and posture of the camera in the world coordinate system. Extrinsic parameters may change at different camera positions or shooting times.

[0053] In this application, external parameters may include rotation matrices.

[0054] The preset installation angles of the target blind spot fisheye camera may include a pitch angle, a yaw angle, and a roll angle.

[0055] In a possible implementation, obtaining the external parameters corresponding to each preset installation angle of the target blind spot fisheye camera may include:

[0056] Initialize the pitch angle, yaw angle and roll angle in each preset installation angle respectively, the yaw angle and roll angle after initialization are both 0 degrees, and the pitch angle in each preset installation angle after initialization is different;

[0057] The rotation matrix of the target blind spot fisheye camera at the corresponding preset installation angle is calculated using each initialized preset installation angle.

[0058] Optionally, after setting a preset installation angle for the target blind spot fisheye camera, it is first necessary to initialize the pitch angle, yaw angle and roll angle in each preset installation angle, and then use the initialized preset installation angle to calculate the rotation matrix of the target blind spot fisheye camera at the preset installation angle.

[0059] Assuming that there are four sets of preset installation angles for the target blind spot fisheye camera, the pitch angle, yaw angle, and roll angle after the four sets of preset installation angles are initialized are shown in the following table:

[0060]

[0061] The calculation process of the rotation matrix for each preset installation angle is as follows:

[0062] Rotating around the X axis gives:

[0063]

[0064] in, is the pitch angle.

[0065] Rotating around the Y axis gives:

[0066]

[0067] in, is the yaw angle.

[0068] Rotating around the Z axis gives:

[0069]

[0070] in, is the roll angle.

[0071] Rotate around the X axis to get , obtained by rotating around the Y axis And the rotation around the Z axis Multiply them together to get the rotation matrix:

[0072]

[0073] in, is the rotation matrix, which is a The matrix of .

[0074] Accordingly, assuming that there are four sets of preset installation angles for the target blind spot fisheye camera, the corresponding four sets of rotation matrices are .

[0075] In a possible implementation, the external reference in the present application may also include a translation matrix.

[0076] In a possible implementation, obtaining the external parameters corresponding to each preset installation angle of the target blind spot fisheye camera may include:

[0077] Obtain the installation position of the target blind spot fisheye camera corresponding to each preset installation angle and the height of the corresponding installation position;

[0078] The translation matrix of the target blind spot fisheye camera at the corresponding preset installation angle is calculated using the rotation matrix of each installation position, the height of the corresponding installation position and the corresponding preset installation angle.

[0079] Optionally, the installation position corresponding to each preset installation angle of the target blind spot fisheye camera and the height of the installation position are obtained, and then for each preset installation angle, the translation matrix of the target blind spot fisheye camera at the preset installation angle is calculated using the installation position corresponding to the preset installation angle, the height of the installation position and the rotation matrix of the preset installation angle.

[0080] The calculation formula of the translation matrix is:

[0081]

[0082] in, is the translation matrix, which is a The matrix of is the X-axis position of the target blind spot camera in the world coordinate system, is the Y-axis position of the target blind spot camera in the world coordinate system, is the Z-axis position of the target blind spot camera in the world coordinate system, is the rotation matrix at the corresponding preset installation angle.

[0083] Accordingly, assuming that there are four sets of preset installation angles for the target blind spot fisheye camera, the corresponding four sets of translation matrices are .

[0084] In a possible implementation, using the external parameters corresponding to each preset installation angle, respectively converting the world coordinates of at least one reference point and the center point at the corresponding preset installation angle from the world coordinate system to the camera coordinate system corresponding to the target blind spot fisheye camera, and obtaining the first coordinates of at least one reference point and the center point may include:

[0085] The first coordinates of at least one reference point and the center point at each preset installation angle are calculated by a first formula, and the first formula is:

[0086]

[0087] in, For the The first preset installation angle The horizontal coordinate of the first coordinate, For the The first preset installation angle The ordinate of the first coordinate, For the The first preset installation angle The vertical axis coordinate of the first coordinate, For the The world abscissa of the world coordinates, For the The world ordinate of the world coordinate, For the The world vertical axis coordinate of the world coordinate, For the The rotation matrix at a preset installation angle, For the A translation matrix for a preset mounting angle.

[0088] Optionally, for each preset installation angle, the first formula Calculate the first coordinates of at least one reference point and the center point under the preset installation angle .

[0089] In a possible implementation, converting the first coordinates of at least one reference point and the center point after dedistortion at each preset installation angle into image coordinates in a pixel plane coordinate system corresponding to the target blind area fisheye camera may include:

[0090] Obtain the internal reference of the fisheye camera in the target blind area, the internal reference including the first internal reference, the second internal reference, the third internal reference and the fourth internal reference;

[0091] For each preset installation angle, the internal parameters of the target blind spot fisheye camera and each first coordinate after dedistortion at the preset installation angle are respectively input into the second formula to obtain the image coordinates of each first coordinate after dedistortion at the preset installation angle in the pixel plane coordinates corresponding to the target blind spot fisheye camera. The second formula is:

[0092]

[0093] in, For the The first preset installation angle The horizontal coordinate of the image coordinates, For the The first preset installation angle The ordinate of the image coordinates, For the The first preset installation angle The horizontal coordinate of the first coordinate, For the The first preset installation angle The ordinate of the first coordinate, is the first internal reference, is the second internal reference, is the third internal reference, It is the fourth internal reference.

[0094] Among them, intrinsic parameters (IP) are parameters that describe the internal properties of the camera, including focal length, principal point (optical center) coordinates, distortion coefficients, etc. Intrinsic parameters are usually determined during camera calibration, because they are usually fixed for a specific camera model and do not change over time. Once the camera intrinsic parameters are determined, they usually remain unchanged during the use of the camera.

[0095] In this application, the internal reference includes the first internal reference , the second internal reference 、The third internal reference And the fourth internal reference .

[0096] Optionally, obtain the first internal parameter of the target blind area fisheye camera , the second internal reference 、The third internal reference And the fourth internal reference For each preset installation angle, the first internal parameter , the second internal reference 、The third internal reference And the fourth internal reference And the first coordinates of at least one reference point and the center point after anti-distortion at the preset installation angle are input into the second formula The image coordinates of each first coordinate after anti-distortion in the pixel plane coordinate system corresponding to the target blind area fisheye camera are obtained. .

[0097] In step 104, for each preset installation angle, an image curve at the preset installation angle is drawn on the original image using the image coordinates of the reference point and the center point at the preset installation angle.

[0098] In an embodiment of the present application, for each preset installation angle, the image coordinates of at least one reference point and a center point at the preset installation angle in step 103 are used to draw an image curve, and the image curve at each preset installation angle is drawn on the original image captured in step 101.

[0099] For example, assuming that there are four sets of preset installation angles for the target blind spot fisheye camera, four image curves are drawn on the original image, referring to Figure 3 The four image curves include image curve A, image curve B, image curve C and image curve D, which respectively represent image curves corresponding to four sets of preset installation angles.

[0100] In step 105, an image curve tangent to the outer ground contact point of the target wheel is selected from all image curves on the original image as the target image curve, and a preset installation angle corresponding to the target image curve is used as the installation angle of the target blind spot fisheye camera. The target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera.

[0101] In an embodiment of the present application, a mounting fixture is used to ensure that the yaw angle and roll angle of the target blind spot fisheye camera are 0, and then the pitch angle of the target blind spot fisheye camera is adjusted. An image curve tangent to the outer ground contact point of the target wheel is selected as the target image curve, and the preset installation angle corresponding to the target image curve is used as the installation angle of the target blind spot fisheye camera, that is, the pitch angle of the target blind spot fisheye camera is determined.

[0102] For example, refer to Figure 3, image curve B is tangent to the outer ground contact point of the target wheel, then image curve B is determined to be the target image curve, and the installation angle corresponding to image curve B is used as the installation angle of the target blind spot fisheye camera.

[0103] The remaining blind spot fisheye cameras of the vehicle are also installed and calibrated using the above calibration method, and finally the installation angles of all blind spot cameras of the vehicle are obtained.

[0104] The present application provides a vehicle body blind spot camera calibration method, which obtains an original image taken by a target blind spot fisheye camera of a vehicle; takes the grounding point of the installation position of the target blind spot fisheye camera as the center point, determines the world coordinates of the center point in the world coordinate system of the vehicle, and selects at least one reference point corresponding to the center point, the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point; at multiple preset installation angles of the target blind spot fisheye camera, converts the world coordinates of at least one reference point and the center point into image coordinates in the pixel plane coordinate system corresponding to the target blind spot fisheye camera; for each preset installation angle, uses the image coordinates of the reference point and the center point at the preset installation angle to draw an image curve at the preset installation angle on the original image; selects an image curve tangent to the grounding point outside the target wheel from all image curves on the original image as the target image curve, and takes the preset installation angle corresponding to the target image curve as the installation angle of the target blind spot fisheye camera, and the target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera. The present application converts the world coordinates of at least one reference point and a center point into image coordinates, draws an image curve corresponding to each preset installation angle on the original image, and selects the preset installation angle of the image curve tangent to the outer ground contact point of the target wheel as the installation angle of the target blind spot fisheye camera. The calculation is simple and there are no special requirements for the calibration site. The calibration can be completed in a few minutes. Moreover, under the premise of ensuring the accuracy of the installation angle of the blind spot camera, the installation and calibration time of the blind spot camera is shortened, and the installation cost of the blind spot camera is reduced.

[0105] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0106] The following is an embodiment of the device of the present application. For details not described in detail, please refer to the corresponding method embodiment described above.

[0107] Figure 4 The structural diagram of the vehicle body blind spot camera calibration device provided in the embodiment of the present application is shown. For the convenience of explanation, only the part related to the embodiment of the present application is shown, which is described in detail as follows:

[0108] like Figure 4 As shown, the vehicle body blind spot camera calibration device 4 includes:

[0109] An image acquisition module 41 is used to acquire an original image taken by a fisheye camera in a target blind spot of a vehicle;

[0110] A reference point determination module 42 is used to determine the world coordinates of the center point in the world coordinate system of the vehicle using the grounding point of the installation position of the target blind spot fisheye camera as the center point, and select at least one reference point corresponding to the center point, wherein the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point;

[0111] An image coordinate determination module 43, configured to convert the world coordinates of at least one reference point and a center point into image coordinates in a pixel plane coordinate system corresponding to the target blind area fisheye camera at multiple preset installation angles of the target blind area fisheye camera;

[0112] An image curve drawing module 44 is used to draw, for each preset installation angle, an image curve at the preset installation angle on the original image using the image coordinates of the reference point and the center point at the preset installation angle;

[0113] The installation angle determination module 45 is used to select an image curve that is tangent to the outer ground contact point of the target wheel from all image curves on the original image as the target image curve, and use the preset installation angle corresponding to the target image curve as the installation angle of the target blind spot fisheye camera. The target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera.

[0114] The present application provides a vehicle body blind spot camera calibration device, which obtains an original image taken by a target blind spot fisheye camera of a vehicle; takes the grounding point of the installation position of the target blind spot fisheye camera as the center point, determines the world coordinates of the center point in the world coordinate system of the vehicle, and selects at least one reference point corresponding to the center point, the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point; at multiple preset installation angles of the target blind spot fisheye camera, converts the world coordinates of at least one reference point and the center point into image coordinates in the pixel plane coordinate system corresponding to the target blind spot fisheye camera; for each preset installation angle, uses the image coordinates of the reference point and the center point at the preset installation angle to draw an image curve at the preset installation angle on the original image; selects an image curve tangent to the grounding point on the outer side of the target wheel from all image curves on the original image as the target image curve, and takes the preset installation angle corresponding to the target image curve as the installation angle of the target blind spot fisheye camera, and the target wheel is the wheel of the vehicle closest to the target blind spot fisheye camera. The present application converts the world coordinates of at least one reference point and a center point into image coordinates, draws an image curve corresponding to each preset installation angle on the original image, and selects the preset installation angle of the image curve tangent to the outer ground contact point of the target wheel as the installation angle of the target blind spot fisheye camera. The calculation is simple and there are no special requirements for the calibration site. The calibration can be completed in a few minutes. Moreover, under the premise of ensuring the accuracy of the installation angle of the blind spot camera, the installation and calibration time of the blind spot camera is shortened, and the installation cost of the blind spot camera is reduced.

[0115] In a possible implementation, the reference point determination module may be used to:

[0116] In the world coordinate system, at least one reference point is selected, which has the same world horizontal coordinate as the center point and a different world vertical coordinate.

[0117] In a possible implementation, the image coordinate determination module may be used to:

[0118] Obtain the external parameters corresponding to each preset installation angle of the target blind spot fisheye camera;

[0119] Using the external parameters corresponding to each preset installation angle, respectively convert the world coordinates of at least one reference point and the center point at the corresponding preset installation angle from the world coordinate system to the camera coordinate system corresponding to the target blind spot fisheye camera, to obtain the first coordinates of at least one reference point and the center point;

[0120] Using the distortion parameters, respectively performing an inverse distortion transformation on the first coordinates of at least one reference point and the center point;

[0121] The dedistorted first coordinates of at least one reference point and a center point at each preset installation angle are respectively converted into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera.

[0122] In a possible implementation, the image coordinate determination module may also be used to:

[0123] The first coordinates of at least one reference point and the center point at each preset installation angle are calculated by a first formula, and the first formula is:

[0124]

[0125] in, For the The first preset installation angle The horizontal coordinate of the first coordinate, For the The first preset installation angle The ordinate of the first coordinate, For the The first preset installation angle The vertical axis coordinate of the first coordinate, For the The world abscissa of the world coordinates, For the The world ordinate of the world coordinate, For the The world vertical axis coordinate of the world coordinate, For the The rotation matrix at a preset installation angle, For the A translation matrix for a preset mounting angle.

[0126] In a possible implementation, the image coordinate determination module may also be used to:

[0127] Obtain the internal reference of the fisheye camera in the target blind area, the internal reference including the first internal reference, the second internal reference, the third internal reference and the fourth internal reference;

[0128] For each preset installation angle, the internal parameters of the target blind spot fisheye camera and each first coordinate after dedistortion at the preset installation angle are respectively input into the second formula to obtain the image coordinates of each first coordinate after dedistortion at the preset installation angle in the pixel plane coordinates corresponding to the target blind spot fisheye camera. The second formula is:

[0129]

[0130] in, For the The first preset installation angle The horizontal coordinate of the image coordinates, For the The first preset installation angle The ordinate of the image coordinates, For the The first preset installation angle The horizontal coordinate of the first coordinate, For the The first preset installation angle The ordinate of the first coordinate, is the first internal reference, is the second internal reference, is the third internal reference, It is the fourth internal reference.

[0131] In a possible implementation, the external parameter may include a rotation matrix, and the preset installation angle may include a pitch angle, a yaw angle, and a roll angle.

[0132] In a possible implementation, the image coordinate determination module may also be used to:

[0133] Initialize the pitch angle, yaw angle and roll angle in each preset installation angle respectively, the yaw angle and roll angle after initialization are both 0 degrees, and the pitch angle in each preset installation angle after initialization is different;

[0134] The rotation matrix of the target blind spot fisheye camera at the corresponding preset installation angle is calculated using each initialized preset installation angle.

[0135] In a possible implementation, the external parameter may include a translation matrix.

[0136] In a possible implementation, the image coordinate determination module may also be used to:

[0137] Obtain the installation position of the target blind spot fisheye camera corresponding to each preset installation angle and the height of the corresponding installation position;

[0138] The translation matrix of the target blind spot fisheye camera at the corresponding preset installation angle is calculated using the rotation matrix of each installation position, the height of the corresponding installation position and the corresponding preset installation angle.

[0139] Figure 5 is a schematic diagram of a terminal provided in an embodiment of the present application. Figure 5As shown, the terminal 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-mentioned vehicle body blind spot camera calibration method embodiments are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 4 The functions of each module are shown.

[0140] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 52 in the terminal 5. For example, the computer program 52 may be divided into Figure 4 The modules shown.

[0141] The terminal 5 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of terminal 5 and does not constitute a limitation on terminal 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0142] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] The memory 51 may be an internal storage unit of the terminal 5, such as a hard disk or memory of the terminal 5. The memory 51 may also be an external storage device of the terminal 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 5. Further, the memory 51 may include both an internal storage unit of the terminal 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the terminal. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0144] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0150] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned embodiments of the vehicle body blind spot camera calibration method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0151] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A vehicle body blind spot camera calibration method, characterized in that: include: Obtain the original image taken by the fisheye camera of the target blind spot of the vehicle; Taking the grounding point of the installation position of the target blind spot fisheye camera as the center point, determining the world coordinates of the center point in the world coordinate system of the vehicle, and selecting at least one reference point corresponding to the center point, the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point; At multiple preset installation angles of the target blind spot fisheye camera, converting the world coordinates of at least one reference point and the center point into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera; For each preset installation angle, using the image coordinates of the reference point and the center point at the preset installation angle, an image curve at the preset installation angle is drawn on the original image; An image curve tangent to the outer ground contact point of a target wheel is selected from all image curves on the original image as a target image curve, and a preset installation angle corresponding to the target image curve is used as an installation angle of the target blind spot fisheye camera, wherein the target wheel is a wheel of a vehicle closest to the target blind spot fisheye camera; The method of converting the world coordinates of at least one reference point and the center point into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera at multiple preset installation angles of the target blind spot fisheye camera comprises: Obtaining external parameters corresponding to each preset installation angle of the target blind spot fisheye camera; Using the external parameters corresponding to each preset installation angle, respectively convert the world coordinates of at least one reference point and the center point at the corresponding preset installation angle from the world coordinate system to the camera coordinate system corresponding to the target blind spot fisheye camera, to obtain the first coordinates of at least one reference point and the center point; Using the distortion parameters, respectively performing an inverse distortion transformation on the first coordinates of at least one reference point and the center point; The dedistorted first coordinates of at least one reference point and the center point at each preset installation angle are respectively converted into image coordinates in the pixel plane coordinate system corresponding to the target blind spot fisheye camera.

2. The vehicle body blind spot camera calibration method according to claim 1, characterized in that: The method uses the external parameters corresponding to each preset installation angle to convert the world coordinates of at least one reference point and the center point at the corresponding preset installation angle from the world coordinate system to the camera coordinate system corresponding to the target blind spot fisheye camera to obtain the first coordinates of at least one reference point and the center point, including: The first coordinates of at least one reference point and the center point at each preset installation angle are calculated by a first formula, wherein the first formula is: in, For the The first preset installation angle The horizontal coordinate of the first coordinate, For the The first preset installation angle The ordinate of the first coordinate, For the The first preset installation angle The vertical axis coordinate of the first coordinate, For the The world abscissa of the world coordinates, For the The world ordinate of the world coordinate, For the The world vertical axis coordinate of the world coordinate, For the The rotation matrix at a preset installation angle, For the A translation matrix for a preset mounting angle.

3. The vehicle body blind spot camera calibration method according to claim 1, characterized in that: The step of converting the first coordinates of at least one reference point and the center point after dedistortion at each preset installation angle into image coordinates in the pixel plane coordinate system corresponding to the target blind area fisheye camera comprises: Acquire the internal reference of the target blind spot fisheye camera, wherein the internal reference includes a first internal reference, a second internal reference, a third internal reference and a fourth internal reference; For each preset installation angle, the internal parameters of the target blind spot fisheye camera and each first coordinate after dedistortion at the preset installation angle are respectively input into the second formula to obtain the image coordinates of each first coordinate after dedistortion at the preset installation angle in the pixel plane coordinates corresponding to the target blind spot fisheye camera. The second formula is: in, For the The first preset installation angle The horizontal coordinate of the image coordinates, For the The first preset installation angle The ordinate of the image coordinates, For the The first preset installation angle The horizontal coordinate of the first coordinate, For the The first preset installation angle The ordinate of the first coordinate, is the first internal reference, is the second internal reference, is the third internal reference, is the fourth internal reference.

4. The vehicle body blind spot camera calibration method according to claim 1, characterized in that: The external parameters include a rotation matrix, and the preset installation angles include a pitch angle, a yaw angle, and a roll angle; The obtaining of external parameters corresponding to each preset installation angle of the target blind spot fisheye camera includes: Initialize the pitch angle, yaw angle and roll angle in each preset installation angle respectively, the yaw angle and roll angle after initialization are both 0 degrees, and the pitch angle in each preset installation angle after initialization is different; The rotation matrix of the target blind spot fisheye camera at the corresponding preset installation angle is calculated using each initialized preset installation angle.

5. The vehicle body blind spot camera calibration method according to claim 4, characterized in that: The external parameters include a translation matrix, and the external parameters corresponding to each preset installation angle of the target blind spot fisheye camera are obtained, including: Obtaining the installation position of the target blind spot fisheye camera corresponding to each preset installation angle and the height of the corresponding installation position; The translation matrix of the target blind spot fisheye camera at the corresponding preset installation angle is calculated using the rotation matrix of each installation position, the height of the corresponding installation position, and the corresponding preset installation angle.

6. A vehicle body blind spot camera calibration device, characterized in that: include: An image acquisition module, used to acquire the original image taken by the fisheye camera of the target blind spot of the vehicle; a reference point determination module, configured to determine the world coordinates of the center point in the world coordinate system of the vehicle by taking the grounding point of the installation position of the target blind spot fisheye camera as the center point, and select at least one reference point corresponding to the center point, wherein the world horizontal coordinate of the reference point is the same as the world horizontal coordinate of the center point, and the world vertical coordinate is different from the world vertical coordinate of the center point; An image coordinate determination module, used for converting the world coordinates of at least one reference point and the center point into image coordinates in a pixel plane coordinate system corresponding to the target blind spot fisheye camera at multiple preset installation angles of the target blind spot fisheye camera; An image curve drawing module is used to draw, for each preset installation angle, an image curve at the preset installation angle on the original image using the image coordinates of the reference point and the center point at the preset installation angle; an installation angle determination module, for selecting an image curve tangent to the outer ground contact point of a target wheel from all image curves on the original image as a target image curve, and using a preset installation angle corresponding to the target image curve as an installation angle of the target blind spot fisheye camera, wherein the target wheel is a wheel of a vehicle closest to the target blind spot fisheye camera; Wherein, the image coordinate determination module is specifically used for: Obtaining external parameters corresponding to each preset installation angle of the target blind spot fisheye camera; Using the external parameters corresponding to each preset installation angle, respectively convert the world coordinates of at least one reference point and the center point at the corresponding preset installation angle from the world coordinate system to the camera coordinate system corresponding to the target blind spot fisheye camera, to obtain the first coordinates of at least one reference point and the center point; Using the distortion parameters, respectively performing an inverse distortion transformation on the first coordinates of at least one reference point and the center point; The dedistorted first coordinates of at least one reference point and the center point at each preset installation angle are respectively converted into image coordinates in the pixel plane coordinate system corresponding to the target blind spot fisheye camera.

7. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the vehicle body blind spot camera calibration method as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the vehicle body blind spot camera calibration method as described in any one of claims 1 to 5 above are implemented.

Images