Parameter calibration method and device, vehicle and storage medium

Abstract

Embodiments of the present application disclose a parameter calibration method and device, a vehicle and a storage medium. The method comprises: acquiring a target image by the camera, the target image being an image comprising a square target and a circular target; determining pixel coordinates of corner points of the square target from the target image; determining pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points; and calibrating external parameters of the camera based on the pixel coordinates of the center and vehicle coordinates, and the pixel coordinates of the corner points and vehicle coordinates. Through the above method, since the center coordinates of the circular target are used for calibration in the calibration, even if the circular target becomes an ellipse in a very serious distortion scene, the center position is very accurate, and therefore, calibration of the external parameters of the camera based on the pixel coordinates and vehicle coordinates of the circular target can improve the accuracy of parameter calibration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and more particularly, to a parameter calibration method and device, a vehicle, and a storage medium. BACKGROUND

[0002] When a surround-view camera of an automatic driving vehicle is shipped or repaired, the parameters of the surround-view camera of the automatic driving vehicle need to be calibrated, but the related parameter calibration method still has the problem that the calibration accuracy needs to be improved. SUMMARY

[0003] In view of the above problems, the present application provides a parameter calibration method, device, vehicle and storage medium to improve the above problems.

[0004] In a first aspect, the present application provides a parameter calibration method applied to a vehicle, wherein the vehicle is installed with a camera, and the method comprises: acquiring a target image by the camera, wherein the target image is an image comprising a square target and a circular target; determining pixel coordinates of corner points of the square target from the target image; determining pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points; and calibrating external parameters of the camera based on the pixel coordinates of the center and vehicle body coordinates, and the pixel coordinates of the corner points and vehicle body coordinates.

[0005] In a second aspect, the present application provides a parameter calibration device running in a vehicle, wherein the vehicle is installed with a camera, and the device comprises: an image acquisition unit configured to acquire a target image by the camera, wherein the target image is an image comprising a square target and a circular target; a first coordinate acquisition unit configured to determine pixel coordinates of corner points of the square target from the target image; a second coordinate determination unit configured to determine pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points; and a calibration unit configured to calibrate external parameters of the camera based on the pixel coordinates of the center and vehicle body coordinates, and the pixel coordinates of the corner points and vehicle body coordinates.

[0006] In a third aspect, the present application provides a vehicle comprising a camera, one or more processors, and a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above method.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores program codes, and the program codes can be invoked by a processor to execute the above method.

[0008] The application provides a parameter calibration method and device, a vehicle and a storage medium. The parameter calibration method comprises the following steps: acquiring a target image comprising a square target and a circular target through a camera; determining pixel coordinates of corner points of the square target from the target image; determining pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points; and calibrating external parameters of the camera based on the pixel coordinates of the center and vehicle coordinates, and the pixel coordinates of the corner points and the vehicle coordinates. The center coordinates of the circular target are used for calibration in the calibration, and the center position is very accurate even if the circular target becomes an ellipse in a very serious distortion scene. Therefore, the calibration of the external parameters of the camera based on the pixel coordinates of the circular target and the vehicle coordinates can improve the calibration accuracy of the parameters. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0010] Figure 1 A schematic diagram of an application scenario of a parameter calibration method proposed by the present application is shown;

[0011] Figure 2 A structural block diagram of a vehicle proposed by the present application is shown;

[0012] Figure 3 A flowchart of a parameter calibration method proposed by the present application is shown;

[0013] Figure 4 A schematic diagram of a planar layout of a calibration board proposed by the present application is shown;

[0014] Figure 5 A schematic diagram of corner points of a square target proposed by the present application is shown;

[0015] Figure 6 A flowchart of another parameter calibration method proposed by the present application is shown;

[0016] Figure 7 A schematic diagram of a target image in another parameter calibration method proposed by the present application is shown;

[0017] Figure 8 A flowchart of still another parameter calibration method proposed by the present application is shown;

[0018] Figure 9 A flowchart of still another parameter calibration method proposed by the present application is shown;

[0019] Figure 10 a structure block diagram of a parameter calibration device proposed in the present application is shown;

[0020] Figure 11 a structure block diagram of a vehicle for performing a parameter calibration method according to an embodiment of the present application is shown in the present application;

[0021] Figure 12 a storage medium for saving or carrying program codes for implementing a parameter calibration method according to an embodiment of the present application is provided in the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] With the development of society, the number of vehicles increases, and the traffic accidents also increase. In the process of vehicle driving, the traditional reversing image system only installs a camera at the tail of the vehicle, and can only cover a limited area around the tail of the vehicle. The blind area around the vehicle and the front of the vehicle undoubtedly increases the hidden danger of safe driving, and collision and scratching events are prone to occur in narrow and congested urban areas and parking lots. In order to avoid the influence of the visual blind area on the safety of vehicle driving and expand the field of vision of the driver, the vehicle-mounted surround view system emerges as the times require.

[0024] The vehicle-mounted surround view system simultaneously collects images around the vehicle by installing cameras around the vehicle, and forms a seamless and complete panoramic bird's-eye view of the vehicle after processing the images. It can effectively assist the driver to judge the environment around the vehicle, and can be used as a basic component of an intelligent product driving assistance system. On this basis, functions such as blind area monitoring and automatic parking can be realized, and the application prospect is very broad.

[0025] In order to synthesize a 360-degree surround view image of the vehicle body, it is necessary to first calibrate the cameras around the front, back and sides of the vehicle body, that is, to calibrate the camera parameters of each camera relative to the vehicle body coordinate system. At present, the vehicle-mounted surround view system usually adopts an offline calibration method to calibrate the parameters used by the system to configure the cameras. The parameters include internal parameters and external parameters. The internal parameters refer to device parameters such as focal length, optical center and lens distortion, and the external parameters refer to the rotation matrix and translation matrix of the camera coordinate system to the world coordinate system. The calibration of the external parameters is the technical key, and the accuracy of the calibration directly affects the performance of the camera.

[0026] The inventor found in the research on related parameter calibration methods that the accuracy of the related parameter calibration methods needs to be improved.

[0027] Therefore, the inventors propose the parameter calibration method, device, vehicle and storage medium in the present application. First, a target image including a square target and a circular target is obtained by a camera, pixel coordinates of corner points of the square target are determined from the target image, pixel coordinates of a center of the circular target are determined based on the pixel coordinates of the corner points, and finally, external parameters of the camera are calibrated based on the pixel coordinates of the center and the vehicle body coordinates, and the pixel coordinates of the corner points and the vehicle body coordinates. Through the above method, since the center coordinates of the circular target are used for calibration in the calibration, even if it becomes an ellipse in a very serious distortion scene, the center position is very accurate, therefore, the external parameters of the camera are calibrated based on the pixel coordinates of the circular target and the vehicle body coordinates, which can improve the accuracy of parameter calibration.

[0028] The application environment of the parameter calibration method provided by the present application will be introduced as follows:

[0029] Please refer to Figure 1 The parameter calibration method provided by the present application can be applied to a parameter calibration system 100, which can include a vehicle 110 and a server 120, wherein the server 120 can establish a communication connection with the vehicle 110. Optionally, the number of vehicles 110 can also be one or more.

[0030] In the embodiments of the present application, the server 120 can run one or more services or software applications that enable the execution of the parameter calibration method. In some embodiments, the server 120 can be a server of a distributed system. The server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0031] In the embodiments of the present application, the parameter calibration method provided by the present application can be executed by the vehicle 110 and the server 120 in cooperation. In this way of cooperation by the vehicle 110 and the server 120, part of the steps in the parameter calibration method provided by the present application can be executed by the vehicle 110, while the other part of the steps can be executed by the server 120.

[0032] For example, the vehicle 110 can execute the steps in the parameter calibration method including: obtaining a target image by a camera, and sending the target image to the server 120, then determining pixel coordinates of corner points of a square target from the target image by the server 120; and determining pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points; and calibrating external parameters of the camera based on the pixel coordinates of the center and the vehicle body coordinates, and the pixel coordinates of the corner points and the vehicle body coordinates, then the server 120 can send the calculated external parameters of the camera to the vehicle 110.

[0033] It should be noted that in this way cooperatively performed by the vehicle 110 and the server 120, the steps performed by the vehicle 110 and the server 120 respectively are not limited to the way introduced in the above examples, and in actual applications, the steps performed by the vehicle 110 and the server 120 respectively can be dynamically adjusted according to actual situations.

[0034] Of course, the parameter calibration method in the embodiment of the present application can also be performed by the vehicle 110 alone.

[0035] Among them, please refer to Figure 2 , the vehicle 110 can include a vehicle-mounted control device 111, a body control module (BCM), a central display device 112 (CDU), an industry bus (ICAN), a central gateway 113 (CGW), and a positioning device 114.

[0036] Among them, the vehicle-mounted control device 111 can be in communication connection with the server 120, the central display device 112, and the positioning device 114 respectively, and the vehicle-mounted control device 111 can be in communication connection with the central gateway 114 through the industry bus. Optionally, the vehicle-mounted control device 111 can include a central interface device (CIU), a processor electrically connected with the central interface device, and a memory electrically connected with the processor, and the memory can be used to store instructions, programs, codes, code sets or instruction sets, etc. Among them, the central interface device can be used to access the bus. Optionally, the processor can include an XPU cloud computing acceleration chip.

[0037] The positioning device 114 is used to detect the position information of the vehicle 110 in real time. Optionally, the positioning device 114 can be a global navigation satellite system (GNSS), which generally refers to all satellite navigation systems, including global, regional and enhanced satellite navigation systems, such as the global positioning system (GPS) of the United States, the global navigation satellite system (Glonass) of Russia, the Galileo satellite navigation system (Galileo) of Europe, the Beidou satellite navigation system of China, and related enhanced systems such as the WAAS (wide area augmentation system) of the United States, the EGNOS (European geostationary navigation overlay service) of Europe, and the MSAS (multi-functional satellite augmentation system) of Japan, and other satellite navigation systems under construction and to be constructed in the future.

[0038] The vehicle-mounted control device 111 can receive the data sent by the server 130, and control the central display device 112 to display corresponding information according to the received data.

[0039] In the above manner, when the vehicle 110 obtains the target image through the camera, the target image can be sent to the vehicle-mounted control device 111 to execute the following steps: determining the pixel coordinates of the corner points of the square target from the target image; determining the pixel coordinates of the center of the circular target based on the pixel coordinates of the corner points; and calibrating the external parameters of the camera based on the pixel coordinates of the center and the vehicle body coordinates, and the pixel coordinates of the corner points and the vehicle body coordinates.

[0040] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0041] Please refer to Figure 3 The present application provides a parameter calibration method, applied to a vehicle, wherein the vehicle is installed with a camera, and the method comprises the following steps:

[0042] Step 110: obtaining a target image through the camera, wherein the target image is an image including a square target and a circular target.

[0043] In the embodiment of the present application, the cameras are installed at the front, rear, left and right positions of the vehicle, and therefore the parameters of the four cameras installed at the front, rear, left and right positions of the vehicle need to be calibrated by using the pre-designed calibration board. When calibrating the parameters of the four cameras installed at the front, rear, left and right positions of the vehicle, the external parameters and internal parameters of the four cameras installed at the front, rear, left and right positions of the vehicle can be calibrated, and in the embodiment of the present application, the external parameters of the four cameras installed at the front, rear, left and right positions of the vehicle are mainly calibrated. The internal parameters can include focal length, optical center and lens distortion, and the external parameters refer to the rotation matrix and translation matrix of the pixel coordinate system to the vehicle coordinate system. The vehicle coordinate system is used to describe the relative position relationship between the objects around the vehicle and the vehicle. The definition of the commonly used vehicle coordinate system can include ISO international standard definition, SAE (Society of Automotive Engineers) definition and IMU (Inertial Measurement Unit) definition.

[0044] The above three definitions of the vehicle coordinate system can be shown in the following table:

[0045] ISO definition SAE definition IMU definition X positive direction Forward Forward Right Y positive direction Left Right Forward Z positive direction Up Down Up Roll positive direction Right Right Right Pitch positive direction Down Up Down Yaw positive direction Counter-clockwise Clockwise Counter-clockwise Center Vehicle center of gravity Vehicle center of gravity IMU position Right hand coordinate system Yes Yes Yes

[0046] The three different vehicle coordinate systems are defined by the above three definitions. In vehicle dynamics analysis, the ISO defined vehicle coordinate system is more common, the SAE defined vehicle coordinate system is consistent with the body coordinate system commonly used in the field of aerospace, and the IMU defined vehicle coordinate system is more common in the related application of IMU. No matter which coordinate system is used, as long as it is used correctly, the description of the vehicle pose and the determination of the relative position relationship between the surrounding objects and the vehicle can be completed, and in the embodiment of the present application, the vehicle coordinate system can be selected according to the application requirements and usage habits.

[0047] Optionally, in the embodiment of the present application, when it is detected that the vehicle is at a specified position, the image of the calibration board is obtained by using the camera installed in the vehicle to obtain a target image. The calibration board is obtained after the markers (circular targets and square targets) are laid on the ground in a pre-designed manner. For example, the planar layout of the calibration board can be as shown in Figure 4 Figure 4 ​In the embodiment, the circular target and the square target are placed in a pre-designed manner, in which the square target has high accuracy in the near distance and the detection is stable, and the circular target is calibrated by using the center coordinates in the calibration, and even if the target becomes an ellipse in a serious distortion scene, the center position of the circular target is very accurate. Therefore, according to the above characteristics, the checkerboard target can be improved, and the calibration board is designed by placing the square target in the near distance and the circular target in the far distance. In the embodiment, the near distance is a position close to the installed camera, and the far distance is a position far from the installed camera.

[0048] In the embodiment, the specified position can be, for example Figure 4 In the embodiment, when the left front wheel of the vehicle is detected to be at the position 1, the target image is acquired by the camera installed in the vehicle. Alternatively, the position 1 is a pre-set position for fixing the left front wheel of the vehicle, and the vehicle can be placed at the position 1 by the slide rail in the embodiment. Figure 4

[0049] As a manner, since the front, rear, left and right positions of the vehicle are provided with cameras, when the target image is acquired by the camera, the target image corresponding to each position can be acquired by the camera installed at the front, rear, left and right positions of the vehicle. Of course, the target image corresponding to each position can not be acquired at the same time, which is not limited herein.

[0050] Alternatively, when the image acquired by the camera includes the direction target and the circular target at the same time, it is determined that the target image is acquired. In this process, as a manner, the shooting angle of the camera can be adjusted to ensure that the target image can be acquired.

[0051] Step 120: determining the pixel coordinates of the corner points of the square target from the target image.

[0052] In the embodiment, the corner points of the square target are the intersection points of the two square targets, as shown in Figure 5 Figure 5 The points in the dashed circle in the embodiment are the corner points of the square target.

[0053] ​​When the target image is acquired through the camera, the target image can be recognized to recognize the pixel coordinates of the corner points of the square target. As a kind of way, the pixel coordinates of the corner points of the square target can be determined by corner point detection algorithm. Wherein, corner point detection algorithm can include three kinds of algorithms based on gray image corner point detection, binary image corner point detection, contour curve corner point detection. Based on gray image corner point detection can be divided into three kinds of methods based on gradient, based on template and based on template gradient combination, wherein the method based on template mainly considers the gray scale change of pixel field point, that is, the change of image brightness, and the point with enough large brightness compared with adjacent point is defined as corner point. Common template-based corner point detection algorithms include Kitchen-Rosenfeld corner point detection algorithm, Harris corner point detection algorithm, KLT corner point detection algorithm and SUSAN corner point detection algorithm. Compared with other corner point detection algorithms, SUSAN corner point detection algorithm has the characteristics of simple algorithm, accurate position and strong anti-noise ability. In the embodiments of the present application, the specific point detection algorithm is not limited.

[0054] Step 130: based on the pixel coordinates of the corner points, the pixel coordinates of the center of the circular target are determined.

[0055] In the embodiments of the present application, since the vehicle body coordinates of the corner points of the square target and the vehicle body coordinates of the center of the circular target are known in advance when designing the calibration board, the positional relationship between the square target and the circular target is also known. Therefore, when the pixel coordinates of the corner points of the square target are determined by the above method, the pixel coordinates of the center of the circular target can be calculated based on the pixel coordinates of the corner points of the square target and the positional relationship between the square target and the circular target.

[0056] Step 140: based on the pixel coordinates and vehicle body coordinates of the center, and the pixel coordinates and vehicle body coordinates of the corner points, the external parameters of the camera are calibrated.

[0057] In the embodiments of the present application, after the pixel coordinates of the center of the circular target and the pixel coordinates of the square target are calculated, since the vehicle body coordinates of the center of the circular target and the vehicle body coordinates of the corner points of the square target are known, the external parameters of the camera can be calculated according to the pixel coordinates and vehicle body coordinates of the center of the circular target, and the pixel coordinates and vehicle body coordinates of the corner points of the square target, that is, the rotation matrix and translation matrix of the pixel coordinate system to the vehicle body coordinate system.

[0058] The parameter calibration method provided in the embodiment first acquires a target image including a square target and a circular target through a camera, determines pixel coordinates of corner points of the square target from the target image, determines pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points, and finally calibrates external parameters of the camera based on the pixel coordinates of the center and body coordinates of the vehicle and the pixel coordinates of the corner points and body coordinates of the vehicle. Through the above method, since the center coordinates of the circular target are used for calibration in calibration, even if the circular target becomes an ellipse in a very serious distortion scene (i.e., a region far away from the vehicle and having large edge distortion in fisheye camera imaging), the center position is very accurate. Therefore, calibration of the external parameters of the camera based on the pixel coordinates and body coordinates of the circular target can improve the accuracy of parameter calibration.

[0059] Referring to Figure 6 The parameter calibration method provided in the application is applied to a vehicle, the vehicle is installed with a camera, and the method comprises the following steps.

[0060] In step 210, a target image is acquired through the camera, and the target image is an image including a square target and a circular target.

[0061] In the embodiment of the application, when the vehicle is at a specified position, the vehicle can start to acquire a target image through the camera installed in the vehicle upon receiving an image acquisition instruction. The image acquisition instruction can be an instruction sent by an electronic device in communication connection with the vehicle, or can be an instruction automatically triggered by the vehicle when a specified function of the vehicle is detected to be used by a user.

[0062] As one of the modes, if the image acquisition instruction is an instruction sent by an electronic device in communication connection with the vehicle, the electronic device can be provided with a specified application program capable of controlling whether the vehicle acquires a target image, the electronic device sends an image acquisition instruction to the vehicle when the specified application program in the electronic device starts to run, and the vehicle starts to acquire a target image through the camera installed in the vehicle upon receiving the image acquisition instruction sent by the electronic device.

[0063] As another mode, if the image acquisition instruction is an instruction automatically triggered by the vehicle when a specified function of the vehicle is detected to be used by a user, the image acquisition instruction can be automatically triggered when a photographing function or an image recognition function of the vehicle is detected to be used by a user, and then the vehicle can control the camera to acquire a target image based on the image acquisition instruction.

[0064] Optionally, the target image can also be an image including the square targets and the circular targets obtained from a local storage area of the vehicle. In this way, the vehicle can be placed at a specified position in advance, and the image including the square targets and the circular targets can be collected by the camera installed in the vehicle. After the image including the square targets and the circular targets is collected, the image including the square targets and the circular targets can be stored. It should be noted that, in order to distinguish which camera in the vehicle collects the target image during subsequent calibration, the images collected by different cameras can be stored separately when stored. The image collected by the front camera installed in the vehicle is stored in file 1, the image collected by the rear camera installed in the vehicle is stored in file 2, the image collected by the left camera installed in the vehicle is stored in file 3, and the image collected by the right camera installed in the vehicle is stored in file 4. Then, when the parameters of the camera are calibrated, the target image is obtained from the corresponding file according to which camera needs to be calibrated. Meanwhile, when the images collected by different cameras are stored, the installation position and the shooting angle of the camera can also be stored.

[0065] Step 220: determining a first region of interest from the target image, the first region of interest being a region including the image of the square target.

[0066] In the embodiments of the present application, in image processing, the region to be processed is outlined in a box, a circle, an ellipse, an irregular polygon, etc. from the processed image, which is called a region of interest (ROI).

[0067] Various operators and functions are commonly used on machine vision software such as Halcon, OpenCV and Matlab to obtain the region of interest and perform the next step of image processing. In the field of image processing, the region of interest is a selected image region from an image, which is the focus of image analysis. Using the region of interest to enclose the image region of interest can reduce the image processing time and increase the accuracy.

[0068] As a way, the first region of interest is a region of interest of a specified size, and the first region of interest is a region for enclosing the image of the square target. When setting the size of the first region of interest, the size of the square target in the designed calibration plate can be set to make the first region of interest as large as possible to ensure that the square target will fall into the first region of interest. It can be understood that the first region of interest in the embodiments of the present application can enclose at least two intersecting square targets.

[0069] In the embodiments of the present application, how many intersecting square targets are detected in the target image can determine how many first regions of interest. For example, as shown in Figure 7 Figure 7 For the target image obtained by the front camera of the vehicle, Figure 7 The dashed circle in the target image can be regarded as a first region of interest. As can be seen, Figure 7 There are four intersecting square targets in the target image, so four first regions of interest can be determined from the target image.

[0070] Step 230: Obtain the pixel coordinates of the corner points of the square target from the first region of interest.

[0071] In the embodiments of the present application, when the first region of interest is determined from the target image by the above method, the corner points of the square target can be detected to obtain the pixel coordinates of the corner points of the square target.

[0072] Step 240: Determine a second region of interest from the target image based on the pixel coordinates of the corner points, the second region of interest being a region including the image of the circular target.

[0073] In the embodiments of the present application, for the detection of the circular target, because the installation angle of the camera of each vehicle can vary, if a fixed second region of interest is manually divided, some circular targets can not be enclosed, resulting in failure of calibration. Therefore, the second region of interest can be adaptively calculated based on the pixel coordinates of the corner points of the square target, so as to improve the success rate of calibration.

[0074] In the embodiments of the present application, the second region of interest is a region that can enclose the image of the circular target.

[0075] As a way, the second region of interest, that is, the approximate region where the circular target is located, can be calculated in real time according to the pixel coordinates and body coordinates of the corner points of the square target, the body coordinates of the circular target, and the positional relationship between the circular target and the square target.

[0076] Step 250: Obtain the pixel coordinates of the center of the circular target from the second region of interest.

[0077] In the embodiments of the present application, after the second region of interest is determined, the circular target can be recognized according to the second region of interest, so as to determine the pixel coordinates of the center of the circular target.

[0078] Step 260: Calibrate the external parameters of the camera based on the pixel coordinates and body coordinates of the center of the circular target, and the pixel coordinates and body coordinates of the corner points.

[0079] ​In the embodiments of the present application, the detailed explanation of the steps included in step 260 can refer to the corresponding steps in the foregoing embodiments, which will not be repeated here.

[0080] The parameter calibration method provided in the embodiment determines a first region of interest from the target image, obtains the pixel coordinates of the corner points of the square target from the first region of interest, then determines a second region of interest from the target image based on the pixel coordinates of the corner points, obtains the pixel coordinates of the center of the circular target from the second region of interest, and finally calibrates the external parameters of the camera based on the pixel coordinates and the vehicle body coordinates of the center and the pixel coordinates and the vehicle body coordinates of the corner points. Through the above method, since the center coordinates of the circular target are used for calibration in the calibration, even if the circular target becomes an ellipse in a very serious distortion scene, the center position is very accurate, and therefore, the calibration of the external parameters of the camera based on the pixel coordinates and the vehicle body coordinates of the circular target can improve the accuracy of the parameter calibration.

[0081] Please refer to Figure 8 The parameter calibration method provided in the present application is applied to a vehicle, the vehicle is provided with a camera, and the method comprises the following steps.

[0082] Step 310: obtaining a target image by the camera, the target image being an image comprising a square target and a circular target.

[0083] Step 320: determining a first region of interest from the target image, the first region of interest being a region comprising the image of the square target.

[0084] In the embodiments of the present application, the detailed explanation of the steps included in step 310 and step 320 can refer to the corresponding steps in the foregoing embodiments, which will not be repeated here.

[0085] Step 330: matching the square target included in the first region of interest with a preset square target template to obtain the pixel coordinates of the corner points of the square target.

[0086] In the embodiments of the present application, the preset square target template is a template used for searching the corner points of the square target in the first region of interest. Different square targets correspond to different preset square target templates.

[0087] As a manner, after the first region of interest is determined, the target image can be cropped according to the size of the first region of interest to obtain an image corresponding to the first region of interest, and then the image of the first region of interest and different preset square target templates are matched one by one. If the difference between the image of the first region of interest and the preset square target template is less than a threshold, it is determined that the image of the first region of interest matches the preset square target template, and then the pixel coordinates of the corner points of the preset square target template can be taken as the pixel coordinates of the corner points of the square target included in the first region of interest.

[0088] Step 340: determining a second region of interest from the target image based on the pixel coordinates of the corner points, the second region of interest being a region including the image of the circular target.

[0089] In the embodiments of the present application, the detailed explanation of the steps included in step 340 can refer to the corresponding steps in the foregoing embodiments, which will not be repeated here.

[0090] Step 350: performing a binarization processing on the image corresponding to the second region of interest to obtain a binarization image.

[0091] In the embodiments of the present application, after the second region of interest is determined, the target image can be cropped based on the size of the second region of interest to obtain an image corresponding to the second region of interest, and then the image corresponding to the second region of interest is binarized. The gray value of the pixel points on the image corresponding to the second region of interest is set to 0 or 255, that is, the entire image corresponding to the second region of interest presents a clear black and white effect.

[0092] As a manner, the gray values of different pixel points are set to 0 or 255 through a preset threshold. In the embodiments of the present application, according to the preset threshold, the gray value of the pixel points included in the circular target in the second region of interest can be set to 0, and the gray value of other pixel points in the second region of interest can be set to 255.

[0093] Step 360: determining a connected domain of the binarization image.

[0094] In the embodiments of the present application, the connected domain of the binarization image refers to a region composed of pixel points with a gray value of 0, that is, a region where the circular target is located.

[0095] Step 370: determining the pixel coordinates of the center of the circular target based on the connected domain.

[0096] In the embodiments of the present application, the center of the circular target is determined by identifying the circular target, so as to determine the pixel coordinates of the center. As a manner, the center of the circular target can be determined by calculating the zeroth moment and the first moment of the contour, and of course, the center of the circular target can also be determined by other methods which can calculate the center of the circular target, which is not limited here.

[0097] Step 380: calibrating the external parameters of the camera based on the pixel coordinates and the vehicle coordinates of the center, and the pixel coordinates and the vehicle coordinates of the corner points.

[0098] Optionally, in the embodiments of the present application, when the regions of interest are determined from the target image, the first region of interest and the second region of interest can be simultaneously determined from the target image according to the preset region size. When the first region of interest and the second region of interest are simultaneously determined from the target image according to the preset region size, the pixel coordinates of the corner points of the square target can be obtained by matching the square target included in the first region of interest with the preset square target template; the image corresponding to the second region of interest is binarized to obtain a binarized image, the connected domain of the binarized image is determined, and the pixel coordinates of the center of the circular target are determined based on the connected domain.

[0099] Then, it is determined whether the number of the square targets included in the target image is consistent with the actual number of the square targets, and whether the number of the circular targets included in the target image is consistent with the actual number of the circular targets. If the number is incorrect and the number is too small, a homography matrix is calculated based on the pixel coordinates and the vehicle coordinates of the corner points of the four square targets, then the vehicle coordinates of the circular target are projected into the pixel coordinate system to obtain a new second region of interest, the pixel coordinates of the center of the circular target are detected based on the new second region of interest, and after the detection, the external parameters of the camera are calibrated based on the vehicle coordinates and the pixel coordinates of the center of the circular target, and the pixel coordinates and the vehicle coordinates of the corner points of the square target.

[0100] The parameter calibration method provided in the embodiment comprises the following steps: obtaining a target image by using a camera; determining a first region of interest from the target image; matching a square target included in the first region of interest with a preset square target template to obtain pixel coordinates of corner points of the square target; determining a second region of interest from the target image based on the pixel coordinates of the corner points; performing binary processing on an image corresponding to the second region of interest to obtain a binary image; determining a connected domain of the binary image; determining pixel coordinates of a center of a circular target based on the connected domain; and finally calibrating external parameters of the camera based on the pixel coordinates of the center and the body coordinates of the center and the pixel coordinates of the corner points and the body coordinates of the corner points. Through the above method, since the center coordinates of the circular target are used for calibration in the calibration, even if the circular target becomes an ellipse in a very serious distortion scene, the center position is very accurate. Therefore, the external parameters of the camera are calibrated based on the pixel coordinates and the body coordinates of the circular target, so that the accuracy of the parameter calibration can be improved.

[0101] Please refer to Figure 9 The parameter calibration method provided in the application is applied to a vehicle, the vehicle is provided with a camera, and the method comprises the following steps.

[0102] Step 410: obtaining a target image by using the camera, the target image being an image comprising a square target and a circular target.

[0103] Step 420: determining a first region of interest from the target image, the first region of interest being a region comprising an image of the square target.

[0104] Step 430: obtaining pixel coordinates of corner points of the square target from the first region of interest.

[0105] In the embodiments of the application, the detailed explanations of the steps included in steps 410, 420 and 430 can be referred to the corresponding steps in the foregoing embodiments, which will not be described herein.

[0106] Step 440: obtaining body coordinates of a center of the circular target and body coordinates of the corner points.

[0107] Since the body coordinates of the center of the circular target and the body coordinates of the corner points of the square target are known when the calibration board is set, in the embodiments of the application, as long as it is determined that the square target and the circular target in the target image are the circular target and the square target at which position of the calibration board, the body coordinates of the center of the circular target and the body coordinates of the corner points of the square target in the target image can be directly obtained.

[0108] Step 450: determining a homography matrix based on the pixel coordinates of the corner points and the body coordinates of the corner points.

[0109] In the embodiment of the present application, the homography matrix is a transformation matrix used to describe the position mapping relationship between the object in the vehicle body coordinate system and the pixel coordinate system.

[0110] As a manner, the transformation matrix of the position mapping relationship between the square target in the vehicle body coordinate system and the pixel coordinate system can be calculated according to the pixel coordinates and the vehicle body coordinates of the square target.

[0111] Step 460: determining the second region of interest from the target image based on the homography matrix and the vehicle body coordinates of the center of the circle.

[0112] As a manner, the second region of interest is determined from the target image based on the homography matrix by projecting the vehicle body coordinates of the center of the circle into the pixel coordinate system.

[0113] In the embodiment of the present application, the vehicle body coordinates of the center of the circle target can be back-projected into the pixel coordinate system according to the transformation matrix of the position mapping relationship between the square target in the vehicle body coordinate system and the pixel coordinate system calculated above, so that the approximate position of the center of the circle target in the pixel coordinate system can be determined, and thus the second region of interest can be determined from the target image.

[0114] Step 470: obtaining the pixel coordinates of the center of the circle target from the second region of interest.

[0115] Step 480: calibrating the external parameters of the camera based on the pixel coordinates and the vehicle body coordinates of the center of the circle, and the pixel coordinates and the vehicle body coordinates of the corner point.

[0116] In the embodiment of the present application, the detailed explanation of the steps included in steps 470 and 480 can refer to the corresponding steps in the foregoing embodiments, which will not be repeated here.

[0117] The parameter calibration method provided in the embodiment comprises the following steps: obtaining a target image by using a camera, determining a first region of interest from the target image, obtaining pixel coordinates of a corner point of a square target from the first region of interest, determining a homography matrix based on the pixel coordinates of the corner point and vehicle body coordinates of the corner point, determining a second region of interest from the target image based on the homography matrix and vehicle body coordinates of the center of the circle, obtaining pixel coordinates of the center of the circle target from the second region of interest, and finally calibrating external parameters of the camera based on the pixel coordinates and the vehicle body coordinates of the center of the circle, and the pixel coordinates and the vehicle body coordinates of the corner point. Through the above method, since the center coordinates of the circle target are used for calibration in the calibration, even if the circle target becomes an ellipse in a very serious distortion scene, the center position is very accurate, and thus the calibration of the external parameters of the camera based on the pixel coordinates and the vehicle body coordinates of the circle target can improve the accuracy of the parameter calibration.

[0118] Referring to Figure 10 The application provides a parameter calibration device 500, which is arranged in a vehicle, wherein the vehicle is provided with a camera, and the parameter calibration device 500 comprises:

[0119] An image acquisition unit 510 is configured to acquire a target image by using the camera, wherein the target image is an image comprising a square target and a circular target.

[0120] A first coordinate acquisition unit 520 is configured to determine pixel coordinates of corner points of the square target from the target image.

[0121] As a manner, the first coordinate acquisition unit 520 is further configured to determine a first region of interest from the target image, wherein the first region of interest is a region comprising an image of the square target; and acquire the pixel coordinates of the corner points of the square target from the first region of interest.

[0122] Optionally, the first coordinate acquisition unit 520 is specifically configured to match the square target comprised in the first region of interest with a preset square target template, so as to obtain the pixel coordinates of the corner points of the square target.

[0123] A second coordinate determination unit 530 is configured to determine pixel coordinates of a center of the circular target based on the pixel coordinates of the corner points.

[0124] As a manner, the second coordinate acquisition unit 530 is configured to determine a second region of interest from the target image based on the pixel coordinates of the corner points, wherein the second region of interest is a region comprising an image of the circular target; and acquire the pixel coordinates of the center of the circular target from the second region of interest.

[0125] Optionally, the second coordinate acquisition unit 530 is further configured to perform binaryzation processing on an image corresponding to the second region of interest, so as to obtain a binaryzation image; determine a connected domain of the binaryzation image; and determine the pixel coordinates of the center of the circular target based on the connected domain.

[0126] Optionally, the second coordinate acquisition unit 530 is further configured to acquire vehicle body coordinates of the center of the circular target and vehicle body coordinates of the corner points; determine a homography matrix based on the pixel coordinates of the corner points and the vehicle body coordinates of the corner points; and determine the second region of interest from the target image based on the homography matrix and the vehicle body coordinates of the center.

[0127] Optionally, the second coordinate obtaining unit 530 is further configured to project the vehicle body coordinate of the circle center into a pixel coordinate system based on the homography matrix, and determine the second region of interest from the target image.

[0128] The calibration unit 540 is configured to calibrate the external parameters of the camera based on the pixel coordinate and the vehicle body coordinate of the circle center, and the pixel coordinate and the vehicle body coordinate of the corner point.

[0129] It should be noted that, for the convenience and brevity of description, the specific working process of the above-described device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here. In the several embodiments provided by the present application, the coupling between the modules can be electrical, mechanical or other forms of coupling. In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically independently, or two or more modules can be integrated into one module. The above integrated module can be realized in the form of hardware or in the form of a software functional module.

[0130] The above will be described in detail below with reference to the accompanying drawings. Figure 11 A vehicle is described.

[0131] Please refer to Figure 11 Based on the above parameter calibration method and device, the present embodiment further provides another vehicle 100 which can execute the foregoing parameter calibration method. The vehicle 100 in the present application can include one or more (only one is shown in the figure) processors 102, a memory 104, a wireless module 106 and a camera 108 which are coupled with each other. The memory 104 stores programs which can execute the contents in the foregoing embodiments, and the processor 102 can execute the programs stored in the memory 104.

[0132] The processor 102 can include one or more processing cores. The processor 102 connects various parts within the vehicle 100 by various interfaces and lines, and performs various functions of the vehicle 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Alternatively, the processor 102 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 102 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 102, but can be implemented separately by a communication chip.

[0133] The memory 104 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 104 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each of the following method embodiments, etc. The data storage area can also store data created by the vehicle 100 in use (such as a phonebook, audio and video data, chat record data), etc.

[0134] The wireless module 106 is used to receive and send electromagnetic waves, realize mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices, for example, communicate with an audio playing device. The wireless module 106 can include various existing circuit elements for performing these functions, for example, an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a subscriber identity module (SIM) card, a memory, etc. The wireless module 106 can communicate with various networks such as the Internet, an intranet, a wireless network, or communicate with other devices through the wireless network. The above-mentioned wireless network can include a cellular phone network, a wireless local area network or metropolitan area network.

[0135] The camera 108 is configured to capture a target image in real time and send the target image to the processor 102 for processing. In the embodiments of the present application, the camera can be a surround-view camera, a monocular camera, or a fisheye camera, etc.

[0136] Please refer to Figure 12 which shows a structural block diagram of a computer readable storage medium provided by the embodiments of the present application. The computer readable storage medium 800 stores program codes, which can be invoked by a processor to execute the methods described in the above method embodiments.

[0137] The computer readable storage medium 800 can be an electronic storage such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer readable storage medium 800 includes a non-transitory computer readable medium. The computer readable storage medium 800 has a storage space for program codes 810 to execute any of the above methods. These program codes can be read from or written into one or more computer program products. The program codes 810 can be compressed in an appropriate form, for example.

[0138] In summary, the parameter calibration method, device, vehicle, and storage medium provided by the present application first acquire a target image including a square target and a circular target through a camera, determine the pixel coordinates of the corner points of the square target from the target image, determine the pixel coordinates of the center of the circular target based on the pixel coordinates of the corner points, and finally calibrate the external parameters of the camera based on the pixel coordinates and the vehicle coordinates of the center, and the pixel coordinates and the vehicle coordinates of the corner points. Through the above method, since the center coordinates of the circular target are used for calibration in the calibration, even if it becomes an ellipse in a very serious distortion scene, the center position is very accurate. Therefore, calibrating the external parameters of the camera based on the pixel coordinates and the vehicle coordinates of the circular target can improve the accuracy of parameter calibration.

[0139] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A parameter calibration method, characterized in that, Applied to a vehicle, the vehicle is installed with a camera, the method comprises: obtaining a target image through the camera, the target image is an image including square targets and a circular target, the circular target is arranged at a position far from the installed camera, and the square target is arranged at a position close to the installed camera; determining pixel coordinates of corner points of the square target from the target image, the corner points being intersection points of two square targets; obtaining vehicle body coordinates of a center of the circular target and vehicle body coordinates of the corner points; determining a homography matrix based on the pixel coordinates of the corner points and the vehicle body coordinates of the corner points; determining a second region of interest from the target image based on the homography matrix and the vehicle body coordinates of the center of the circular target, the second region of interest being a region including an image of the circular target; obtaining pixel coordinates of the center of the circular target from the second region of interest; calibrating external parameters of the camera based on the pixel coordinates and vehicle body coordinates of the center of the circular target and the pixel coordinates and vehicle body coordinates of the corner points.

2. The method of claim 1, wherein, The determination of the pixel coordinates of the corner points of the square target from the target image comprises: determining a first region of interest from the target image, the first region of interest being a region including an image of the square target; obtaining pixel coordinates of the corner points of the square target from the first region of interest.

3. The method of claim 2, wherein, The obtaining of the pixel coordinates of the corner points of the square target from the first region of interest comprises: matching the square target included in the first region of interest with a preset square target template to obtain the pixel coordinates of the corner points of the square target.

4. The method of claim 1, wherein, The obtaining of the pixel coordinates of the center of the circular target from the second region of interest comprises: performing binaryzation processing on an image corresponding to the second region of interest to obtain a binaryzation image; determining a connected domain of the binaryzation image; determining the pixel coordinates of the center of the circular target based on the connected domain.

5. The method of claim 1, wherein, The determination of the second region of interest from the target image based on the homography matrix and the vehicle body coordinates of the center of the circular target comprises: projecting the vehicle body coordinates of the center of the circular target into a pixel coordinate system based on the homography matrix to determine the second region of interest from the target image.

6. A parameter calibration apparatus characterized by comprising: Running on a vehicle, the vehicle is installed with a camera, the device comprises: an image acquisition unit, configured to obtain a target image through the camera, the target image being an image including square targets and a circular target, the circular target being arranged at a position far from the installed camera, and the square target being arranged at a position close to the installed camera; a first coordinate acquisition unit, configured to determine pixel coordinates of corner points of the square target from the target image, the corner points being intersection points of two square targets; A second coordinate determining unit is configured to obtain a vehicle body coordinate of a center of the circular target and a vehicle body coordinate of the corner point; determine a homography matrix based on the pixel coordinate of the corner point and the vehicle body coordinate of the corner point; determine a second region of interest from the target image based on the homography matrix and the vehicle body coordinate of the center, the second region of interest being a region of the image including the circular target; and obtain a pixel coordinate of the center of the circular target from the second region of interest. A calibration unit is configured to calibrate external parameters of the camera based on the pixel coordinate and the vehicle body coordinate of the center and the pixel coordinate and the vehicle body coordinate of the corner point.

7. A vehicle characterized by comprising: A computer readable storage medium stores one or more programs configured to be executed by one or more processors of a camera, the one or more programs including instructions for performing any of the methods of claims 1-5.

8. A computer readable storage medium, characterized in that, A computer readable storage medium stores one or more programs configured to be executed by one or more processors of a camera, the one or more programs including instructions for performing any of the methods of claims 1-5.

Images