Calibration method and device of vehicle-mounted surround view system

Abstract

The application provides a calibration method and device of a vehicle-mounted surround view system, and relates to the technical field of intelligent vehicles. The method comprises the following steps: acquiring an image of a calibration object collected by a vehicle-mounted camera, the calibration object comprising multiple regions, at least one region comprising a pattern containing identification information, identifying the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern, and determining the calibration external parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner points. Thus, the external parameter calibration of the camera is performed by identifying the corner points of the pattern of the calibration object, the distortion generated when the edge region of the image is identified is reduced, the requirement for ambient light and the arrangement difficulty of the calibration object are reduced, the identification rate of the corner points is improved, and the efficiency and accuracy of the calibration are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle technology, and in particular to a calibration method and apparatus for an in-vehicle surround view system. Background Technology

[0002] Currently, most automotive surround view systems use checkerboard calibration for their image acquisition devices. This involves arranging a checkerboard pattern around the front, rear, left, and right sides of a vehicle parked in a fixed position, recording the checkerboard parameters, locating the checkerboard directly opposite the image acquisition device, and automatically calibrating the camera's extrinsic parameters by combining the physical coordinates of the checkerboard corners with these parameters. However, the checkerboard pattern located at the edge of the camera image produces significant distortion and requires high ambient light, making corner identification difficult. This results in low efficiency and accuracy in calibrating extrinsic parameters, and the checkerboard arrangement is relatively complex. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in the related art.

[0004] Therefore, one objective of this application is to propose a calibration method for an in-vehicle surround view system.

[0005] The second objective of this application is to provide a calibration device for an in-vehicle surround view system.

[0006] The third objective of this application is to propose a vehicle.

[0007] The fourth objective of this application is to propose a calibrator.

[0008] The fifth objective of this application is to propose a vehicle.

[0009] The sixth objective of this application is to provide an electronic device.

[0010] The seventh objective of this application is to propose a vehicle.

[0011] The eighth objective of this application is to provide a computer-readable storage medium.

[0012] To achieve the above objectives, a first aspect of this application proposes a calibration method for an in-vehicle surround view system, comprising: acquiring an image of a calibration object captured by an in-vehicle camera, the calibration object comprising multiple regions, at least one of the regions comprising a code pattern containing identification information; identifying the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern; and determining the calibration extrinsic parameters of the camera based on the identification information of the pattern and the two-dimensional coordinates of the corner points.

[0013] According to one embodiment of this application, the plurality of regions include: a plurality of first regions corresponding to the image edges in the calibration object and a plurality of second regions arranged at intervals from the first regions, wherein the first regions and the second regions include at least one of the patterns.

[0014] According to one embodiment of this application, the density of the pattern in the first region is different from the density of the pattern in the second region.

[0015] According to one embodiment of this application, the size of the pattern in the first region is different from the size of the pattern in the second region.

[0016] According to one embodiment of this application, identifying the two-dimensional coordinates of the corner points of the pattern includes: identifying the pattern in the image; when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, the image is re-acquired or the recognition results of the pattern in multiple frames of the image are superimposed for recognition until the recognition rate reaches the preset recognition rate threshold.

[0017] According to one embodiment of this application, the calibration method of the vehicle surround view system of this application further includes: identifying the pattern of no less than three regions in the image, wherein the three regions include at least one first region and at least one second region.

[0018] According to one embodiment of this application, before determining the calibration extrinsic parameters of the camera based on the identification information of the pattern and the two-dimensional coordinates of the corner points, the method further includes: performing distortion removal processing on the two-dimensional coordinates of the corner points.

[0019] According to one embodiment of this application, determining the calibration extrinsic parameters of the camera based on the identification information of the pattern and the two-dimensional coordinates of the corner points includes: determining the calibration extrinsic parameters of the camera based on the identification information of the pattern, the two-dimensional coordinates of the corner points, and the three-dimensional coordinates of the corresponding points in the scaling pattern.

[0020] According to one embodiment of this application, determining the camera's calibration extrinsic parameters based on the pattern's identification information and the two-dimensional coordinates of the corner points includes: determining candidate calibration extrinsic parameters of the camera corresponding to a single region in the image based on the pattern's identification information and the two-dimensional coordinates of the corner points; determining the projection error of the corner points based on the candidate calibration extrinsic parameters, the pattern's identification information, and the two-dimensional coordinates of the corner points; and if the projection error is less than a preset projection error threshold, then determining the camera's calibration extrinsic parameters based on the pattern's identification information and the two-dimensional coordinates of the corner points.

[0021] According to one embodiment of this application, the calibration method of the vehicle surround view system of this application further includes: if the projection error is equal to or greater than the projection error threshold, outputting a reminder message, the reminder message being used to remind the user to change the calibration site or re-lay the calibration object.

[0022] To achieve the above objectives, a second aspect of this application provides a calibration device for a vehicle surround view system, comprising: a first acquisition module for acquiring an image of a calibration object captured by a vehicle camera, the calibration object comprising multiple regions, at least one of the regions comprising a pattern containing identification information; an identification module for identifying the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern; and a determination module for determining the calibration extrinsic parameters of the camera based on the identification information of the pattern and the two-dimensional coordinates of the corner points.

[0023] According to one embodiment of this application, the recognition module is specifically used for: recognizing the pattern in the image; when the recognition rate of the pattern in the image is lower than a preset recognition rate threshold, the image is re-acquired or the recognition results of the pattern in multiple frames of the image are superimposed for recognition until the recognition rate reaches the preset recognition rate threshold.

[0024] According to one embodiment of this application, the determining module is further configured to: perform distortion removal processing on the two-dimensional coordinates of the corner point before determining the calibration extrinsic parameters of the camera based on the two-dimensional coordinates of the corner point.

[0025] According to one embodiment of this application, the determining module is specifically used to: determine the calibration extrinsic parameters of the camera based on the identification information of the pattern, the two-dimensional coordinates of the corner point, and the three-dimensional coordinates of the corresponding point in the scaling pattern.

[0026] According to one embodiment of this application, the determining module is specifically configured to: determine candidate calibration extrinsic parameters of the camera corresponding to a single region in the image based on the identification information of the pattern and the two-dimensional coordinates of the corner point; determine the projection error of the corner point based on the candidate calibration extrinsic parameters, the identification information of the pattern, and the two-dimensional coordinates of the corner point; if the projection error is less than a preset projection error threshold, then determine the calibration extrinsic parameters of the camera based on the identification information of the pattern and the two-dimensional coordinates of the corner point.

[0027] According to one embodiment of this application, the determining module is further configured to: if the projection error is equal to or greater than the projection error threshold, output a reminder message, the reminder message being used to remind the user to change the calibration site or re-lay the calibration object.

[0028] To achieve the above objectives, a third aspect of this application provides a vehicle including a calibration device for an in-vehicle surround view system as described in a second aspect of this application.

[0029] To achieve the above objectives, a fourth aspect of this application provides a calibration object for calibrating an in-vehicle surround view system, comprising: a plurality of first regions located in the calibration object corresponding to the edge of the image and a plurality of second regions arranged at intervals from the first regions, wherein the first regions and the second regions include at least one pattern containing identification information.

[0030] According to one embodiment of this application, the density of the pattern in the first region is different from the density of the pattern in the second region. According to one embodiment of this application, the size of the pattern in the first region is different from the size of the pattern in the second region.

[0031] To achieve the above objectives, a fifth aspect of this application provides a vehicle in which the vehicle's onboard surround view system performs calibration by acquiring images of a calibration object as described in the fourth aspect of this application.

[0032] To achieve the above objectives, a sixth aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the calibration method for an in-vehicle surround view system as described in the first aspect of this application. To achieve the above objectives, a seventh aspect of this application provides a vehicle, including the electronic device as described in the sixth aspect of this application.

[0033] To achieve the above objectives, an eighth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the calibration method for an in-vehicle surround view system as described in the first aspect of this application. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a calibration method for an in-vehicle surround view system according to an exemplary embodiment of this application;

[0035] Figure 2 This is a schematic diagram illustrating a calibration method for a vehicle surround view system according to an exemplary embodiment of this application;

[0036] Figure 3 This is a flowchart illustrating another calibration method for an in-vehicle surround view system according to an exemplary embodiment of this application;

[0037] Figure 4 This is a flowchart illustrating another calibration method for an in-vehicle surround view system according to an exemplary embodiment of this application;

[0038] Figure 5 This is a block diagram illustrating a calibration device for a vehicle surround view system according to an exemplary embodiment of this application;

[0039] Figure 6 This is a schematic diagram of the structure of a vehicle according to an exemplary embodiment of this application;

[0040] Figure 7 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of this application;

[0041] Figure 8 This is a schematic diagram of the structure of another vehicle according to an exemplary embodiment of this application. Detailed Implementation

[0042] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0043] Figure 1 This is a flowchart illustrating a calibration method for a vehicle surround view system according to an exemplary embodiment of this application, such as... Figure 1 As shown, the calibration method for this vehicle surround view system includes the following steps:

[0044] S101, acquire an image of a calibration object captured by an onboard camera. The calibration object includes multiple regions, and at least one region includes a pattern containing identification information.

[0045] Specifically, the calibration method of the vehicle surround view system in this application embodiment can be executed by the vehicle surround view system calibration device provided in this application embodiment. The vehicle surround view system calibration device can be set in the vehicle surround view system to provide calibration services for the system.

[0046] The image of the calibration object is acquired from a vehicle-mounted camera. The vehicle-mounted camera is located within a vehicle surround-view system, which is typically a panoramic surround-view system composed of images and videos stitched together from cameras (such as fisheye or wide-angle cameras) installed in the four directions (front, rear, left, and right) of the vehicle body. The calibration object is the object used to calibrate the vehicle surround-view system. The calibration object includes multiple regions, including multiple first regions corresponding to the edges of the image and multiple second regions spaced apart from the first regions. Each first and second region includes at least one pattern. For example... Figure 2As shown, for example, eight regions can be arranged in the four directions of the calibration object: top, bottom, left, and right. The four regions corresponding to the image edges (i.e., the four corners) are designated as the first region 21, and the four regions corresponding to the middle of the four sides and spaced apart from the first regions are designated as the second region 22. For ease of description, the region containing the first region 21 is called the edge region of the calibration object, and the region containing the second region 22 is called the non-edge region of the calibration object. These regions include patterns containing identification information. The patterns can be April Tags, one-dimensional barcodes, two-dimensional barcodes, or any pattern with information recording function, or a pattern with a certain degree of recognizability. The patterns in these regions can be one, multiple, or a combination of multiple patterns. The pattern density varies in different regions. The edge region of the calibration object differs from the non-edge region; that is, the pattern density in the first edge region is less than or greater than the pattern density in the second non-edge region. The pattern size also varies in different regions; the pattern size in the first edge region is less than or greater than the pattern size in the second non-edge region.

[0047] The identification information recorded in the pattern can be numbered information, text information, location information, or other information, such as 1, 2, 3, 4... or one, two, three, four... etc. The pattern can also be a pattern with a certain degree of recognizability, such as animal image, human image, plant image, geometric shape, etc. All patterns on the marker are arranged in a preset order so that each pattern or pattern combination has a corresponding position. The vehicle stores or temporarily stores the preset order information of all patterns. By recognizing the image captured by the recognition camera, the coordinate position of all patterns can be obtained by recognizing the pattern or pattern combination.

[0048] The calibration object may also have a central area for parking the vehicle to be calibrated.

[0049] Taking the AprilTag pattern as an example, each camera in a vehicle surround view system can collect three sets of AprilTags for different areas. Since each camera covers three areas—left, middle, and right—the three sets of AprilTags represent these three areas respectively. Therefore, collecting three sets of AprilTags ensures the comprehensiveness and completeness of the images captured by each camera, covering the left, middle, and right sides of the image. This makes the collected data more accurate and reliable, and the calibration stability better, thus further improving the calibration effect. For example... Figure 2The three April Tags captured by the camera on the left are located in the upper left, left, and lower left regions, respectively. In terms of April Tag density, the density of April Tags in the edge region of the calibration object is smaller than that in the non-edge region of the calibration object. That is, the density of April Tags in the first edge region is less than that in the second non-edge region. In terms of the grid size corresponding to April Tags, the grid size of April Tags in the edge region of the calibration object is larger than that in the non-edge region of the calibration object. That is, the grid size of April Tags in the first edge region is larger than that of April Tags in the second non-edge region. In addition, the ID (Identity document) of April Tags is also unique.

[0050] S102, identifies the pattern's identification information and the two-dimensional coordinates of the pattern's corner points.

[0051] Specifically, the April Tag is a rectangular pattern with four corners, each corresponding to a corner point. Adjacent April Tags may share corner points. The image of the calibration object acquired by the vehicle-mounted camera in step S101 is identified to obtain the identification information recorded in each April Tag in the image and the two-dimensional coordinates of each corner point of each April Tag. The corner points are the four vertices and the center point of each April Tag. The sequence numbers of the four vertices are based on pre-defined real sequence numbers in physical space, i.e., the vertex sequence numbers are known. The coordinates of the corner points are the coordinates of the corner points on the plane where the image of the calibration object is located.

[0052] It should be noted that April Tag's corner recognition does not have high requirements for the lighting conditions around the vehicle. It does not need to detect all corners in one frame, but can extract the coordinates of corners based on the cumulative results of multiple frames, thereby calibrating the vehicle surround view system.

[0053] S103 determines the camera's calibration extrinsic parameters based on the pattern's identification information and the two-dimensional coordinates of the corner points.

[0054] Specifically, based on the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern identified in step S102, the calibration extrinsic parameters of the corresponding vehicle camera are determined.

[0055] It should be noted that calibrating extrinsic parameters refers to calibrating and determining the extrinsic parameters of the vehicle-mounted camera. The camera extrinsic parameters, also known as camera pose, consist of a rotation matrix R and a translation matrix t. Calibrating the camera extrinsic parameters means determining a rotation matrix and a translation matrix corresponding to the camera, which is used to describe the transformation relationship between the camera coordinate system and other coordinate systems, such as the transformation relationship between the camera coordinate system and the vehicle body coordinate system.

[0056] Here, we take solving the extrinsic parameters of the panoramic camera coordinate system and the world coordinate system as an example. Camera extrinsic parameter calibration is to establish a world coordinate system with the help of a calibration object and fix the transformation relationship between the vehicle body coordinate system and the world coordinate system. The vehicle body coordinate system requires that the camera be fixed on the vehicle body and kept relatively stationary. The world coordinate system can be the state in which the position and attitude relationship between the vehicle body and the calibration object are always fixed when the vehicle is parked. In this way, the extrinsic parameters of the camera coordinate system and the world coordinate system are determined by using the imaging information of the calibration plate on the image plane and the inherent size information on the calibration object.

[0057] In this embodiment, an image of a calibration object comprising multiple regions is acquired from a vehicle-mounted camera, with at least one region including a pattern containing identification information. The identification information of the pattern and the two-dimensional coordinates of the corner points in the pattern are identified. Based on the identification information of the pattern and the two-dimensional coordinates of the corner points, the calibration extrinsic parameters of the camera are determined. Therefore, by identifying the corner points of the calibration object pattern, the extrinsic parameters of the camera are calibrated, reducing distortion caused during image edge region identification, lowering the requirements for ambient light and the difficulty of arranging the calibration objects, and improving the corner point recognition rate, as well as the efficiency and accuracy of calibration.

[0058] Figure 3 This is a flowchart illustrating the calibration method for an in-vehicle surround view system according to the second embodiment of this application.

[0059] Based on the above embodiments, such as Figure 3 As shown, the calibration method for the vehicle surround view system in this application embodiment may specifically include the following steps:

[0060] S301, acquire an image of a calibration object captured by a vehicle-mounted camera. The calibration object includes multiple regions, and at least one region includes a pattern containing identification information.

[0061] Specifically, step S301 in this embodiment is the same as step S101 in the above embodiment, and will not be repeated here.

[0062] Step S102 in the above embodiment, "identifying the pattern's identifier information and the two-dimensional coordinates of the pattern's corner points," may specifically include the following steps S302 and S303:

[0063] S302, Recognize patterns in an image.

[0064] Specifically, the image of the calibration object acquired by the vehicle-mounted camera in step S301 is identified to obtain the corresponding pattern in the image. That is, after the vehicle is parked in a fixed pose on the calibration object, the images acquired by each camera are obtained and the pattern in the image is detected and identified.

[0065] As one possible implementation, a pattern in an image with no fewer than three regions can be identified, the three regions including at least one first region and at least one second region.

[0066] S303, when the recognition rate of the pattern in the image is lower than the preset recognition rate threshold, the image is re-acquired or the recognition results of the patterns in multiple frames are superimposed for recognition until the recognition rate reaches the preset recognition rate threshold.

[0067] Specifically, a recognition rate threshold is preset. The recognition rate of the pattern in the image of the calibrator identified in step S302 is compared with the preset recognition rate threshold. When the recognition rate of the pattern is lower than the preset recognition rate threshold, the pattern in the image of the calibrator is re-acquired or the recognition results of the pattern in multiple frames of images are deduplicated and then superimposed until the recognition rate of the pattern is higher than the preset recognition rate threshold.

[0068] It should be noted that, in order to maintain calibration accuracy, the recognition rate of the patterns in the three regions of each image must be greater than the preset recognition rate threshold. The preset threshold can be set to 70%-80%, for example, 75%.

[0069] S304 performs distortion removal processing on the two-dimensional coordinates of the corner points.

[0070] Specifically, the two-dimensional coordinates of the corner points of the pattern in the image identified in step S302 can be processed to remove distortion according to the distortion table or the calibrated camera intra-camera distortion parameters to obtain the two-dimensional coordinates of the corner points after distortion removal. This application does not impose too many restrictions on the specific method of distortion removal.

[0071] Step S103 in the above embodiment, "determining the camera's calibration extrinsic parameters based on the pattern's identification information and the two-dimensional coordinates of the corner points," may specifically include the following step S305:

[0072] S305 determines the candidate calibration extrinsic parameters of the camera corresponding to a single region in the image based on the pattern's identification information and the two-dimensional coordinates of the corner points.

[0073] Specifically, based on the two-dimensional coordinates of the distorted corner points obtained in step S304, the identification information of the pattern identified in step S302, and the three-dimensional coordinates of the corresponding points in the scaling pattern, the extrinsic parameters of the camera corresponding to a single region in the image can be calibrated. The obtained extrinsic parameters are candidate calibration extrinsic parameters. For example, three sets of candidate calibration extrinsic parameters of a camera at a certain position can be marked and named T1-T3 respectively. That is, take the corner point candidate calibration extrinsic parameter T1 of the middle region, take the corner point candidate calibration extrinsic parameter T2 of the left region, and take the corner point candidate calibration extrinsic parameter T3 of the right region. The candidate calibration extrinsic parameters of the other three cameras can be marked in this way.

[0074] It should be noted that the two-dimensional coordinates of the corner points after distortion removal correspond one-to-one with the known three-dimensional coordinates of the points on the established scaling pattern. For distortion-free cameras, the opencvpnp function can be used to calibrate the extrinsic parameters.

[0075] S306, determine the projection error of the corner point based on the candidate calibration extrinsic parameters, the pattern identification information, and the two-dimensional coordinates of the corner point.

[0076] Specifically, based on the candidate calibration extrinsic parameters of the camera corresponding to a single region in the image determined in step S305, the identification information of the pattern identified in step S302, and the two-dimensional coordinates of the corner points after distortion removal obtained in step S304, the projection error corresponding to the corner points is further determined. That is, the three-dimensional world coordinates of the three-dimensional coordinate points, namely the four vertices and the center point on the calibration pattern, are used to calculate the projection points on the image based on the camera's intrinsic and extrinsic parameters, thereby obtaining the projection error, which is the difference between the coordinates of the projection points and the points extracted from the real image. For example, for the three candidate calibration extrinsic parameters T1, T2, and T3, the projection error of the corner points of a single region is solved using the three-dimensional points on the calibration patterns of the three regions.

[0077] S307, if the projection error is less than the preset projection error threshold, the camera calibration extrinsic parameters are determined based on the two-dimensional coordinates of the corner points.

[0078] Specifically, a projection error threshold is preset. Based on the candidate calibration extrinsic parameters, the pattern identification information, and the two-dimensional coordinates of the corner points, the projection error of the corner points is determined. The projection error is compared with the preset projection error threshold. If the projection error is less than the preset projection error threshold, the camera calibration extrinsic parameters are determined based on the two-dimensional coordinates of the corner points. The projection error threshold can be set as needed, and this application does not impose excessive restrictions on the specific value of the preset projection error threshold.

[0079] It should be noted that when the projection error is less than the set threshold, it is considered that the accuracy of the extrinsic parameters calibrated in each region is high, and the three-dimensional coordinates and extracted angle errors of the overall calibrated object are very small, which can meet the accuracy requirements of calibration. At this time, the calibration extrinsic parameter T4 of the three regions is selected as the final calibration extrinsic parameter result.

[0080] S308 If the projection error is equal to or greater than the projection error threshold, an alert message will be output. The alert message is used to remind the user to change the calibration site or re-lay the calibration object.

[0081] Specifically, based on the candidate calibration extrinsic parameters, the pattern identification information, and the two-dimensional coordinates of the corner points, the projection error of the corner points is determined. The projection error is compared with the preset projection error threshold. If the projection error is not less than, equal to, or greater than the preset projection error threshold, a reminder message is output to the user that the calibration site needs to be changed or the calibration object needs to be re-laid for subsequent processing.

[0082] It should be noted that when the projection error is equal to or greater than the set threshold, it may be due to the size error of the calibration object itself, or it may be because the entire calibration object is not on the same plane, or the relative positions of the calibration objects in the three areas are inconsistent with the preset, etc. In this case, it is necessary to change the site or re-lay the calibration object until the corresponding requirements are met.

[0083] In this embodiment, an image of a calibration object comprising multiple regions is acquired from a vehicle-mounted camera, with at least one region including a pattern containing identification information. The pattern in the image is identified, and if the recognition rate is lower than a preset threshold, the image is reacquired or the recognition results of patterns from multiple frames are superimposed for identification until the recognition rate reaches the preset threshold. Then, distortion removal processing is performed on the two-dimensional coordinates of the corner points. Based on the pattern's identification information and the two-dimensional coordinates of the corner points, candidate calibration extrinsic parameters for the camera corresponding to a single region in the image are determined. The projection error of the corner points is determined based on the candidate calibration extrinsic parameters, the pattern's identification information, and the two-dimensional coordinates of the corner points. If the projection error is less than a preset projection error threshold, the camera's calibration extrinsic parameters are determined based on the two-dimensional coordinates of the corner points. If the projection error is equal to or greater than the projection error threshold, information is output to remind the user to change the calibration site or re-lay the calibration object. Therefore, by identifying the corner points of the calibration object's pattern, the camera's extrinsic parameters are calibrated, reducing distortion during image edge region recognition, lowering the requirements for ambient light and the difficulty of arranging the calibration object, and improving the corner point recognition rate, calibration efficiency, and accuracy. Meanwhile, by judging and processing the pattern according to the preset recognition rate threshold, the recognition rate of corner points is further improved. By calculating the projection error, problems of the calibration object or calibration site can be found, which further improves the calibration accuracy from the perspective of algorithm and feedback mechanism.

[0084] To clearly describe the calibration method of the vehicle surround view system according to the embodiments of this application, the following is combined with Figure 4 The specific implementation process of the calibration method for the vehicle surround view system according to the embodiments of this application is described in detail. For example... Figure 4 As shown, the method may specifically include the following steps:

[0085] S401, acquire the image of the calibration object captured by the vehicle-mounted camera.

[0086] S402, Identify patterns in an image.

[0087] S403, determine whether the recognition rate of the pattern in the image is lower than a preset recognition rate threshold. If yes, proceed to step S404; otherwise, proceed to step S405.

[0088] S404, reacquire the image or superimpose the recognition results of patterns from multiple frames for recognition until the recognition rate reaches the preset recognition rate threshold. Execute step S402.

[0089] S405 performs distortion removal processing on the two-dimensional coordinates of the corner points.

[0090] S406, based on the pattern's identification information, the two-dimensional coordinates of the corner points, and the three-dimensional coordinates of the corresponding points in the scaling pattern, determine the candidate calibration extrinsic parameters of the camera corresponding to a single region in the image.

[0091] S407, determine the projection error of the corner point based on the candidate calibration extrinsic parameters, the pattern identification information, and the two-dimensional coordinates of the corner point.

[0092] S408, determine whether the projection error is less than the preset projection error threshold. If yes, proceed to step S409; otherwise, proceed to step S410.

[0093] S409 determines the camera's calibration extrinsic parameters based on the two-dimensional coordinates of the corner points.

[0094] S410 outputs a reminder message to remind the user to change the calibration site or re-lay the calibration material.

[0095] Figure 5 This is a block diagram illustrating a calibration device for a vehicle surround view system according to an exemplary embodiment of this application, such as... Figure 5 As shown, the calibration device 500 of the vehicle surround view system includes: an acquisition module 501, an identification module 502, and a determination module 503.

[0096] The acquisition module 501 is used to acquire images of a calibration object captured by a vehicle-mounted camera. The calibration object includes multiple regions, and at least one region includes a pattern containing identification information.

[0097] The recognition module 502 is used to recognize the identification information of the pattern and the two-dimensional coordinates of the corner points of the pattern.

[0098] The determination module 503 is used to determine the calibration extrinsic parameters of the camera based on the pattern's identification information and the two-dimensional coordinates of the corner points.

[0099] According to one embodiment of this application, the recognition module 502 is specifically used for: recognizing patterns in an image; when the recognition rate of a pattern in an image is lower than a preset recognition rate threshold, the image is re-acquired or the recognition results of patterns from multiple frames of images are superimposed for recognition until the recognition rate reaches the preset recognition rate threshold.

[0100] According to one embodiment of this application, the determining module 503 is further configured to: perform distortion removal processing on the two-dimensional coordinates of the corner points before determining the calibration extrinsic parameters of the camera based on the pattern identification information and the two-dimensional coordinates of the corner points.

[0101] According to one embodiment of this application, the determining module 503 is specifically used to: determine the calibration extrinsic parameters of the camera based on the pattern's identification information, the two-dimensional coordinates of the corner points, and the three-dimensional coordinates of the corresponding points in the scaling pattern.

[0102] According to one embodiment of this application, the determining module 503 is specifically used to: determine the candidate calibration extrinsic parameters of the camera corresponding to a single region in the image based on the pattern identification information and the two-dimensional coordinates of the corner point; determine the projection error of the corner point based on the candidate calibration extrinsic parameters, the pattern identification information and the two-dimensional coordinates of the corner point; if the projection error is less than a preset projection error threshold, then determine the calibration extrinsic parameters of the camera based on the pattern identification information and the two-dimensional coordinates of the corner point.

[0103] According to one embodiment of this application, the determining module 503 is further configured to: if the projection error is equal to or greater than the projection error threshold, output a reminder message, the reminder message being used to remind the user to change the calibration site or re-lay the calibration object.

[0104] It should be noted that the above explanation of the calibration method embodiment for the vehicle surround view system also applies to the calibration device of the vehicle surround view system in the embodiments of this application, and the specific process will not be repeated here.

[0105] In this embodiment, an image of a calibration object comprising multiple regions is acquired from a vehicle-mounted camera, with at least one region including a pattern containing identification information. The pattern in the image is identified, and if the recognition rate is lower than a preset threshold, the image is reacquired or the recognition results of patterns from multiple frames are superimposed for identification until the recognition rate reaches the preset threshold. Then, distortion removal processing is performed on the two-dimensional coordinates of the corner points. Based on the pattern's identification information and the two-dimensional coordinates of the corner points, candidate calibration extrinsic parameters for the camera corresponding to a single region in the image are determined. The projection error of the corner points is determined based on the candidate calibration extrinsic parameters, the pattern's identification information, and the two-dimensional coordinates of the corner points. If the projection error is less than a preset projection error threshold, the camera's calibration extrinsic parameters are determined based on the two-dimensional coordinates of the corner points. If the projection error is equal to or greater than the projection error threshold, information is output to remind the user to change the calibration site or re-lay the calibration object. Therefore, by identifying the corner points of the calibration object's pattern, the camera's extrinsic parameters are calibrated, reducing distortion during image edge region recognition, lowering the requirements for ambient light and the difficulty of arranging the calibration object, and improving the corner point recognition rate, calibration efficiency, and accuracy. Meanwhile, by judging and processing the pattern according to the preset recognition rate threshold, the recognition rate of corner points is further improved. By calculating the projection error, problems of the calibration object or calibration site can be found, which further improves the calibration accuracy from the perspective of algorithm and feedback mechanism.

[0106] To implement the above embodiments, this application also proposes a vehicle 600, such as... Figure 6 As shown, the vehicle 600 may specifically include: a calibration device 500 for the vehicle surround view system as described in the above embodiment.

[0107] To achieve the above embodiments, this application also proposes a calibration object for calibrating an in-vehicle surround view system. The calibration object may specifically include: a plurality of first regions located in the calibration object corresponding to the edge of an image and a plurality of second regions arranged at intervals from the first regions. The first regions and the second regions include at least one pattern containing identification information.

[0108] According to one embodiment of this application, the density of the pattern in the first region is different from the density of the pattern in the second region.

[0109] According to one embodiment of this application, the size of the pattern in the first region is different from the size of the pattern in the second region.

[0110] To achieve the above embodiments, this application also proposes a vehicle in which the vehicle's onboard surround view system performs calibration by acquiring images of the calibration objects shown in the above embodiments.

[0111] To implement the above embodiments, this application also proposes an electronic device 700, such as... Figure 7 As shown, the electronic device 700 may specifically include: a memory 701, a processor 702, and a computer program stored in the memory 701 and executable on the processor 702. When the processor 702 executes the program, it implements the calibration method of the vehicle surround view system as shown in the above embodiment.

[0112] To achieve the above embodiments, this application also proposes a vehicle 800, such as... Figure 8 As shown, the vehicle 800 may specifically include: electronic equipment 700 as shown in the above embodiment.

[0113] To implement the above embodiments, this application also proposes a computer-readable storage medium storing a computer program that is executed by a processor to implement the calibration method for the vehicle surround view system as shown in the above embodiments.

[0114] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0115] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0116] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A calibration method for a vehicle surround view system, characterized in that, The method comprises: acquiring an image of a calibration object collected by a vehicle-mounted camera, the calibration object comprising a plurality of regions, at least one of the regions comprising a pattern containing identification information; the plurality of regions comprising a plurality of first regions corresponding to edges of the image in the calibration object and a plurality of second regions arranged at intervals from the first regions, the first regions and the second regions comprising at least one of the patterns, the pattern of the first regions being different from the pattern of the second regions in density and / or size; identifying identification information of the pattern and two-dimensional coordinates of a corner point of the pattern; determining calibration extrinsic parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point, wherein the determination comprises: determining candidate calibration extrinsic parameters of the camera corresponding to an individual one of the regions in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner point; determining a projection error of the corner point according to the candidate calibration extrinsic parameters, the identification information of the pattern and the two-dimensional coordinates of the corner point; if the projection error is less than a preset projection error threshold, determining the calibration extrinsic parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point.

2. The calibration method of claim 1, wherein The identification of the identification information of the pattern and the two-dimensional coordinates of the corner point of the pattern comprises: identifying the pattern in the image; if an identification rate of the pattern in the image is lower than a preset identification rate threshold, re-acquiring the image or superimposing identification results of the pattern in a plurality of frames of the image to identify the pattern until the identification rate reaches the preset identification rate threshold.

3. The calibration method of claim 1, wherein Further comprising: identifying the pattern in at least three of the regions in the image, the three regions comprising at least one of the first regions and at least one of the second regions.

4. The calibration method of claim 1, wherein Before the determination of the calibration extrinsic parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point, further comprising: performing distortion removal processing on the two-dimensional coordinates of the corner point.

5. The calibration method of claim 1, wherein The determination of the calibration extrinsic parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point comprises: determining the calibration extrinsic parameters of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner point and three-dimensional coordinates of a corresponding point in a scale pattern.

6. The calibration method of claim 1, wherein Further comprising: if the projection error is equal to or greater than the projection error threshold, outputting reminding information, the reminding information being used to remind a user to replace a calibration site or to re-lay the calibration object.

7. A calibration device for a surround view system of a vehicle, characterized in that The method comprises: an acquiring module configured to acquire an image of a calibration object collected by a vehicle-mounted camera, the calibration object comprising a plurality of regions, at least one of the regions comprising a pattern containing identification information, the plurality of regions comprising a plurality of first regions corresponding to edges of the image in the calibration object and a plurality of second regions arranged at intervals from the first regions, the first regions and the second regions comprising at least one of the patterns, the pattern of the first regions being different from the pattern of the second regions in density and / or size; an identifying module configured to identify identification information of the pattern and two-dimensional coordinates of a corner point of the pattern; a determining module configured to determine calibration extrinsic parameters of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point. The determining module is specifically configured to: determine a candidate calibration extrinsic parameter of the camera corresponding to a single region in the image according to the identification information of the pattern and the two-dimensional coordinates of the corner point; determine a projection error of the corner point according to the candidate calibration extrinsic parameter, the identification information of the pattern and the two-dimensional coordinates of the corner point; if the projection error is less than a preset projection error threshold, determine the calibration extrinsic parameter of the camera according to the identification information of the pattern and the two-dimensional coordinates of the corner point.

8. The calibration device of claim 7, wherein The identifying module is specifically configured to: identify the pattern in the image; if an identification rate of the pattern in the image is lower than a preset identification rate threshold, reacquire the image or superimpose identification results of the pattern in multiple frames of the image to identify, until the identification rate reaches the preset identification rate threshold.

9. The calibration device of claim 7, wherein, The determining module is further configured to: perform distortion removal processing on the two-dimensional coordinates of the corner point before determining the calibration extrinsic parameter of the camera according to the two-dimensional coordinates of the corner point.

10. The calibration device of claim 7, wherein, The determining module is specifically configured to: determine the calibration extrinsic parameter of the camera according to the identification information of the pattern, the two-dimensional coordinates of the corner point and three-dimensional coordinates of a corresponding point in a scale pattern.

11. The calibration device of claim 7, wherein, The determining module is further configured to: if the projection error is equal to or greater than the projection error threshold, output a prompt information, the prompt information being used to prompt a user to replace a calibration site or re-lay the calibration object.

12. A vehicle characterized by comprising: The calibration device of the vehicle-mounted surround view system according to any one of claims 7-11. The calibration object comprises a plurality of first regions corresponding to edges of the image and a plurality of second regions arranged at intervals from the first regions, the first regions and the second regions comprising at least one pattern containing identification information; the pattern in the first region is different from the pattern in the second region in density and / or size.

13. A calibration object for use in a calibration method of a surround view system according to any one of claims 1 to 6, characterized in that The vehicle-mounted surround view system of the vehicle calibrates by collecting images of the calibration object according to claim 13.

14. A vehicle characterized by comprising: The calibration device of the vehicle-mounted surround view system according to any one of claims 7-11.

15. An electronic device, comprising: The calibration device of the vehicle-mounted surround view system according to any one of claims 7-11. The calibration device of the vehicle-mounted surround view system according to any one of claims 7-11.

16. A vehicle characterized by comprising: The calibration device of the vehicle-mounted surround view system according to any one of claims 7-11.

17. A computer readable storage medium having stored thereon a computer program, characterized in that, The calibration device of the vehicle-mounted surround view system according to any one of claims 7-11.

Images