External parameter calibration method of vehicle-mounted camera, electronic equipment and storage medium

Abstract

The invention discloses an external parameter calibration method for a vehicle-mounted camera, the vehicle-mounted camera comprises a first camera and a second camera, and the method comprises the following steps: collecting a first image of the first camera and a second image of the second camera, the acquisition states of the first image and the second image are the same; acquiring linear features of the first image and the second image, wherein the linear features comprise a vertical linear feature, a horizontal linear feature and a ground linear feature; performing external parameter calculation by using the vertical straight line features and the horizontal straight line features to obtain initial external parameters of the vehicle-mounted camera; and verifying the initial external parameters by using the ground linear features so as to determine target external parameters of the vehicle-mounted camera. The invention further discloses electronic equipment and a storage medium. According to the invention, the efficiency and accuracy of external parameter calibration are improved.

Description

Technical Field

[0001] The disclosed embodiments of the present application relate to the field of vehicle-mounted technology, and more specifically, to an external parameter calibration method, electronic device, and storage medium for a vehicle-mounted camera. Background Art

[0002] In vehicle intelligent driving tasks, binocular cameras can be used to make up for the shortcomings of insufficient perception accuracy of single cameras. However, for the use of binocular cameras, the calibration of binocular camera external parameters is particularly important. At present, the calibration of binocular camera external parameters is mostly done in the calibration room when the vehicle leaves the factory. The calibration process is complicated and cumbersome, which is extremely inconvenient for both manufacturers and users. Summary of the invention

[0003] According to the embodiments of the present application, the present application proposes an external parameter calibration method, an electronic device and a storage medium for a vehicle-mounted camera to solve the above-mentioned problems.

[0004] The first aspect of the present application discloses an extrinsic parameter calibration method for a vehicle-mounted camera, wherein the vehicle-mounted camera includes a first camera and a second camera, and the method includes: acquiring a first image of the first camera and a second image of the second camera, wherein the acquisition states of the first image and the second image are the same: acquiring straight line features of the first image and the second image, wherein the straight line features include vertical straight line features, horizontal straight line features, and ground straight line features; performing extrinsic parameter calculation using the vertical straight line features and the horizontal straight line features to obtain initial extrinsic parameters of the vehicle-mounted camera; and verifying the initial extrinsic parameters using the ground straight line features to determine target extrinsic parameters of the vehicle-mounted camera.

[0005] In some embodiments, the obtaining of straight line features of the first image and the second image includes: performing dedistortion processing on the first image and the second image to obtain a first image of interest and a second image of interest from the first image and the second image, respectively, wherein the fields of view of the first image of interest and the second image of interest are consistent; and using Hough transform to extract straight line segments from the first image of interest and the second image of interest to serve as the straight line features, wherein the straight line features are represented by the endpoint coordinates of the straight line segments.

[0006] In some embodiments, the absolute value of the angle between the straight line segment corresponding to the vertical straight line feature and the X-axis of the image coordinate system is within a first preset angle range; the absolute value of the angle between the straight line segment corresponding to the horizontal straight line feature and the X-axis of the image coordinate system is within a second preset angle range; and the absolute value of the distance between the straight line segment corresponding to the ground straight line feature and the center point of the image is within a preset distance range.

[0007] In some embodiments, the straight line features of the first image include first vertical straight line features, and the straight line features of the second image include second vertical straight line features; using the vertical straight line features to calculate external parameters includes: obtaining at least one first homography matrix, the first homography matrix is ​​calculated based on a preset yaw angle; using the at least one first homography matrix, mapping the first vertical straight line feature to the second image to obtain the mapped first vertical straight line feature; calculating the horizontal distance between the mapped first vertical straight line feature and the second vertical straight line feature; based on the horizontal distance, determining the overlap between the mapped first vertical straight line feature and the second vertical straight line feature; and using the yaw angle and roll angle corresponding to the overlap that meets the preset conditions as part of the initial external parameters.

[0008] In some embodiments, the roll angle is determined by an average value of angles between the corresponding mapped first vertical straight line feature and the second vertical straight line feature.

[0009] In some embodiments, the straight line features of the first image include first horizontal straight line features, and the straight line features of the second image include second horizontal straight line features; using the horizontal straight line features to perform external parameter calculation, including: obtaining at least one second homography matrix, the second homography matrix is ​​calculated based on a preset pitch angle; using the at least one second homography matrix, mapping the first horizontal straight line feature to the second image to obtain the mapped first horizontal straight line feature; calculating the vertical distance between the mapped first horizontal straight line feature and the second horizontal straight line feature; based on the vertical distance, determining the overlap between the mapped first horizontal straight line feature and the second horizontal straight line feature; and using the pitch angle corresponding to the overlap that meets the preset conditions as part of the initial external parameter.

[0010] In some embodiments, the straight line features of the first image include first ground straight line features, and the straight line features of the second image include second ground straight line features; the initial extrinsic parameters are verified using the ground straight line features, including: obtaining a third homography matrix, wherein the third homography matrix is ​​calculated based on the initial extrinsic parameters; using the third homography matrix, mapping the first ground straight line feature to the second image to obtain the mapped first ground straight line feature; calculating the endpoint distance between the mapped first ground straight line feature and the second ground straight line feature; based on the endpoint distance, determining the overlap between the mapped first ground straight line feature and the second ground straight line feature; and determining the initial extrinsic parameter corresponding to the overlap that meets a preset condition as the target extrinsic parameter of the vehicle-mounted camera.

[0011] In some embodiments, in response to the number of straight line features of the first image and the second image being greater than a preset value, it is determined that the first image and the second image conform to a calibration scene.

[0012] The second aspect of the present application discloses an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the external parameter calibration method of the vehicle-mounted camera described in the first aspect.

[0013] The third aspect of the present application discloses a non-volatile computer-readable storage medium having program instructions stored thereon. When the program instructions are executed by a processor, the extrinsic parameter calibration method for a vehicle-mounted camera described in the first aspect is implemented.

[0014] The beneficial effects of the present application include: acquiring a first image of a first camera and a second image of a second camera, the first image and the second image having the same acquisition state, extracting straight line features from the first image and the second image, the straight line features including vertical straight line features, horizontal straight line features and ground straight line features, further, using the vertical straight line features and the horizontal straight line features to perform external parameter calculation to obtain initial external parameters of the vehicle-mounted camera, and using the ground straight line features to verify the initial external parameters to determine the target external parameters of the vehicle-mounted camera, and by using the straight line features in the first image and the second image to calibrate the external parameters of the vehicle-mounted binocular camera, the process of external parameter calibration of the vehicle-mounted binocular camera is simplified, and the efficiency and accuracy of the external parameter calibration are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present application will be further described below in conjunction with the accompanying drawings and implementation methods, in which:

[0016] Figure 1 is a flow chart of an external parameter calibration method for a vehicle-mounted camera according to an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of a first image according to an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of a second image according to an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of a first image of interest according to an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a second image of interest according to an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of a straight line feature in a first image of interest according to an embodiment of the present application;

[0022] Figure 7is a schematic diagram of a straight line feature in a second image of interest in an embodiment of the present application;

[0023] Figure 8 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0024] Fig. 9 It is a schematic diagram of the structure of the non-volatile computer-readable storage medium of an embodiment of the present application. DETAILED DESCRIPTION

[0025] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] The term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, and can mean including any one or more elements selected from the set consisting of A, B, and C. In addition, the terms "first", "second", and "third" in this application are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0027] In order to enable those skilled in the art to better understand the technical solution of the present application, the technical solution of the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0028] See also Figure 1 , Figure 1 1 is a flow chart of an external parameter calibration method for a vehicle-mounted camera according to an embodiment of the present application. The vehicle-mounted camera includes at least a first camera and a second camera, wherein the first camera may be a telephoto camera and the second camera may be a short-focus camera. The execution subject of the method may be an electronic device with computing functions, such as a microcomputer, a server, and a mobile device such as a laptop computer and a tablet computer.

[0029] It should be noted that if there are substantially the same results, the method of the present application is not limited to Figure 1 The process sequence shown is limited.

[0030] In some possible implementations, the method may be implemented by a processor calling computer-readable instructions stored in a memory, such as Figure 1 As shown, the method may include the following steps:

[0031] S11: Acquire a first image of a first camera and a second image of a second camera, wherein the first image and the second image are acquired in the same state.

[0032] The first image of the first camera may be an image captured by a telephoto camera, i.e., a narrow-angle image, and the second image may be an image captured by a short-focus camera, i.e., a wide-angle image, wherein the first image and the second image are captured in the same state, such as a vehicle driving state, and the time when the first camera and the second camera capture the images is synchronized, i.e., the difference in timestamps of the captured binocular images is very small; or the images captured by the first camera and the second camera when the vehicle is stationary, etc.

[0033] S12: Acquire straight line features of the first image and the second image, where the straight line features include vertical straight line features, horizontal straight line features, and ground straight line features.

[0034] Straight line features are extracted from the first image and the second image, wherein the straight line features in the image include at least one of vertical straight line features, horizontal straight line features, and ground straight line features, wherein the straight line features can be determined from road features such as lane lines, lamp posts, road signs, and buildings in the image.

[0035] S13: Calculate external parameters using vertical straight line features and horizontal straight line features to obtain initial external parameters of the vehicle-mounted camera.

[0036] External parameters are calculated using vertical straight line features and horizontal straight line features in the first image and the second image to obtain initial external parameters of the vehicle-mounted camera, wherein the initial external parameters of the vehicle-mounted camera may include calculated yaw angle, pitch angle, roll angle, etc.

[0037] S14: Use the ground straight line feature to verify the initial external parameters to determine the target external parameters of the vehicle-mounted camera.

[0038] The initial extrinsic parameters are verified using the ground straight line features in the first image and the second image. That is, based on the initial extrinsic parameters, verification calculations are performed using the ground straight line features in the first image and the second image. If the calculation results meet the preset conditions, the initial extrinsic parameters can be determined as the target extrinsic parameters of the vehicle-mounted camera.

[0039] In this embodiment, a first image of a first camera and a second image of a second camera are acquired, and the acquisition states of the first image and the second image are the same. Straight line features are extracted from the first image and the second image, and the straight line features include vertical straight line features, horizontal straight line features, and ground straight line features. Furthermore, extrinsic parameters are calculated using the vertical straight line features and the horizontal straight line features to obtain initial extrinsic parameters of the vehicle-mounted camera, and the initial extrinsic parameters are verified using the ground straight line features to determine the target extrinsic parameters of the vehicle-mounted camera. By using the straight line features in the first image and the second image to calibrate the extrinsic parameters of the vehicle-mounted binocular camera, the process of extrinsic parameter calibration of the vehicle-mounted binocular camera is simplified, and the efficiency and accuracy of the extrinsic parameter calibration are improved.

[0040] In some embodiments, obtaining straight line features of the first image and the second image includes: performing dedistortion processing on the first image and the second image to obtain a first image of interest and a second image of interest from the first image and the second image, respectively, wherein the fields of view of the first image of interest and the second image of interest are consistent; and using Hough transform to extract straight line segments from the first image of interest and the second image of interest to serve as straight line features, wherein the straight line features are represented by the endpoint coordinates of the straight line segments.

[0041] Get the first image of the first camera (such as Figure 2 ) and a second image from a second camera (as shown Figure 3 The first image and the second image are subjected to dedistortion processing, for example, the input image is dedistorted using known camera intrinsic parameters, and a binocular region of interest (ROI) image with approximately the same field of view is selected and converted into a grayscale image, so as to obtain a first image of interest (as shown in FIG. Figure 4 ) and the second image of interest (as shown Figure 5 shown).

[0042] Line feature extraction is performed on the first image of interest and the second image of interest, for example, using Hough transform to extract line features, and the corresponding line features in the first image of interest (such as Figure 6 ) and the corresponding straight line features in the second image of interest (as shown in Figure 7 As shown in the figure), the extracted straight line features can be represented by the coordinates of the two end points of the straight line segment on the image.

[0043] In some embodiments, the absolute value of the angle between the straight line segment corresponding to the vertical straight line feature and the X-axis of the image coordinate system is within a first preset angle range; the absolute value of the angle between the straight line segment corresponding to the horizontal straight line feature and the X-axis of the image coordinate system is within a second preset angle range; and the absolute value of the distance between the straight line segment corresponding to the ground straight line feature and the center point of the image is within a preset distance range.

[0044] The straight line features in the image include vertical straight line features, horizontal straight line features and ground straight line features. Among them, the absolute value of the angle between the straight line segment corresponding to the vertical straight line feature and the X-axis of the image coordinate system is within the first preset angle range. For example, the angle between the straight line segment and the X-axis in the image coordinate system is calculated. If the approximate absolute value of the angle between the straight line segment and the X-axis of the image coordinate system is between (89° and 91°), the corresponding straight line is classified as a vertical straight line set V, and the vertical straight line feature is obtained. The absolute value of the angle between the straight line segment corresponding to the horizontal straight line feature and the X-axis of the image coordinate system is within the second preset angle range. For example, the angle between the straight line segment and the X-axis in the image coordinate system is calculated. If the approximate absolute value of the angle between the straight line segment and the X-axis of the image coordinate system is between (0° and 1°), the corresponding straight line is classified as a horizontal straight line set V, and the horizontal straight line feature is obtained. The absolute value of the distance between the straight line segment corresponding to the ground straight line feature and the center point of the image is within the preset distance range. For example, the vertical distance from the center point of the image to the straight line is calculated. If the vertical distance from the center point of the image to the straight line is less than a threshold (for example, 2 pixels), the corresponding straight line is classified as a ground parallel straight line set G, and the ground straight line feature is obtained.

[0045] Further, in some embodiments, in response to the number of straight line features of the first image and the second image being greater than a preset value, it is determined that the first image and the second image conform to the calibration scene.

[0046] The number of straight line features in the first image and the second image is used to determine whether the current scene meets the calibration requirements. For example, in response to the number of three types of straight line segments (vertical, horizontal, and ground) in the first image and the second image being greater than a preset value, it is determined that the current scene meets the calibration requirements. If it is less than the preset value, the current scene does not meet the calibration requirements and data needs to be re-collected. In some embodiments, the preset value can be 10.

[0047] For ease of understanding, the external parameters of the vehicle-mounted camera are further explained. The external parameters of the vehicle-mounted camera include rotation R and translation t, where rotation R uses Euler angles yaw, pitch, roll or rotation matrix R wide_narrow Indicates that the two can be converted into each other; translation t is a three-dimensional translation vector, which can be obtained from the installation position of the camera.

[0048] Among them, the homography matrix of the image can be calculated using the internal and external parameters of the binocular camera to achieve the mapping of binocular image pixels. For example, the homography matrix H from narrow angle to wide angle wide_narrow The calculation method is as follows:

[0049]

[0050] Among them, K wide is the internal parameter matrix of the short-focus camera, K narrow is the internal parameter matrix of the telephoto camera, Rwide_narrow is the rotation matrix from the telephoto camera to the short focal length camera; the homography matrix can transform the pixel point [u1, v1, 1] of the telephoto camera to [u2, v2, 1] on the short focal length camera image, and the relationship is:

[0051] [u2,v2,1] T =H wide_narrow *[u1,v1,1] T

[0052] In some embodiments, the straight line features of the first image include first vertical straight line features, such as a narrow-angle vertical straight line set V narrow , the straight line features of the second image include a second vertical straight line feature, such as a wide-angle vertical straight line set V wide .

[0053] At this time, the vertical straight line feature is used to calculate the external parameters, including: obtaining at least one first homography matrix, the first homography matrix is ​​calculated based on a preset yaw angle; using at least one first homography matrix, the first vertical straight line feature is mapped to the second image to obtain the mapped first vertical straight line feature; calculating the horizontal distance between the mapped first vertical straight line feature and the second vertical straight line feature; based on the horizontal distance, determining the overlap between the mapped first vertical straight line feature and the second vertical straight line feature; and using the yaw angle and roll angle corresponding to the overlap that meets the preset conditions as part of the initial external parameters.

[0054] For example, the installation value of yaw is yaw_install, the installation error is E, and then M preset yaw angles (yaw_pr) are selected at intervals of 0.1° within the range of (yaw_install-E, yaw_install+E).

[0055] Get at least one first homography matrix, which is calculated based on the preset yaw angle, that is, for M yaw_pr, set the Euler angle YPR to (yaw_pr, 0, 0) and convert it into a rotation matrix to further obtain M binocular image homography matrices H wide_narrow .

[0056] Using at least one first homography matrix, the first vertical straight line feature is mapped to the second image to obtain the mapped first vertical straight line feature. For example, based on M H wide_narrow One of the narrow-angle image (i.e., the first image) is converted into V narrow Mapped to the wide-angle image (i.e., the second image), the mapped first vertical straight line feature is obtained, i.e., the homographic vertical straight line set V homography , at this time, set the overlap Y to 0. Calculate the horizontal distance between the mapped first vertical line feature and the second vertical line feature, for example, V homographyThere are multiple straight lines v1, V wide There are multiple straight lines v2 in the equation. Based on the midpoint of straight line v1, the horizontal distance Dh from straight line v1 to straight line v2 is calculated respectively.

[0057] Further, based on the horizontal distance, the overlap between the mapped first vertical straight line feature and the second vertical straight line feature is determined. For example, if the calculated horizontal distance Dh is less than a preset value (e.g., 3 pixels), the corresponding straight lines v1 and v2 are considered to overlap, and the overlap Y is increased by one. Also, for the overlapping straight lines, the angle r between the straight line v1 and the straight line v2 is calculated, and then the average roll_mean of the multiple angles r can be obtained as the corresponding roll angle. It can be understood that for M yaw_pr, there are M overlaps Y and M roll_mean.

[0058] The yaw angle and roll angle corresponding to the overlap that meets the preset conditions are used as part of the initial external parameters. For example, the yaw_pr angle and roll_mean angle corresponding to the maximum value of multiple overlaps Y are set as best_yaw and best_roll, and the initial yaw angle and initial roll angle are obtained.

[0059] In some embodiments, the straight line features of the first image include first horizontal straight line features, such as a narrow-angle horizontal straight line set H narrow , the straight line features of the second image include second horizontal straight line features, such as a wide-angle horizontal straight line set H wide .

[0060] At this time, the horizontal straight line feature is used to calculate the external parameters, including: obtaining at least one second homography matrix, the second homography matrix is ​​calculated based on a preset pitch angle; using at least one second homography matrix, the first horizontal straight line feature is mapped to the second image to obtain the mapped first horizontal straight line feature; calculating the vertical distance between the mapped first horizontal straight line feature and the second horizontal straight line feature; based on the vertical distance, determining the overlap between the mapped first horizontal straight line feature and the second horizontal straight line feature; and using the pitch angle corresponding to the overlap that meets the preset conditions as part of the initial external parameters.

[0061] For example, the installation value of pitch is pitch_install, and the installation error is E. Then, within the range of (pitch_install-E, pitch_install+E), N preset pitch angles (pitch_pr) are selected at intervals of 0.1°.

[0062] Get at least one second homography matrix, which is calculated based on the preset pitch angle, that is, for N pitch_pr, set the Euler angle YPR to (0, pitch_pr, 0) and convert it into a rotation matrix to further obtain N binocular image homography matrices H wide_narrow .

[0063] Using at least one second homography matrix, the first horizontal straight line feature is mapped to the second image to obtain the mapped first horizontal straight line feature. For example, using N H wide_narrow One of the narrow-angle image (i.e., the first image) is H narrow Mapped to the wide-angle image (i.e., the second image), the mapped first horizontal straight line feature is obtained, i.e., the homography horizontal straight line set H homography , at this time, set the overlap X to 0. Calculate the vertical distance between the mapped first horizontal straight line feature and the second horizontal straight line feature, for example, H homography There are multiple straight lines h1, H wide There are multiple straight lines h2 in the equation. Based on the midpoint of the straight line h1, the vertical distance Dv from the straight line h1 to the straight line h2 is calculated respectively.

[0064] Further, based on the vertical distance, the overlap between the mapped first horizontal straight line feature and the second horizontal straight line feature is determined. For example, if the calculated vertical distance Dv is less than a preset value (for example, 3 pixels), it is considered that the corresponding straight lines h1 and h2 overlap, and the overlap X is increased by one. It can be understood that for N pitch_prs, there are N overlaps X. Further, the pitch angle corresponding to the overlap that meets the preset conditions is used as part of the initial external parameter. For example, the pitch_pr corresponding to the maximum value of the multiple overlaps X is set as best_pitch, and the initial pitch angle is obtained.

[0065] In some embodiments, the straight line features of the first image include first ground straight line features, such as a narrow-angle ground straight line set G narrow , the straight line features of the second image include the second ground straight line features, such as the wide-angle ground straight line set G wide .

[0066] At this time, the initial extrinsic parameters are verified using the ground straight line features, including: obtaining a third homography matrix, which is calculated based on the initial extrinsic parameters; using the third homography matrix, mapping the first ground straight line feature to the second image to obtain the mapped first ground straight line feature; calculating the endpoint distance between the mapped first ground straight line feature and the second ground straight line feature; based on the endpoint distance, determining the overlap between the mapped first ground straight line feature and the second ground straight line feature; and determining the initial extrinsic parameter corresponding to the overlap that meets the preset conditions as the target extrinsic parameter of the vehicle-mounted camera.

[0067] Get the third homography matrix. The third homography matrix is ​​calculated based on the initial external parameters. For example, based on the best_yaw, best_pitch, and best_roll obtained above, the corresponding binocular image homography matrix H is obtained. wide_narrow The first ground straight line feature is mapped to the second image to obtain the mapped first ground straight line feature. For example, using the obtained H wide_narrow , G in the narrow-angle image (i.e., the first image) narrow Mapped to the wide-angle image (i.e., the second image), the mapped first ground straight line feature is obtained, i.e., the homography ground straight line set G homography , at this time, set the overlap Z to 0. Calculate the endpoint distance between the first ground straight line feature and the second ground straight line feature after mapping, for example, G homography There are multiple straight lines g1, G wide There are multiple straight lines g2 in the equation, and the distances from the two endpoints of straight line g1 to straight line g2, namely d1 and d2, are calculated respectively.

[0068] Furthermore, based on the endpoint distance, the degree of overlap between the mapped first ground straight line feature and the second ground straight line feature is determined. For example, if the calculated distances d1 and d2 are both less than a preset value (for example, 3 pixels), it is considered that the corresponding straight line g1 overlaps with the straight line g2, and the degree of overlap Z is increased by one.

[0069] The initial external parameter corresponding to the overlap degree that meets the preset conditions is determined as the target external parameter of the vehicle-mounted camera. For example, traversing G homography All straight lines, calculated coincidence Z and G homography The ratio p of the number of straight lines C is (Z / C), wherein, if the ratio is greater than 0.5, the initial external parameter value of this group is considered normal, and the calibration is completed and the calibration result is output; otherwise, the data is collected again for calibration.

[0070] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0071] See also Figure 8 , Figure 8 80 is a schematic diagram of the structure of an electronic device of an embodiment of the present application. The electronic device 80 includes a memory 81 and a processor 82 coupled to each other, and the processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of the embodiment of the external parameter calibration method of the above-mentioned vehicle-mounted camera. In a specific implementation scenario, the electronic device 80 may include but is not limited to: a microcomputer, a server, which is not limited here.

[0072] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps of the embodiment of the external parameter calibration method for the above-mentioned vehicle-mounted camera. The processor 82 can also be called a CPU (Central Processing Unit), and the processor 82 may be an integrated circuit chip with signal processing capabilities. The processor 82 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 82 can be implemented by an integrated circuit chip.

[0073] See also Fig. 9 , Fig. 9 Schematic diagram of the structure of a non-volatile computer-readable storage medium according to an embodiment of the present application. The non-volatile computer-readable storage medium 90 is used to store a computer program 901. When the computer program 901 is executed by a processor, for example, Figure 8 When the processor 82 in the embodiment is executed, it is used to implement the steps of the embodiment of the above-mentioned external parameter calibration method for a vehicle-mounted camera.

[0074] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0075] In the several embodiments provided in the present application, it should be understood that the disclosed methods and related devices can be implemented in other ways. For example, the above-described related device implementations are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication disconnection shown or discussed can be through some interfaces, indirect coupling or communication disconnection of devices or units, which can be electrical, mechanical or other forms.

[0076] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0077] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0078] It is easy for a person skilled in the art to know that many modifications and changes can be made to the device and method while maintaining the teaching content of the present application. Therefore, the above disclosure should be considered as limited only by the scope of the appended claims.

Claims

1. A method for calibrating external parameters of a vehicle-mounted camera, characterized in that: The vehicle-mounted camera includes a first camera and a second camera, and the method includes: A first image of the first camera and a second image of the second camera are acquired, wherein the first image and the second image are acquired in the same state: Acquire straight line features of the first image and the second image, wherein the straight line features include vertical straight line features, horizontal straight line features, and ground straight line features; Calculating external parameters using the vertical straight line feature and the horizontal straight line feature to obtain initial external parameters of the vehicle-mounted camera; The initial extrinsic parameters are verified using the ground straight line features to determine the target extrinsic parameters of the vehicle-mounted camera.

2. The method according to claim 1, characterized in that The acquiring straight line features of the first image and the second image includes: Performing dedistortion processing on the first image and the second image to obtain a first image of interest and a second image of interest from the first image and the second image, respectively, wherein the first image of interest and the second image of interest have the same field of view; Straight line segments are extracted from the first image of interest and the second image of interest using Hough transform to serve as the straight line features, where the straight line features are represented by the coordinates of the endpoints of the straight line segments.

3. The method according to claim 2, characterized in that The absolute value of the angle between the straight line segment corresponding to the vertical straight line feature and the X-axis of the image coordinate system is within a first preset angle range; The absolute value of the angle between the straight line segment corresponding to the horizontal straight line feature and the X-axis of the image coordinate system is within a second preset angle range; The absolute value of the distance between the straight line segment corresponding to the ground straight line feature and the center point of the image is within a preset distance range.

4. The method according to claim 1, characterized in that The straight line features of the first image include a first vertical straight line feature, and the straight line features of the second image include a second vertical straight line feature; The vertical straight line feature is used to calculate external parameters, including: Acquire at least one first homography matrix, where the first homography matrix is ​​calculated based on a preset yaw angle; Mapping the first vertical straight line feature to the second image using the at least one first homography matrix to obtain a mapped first vertical straight line feature; Calculating a horizontal distance between the mapped first vertical straight line feature and the second vertical straight line feature; Determining, based on the horizontal distance, a degree of overlap between the mapped first vertical straight line feature and the second vertical straight line feature; The yaw angle and the roll angle corresponding to the overlap degree that meets the preset conditions are used as part of the initial external parameters.

5. The method according to claim 4, characterized in that The roll angle is determined by an average value of the angles between the first vertical straight line feature and the second vertical straight line feature after the corresponding mapping.

6. The method according to claim 1, characterized in that The straight line features of the first image include first horizontal straight line features, and the straight line features of the second image include second horizontal straight line features; The horizontal straight line feature is used to calculate external parameters, including: Acquire at least one second homography matrix, where the second homography matrix is ​​calculated based on a preset pitch angle; Mapping the first horizontal straight line feature to the second image using the at least one second homography matrix to obtain a mapped first horizontal straight line feature; Calculating a vertical distance between the mapped first horizontal straight line feature and the second horizontal straight line feature; Determining, based on the vertical distance, a degree of overlap between the mapped first horizontal straight line feature and the second horizontal straight line feature; The pitch angle corresponding to the overlap degree that meets the preset conditions is used as part of the initial external parameters.

7. The method according to claim 1, characterized in that The straight line features of the first image include first ground straight line features, and the straight line features of the second image include second ground straight line features; Using the ground straight line feature to verify the initial external parameter includes: Obtaining a third homography matrix, wherein the third homography matrix is ​​calculated based on the initial extrinsic parameters; Mapping the first ground straight line feature to the second image using the third homography matrix to obtain a mapped first ground straight line feature; Calculating the distance between the endpoints of the mapped first ground straight line feature and the second ground straight line feature; Determining the degree of overlap between the mapped first ground straight line feature and the second ground straight line feature based on the endpoint distance; The initial external parameter corresponding to the overlap degree that meets the preset condition is determined as the target external parameter of the vehicle-mounted camera.

8. The method according to claim 1, characterized in that Also includes: In response to the number of straight line features of the first image and the second image being greater than a preset value, it is determined that the first image and the second image conform to a calibration scene.

9. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the external parameter calibration method of the vehicle-mounted camera according to any one of claims 1 to 8.

10. A non-volatile computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by the processor, the external parameter calibration method of the vehicle-mounted camera according to any one of claims 1 to 8 is implemented.

Images