Method, device and equipment for calibrating external parameters of image acquisition component and storage medium

Abstract

The present disclosure provides a method and device for calibrating extrinsic parameters of an image acquisition component, and a storage medium, relating to the technical field of artificial intelligence, in particular to the field of intelligent transportation and vehicle-road cooperation. The specific implementation scheme is as follows: registering a first corner point and a second corner point to obtain a registration result; obtaining target point pair information corresponding to initial point pair information by using the registration result; and calibrating a current extrinsic parameter of the image acquisition component based on the target point pair information.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and further to the fields of intelligent transportation and vehicle-road cooperation, and particularly to a method, apparatus, device and storage medium for calibrating the external parameters of an image acquisition component. Background Technology

[0002] Vehicle-to-Everything (V2X) wireless communication technology is one of the supporting technologies for intelligent vehicles and intelligent transportation. Under the backdrop of the new infrastructure initiative, V2X roadside perception systems can provide vehicles in vehicle-road cooperative systems with beyond-line-of-sight perception information. As one of the most important sensors in V2X roadside perception systems, the accurate calibration of the camera's intrinsic and extrinsic parameters is crucial for obtaining reliable perception information. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for calibrating the extrinsic parameters of an image acquisition component, so as to at least solve the technical problems of poor timeliness and high calibration cost of vehicle-road cooperative camera parameters in related technologies.

[0004] According to one aspect of this disclosure, a method for calibrating the extrinsic parameters of an image acquisition component is provided, comprising: registering a first corner point and a second corner point to obtain a registration result, wherein the first corner point is a two-dimensional dashed lane line corner point in a first frame image, the second corner point is a two-dimensional dashed lane line corner point in a second frame image, the first frame image is an image acquired and calibrated by the image acquisition component at a first position before movement, and the second frame image is an image acquired by the image acquisition component at a second position after movement; using the registration result, obtaining target point pair information corresponding to initial point pair information, wherein the initial point pair information is used to determine the correspondence between two-dimensional pixel coordinates in the first frame image and three-dimensional spatial point coordinates in the target electronic map, and the target point pair information is used to determine the correspondence between two-dimensional pixel coordinates in the second frame image and three-dimensional spatial point coordinates in the target electronic map; calibrating the current extrinsic parameters of the image acquisition component based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after movement.

[0005] According to another aspect of this disclosure, an apparatus for calibrating the extrinsic parameters of an image acquisition component is provided, comprising: a registration module for registering a first corner point and a second corner point to obtain a registration result, wherein the first corner point is a two-dimensional dashed lane line corner point in a first frame image, the second corner point is a two-dimensional dashed lane line corner point in a second frame image, the first frame image is an image acquired and calibrated by the image acquisition component at a first position before movement, and the second frame image is an image acquired by the image acquisition component at a second position after movement; an acquisition module for acquiring target point pair information corresponding to initial point pair information using the registration result, wherein the initial point pair information is used to determine the correspondence between two-dimensional pixel coordinates in the first frame image and three-dimensional spatial point coordinates in the target electronic map, and the target point pair information is used to determine the correspondence between two-dimensional pixel coordinates in the second frame image and three-dimensional spatial point coordinates in the target electronic map; and a calibration module for calibrating the current extrinsic parameters of the image acquisition component based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after movement.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method for calibrating extrinsic parameters of an image acquisition component as proposed in this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method for calibrating extrinsic parameters of an image acquisition component as proposed in this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that is executed by a processor to perform the method for calibrating extrinsic parameters of an image acquisition component as disclosed in this disclosure.

[0009] In this disclosure, by registering the first corner point and the second corner point, a registration result is obtained. Then, using the registration result, the target point pair information corresponding to the initial point pair information is obtained. Finally, the current extrinsic parameters of the image acquisition component are calibrated based on the target point pair information. This achieves the purpose of quickly and automatically calibrating the extrinsic parameters of the image acquisition component, thereby improving the timeliness of calibration of vehicle-road cooperative camera parameters and reducing calibration costs. This solves the technical problems of poor timeliness and high calibration costs of calibration of vehicle-road cooperative camera parameters in related technologies.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for calibrating the extrinsic parameters of an image acquisition component according to an embodiment of the present disclosure;

[0013] Figure 2 This is a flowchart of a method for calibrating the extrinsic parameters of an image acquisition component according to an embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram of an image acquisition component acquiring an image according to an embodiment of the present disclosure;

[0015] Figure 4 This is a schematic diagram of an image acquisition component acquiring an image according to another embodiment of the present disclosure;

[0016] Figure 5 This is a structural block diagram of an apparatus for calibrating the extrinsic parameters of an image acquisition component according to an embodiment of the present disclosure. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] In related technologies, vehicle-to-infrastructure (V2I) cameras are typically installed at fixed locations on roads, such as in positions similar to speed cameras. After a roadside algorithm identifies a displaced V2I camera, the latest image of the camera after the movement is usually manually acquired offline for extrinsic parameter calibration, and then the camera parameters in the V2I system are manually replaced. However, this manual parameter calibration method is inefficient and requires significant manpower.

[0020] According to embodiments of this disclosure, a method for calibrating extrinsic parameters of an image acquisition component is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0021] The method embodiments provided in this disclosure can be performed in a mobile terminal, computer terminal, or similar electronic device. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the disclosure described and / or claimed herein. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for calibrating the extrinsic parameters of an image acquisition component is shown.

[0022] like Figure 1 As shown, the computer terminal 100 includes a computing unit 101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 102 or a computer program loaded from a storage unit 108 into a random access memory (RAM) 103. The RAM 103 may also store various programs and data required for the operation of the computer terminal 100. The computing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0023] Multiple components in the computer terminal 100 are connected to the I / O interface 105, including: an input unit 106, such as a keyboard and mouse; an output unit 107, such as various types of displays and speakers; a storage unit 108, such as a hard disk and optical disk; and a communication unit 109, such as a network interface card (NIC), a modem, or a wireless transceiver. The communication unit 109 allows the computer terminal 100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0024] The computing unit 101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 101 performs the method described herein for calibrating the extrinsic parameters of the image acquisition component. For example, in some embodiments, the method for calibrating the extrinsic parameters of the image acquisition component may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 108. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer terminal 100 via ROM 102 and / or communication unit 109. When the computer program is loaded into RAM 103 and executed by the computing unit 101, one or more steps of the method described herein for calibrating the extrinsic parameters of the image acquisition component may be performed. Alternatively, in other embodiments, the computing unit 101 may be configured by any other suitable means (e.g., by means of firmware) to perform a method for calibrating the extrinsic parameters of the image acquisition component.

[0025] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0026] It should be noted here that, in some optional embodiments, the above... Figure 1The electronic device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular example, and is intended to illustrate the types of components that may exist in the aforementioned electronic devices.

[0027] Under the aforementioned operating environment, this disclosure provides, for example... Figure 2 The method shown is for calibrating the extrinsic parameters of the image acquisition component. This method can be performed by... Figure 1 The computer terminal or similar electronic device shown is used for execution. Figure 2 This is a flowchart illustrating a method for calibrating the extrinsic parameters of an image acquisition component according to an embodiment of this disclosure. Figure 2 As shown, the method may include the following steps:

[0028] Step S22: Register the first corner point and the second corner point to obtain the registration result. The first corner point is the corner point of the two-dimensional dashed lane line in the first frame image, and the second corner point is the corner point of the two-dimensional dashed lane line in the second frame image. The first frame image is the image acquired and calibrated by the image acquisition component at the first position before the movement, and the second frame image is the image acquired by the image acquisition component at the second position after the movement.

[0029] The aforementioned image acquisition component can be a vehicle-road cooperative camera, which can be installed on the cross arm of a roadside monitoring pole. The vehicle-road cooperative camera is equipped with a displacement alarm module to determine whether the vehicle-road cooperative camera has moved.

[0030] The first frame image is acquired and calibrated at the first position before the vehicle-road cooperative camera moves. The first corner point is the four vertices corresponding to each dashed lane line in the first frame image. The extrinsic parameters of the vehicle-road cooperative camera at the first position have been calibrated. The second frame image is acquired at the second position after the vehicle-road cooperative camera moves. The second corner point is the four vertices corresponding to each dashed lane line in the second frame image. The extrinsic parameters of the vehicle-road cooperative camera at the second position have not been calibrated.

[0031] Figure 3 This is a schematic diagram of an image acquisition component acquiring an image according to an embodiment of this disclosure, such as... Figure 3 As shown, the first frame image is acquired at the first position using a vehicle-road cooperative camera. The first corner point is the corner point of the two-dimensional dashed lane line in the first frame image. For example, the first corner point may include the four vertices A, B, C, and D corresponding to the dashed lane line L1 in the first frame image.

[0032] Figure 4 This is a schematic diagram illustrating the acquisition of an image by an image acquisition component according to an embodiment of this disclosure, as shown below. Figure 4As shown, the displacement alarm module in the vehicle-road cooperative camera detects that the camera position has moved from the first position to the second position. At the second position, the vehicle-road cooperative camera acquires a second frame image. The second corner point is the corner point of the two-dimensional dashed lane line in the second frame image. For example, the second corner point may include the four vertices A', B', C', and D' corresponding to the dashed lane line L2 in the second frame image.

[0033] The above registration results are used to represent the registration of the corresponding two-dimensional lane line corner points before and after the vehicle-road cooperative camera moves. For example, the registration results can represent the registration between the four vertices A, B, C, and D corresponding to the dashed lane line L1 in the first frame image and the four vertices A', B', C', and D' corresponding to the dashed lane line L2 in the second frame image.

[0034] Step S24: Using the registration result, obtain the target point pair information corresponding to the initial point pair information. The initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image and the three-dimensional spatial point coordinates in the target electronic map. The target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image and the three-dimensional spatial point coordinates in the target electronic map.

[0035] Specifically, the aforementioned initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image acquired by the vehicle-road cooperative camera before movement and the three-dimensional spatial point coordinates in the high-precision visual map. The aforementioned target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image acquired by the vehicle-road cooperative camera after movement and the three-dimensional spatial point coordinates in the high-precision visual map.

[0036] Step S26: Based on the target point pair information, calibrate the current extrinsic parameters of the image acquisition component, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after the movement.

[0037] Specifically, extrinsic parameters can include the position information and rotation angle of the vehicle-to-infrastructure (V2I) camera in the world coordinate system. By establishing the correspondence between the two-dimensional pixel coordinates in the second frame image acquired after the V2I camera has moved and the three-dimensional spatial point coordinates in the high-precision visual map, the current extrinsic parameters of the V2I camera can be accurately calibrated automatically. This enables the V2I system to obtain reliable beyond-line-of-sight perception information based on the V2I camera after its position has moved, thereby providing information assistance for autonomous vehicles.

[0038] According to steps S20 to S24 of this disclosure, by registering the first corner point and the second corner point, a registration result is obtained. Then, using the registration result, the target point pair information corresponding to the initial point pair information is obtained. Finally, the current extrinsic parameters of the image acquisition component are calibrated based on the target point pair information. This achieves the purpose of quickly and automatically calibrating the extrinsic parameters of the image acquisition component, thereby improving the timeliness of calibration of vehicle-road cooperative camera parameters and reducing calibration costs. This solves the technical problems of poor timeliness and high calibration costs of calibration of vehicle-road cooperative camera parameters in related technologies.

[0039] The method for calibrating the extrinsic parameters of the image acquisition component in the above embodiments will be further described below.

[0040] As an optional implementation, in step S22, the first corner point and the second corner point are registered to obtain the registration result, including:

[0041] Step S221: Obtain the first three-dimensional coordinates corresponding to the first pixel coordinates, wherein the first pixel coordinates are the two-dimensional pixel coordinates of the first corner point in the first frame image;

[0042] Specifically, the aforementioned first pixel coordinates are the two-dimensional pixel coordinates of the four vertices corresponding to each dashed lane line in the first frame image captured by the vehicle-to-infrastructure (V2I) camera before it moves. For example, the first pixel coordinates are as described above. Figure 3 The two-dimensional pixel coordinates of the four vertices A, B, C, and D corresponding to the dashed lane line L1 can also be alternatively described as the original corner point two-dimensional pixel coordinates (points_origin). Based on points_origin, the corresponding first three-dimensional coordinates are obtained. The first three-dimensional coordinates are the three-dimensional spatial coordinates of points_origin in the high-precision visual map.

[0043] Step S222: Use a corner detection model to detect the coordinates of the second pixel in the second frame image, where the coordinates of the second pixel are the two-dimensional pixel coordinates of the second corner in the second frame image;

[0044] Specifically, the aforementioned second pixel coordinates are the two-dimensional pixel coordinates of the four vertices corresponding to each dashed lane line detected from the second frame image using a corner detection model. For example, the second pixel coordinates are as described above. Figure 4 The two-dimensional pixel coordinates of the four vertices A', B', C', and D' corresponding to the dashed lane line L2 can also be alternatively described as the two-dimensional pixel coordinates of the detected corner (pts2d_corner_detect).

[0045] Step S223: Project the first three-dimensional coordinates onto the second frame image using the historical extrinsic parameters of the image acquisition component to obtain the registration result of the second pixel coordinates and the third pixel coordinates. The historical extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the first position before movement, and the third pixel coordinates are the two-dimensional pixel coordinates obtained by projecting the first three-dimensional coordinates onto the second frame image.

[0046] Specifically, the aforementioned historical extrinsic parameters are the extrinsic parameters calibrated by the vehicle-road cooperative camera at the first position. The historical extrinsic parameters are used to project points_origin onto the second frame image acquired by the vehicle-road cooperative camera at the second position, and the combination-sorting-matching algorithm is used to obtain the registration result between the third pixel coordinates (pts2d_corner) and pts2d_corner_detect.

[0047] Based on the above optional implementation, by obtaining the first three-dimensional coordinates corresponding to the first pixel coordinates, and then using a corner detection model to detect the second pixel coordinates in the second frame image, and finally using the historical extrinsic parameters of the image acquisition component to project the first three-dimensional coordinates onto the second frame image, the registration results of the second pixel coordinates and the third pixel coordinates can be quickly obtained, thereby further realizing the automated calibration of vehicle-road cooperative camera parameters and reducing calibration costs.

[0048] As an optional implementation, in step S223, projecting the first three-dimensional coordinates onto the second frame image using historical extrinsic parameters to obtain the registration result of the second pixel coordinates and the third pixel coordinates includes:

[0049] Step S2231: Project the first three-dimensional coordinates onto the second frame image using historical extrinsic parameters to obtain the fourth pixel coordinates;

[0050] Step S2232: Using the second pixel coordinates as a reference, select the fifth pixel coordinates from the fourth pixel coordinates within a preset range around the second pixel coordinates;

[0051] Step S2233: Combine the coordinates of the fifth pixel based on the number of corner points of the two-dimensional dashed lane line to obtain multiple target contours;

[0052] Step S2234: Sort multiple target contours according to a preset direction to obtain a sorting result;

[0053] Step S2235: Obtain the registration result of the second pixel coordinate and the third pixel coordinate based on the sorting result.

[0054] Specifically, the first 3D coordinates are projected onto the second frame image using historical extrinsic parameters to obtain the fourth pixel coordinates. Then, using the pts2d_corner_detect obtained through the corner detection model as a reference, n fifth pixel coordinates within a 50-pixel range around each pts2d_corner_detect are selected from the fourth pixel coordinates. The pts2d_corner_detect obtained through the corner detection model is grouped into sets of four coordinates, representing the four vertices corresponding to each dashed lane line in the second frame image. Subsequently, the fifth pixel coordinates are combined in groups of four to obtain multiple bounding boxes. Each group of four coordinates is sorted clockwise to obtain a bounding box. Finally, contour matching is performed on multiple bounding boxes to obtain accurate 2D-2D point pairs between the corner coordinates of the dashed lane lines in the moved image and the 2D coordinates of the high-precision visual map point projection, i.e., the registration result of the second pixel coordinates and the third pixel coordinates.

[0055] Based on the above optional implementation methods, the accurate registration results of the second pixel coordinates and the third pixel coordinates are obtained by using the combination-sorting-matching algorithm, thereby further realizing the accurate calibration of the vehicle-road cooperative camera parameters.

[0056] As an optional implementation, in step S24, using the registration results, obtaining the target point pair information corresponding to the initial point pair information includes:

[0057] Step S241: Determine the homography matrix between the first frame image and the second frame image using the registration results;

[0058] Step S242: Obtain the target point pair information corresponding to the initial point pair information based on the homography matrix.

[0059] Specifically, the homography matrix mentioned above is a 3*3 matrix. By calling a preset function in the cross-platform computer vision library (OpenCV), the homography matrix between the first frame image and the second frame image is determined using the registration result, thereby obtaining the target point pair information corresponding to the initial point pair information.

[0060] Based on the above optional implementation methods, the homography matrix of the first frame image and the second frame image is determined by the registration result. Then, the target point pair information corresponding to the initial point pair information can be obtained based on the homography matrix, which can be used for the rapid calibration of the current extrinsic parameters of the vehicle-road cooperative camera to further improve the calibration efficiency.

[0061] As an optional implementation, in step S241, determining the homography matrix between the first frame image and the second frame image using the registration result includes:

[0062] Using the registration results, the coordinates of the second and third pixels are set as input parameters of the homography function to calculate the homography matrix.

[0063] Specifically, the homography function mentioned above can be the findHomography function.

[0064] Based on the above optional implementation methods, by using the registration results, the coordinates of the second pixel and the third pixel are set as input parameters of the homography function, and the homography matrix is ​​quickly calculated, thereby further improving the timeliness of the calibration of the current extrinsic parameters of the vehicle-road cooperative camera.

[0065] As an optional implementation, in step S242, obtaining the target point pair information corresponding to the initial point pair information based on the homography matrix includes:

[0066] Step S2421: Calculate the sixth pixel coordinates corresponding to the first pixel coordinates based on the homography matrix, where the sixth pixel coordinates are the two-dimensional pixel coordinates corresponding to the first pixel coordinates in the second frame image;

[0067] Step S2422: Determine the target point pair information using the sixth pixel coordinate and the second three-dimensional coordinate, wherein the second three-dimensional coordinate is the coordinate of the three-dimensional spatial point in the target electronic map that corresponds to the sixth pixel coordinate.

[0068] Specifically, the sixth pixel coordinate (points_curr) of points_origin in the second frame image is calculated using the homography matrix. Then, the three-dimensional spatial coordinates of the corresponding point in the high-precision visual map, i.e. the second three-dimensional coordinates, are determined using points_curr and the second three-dimensional coordinates. Finally, the target point pair information is determined based on points_curr and the second three-dimensional coordinates.

[0069] Based on the above optional implementation, by calculating the sixth pixel coordinate corresponding to the first pixel coordinate based on the homography matrix, the target point pair information can be quickly determined using the sixth pixel coordinate and the second three-dimensional coordinate, thereby further improving the timeliness of the calibration of the current external parameters of the vehicle-road cooperative camera.

[0070] As an optional implementation, step S26, calibrating the current extrinsic parameters of the image acquisition component based on the target point pair information, includes:

[0071] The target point pair information is set as the input parameter of the monocular ranging algorithm to calibrate the current extrinsic parameters of the image acquisition component.

[0072] The monocular ranging algorithm mentioned above can be the Perspective-n-Point (PNP) algorithm in OpenCV. Using the PNP algorithm, the pose of the vehicle-road cooperative camera in space can be calculated, including the position information and rotation angle of the vehicle-road cooperative camera at the second position. That is, the current extrinsic parameters of the vehicle-road cooperative camera at the second position can be obtained, thereby automatically calibrating the current extrinsic parameters of the vehicle-road cooperative camera in the vehicle-road cooperative system.

[0073] The method based on the embodiments of this disclosure obtains a registration result by registering the first corner point and the second corner point, and then uses the registration result to obtain the target point pair information corresponding to the initial point pair information. Finally, the current extrinsic parameters of the image acquisition component are calibrated based on the target point pair information. This method can achieve automated camera extrinsic parameter calibration at the daily level for moving vehicle-road cooperative cameras, which is highly efficient and reduces a lot of manpower and material costs. This effectively solves the technical problems of poor timeliness and high calibration cost of vehicle-road cooperative camera parameters in related technologies.

[0074] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0075] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0076] This disclosure also provides an apparatus for calibrating the extrinsic parameters of an image acquisition component, which is used to implement the above embodiments and preferred embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0077] Figure 5 This is a structural block diagram of an apparatus for calibrating the extrinsic parameters of an image acquisition component according to one embodiment of the present disclosure, such as... Figure 5 As shown, a device 500 for calibrating the extrinsic parameters of an image acquisition component includes:

[0078] The registration module 501 is used to register the first corner point and the second corner point to obtain the registration result. The first corner point is the corner point of the two-dimensional dashed lane line in the first frame image, and the second corner point is the corner point of the two-dimensional dashed lane line in the second frame image. The first frame image is the image acquired and calibrated by the image acquisition component at the first position before movement, and the second image is the image acquired by the image acquisition component at the second position after movement.

[0079] The acquisition module 502 is used to acquire target point pair information corresponding to the initial point pair information using the registration result. The initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image and the three-dimensional spatial point coordinates in the target electronic map. The target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image and the three-dimensional spatial point coordinates in the target electronic map.

[0080] The calibration module 503 is used to calibrate the current extrinsic parameters of the image acquisition component based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after the movement.

[0081] Optionally, the registration module 501 is further configured to: obtain the first three-dimensional coordinates corresponding to the first pixel coordinates, wherein the first pixel coordinates are the two-dimensional pixel coordinates of the first corner point in the first frame image; detect the second pixel coordinates in the second frame image using a corner detection model, wherein the second pixel coordinates are the two-dimensional pixel coordinates of the second corner point in the second frame image; project the first three-dimensional coordinates onto the second frame image using the historical extrinsic parameters of the image acquisition component, and obtain the registration result of the second pixel coordinates and the third pixel coordinates, wherein the historical extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the first position before movement, and the third pixel coordinates are the two-dimensional pixel coordinates obtained by projecting the first three-dimensional coordinates onto the second frame image.

[0082] Optionally, the registration module 501 is further configured to: project the first three-dimensional coordinates onto the second frame image using the historical extrinsic parameters to obtain the fourth pixel coordinates; select a fifth pixel coordinate within a preset range around the second pixel coordinates from the fourth pixel coordinates, using the second pixel coordinates as a reference; combine the fifth pixel coordinates based on the number of corner points of the two-dimensional dashed lane line to obtain multiple target contours; sort the multiple target contours according to a preset direction to obtain a sorting result; and obtain the registration result of the second pixel coordinates and the third pixel coordinates based on the sorting result.

[0083] Optionally, the acquisition module 502 is further configured to: determine the homography matrix between the first frame image and the second frame image using the registration result; and acquire the target point pair information corresponding to the initial point pair information based on the homography matrix.

[0084] Optionally, the acquisition module 502 is further configured to: use the registration result to set the second pixel coordinates and the third pixel coordinates as input parameters of the homography function, and calculate the homography matrix.

[0085] Optionally, the acquisition module 502 is further configured to: calculate the sixth pixel coordinate corresponding to the first pixel coordinate based on the homography matrix, wherein the sixth pixel coordinate is the two-dimensional pixel coordinate corresponding to the first pixel coordinate in the second frame image; and determine the target point pair information using the sixth pixel coordinate and the second three-dimensional coordinate, wherein the second three-dimensional coordinate is the three-dimensional spatial point coordinate corresponding to the sixth pixel coordinate in the target electronic map.

[0086] Optionally, the calibration module 503 is further configured to: set the target point pair information as the input parameters of the monocular ranging algorithm, and calibrate the current extrinsic parameters of the image acquisition component.

[0087] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0088] According to embodiments of this disclosure, this disclosure also provides an electronic device including a memory and at least one processor, the memory storing computer instructions, the processor being configured to execute the computer instructions to perform the steps in the above method embodiments.

[0089] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0090] Optionally, in this disclosure, the processor described above can be configured to perform the following steps via a computer program:

[0091] S1, register the first corner point and the second corner point to obtain the registration result, wherein the first corner point is the corner point of the two-dimensional dashed lane line in the first frame image, the second corner point is the corner point of the two-dimensional dashed lane line in the second frame image, the first frame image is the image acquired and calibrated by the image acquisition component at the first position before movement, and the second image is the image acquired by the image acquisition component at the second position after movement;

[0092] S2, using the registration result, obtain the target point pair information corresponding to the initial point pair information. The initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image and the three-dimensional spatial point coordinates in the target electronic map. The target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image and the three-dimensional spatial point coordinates in the target electronic map.

[0093] S3, calibrate the current extrinsic parameters of the image acquisition component based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after the movement.

[0094] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0095] According to embodiments of this disclosure, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to execute the steps in the above method embodiments at runtime.

[0096] Optionally, in this embodiment, the aforementioned non-transitory computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0097] S1, register the first corner point and the second corner point to obtain the registration result, wherein the first corner point is the corner point of the two-dimensional dashed lane line in the first frame image, the second corner point is the corner point of the two-dimensional dashed lane line in the second frame image, the first frame image is the image acquired and calibrated by the image acquisition component at the first position before movement, and the second image is the image acquired by the image acquisition component at the second position after movement;

[0098] S2, using the registration result, obtain the target point pair information corresponding to the initial point pair information. The initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image and the three-dimensional spatial point coordinates in the target electronic map. The target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image and the three-dimensional spatial point coordinates in the target electronic map.

[0099] S3, calibrate the current extrinsic parameters of the image acquisition component based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after the movement.

[0100] Optionally, in this embodiment, the aforementioned non-transitory computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] According to embodiments of this disclosure, a computer program product is also provided. Program code for implementing embodiments of the methods of this disclosure can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on a machine, partially on a machine, partially on a remote machine as a standalone software package, or entirely on a remote machine or server.

[0102] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0107] The above description is only a preferred embodiment of this disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure.

Claims

1. A method for calibrating the extrinsic parameters of an image acquisition component, comprising: The first corner point and the second corner point are registered to obtain the registration result. The first corner point is the corner point of the two-dimensional dashed lane line in the first frame image, and the second corner point is the corner point of the two-dimensional dashed lane line in the second frame image. The first frame image is the image acquired and calibrated by the image acquisition component at the first position before the movement, and the second frame image is the image acquired by the image acquisition component at the second position after the movement. Using the registration result, target point pair information corresponding to the initial point pair information is obtained, wherein the initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image and the three-dimensional spatial point coordinates in the target electronic map, and the target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image and the three-dimensional spatial point coordinates in the target electronic map; The current extrinsic parameters of the image acquisition component are calibrated based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after the movement; The registration of the first corner point and the second corner point to obtain the registration result includes: Obtain the first three-dimensional coordinates corresponding to the first pixel coordinates, wherein the first pixel coordinates are the two-dimensional pixel coordinates of the first corner point in the first frame image; A corner detection model is used to detect the coordinates of the second pixel in the second frame image, wherein the coordinates of the second pixel are the two-dimensional pixel coordinates of the second corner in the second frame image; The first three-dimensional coordinates are projected onto the second frame image using the historical extrinsic parameters of the image acquisition component to obtain the registration result of the second pixel coordinates and the third pixel coordinates. The historical extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the first position before movement, and the third pixel coordinates are the two-dimensional pixel coordinates obtained by projecting the first three-dimensional coordinates onto the second frame image.

2. The method according to claim 1, wherein, Projecting the first three-dimensional coordinates onto the second frame image using the historical extrinsic parameters, and obtaining the registration result of the second pixel coordinates and the third pixel coordinates, includes: The first three-dimensional coordinates are projected onto the second frame image using the historical extrinsic parameters to obtain the fourth pixel coordinates; Using the second pixel coordinate as a reference, select the fifth pixel coordinate from the fourth pixel coordinate, which is located within a preset range around the second pixel coordinate; Based on the number of corner points of the two-dimensional dashed lane line, the coordinates of the fifth pixel are combined to obtain multiple target contours; The multiple target contours are sorted according to a preset direction to obtain a sorting result; The registration result of the second pixel coordinate and the third pixel coordinate is obtained based on the sorting result.

3. The method according to claim 1, wherein, Using the registration result, obtaining the target point pair information corresponding to the initial point pair information includes: The homography matrix between the first frame image and the second frame image is determined using the registration result; The target point pair information corresponding to the initial point pair information is obtained based on the homography matrix.

4. The method according to claim 3, wherein, Determining the homography matrix between the first frame image and the second frame image using the registration result includes: Using the registration result, the second pixel coordinates and the third pixel coordinates are set as input parameters of the homography function to calculate the homography matrix.

5. The method according to claim 3, wherein, Obtaining the target point pair information corresponding to the initial point pair information based on the homography matrix includes: The sixth pixel coordinate corresponding to the first pixel coordinate is calculated based on the homography matrix, wherein the sixth pixel coordinate is the two-dimensional pixel coordinate corresponding to the first pixel coordinate in the second frame image; The target point pair information is determined using the sixth pixel coordinate and the second three-dimensional coordinate, wherein the second three-dimensional coordinate is the coordinate of a three-dimensional spatial point in the target electronic map that corresponds to the sixth pixel coordinate.

6. The method according to claim 1, wherein, The calibration of the current extrinsic parameters of the image acquisition component based on the target point pair information includes: The target point pair information is set as the input parameter of the monocular ranging algorithm to calibrate the current extrinsic parameters of the image acquisition component.

7. A device for calibrating the extrinsic parameters of an image acquisition component, comprising: The registration module is used to register the first corner point and the second corner point to obtain the registration result. The first corner point is the corner point of the two-dimensional dashed lane line in the first frame image, and the second corner point is the corner point of the two-dimensional dashed lane line in the second frame image. The first frame image is the image acquired and calibrated by the image acquisition component at the first position before movement, and the second image is the image acquired by the image acquisition component at the second position after movement. The acquisition module is used to acquire target point pair information corresponding to the initial point pair information using the registration result. The initial point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the first frame image and the three-dimensional spatial point coordinates in the target electronic map. The target point pair information is used to determine the correspondence between the two-dimensional pixel coordinates in the second frame image and the three-dimensional spatial point coordinates in the target electronic map. The calibration module is used to calibrate the current extrinsic parameters of the image acquisition component based on the target point pair information, wherein the current extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the second position after the movement; The registration module is further used for: Obtain the first three-dimensional coordinates corresponding to the first pixel coordinates, wherein the first pixel coordinates are the two-dimensional pixel coordinates of the first corner point in the first frame image; A corner detection model is used to detect the coordinates of the second pixel in the second frame image, wherein the coordinates of the second pixel are the two-dimensional pixel coordinates of the second corner in the second frame image; The first three-dimensional coordinates are projected onto the second frame image using the historical extrinsic parameters of the image acquisition component to obtain the registration result of the second pixel coordinates and the third pixel coordinates. The historical extrinsic parameters are the extrinsic parameters calibrated by the image acquisition component at the first position before movement, and the third pixel coordinates are the two-dimensional pixel coordinates obtained by projecting the first three-dimensional coordinates onto the second frame image.

8. The apparatus according to claim 7, wherein, The registration module is also used for: The first three-dimensional coordinates are projected onto the second frame image using the historical extrinsic parameters to obtain the fourth pixel coordinates; Using the second pixel coordinate as a reference, select the fifth pixel coordinate from the fourth pixel coordinate, which is located within a preset range around the second pixel coordinate; Based on the number of corner points of the two-dimensional dashed lane line, the coordinates of the fifth pixel are combined to obtain multiple target contours; The multiple target contours are sorted according to a preset direction to obtain a sorting result; The registration result of the second pixel coordinate and the third pixel coordinate is obtained based on the sorting result.

9. The apparatus according to claim 7, wherein, The acquisition module is also used for: The homography matrix between the first frame image and the second frame image is determined using the registration result; The target point pair information corresponding to the initial point pair information is obtained based on the homography matrix.

10. The apparatus according to claim 9, wherein, The acquisition module is also used for: Using the registration result, the second pixel coordinates and the third pixel coordinates are set as input parameters of the homography function to calculate the homography matrix.

11. The apparatus according to claim 9, wherein, The acquisition module is also used for: The sixth pixel coordinate corresponding to the first pixel coordinate is calculated based on the homography matrix, wherein the sixth pixel coordinate is the two-dimensional pixel coordinate corresponding to the first pixel coordinate in the second frame image; The target point pair information is determined using the sixth pixel coordinate and the second three-dimensional coordinate, wherein the second three-dimensional coordinate is the coordinate of a three-dimensional spatial point in the target electronic map that corresponds to the sixth pixel coordinate.

12. The apparatus according to claim 7, wherein, The calibration module is also used for: The target point pair information is set as the input parameter of the monocular ranging algorithm to calibrate the current extrinsic parameters of the image acquisition component.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images