Method, device and equipment for determining camera extrinsic parameter and storage medium

Abstract

The present disclosure provides a method and device for determining camera extrinsic parameters, an equipment and a storage medium, relating to the technical field of vehicle-road cooperation, and particularly to the technical fields of camera parameter calibration and camera coordinate transformation. The specific implementation scheme is as follows: obtaining pixel coordinates of at least one feature point in an image, and matching corresponding world coordinates for the at least one feature point based on preset map information to obtain a coordinate pair of the at least one feature point. The coordinate pair includes pixel coordinates and world coordinates corresponding to the respective feature points, and the map information includes world coordinates of each position in a region photographed by the camera. The extrinsic parameters of the camera are determined according to the coordinate pair. The extrinsic parameters of the camera can be automatically determined based on the preset map information, so as to automatically update the extrinsic parameters of the camera subsequently, thereby saving manpower, shortening the time for determining the extrinsic parameters of the camera, and improving the timeliness of the extrinsic parameters of the camera.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle-road cooperative technology, and in particular to the field of camera parameter calibration and camera coordinate transformation technology, specifically to a method, apparatus, device, and storage medium for determining camera extrinsic parameters. Background Technology

[0002] Against the backdrop of the new infrastructure initiative, V2X roadside perception systems provide vehicles with beyond-line-of-sight perception information for vehicle-to-infrastructure (V2I) communication. As one of the most crucial sensors in roadside perception systems, the accurate intrinsic and extrinsic parameters of cameras are vital for the perception capabilities of V2I systems.

[0003] The current method for calibrating (or determining) camera extrinsic parameters involves manually calibrating the camera extrinsic parameters based on the camera coordinates and pitch angle after the camera rotates or translates. This method is labor-intensive and time-consuming. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and storage medium for determining camera extrinsic parameters. It can automatically determine the camera extrinsic parameters based on preset map information, so as to automatically update the camera extrinsic parameters in the future, thereby saving manpower, shortening the time for determining camera extrinsic parameters, and improving the timeliness of camera extrinsic parameters.

[0005] According to a first aspect of this disclosure, a method for determining camera extrinsic parameters is provided, comprising:

[0006] Acquire images captured by the camera; acquire the pixel coordinates of at least one feature point in the image, and match the corresponding world coordinates for the at least one feature point based on preset map information to obtain a coordinate pair of at least one feature point; the coordinate pair includes the pixel coordinates and world coordinates corresponding to the feature point, and the map information includes the world coordinates of each location in the area captured by the camera; determine the camera's extrinsic parameters based on the coordinate pair.

[0007] According to a second aspect of this disclosure, an apparatus for determining camera extrinsic parameters is provided, comprising: an acquisition module for acquiring an image captured by a camera; a processing module for acquiring the pixel coordinates of at least one feature point in the image, and matching the corresponding world coordinates for the at least one feature point based on preset map information to obtain a coordinate pair of at least one feature point; the coordinate pair includes the pixel coordinates and world coordinates corresponding to the respective feature point, and the map information includes the world coordinates of each location in the area captured by the camera; and a determination module for determining the extrinsic parameters of the camera based on the coordinate pair.

[0008] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method provided in the first aspect.

[0009] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the method provided according to the first aspect.

[0010] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to the first aspect.

[0011] This disclosure describes a method for obtaining the pixel coordinates of at least one feature point in an image captured by a camera. Then, based on a preset map containing world coordinates of various locations within the area captured by the camera, the corresponding world coordinates are matched to the at least one feature point in the acquired image, resulting in a coordinate pair consisting of the pixel coordinates and world coordinates of that feature point. The camera's extrinsic parameters can then be determined based on the obtained coordinate pair. In this way, the camera's extrinsic parameters can be automatically determined based on the captured image and the preset map information, saving manpower, shortening the time required to determine the camera's extrinsic parameters, and improving the timeliness of the determination. Therefore, when the camera moves (such as by translation or rotation), the extrinsic parameters can be determined more quickly for subsequent updates.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0014] Figure 1 This is one of the flowcharts illustrating a method for determining camera extrinsic parameters provided in this embodiment of the disclosure;

[0015] Figure 2 This is a second schematic flowchart illustrating a method for determining camera extrinsic parameters provided in an embodiment of this disclosure.

[0016] Figure 3 The third schematic flowchart of the method for determining camera extrinsic parameters provided in this embodiment of the disclosure;

[0017] Figure 4 A schematic diagram illustrating the composition of a device for determining camera extrinsic parameters provided in an embodiment of this disclosure;

[0018] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] The method and apparatus for determining camera extrinsic parameters disclosed herein are applicable to situations requiring the determination and updating of camera extrinsic parameters, particularly when the camera needs to be re-determined and updated after movement. The method for determining camera extrinsic parameters disclosed herein can be executed by the apparatus, which can be implemented in software and / or hardware and specifically configured in an electronic device. This electronic device can be a camera, server, smartphone, laptop, computer, microcontroller, or other computing device, without limitation. Alternatively, the apparatus can also be specifically configured in a system consisting of a camera and a server, so that the method for determining camera extrinsic parameters disclosed herein can be implemented jointly by the camera and the server, without limitation.

[0021] The following section will first provide a detailed description of the method for determining camera extrinsic parameters provided in this disclosure.

[0022] Against the backdrop of the new infrastructure initiative, V2X roadside perception systems provide vehicles with beyond-line-of-sight perception information for vehicle-to-infrastructure (V2I) communication. As one of the most crucial sensors in roadside perception systems, the accurate intrinsic and extrinsic parameters of cameras are vital for the perception capabilities of V2I systems.

[0023] Camera extrinsic parameters, also known as camera pose, consist of rotation and translation matrices. Determining camera extrinsic parameters refers to defining a rotation and translation matrix to describe the transformation relationship between the pixel coordinate system and other coordinate systems (such as the world coordinate system). Current camera extrinsic parameter calibration involves manually calibrating the parameters based on the camera coordinates and pitch angle after rotation or translation, which is labor-intensive and time-consuming.

[0024] In response, this disclosure provides a method for determining camera extrinsic parameters, comprising: acquiring an image captured by the camera; acquiring the pixel coordinates of at least one feature point in the image, and matching the corresponding world coordinates for the at least one feature point based on preset map information to obtain a coordinate pair of at least one feature point; the coordinate pair includes the pixel coordinates and world coordinates corresponding to the feature point, and the map information includes the world coordinates of each location in the area captured by the camera; and determining the camera extrinsic parameters based on the coordinate pair.

[0025] This disclosure describes a method for obtaining the pixel coordinates of at least one feature point in an image captured by a camera. Then, based on a preset map containing world coordinates of various locations within the area captured by the camera, the corresponding world coordinates are matched to the at least one feature point in the acquired image, resulting in a coordinate pair consisting of the pixel coordinates and world coordinates of that feature point. The camera's extrinsic parameters can then be determined based on the obtained coordinate pair. In this way, the camera's extrinsic parameters can be automatically determined based on the captured image and the preset map information, saving manpower, shortening the time required to determine the camera's extrinsic parameters, and improving the timeliness of the determination. Therefore, when the camera moves (such as by translation or rotation), the extrinsic parameters can be determined more quickly for subsequent updates.

[0026] Figure 1 This is a flowchart illustrating a method for determining camera extrinsic parameters provided in an embodiment of this disclosure. Figure 1 As shown, the method may include the following S101-S103.

[0027] S101. Acquire images captured by the camera.

[0028] Optionally, when camera movement is detected, images captured by the camera can be acquired and subsequent steps can be performed to determine camera extrinsic parameters, thereby enabling timely determination and updating of the latest camera extrinsic parameters when camera movement occurs. Detecting camera movement can be implemented based on roadside algorithms. For example, the pixel coordinates of fixed features (such as lane lines, lane markings, etc.) in the images captured by the camera can be continuously detected; when a change in the pixel coordinates of a fixed feature is detected, camera movement can be determined. Of course, in other feasible embodiments of this application, sensors can also be installed on the camera to monitor changes in camera posture to determine whether camera movement has occurred; this is not a limitation.

[0029] Optionally, images captured by the camera can be acquired periodically, and subsequent steps can be performed to determine the camera's extrinsic parameters. This allows for the periodic determination and updating of the latest camera extrinsic parameters to maintain their timeliness. For example, the period can be set to one day, allowing the camera's extrinsic parameters to be determined daily and updated promptly when they change. Of course, the period described above in this embodiment can be set according to actual circumstances and is not limited here.

[0030] For example, in a vehicle-to-infrastructure (V2I) scenario, the camera whose extrinsic parameters need to be determined can be a roadside camera, and the image captured by the camera can be a road surface image. The image may include the road surface, lane lines (solid lane lines and dashed lane lines, etc.), lane markings, etc., within a preset shooting area. Of course, in some other embodiments, this method can also be applied to scenarios where other cameras are positioned at fixed locations; this is not limited here. Correspondingly, the image captured by the camera can also be other images, without limitation here. For example, the camera could be an indoor monitoring system in a shopping mall; in this case, for example, the image captured by the camera could be an indoor image of the shopping mall, which may include shop fronts, etc., within a preset shooting area.

[0031] S102. Obtain the pixel coordinates of at least one feature point in the image, and match the corresponding world coordinates for the at least one feature point based on preset map information, so as to obtain a coordinate pair of at least one feature point.

[0032] The coordinate pairs include the pixel coordinates and world coordinates of the corresponding feature points, and the map information includes the world coordinates of each location in the area captured by the camera.

[0033] For example, feature points in an image can be points on feature elements within the image. For instance, when the image is a road surface image, feature points can be points on lane lines, lane markings, or road barriers, etc. As another example, when the image is an indoor image of a shopping mall, feature points can be points on shop doors, points on mall escalators, etc., without any restrictions.

[0034] Accordingly, the world coordinates of each location in the shooting area included in the preset map information can be the world coordinates of all points in the shooting area, or the world coordinates of points on corresponding feature elements in the shooting area. For example, when the image is a road surface image, the map information may include the world coordinates of each point on the lane lines, the world coordinates of each point on the lane markings, the world coordinates of each point on the shoulder, and the world coordinates of each point on the guardrail, etc. Of course, the map information may also include the world coordinates of all points in the area captured by the image.

[0035] Optionally, the method for obtaining the pixel coordinates of at least one feature point in the image may be to perform feature segmentation detection or semantic recognition on the image to obtain feature elements in the image, thereby determining the pixel coordinates of at least one point on the feature element as the pixel coordinates of at least one feature point in the image.

[0036] For example, based on preset map information, to match the corresponding world coordinates of feature points with obtained pixel coordinates, one could extract the corresponding feature points from the map information based on feature recognition, and then match the feature points in the map information with the feature points in the image with obtained pixel coordinates, thereby matching the world coordinates of the corresponding feature points in the map information to the corresponding feature points in the image. Of course, in this application, other methods can also be used to match world coordinates of feature points with obtained pixel coordinates based on preset map information; this is not limited here. For example, the world coordinates of each point in the map information can be projected onto the image, and then the coordinates of the projected points can be matched with the pixel coordinates of the obtained feature points. The world coordinates corresponding to the matched projected points can then be used as the world coordinates for the corresponding feature point matching.

[0037] S103. Determine the camera's extrinsic parameters based on the coordinate pairs.

[0038] For example, a loss function can be established based on coordinate pairs, and then the camera extrinsic parameters can be obtained through optimization based on the loss function. Alternatively, the camera extrinsic parameters can also be obtained based on coordinate pairs using least-squares nonlinear optimization; there is no restriction here.

[0039] Optionally, when the image acquired by the camera is a road surface image, since the road surface image may include feature elements such as the road surface, lane lines (solid lane lines and dashed lane lines, etc.), and lane markings, the pixel coordinates of each feature point on the lane lines in the image can be obtained, as well as the world coordinates corresponding to each feature point on the lane lines in the matching image. And / or, the pixel coordinates of each corner point of the dashed lane lines in the image can be obtained, as well as the world coordinates corresponding to each corner point of the dashed lane lines in the matching image.

[0040] That is, the image is a road surface image. The pixel coordinates of at least one feature point in the image are obtained, and based on preset map information, the corresponding world coordinates are matched for at least one feature point to obtain a coordinate pair of at least one feature point. This can include:

[0041] Obtain the pixel coordinates of each feature point on the lane line in the image, and based on map information, match the corresponding world coordinates for each feature point on the lane line in the image to obtain the coordinate pairs of each feature point on the lane line in the image.

[0042] And / or, obtain the pixel coordinates of each dashed lane corner point in the image, and based on map information, match the corresponding world coordinates for each dashed lane corner point in the image to obtain the coordinate pairs of each dashed lane corner point in the image.

[0043] Optionally, obtaining the pixel coordinates of each feature point on the lane line in the image can be done by identifying the lane lines in the image based on a lane line segmentation model, and then obtaining the pixel coordinates of each feature point on the identified lane line. Similarly, obtaining the pixel coordinates of the corner points of each dashed lane line in the image can be done by identifying the corner points of the dashed lane lines in the image based on a lane line corner detection model, and then obtaining the pixel coordinates of the identified dashed lane line corner points.

[0044] Since the map information includes the coordinates of each location in the shooting area, the lane lines on the map correspond to the lane lines (including dashed lane lines) in the image. Therefore, based on the map information, it is possible to match the world coordinates of each feature point on the lane line in the image.

[0045] Since lane line detection and dashed lane line corner point detection are relatively simple and accurate, obtaining the corresponding coordinate pairs based on each feature point on the lane line and / or each dashed lane line corner point is relatively convenient and has high accuracy.

[0046] Optionally, when the feature points in the acquired image are feature points on the lane lines, the world coordinates of each feature point on the lane lines in the map information can be projected onto the image, and then the projected points can be matched for each feature point on the lane lines in the image, thereby matching the corresponding world coordinates for each feature point on the lane lines in the image.

[0047] For example, take the first feature point among the feature points on the lane line in the image.

[0048] That is, the first feature point is included among the feature points on the lane lines in the image. Then, based on map information, the corresponding world coordinates are matched for each feature point on the lane lines in the image, such as... Figure 2 As shown, it may include:

[0049] S201. Obtain the world coordinates of each feature point on the lane line in the map information.

[0050] The map information can be a high-resolution map of the camera's shooting area, or it can be a visual map (i.e., a visual SLAM map) of the camera's shooting area; there is no limitation here. Since the map information corresponds to the camera's shooting area, the lane lines in the map information also correspond to the lane lines in the camera's image. Therefore, the world coordinates of each feature point on the corresponding lane line in the map information can be directly extracted.

[0051] S202. Based on the camera's initial extrinsic parameters, convert the world coordinates of each feature point on the lane line in the map information into the first projected coordinates.

[0052] Wherein, the first projection coordinates are the pixel coordinates of each feature point on the lane line in the map information when it is projected onto the image according to the initial extrinsic parameters.

[0053] The initial extrinsic parameters can be the extrinsic parameters calibrated during the initial camera setup, or the extrinsic parameters currently being used by the camera; there are no restrictions here.

[0054] S203. Match the world coordinates corresponding to the first projection coordinates that are closest to the pixel coordinates of the first feature point to the world coordinates corresponding to the first feature point.

[0055] Optionally, a nearest neighbor search can be used to match the first projected coordinates that are closest to the pixel coordinates of the first feature point. Alternatively, the distance between each first projected coordinate and the pixel coordinates of the first feature point can be directly calculated, and then the closest first projected coordinate can be determined.

[0056] It should be noted that the above steps are illustrated using the first feature point as an example. Therefore, by performing S203 once for each feature point on the lane line in the image, the corresponding world coordinates can be matched for each feature point on the lane line in the image.

[0057] Based on initial extrinsic parameters, feature points along lane lines in the map information can be projected onto the image. When the camera moves, the points projected onto the image according to the initial extrinsic parameters will not coincide with the lane lines in the image. Therefore, the point corresponding to the first projected coordinates closest to the first feature point in the image can be identified as the point corresponding to the first feature point in the map information. Thus, the world coordinates corresponding to these first projected coordinates can be matched as the world coordinates of the first feature point. In this way, the first feature point in the image can be matched with points in the map information relatively accurately, thereby matching its corresponding world coordinates. This improves the accuracy of the matched feature point coordinate pairs, thus improving the accuracy of the camera extrinsic parameters subsequently obtained based on the coordinate pairs.

[0058] Optionally, when the feature points in the acquired image are the corner points of each dashed lane line, the world coordinates of each dashed lane line corner point in the map information can be projected onto the image, and then the projected points can be matched for each dashed lane line corner point in the image, thereby matching the corresponding world coordinates for each dashed lane line corner point in the image.

[0059] For example, consider the first dashed lane line in the image, where each dashed lane line includes four corner points.

[0060] That is, each dashed lane line includes four corner points, and the image includes the first dashed lane line. Based on map information, the corresponding world coordinates are matched for each corner point of the dashed lane line in the image, such as... Figure 3 As shown, it may include:

[0061] S301. Obtain the world coordinates of the corner points of each dashed lane line in the map information.

[0062] The map information can be a visual map (i.e., a visual SLAM map) of the camera's shooting area, or a high-resolution map of the camera's shooting area; there is no limitation here. Since the map information corresponds to the camera's shooting area, the dashed lane lines in the map information also correspond to the dashed lane lines in the image captured by the camera. Therefore, the world coordinates of the corner points of the corresponding dashed lane lines in the map information can be directly extracted.

[0063] S302. Based on the camera's initial extrinsic parameters, convert the world coordinates of the corner points of each dashed lane line in the map information into second projected coordinates.

[0064] The second projection coordinates are the pixel coordinates of the corner points of each dashed lane line in the map information when they are projected onto the image according to the initial extrinsic parameters.

[0065] The initial extrinsic parameters can be the extrinsic parameters calibrated during the initial camera setup, or the extrinsic parameters currently being used by the camera; there are no restrictions here.

[0066] S303. The second projection coordinates located within a preset distance range of each corner point of the first dashed lane line are combined in groups of four.

[0067] The first dashed lane line may include four corner points, and each corner point may include at least one second projected coordinate within a preset distance range. Therefore, the second projected coordinates located within the preset distance range of each corner point of the first dashed lane line include at least four coordinates, which can be combined into at least one combination.

[0068] For example, since pixel coordinates are usually measured in pixels, the preset distance range can also be set in pixels. For instance, the preset distance range could be 50 pixels. That is, all second projected coordinates within a circle with a radius of 50 pixels, centered at a corner point of the first dashed lane line, are the second projected coordinates within the preset distance range of that corner point.

[0069] S304. The contours of the closed figures enclosed by the points corresponding to the four second projection coordinates of each combination are matched with the contours of the closed figures enclosed by the corner points of the first dashed lane line. The world coordinates corresponding to the four second projection coordinates of the combination with the highest similarity are matched with the world coordinates corresponding to the corresponding corner points of the first dashed lane line.

[0070] Optionally, contour similarity matching can be achieved using a matrix contour matching algorithm, and there are no restrictions here.

[0071] Since each combination of four second-projection coordinates can be considered as one of the four corner points of a dashed lane line, the combination with the highest similarity between the outline of the closed shape formed by the points corresponding to the four second-projection coordinates and the outline of the closed shape formed by the corner points of the first dashed lane line (i.e., the outline of the first dashed lane line) can be considered identical to the first dashed lane line. Therefore, the points corresponding to the four second-projection coordinates of this combination can be regarded as points in the map information that match the four corner points of the first dashed lane line. Thus, the world coordinates corresponding to the four second-projection coordinates of this combination can be matched with the corner points of the first dashed lane line.

[0072] Since the corner points of dashed lane lines are typically detected starting from the bottom left corner of the parallelogram-shaped dashed lane line and proceeding clockwise, before matching the world coordinates corresponding to the four second-projected coordinates of the highest similarity combination with the corresponding corner points of the first dashed lane line, the four second-projected coordinates of the highest similarity combination can be sorted clockwise starting from the bottom left corner. This facilitates accurate mapping of the four second-projected coordinates of this combination to the four corner points of the first dashed lane line. Alternatively, before step S304, the four second-projected coordinates of each combination can be sorted clockwise starting from the bottom left corner. This facilitates the subsequent mapping of the four second-projected coordinates of the combination with the highest contour similarity to the four corner points of the first dashed lane line after the highest contour similarity combination is determined.

[0073] Since the pixel coordinates of the corner points of the dashed lane lines in the image and the world coordinates of the corner points of the dashed lane lines in the map information are relatively accurate, obtaining the coordinate pairs of the corresponding feature points based on the corner points of the dashed lane lines in this step is more accurate, thereby improving the accuracy of the camera extrinsic parameters obtained from the coordinate pairs in the subsequent steps.

[0074] It should be noted that the above steps are illustrated using the first dashed lane line as an example. Therefore, by performing S303-S304 once for each dashed lane line in the image, the corresponding world coordinates can be matched for the corner points of each dashed lane line in the image.

[0075] Optionally, determining the camera's extrinsic parameters based on the coordinate pairs may include:

[0076] The camera's extrinsic parameters are obtained based on the coordinate pairs using least-squares nonlinear optimization.

[0077] For example, based on least squares, a loss function can be constructed by inputting each pair of coordinates as follows:

[0078]

[0079] Where m represents the number of coordinate pairs; (uo) i ,vo i () represents the pixel coordinate in the i-th coordinate pair; (Xw i ,Yw i Zw i ) represents the world coordinates in the i-th coordinate pair; R represents the rotation matrix of the camera extrinsic parameters, t represents the translation matrix of the camera extrinsic parameters; K represents the intrinsic parameter matrix of the camera extrinsic parameters.

[0080] Therefore, optimization algorithms such as gradient descent, Gauss-Newton method, and LM algorithm can be used to solve the above loss function to obtain the rotation matrix R and translation matrix t of the camera extrinsic parameters.

[0081] The camera's extrinsic parameters are obtained by using least-squares nonlinear optimization, which can more accurately express the correspondence between pixel coordinates and world coordinates in each coordinate pair, thus resulting in more accurate camera extrinsic parameters.

[0082] Optionally, after determining the camera's extrinsic parameters based on the coordinate pairs, the method may further include updating the camera's extrinsic parameters to the determined extrinsic parameters. Thus, after determining the camera's extrinsic parameters, the currently used extrinsic parameters of the camera can be updated to the determined extrinsic parameters, thereby keeping the camera's extrinsic parameters up-to-date and improving their timeliness.

[0083] In an exemplary embodiment, this disclosure also provides an apparatus for determining camera extrinsic parameters, which can be used to implement the method for determining camera extrinsic parameters as described in the foregoing embodiments.

[0084] Figure 4 A schematic diagram of the composition of a device for determining camera extrinsic parameters provided in an embodiment of this disclosure.

[0085] like Figure 4 As shown, the device may include:

[0086] The acquisition module 401 is used to acquire images captured by the camera;

[0087] The processing module 402 is used to obtain the pixel coordinates of at least one feature point in the image, and match the corresponding world coordinates for at least one feature point based on preset map information to obtain a coordinate pair of at least one feature point; the coordinate pair includes the pixel coordinates and world coordinates corresponding to the feature point, and the map information includes the world coordinates of each location in the area captured by the camera;

[0088] The determination module 403 is used to determine the extrinsic parameters of the camera based on the coordinate pairs.

[0089] In some possible implementations, the image is a road surface image; the processing module 402 is specifically used to obtain the pixel coordinates of each feature point on the lane line in the image, and based on map information, match the corresponding world coordinates for each feature point on the lane line in the image to obtain coordinate pairs of each feature point on the lane line in the image; and / or, obtain the pixel coordinates of each corner point of the dashed lane line in the image, and based on map information, match the corresponding world coordinates for each corner point of the dashed lane line in the image to obtain coordinate pairs of each corner point of the dashed lane line in the image.

[0090] In some possible implementations, each feature point on the lane line in the image includes a first feature point; the processing module 402 is specifically used to obtain the world coordinates of each feature point on the lane line in the map information; based on the camera's initial extrinsic parameters, the world coordinates of each feature point on the lane line in the map information are converted into first projected coordinates, the first projected coordinates being the pixel coordinates corresponding to each feature point on the lane line in the map information when projected onto the image according to the initial extrinsic parameters; and the world coordinates corresponding to the first projected coordinates closest to the pixel coordinates of the first feature point are matched as the world coordinates corresponding to the first feature point.

[0091] In some possible implementations, each dashed lane line includes four corner points, and the image includes a first dashed lane line. The processing module 402 is specifically used to acquire the world coordinates of each dashed lane line corner point in the map information; based on the camera's initial extrinsic parameters, it converts the world coordinates of each dashed lane line corner point in the map information into second projected coordinates, where the second projected coordinates are the pixel coordinates corresponding to each dashed lane line corner point projected onto the image according to the initial extrinsic parameters; it combines the second projected coordinates located within a preset distance range of each corner point of the first dashed lane line into groups of four; it performs similarity matching between the contours of the closed shapes formed by the points corresponding to the four combined second projected coordinates and the contours of the closed shapes formed by the corner points of the first dashed lane line, and matches the world coordinates corresponding to the four second projected coordinates of the combination with the highest similarity to the world coordinates corresponding to the respective corner points of the first dashed lane line.

[0092] In some possible implementations, the determination module 403 is specifically used to obtain the camera's extrinsic parameters based on least-squares nonlinear optimization according to the coordinate pairs.

[0093] In some possible implementations, the apparatus further includes an update module 404 for updating the camera's extrinsic parameters to the determined extrinsic parameters.

[0094] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0095] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0096] In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in the above embodiments.

[0097] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method described in the above embodiments.

[0098] In an exemplary embodiment, the computer program product includes a computer program that, when executed by a processor, implements the method described in the above embodiments.

[0099] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The electronic device may also be an image acquisition device such as a camera or camcorder. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0100] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0101] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0102] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for determining camera extrinsic parameters. For example, in some embodiments, the method for determining camera extrinsic parameters may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the method for determining camera extrinsic parameters described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to perform the method for determining camera extrinsic parameters by any other suitable means (e.g., by means of firmware).

[0103] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0107] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0108] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0109] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining camera extrinsic parameters, characterized in that, include: Acquire images captured by the camera; The pixel coordinates of at least one feature point in the image are obtained, and based on preset map information, the corresponding world coordinates are matched for the at least one feature point to obtain a coordinate pair of the at least one feature point; the coordinate pair includes the pixel coordinates and world coordinates corresponding to the feature point, and the map information includes the world coordinates of each location in the area captured by the camera; Based on the coordinate pairs, determine the extrinsic parameters of the camera; The image is a road surface image; the image includes a first dashed lane line; the process of matching the corresponding world coordinates for the at least one feature point based on preset map information includes: Obtain the world coordinates of the corner points of each dashed lane line in the map information; each dashed lane line includes 4 corner points; Based on the initial extrinsic parameters of the camera, the world coordinates of each corner point of the dashed lane line in the map information are transformed into second projected coordinates. The second projected coordinates are the pixel coordinates corresponding to each corner point of the dashed lane line in the map information when projected onto the image according to the initial extrinsic parameters. The second projected coordinates located within a preset distance range of each corner point of the first dashed lane line are combined in groups of four. The contours of the closed figures formed by the points corresponding to the four second projection coordinates of each combination are matched with the contours of the closed figures formed by the corner points of the first dashed lane line. The world coordinates corresponding to the four second projection coordinates of the combination with the highest similarity are matched with the world coordinates corresponding to the corresponding corner points of the first dashed lane line.

2. The method according to claim 1, characterized in that, The step of obtaining the pixel coordinates of at least one feature point in the image and matching the corresponding world coordinates for the at least one feature point based on preset map information to obtain the coordinate pairs of the at least one feature point further includes: The pixel coordinates of each feature point on the lane line in the image are obtained, and based on the map information, the corresponding world coordinates are matched for each feature point on the lane line in the image to obtain the coordinate pairs of each feature point on the lane line in the image.

3. The method according to claim 2, characterized in that, The image includes a first feature point among the feature points on the lane lines; the step of matching corresponding world coordinates for each feature point on the lane lines in the image based on the map information includes: Obtain the world coordinates of each feature point on the lane line in the map information; Based on the initial extrinsic parameters of the camera, the world coordinates of each feature point on the lane line in the map information are transformed into first projected coordinates. The first projected coordinates are the pixel coordinates corresponding to each feature point on the lane line in the map information when projected onto the image according to the initial extrinsic parameters. Match the world coordinates corresponding to the first projection coordinates that are closest to the pixel coordinates of the first feature point to the world coordinates corresponding to the first feature point.

4. The method according to claim 1, characterized in that, Determining the extrinsic parameters of the camera based on the coordinate pair includes: The extrinsic parameters of the camera are obtained based on the coordinate pairs using least-squares nonlinear optimization.

5. The method according to claim 1, characterized in that, After determining the extrinsic parameters of the camera based on the coordinate pair, the method further includes: Update the camera's extrinsic parameters to the determined extrinsic parameters.

6. A device for determining camera extrinsic parameters, characterized in that, include: The acquisition module is used to acquire images captured by the camera; The processing module is used to obtain the pixel coordinates of at least one feature point in the image, and match the corresponding world coordinates for the at least one feature point based on preset map information to obtain a coordinate pair of the at least one feature point; the coordinate pair includes the pixel coordinates and world coordinates corresponding to the feature point, and the map information includes the world coordinates of each location in the area captured by the camera; The determination module is used to determine the extrinsic parameters of the camera based on the coordinate pair; The image is a road surface image; the image includes a first dashed lane line; the processing module is specifically used to obtain the world coordinates of the corner points of each dashed lane line in the map information; each dashed lane line includes 4 corner points; based on the initial extrinsic parameters of the camera, the world coordinates of the corner points of each dashed lane line in the map information are converted into second projected coordinates, the second projected coordinates being the pixel coordinates corresponding to the corner points of each dashed lane line in the map information when projected onto the image according to the initial extrinsic parameters; the second projected coordinates located within a preset distance range of each corner point of the first dashed lane line are combined in groups of 4; the contour of the closed figure enclosed by the points corresponding to the 4 second projected coordinates of each combination is matched with the contour of the closed figure enclosed by each corner point of the first dashed lane line for similarity matching, and the world coordinates corresponding to the 4 second projected coordinates of the combination with the highest similarity are matched as the world coordinates corresponding to the corresponding corner points of the first dashed lane line.

7. The apparatus according to claim 6, characterized in that, The processing module is specifically used to obtain the pixel coordinates of each feature point on the lane line in the image, and based on the map information, to match the corresponding world coordinates for each feature point on the lane line in the image, so as to obtain the coordinate pairs of each feature point on the lane line in the image.

8. The apparatus according to claim 7, characterized in that, The image includes a first feature point among the feature points on the lane line; The processing module is specifically used to obtain the world coordinates of each feature point on the lane line in the map information; based on the initial extrinsic parameters of the camera, convert the world coordinates of each feature point on the lane line in the map information into first projected coordinates, wherein the first projected coordinates are the pixel coordinates corresponding to each feature point on the lane line in the map information when projected onto the image according to the initial extrinsic parameters; and match the world coordinates corresponding to the first projected coordinates that are closest to the pixel coordinates of the first feature point as the world coordinates corresponding to the first feature point.

9. The apparatus according to claim 6, characterized in that, The determining module is specifically used to obtain the extrinsic parameters of the camera based on the coordinate pair and least-squares nonlinear optimization.

10. The apparatus according to claim 6, characterized in that, The device further includes an update module for updating the camera's extrinsic parameters to the determined extrinsic parameters.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images