An automatic calibration method, device and server for a laser camera and a spherical screen camera

Abstract

The application is suitable for the technical field of camera calibration, and provides an automatic calibration method, device and server for a laser camera and a dome camera, the method comprising: acquiring a to-be-processed image corresponding to a calibration object; extracting feature points of the to-be-processed image; and calibrating internal parameters of the dome camera and external parameters between the laser camera and the dome camera according to the feature points, wherein the relative positions between the laser camera and the dome camera are fixed. It can be seen that the internal parameters of the dome camera and the external parameters between the laser camera and the dome camera are calibrated through the feature point processing of the calibration object, the automatic calibration of the laser camera and the dome camera is realized, the internal and external parameter calibration precision of the laser camera and the dome camera is improved in combination with the characteristics of the laser camera and the dome camera, and the calibration process is time-saving and efficient.

Description

Technical Field

[0001] This application belongs to the field of camera calibration technology, and in particular relates to an automatic calibration method, device and server for laser cameras and dome cameras. Background Technology

[0002] Currently, the use of laser camera and dome camera fusion technology to create various high-end technological products is becoming an increasingly popular trend in China, leading to rapid development in related technologies. The most fundamental technology for this fusion is the calibration of the intrinsic and extrinsic parameters between the two cameras, which is essential for ensuring the consistency of the environmental information perceived by both. However, existing technologies suffer from low accuracy during the calibration process for both laser cameras and dome cameras. Summary of the Invention

[0003] This application provides an automatic calibration method, apparatus, and server for laser cameras and dome cameras, which can solve the problems of low accuracy in the calibration process of laser cameras and dome cameras in the prior art.

[0004] In a first aspect, embodiments of this application provide an automatic calibration method for a laser camera and a dome camera, comprising:

[0005] Obtain the image to be processed corresponding to the calibration object;

[0006] Extract feature points from the image to be processed;

[0007] The intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera are determined based on the feature points, wherein the relative positions between the laser camera and the dome camera are fixed.

[0008] In one possible implementation of the first aspect, the calibration object includes a first structural component and a second structural component;

[0009] The image to be processed includes a first RGB image, a second RGB image, a first intensity image, and a first depth image;

[0010] The laser camera includes a first laser camera, a second laser camera, and a third laser camera with fixed relative positions;

[0011] Obtain the image to be processed corresponding to the calibration object, including:

[0012] Obtain the first RGB image corresponding to the first structural component;

[0013] Wherein, the first RGB image is the image corresponding to the first structural component obtained by the dome camera at a preset distance from the first structural component;

[0014] Obtain the image to be processed corresponding to the calibration object, including:

[0015] Obtain the second RGB image, the first intensity image, and the first depth image corresponding to the second structural component;

[0016] The second RGB image, the first intensity image, and the first depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos of the second structural component from a preset distance from the dome camera.

[0017] In one possible implementation of the first aspect, the feature points include 2D-2D matching points and a first 2D-3D matching point;

[0018] Extracting feature points from the image to be processed includes:

[0019] Extract the first 2D corner points of the first RGB image;

[0020] Based on the first 2D corner point, the corresponding 2D-2D matching point of the dome camera is determined;

[0021] Extracting feature points from the image to be processed includes:

[0022] Extract the second 2D corner points of the second RGB image and the third 2D corner points of the first intensity image, wherein there is a one-to-one correspondence between the second 2D corner points and the third 2D corner points;

[0023] Based on the pixel correspondence between the first intensity image and the first depth image, the first position point of the third 2D corner point on the first depth image is found, and the third 2D corner point is converted into a first 3D corner point.

[0024] Based on the second 2D corner point of the second RGB image and the first 2D corner point of the first intensity image, the first 2D-3D matching points of the dome camera with the first laser camera, the second laser camera and the third laser camera are determined respectively.

[0025] In one possible implementation of the first aspect, calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points includes:

[0026] The first cost function is determined based on the 2D-2D matching points;

[0027] The second cost function is determined based on the first 2D-3D matching point;

[0028] By combining the first cost function and the second cost function, the first intrinsic parameter of the dome camera, the first extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the third extrinsic parameter between the third laser camera and the dome camera are optimized.

[0029] In one possible implementation of the first aspect, the image to be processed includes a second intensity image and a second depth image;

[0030] Obtain the image to be processed corresponding to the calibration object, including:

[0031] Acquire the second intensity image and the second depth image corresponding to the calibration object. The second intensity image and the second depth image are images obtained by the first laser camera and the second laser camera, and the second laser camera and the third camera taking photos simultaneously.

[0032] The feature points include 3D-3D matching points;

[0033] Extracting feature points from the image to be processed includes:

[0034] Extract the fourth 2D corner point of the second intensity image;

[0035] Based on the pixel correspondence between the second intensity image and the second depth image, the second position point of the fourth 2D corner point on the second depth image is found, and the fourth 2D corner point is converted into a second 3D corner point.

[0036] Based on the fourth 2D corner point and the second 3D corner point of the second intensity image, the 3D-3D matching points between the first laser camera and the second laser camera, and between the second laser camera and the third laser camera, are determined.

[0037] After calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points, the method further includes:

[0038] Set the first intrinsic parameter of the dome camera and the second extrinsic parameter between the second laser camera and the dome camera to fixed values;

[0039] Initialize the first extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera;

[0040] The third cost function is determined based on the 3D-3D matching points;

[0041] Based on the third cost function, and on the first extrinsic parameter between the first laser camera and the dome camera and the third extrinsic parameter between the third laser camera and the dome camera after initialization, the fourth extrinsic parameter between the first laser camera and the dome camera and the fifth extrinsic parameter between the third laser camera and the dome camera are optimized.

[0042] In one possible implementation of the first aspect, the image to be processed includes a third RGB image, a third intensity image, and a third depth image;

[0043] Obtain the image to be processed corresponding to the calibration object, including:

[0044] Obtain the third RGB image, third intensity image, and third depth image corresponding to the second structural component;

[0045] The third RGB image, the third intensity image, and the third depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos together with the dome camera during rotation at a preset position away from the second structural component.

[0046] Based on the third cost function, and after optimizing the fourth and fifth extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera, according to the initialized first extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera, the following further steps are taken:

[0047] Extract 2D line features from the third RGB image;

[0048] Extract 3D line features corresponding to the third depth image;

[0049] Initialize the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera;

[0050] Based on the 2D line features and the 3D line features, the second 2D-3D matching points between the dome camera and the first laser camera, the second laser camera, and the third laser camera are determined respectively.

[0051] The fourth cost function is determined based on the second 2D-3D matching point;

[0052] Based on the fourth cost function, and on the initial first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera, the second intrinsic parameter of the dome camera, the sixth extrinsic parameter between the first laser camera and the dome camera, the seventh extrinsic parameter between the second laser camera and the dome camera, and the eighth extrinsic parameter between the third laser camera and the dome camera are calibrated.

[0053] Secondly, embodiments of this application provide an automatic calibration device for laser cameras and dome cameras, comprising:

[0054] The acquisition module is used to acquire the image to be processed corresponding to the calibration object;

[0055] The extraction module is used to extract feature points from the image to be processed;

[0056] The calibration module is used to calibrate the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points, wherein the relative positions between the laser camera and the dome camera are fixed.

[0057] In one possible implementation of the second aspect, the calibration object includes a first structural member and a second structural member;

[0058] The image to be processed includes a first RGB image, a second RGB image, a first intensity image, and a first depth image;

[0059] The acquisition module includes:

[0060] The first acquisition module is used to acquire the first RGB image corresponding to the first structural component;

[0061] Wherein, the first RGB image is the image corresponding to the first structural component obtained by the dome camera at a preset distance from the first structural component;

[0062] The acquisition module includes:

[0063] The second acquisition module is used to acquire the second RGB image, the first intensity image, and the first depth image corresponding to the second structural component;

[0064] The second RGB image, the first intensity image, and the first depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos of the second structural component from a preset distance from the dome camera.

[0065] In one possible implementation of the second aspect, the feature points include 2D-2D matching points and a first 2D-3D matching point;

[0066] The extraction module includes:

[0067] The first extraction submodule is used to extract the first 2D corner points of the first RGB image;

[0068] The first determining submodule is used to determine the 2D-2D matching point corresponding to the dome camera based on the first 2D corner point;

[0069] The extraction module includes:

[0070] The second extraction submodule is used to extract the second 2D corner points of the second RGB image and the third 2D corner points of the first intensity image, wherein there is a one-to-one correspondence between the second 2D corner points and the third 2D corner points;

[0071] The first conversion submodule is used to find the first position point of the third 2D corner point on the first depth image according to the pixel correspondence between the first intensity image and the first depth image, and convert the third 2D corner point into a first 3D corner point.

[0072] The second determining submodule is used to determine the first 2D-3D matching points of the dome camera with the first laser camera, the second laser camera, and the third laser camera respectively, based on the second 2D corner points of the second RGB image and the first 2D corner points of the first intensity image.

[0073] In one possible implementation of the second aspect, the calibration module includes:

[0074] The third determining submodule is used to determine the first cost function based on the 2D-2D matching points;

[0075] The fourth determining submodule is used to determine the second cost function based on the first 2D-3D matching points;

[0076] The joint calibration submodule is used to combine the first cost function and the second cost function to optimize the first intrinsic parameter of the dome camera, the first extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the third extrinsic parameter between the third laser camera and the dome camera.

[0077] In one possible implementation of the second aspect, the image to be processed includes a second intensity image and a second depth image;

[0078] The acquisition module includes:

[0079] The third acquisition module is used to acquire the second intensity image and the second depth image corresponding to the calibration object. The second intensity image and the second depth image are images obtained by the first laser camera and the second laser camera, and the second laser camera and the third camera taking photos together.

[0080] The extraction module includes:

[0081] The third extraction submodule is used to extract the fourth 2D corner point of the second intensity image;

[0082] The second conversion submodule is used to find the second position point of the fourth 2D corner point on the second depth image according to the pixel correspondence between the second intensity image and the second depth image, and convert the fourth 2D corner point into a second 3D corner point;

[0083] The fifth determining submodule is used to determine the 3D-3D matching points between the first laser camera and the second laser camera, and between the second laser camera and the third laser camera, based on the fourth 2D corner point and the second 3D corner point of the second intensity image;

[0084] The device further includes:

[0085] The setting module is used to set the first intrinsic parameter of the dome camera and the second extrinsic parameter between the second laser camera and the dome camera to fixed values;

[0086] An initialization module is used to initialize the first extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera;

[0087] The second determining module is used to determine the third cost function based on the 3D-3D matching points;

[0088] The second calibration module is used to optimize the fourth extrinsic parameter between the first laser camera and the dome camera and the fifth extrinsic parameter between the third laser camera and the dome camera based on the third cost function and the first extrinsic parameter between the first laser camera and the dome camera after initialization, as well as the third extrinsic parameter between the third laser camera and the dome camera.

[0089] In one possible implementation of the second aspect, the image to be processed includes a third RGB image, a third intensity image, and a third depth image;

[0090] The acquisition module includes:

[0091] The third acquisition submodule is used to acquire the third RGB image, the third intensity image and the third depth image corresponding to the second structural component;

[0092] The third RGB image, the third intensity image, and the third depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos together with the dome camera during rotation at a preset position away from the second structural component.

[0093] The device further includes:

[0094] The second extraction submodule is used to extract 2D line features from the third RGB image;

[0095] The third extraction submodule is used to extract the 3D line features corresponding to the third depth image;

[0096] An initialization module is used to initialize the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera;

[0097] The third determining module is used to determine the second 2D-3D matching points of the dome camera with the first laser camera, the second laser camera and the third laser camera respectively, based on the 2D line features and the 3D line features;

[0098] The fourth determining module is used to determine the fourth cost function based on the second 2D-3D matching point;

[0099] The third calibration module is used to calibrate the second intrinsic parameter of the dome camera, the sixth extrinsic parameter between the first laser camera and the dome camera, the seventh extrinsic parameter between the second laser camera and the dome camera, and the eighth extrinsic parameter between the third laser camera and the dome camera, based on the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera, according to the fourth cost function.

[0100] Thirdly, embodiments of this application provide a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0101] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0102] The beneficial effects of the embodiments in this application compared with the prior art are:

[0103] In this embodiment, the process involves acquiring the image to be processed corresponding to the calibration object; extracting feature points from the image to be processed; and calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points, wherein the relative positions between the laser camera and the dome camera are fixed. It is evident that this application calibrates the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera by processing the feature points of the calibration object, thereby achieving automated calibration of the laser camera and the dome camera. Furthermore, by combining the characteristics of both the laser camera and the dome camera, the calibration accuracy of their intrinsic and extrinsic parameters is improved. The calibration process is time-efficient and highly effective.

[0104] Image descriptions included

[0105] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying images used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying images described below are only some embodiments of this application. For those skilled in the art, other accompanying images can be obtained based on these accompanying images without creative effort.

[0106] Figure 1 This is a flowchart illustrating the automatic calibration method for laser cameras and dome cameras provided in the embodiments of this application;

[0107] Figure 2 This is a schematic diagram of the automatic calibration device for laser cameras and dome cameras provided in the embodiments of this application;

[0108] Figure 3 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0109] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0110] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0111] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0112] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0113] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0114] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0115] The automatic calibration method for laser cameras and dome cameras provided in this application embodiment can be applied to a server, preferably a cloud server. This application embodiment does not impose any restrictions on the specific type of terminal device. The server is connected to 3D cameras, wherein the 3D cameras include laser cameras and dome cameras, and the relative positions between the laser cameras and dome cameras are fixed. The laser cameras include a first laser camera (i.e., an UpLidar laser camera), a second laser camera (i.e., a MidLidar laser camera), and a third laser camera (i.e., a DownLidar laser camera) with fixed relative positions. For example, the UpLidar laser camera, the MidLidar laser camera, and the DownLidar laser camera are arranged alternately in the order of top, middle, and bottom on the same vertical line.

[0116] The technical solutions provided in the embodiments of this application will be described below through specific examples.

[0117] Referring to Figure 1, which is a flowchart illustrating the automatic calibration method for a laser camera and a dome camera provided in an embodiment of this application, this method is provided as an example and not as a limitation. This method can be applied to a server and may include the following steps:

[0118] Step S102: Obtain the image to be processed corresponding to the calibration object.

[0119] In specific applications, the calibration objects include a first structural component and a second structural component, and the images to be processed include a first RGB image, a second RGB image, a first intensity image, and a first depth image.

[0120] The first structural component consists of multiple vertically arranged checkerboard grids, and the second structural component consists of multiple squarely arranged checkerboard grids; RGB image refers to an image representing color information, intensity image refers to an image representing reflection intensity information, and depth image refers to an image representing depth information; laser camera refers to a laser sensor, such as a lidar laser sensor.

[0121] Obtain the image to be processed corresponding to the calibration object, including:

[0122] Obtain the first RGB image corresponding to the first structural component.

[0123] The first RGB image is an image of the first structural component obtained by a dome camera at a preset distance (e.g., 2.5 meters) from the first structural component.

[0124] In a specific application, the 3D camera (laser camera and dome camera system) is placed at a distance of about 2.5m from the structure, so that the dome camera can capture RGB images of the M1 group containing overlapping areas with a checkerboard pattern.

[0125] Obtain the image to be processed corresponding to the calibration object, including:

[0126] Obtain the second RGB image, the first intensity image, and the first depth image corresponding to the second structural component.

[0127] The second RGB image, the first intensity image, and the first depth image are obtained by taking photos of the second structural component from a pre-set distance (e.g., 2.5 meters) between the first laser camera, the second laser camera, and the third laser camera, respectively, and the dome camera. It should be noted that "co-viewing" refers to viewing the component from an overlapping area with a checkerboard pattern.

[0128] In a specific application, the 3D camera is placed at a distance of about 2.5m from the second structural component, and the UpLidar laser camera, MidLidar laser camera, and DownLidar laser camera respectively capture the first intensity image, the first depth image, and the second RGB image of the overlapping area with the dome camera, which are displayed in a checkerboard pattern. Among them, the UpLidar-Cam (i.e., the UpLidar laser camera and the Cam dome camera are captured together) contains M2 sets of the first intensity image, the first depth image, and the second RGB image; the DownLidar-Cam (i.e., the DownLidar laser camera and the Cam dome camera are captured together) contains M2 sets of the first intensity image, the first depth image, and the second RGB image; and the MidLidar-Cam (i.e., the MidLidar laser camera and the Cam dome camera are captured together) contains M3 sets of the first intensity image, the first depth image, and the second RGB image.

[0129] In one possible implementation, the image to be processed includes a second intensity image and a second depth image.

[0130] Obtain the image to be processed corresponding to the calibration object, including:

[0131] The second intensity image and the second depth image corresponding to the calibrated object are acquired. The second intensity image and the second depth image are images obtained by co-viewing the first laser camera and the second laser camera, and the second laser camera and the third camera. Co-viewing refers to images with overlapping areas and a checkerboard pattern.

[0132] In practical applications, the orientation of the 3D camera is adjusted to capture M4 intensity and depth images of overlapping areas with a checkerboard pattern, where the UpLidar and MidLidar laser cameras and the MidLidar and DownLidar laser cameras share the same view. The principle of adjustment is to adjust the UpLidar and MidLidar laser cameras to capture the complete checkerboard pattern when capturing the shared view area between them. Similarly, the same applies when capturing images of the MidLidar and DownLidar laser cameras.

[0133] In one possible implementation, the image to be processed includes a third RGB image, a third intensity image, and a third depth image.

[0134] Obtain the image to be processed corresponding to the calibration object, including:

[0135] Obtain the third RGB image, third intensity image, and third depth image corresponding to the second structural component;

[0136] Among them, the third RGB image, the third intensity image, and the third depth image are images of the second structure obtained by the first laser camera, the second laser camera, and the third laser camera taking photos together with the dome camera during rotation at a preset position (e.g., 2.5 meters) away from the second structure.

[0137] In practical applications, the 3D camera orientation is adjusted to capture the M5 group, which includes the third RGB image, the second intensity image, and the second depth image, all rotated six times. The principle of adjustment is to ensure that the third RGB image, the second intensity image, and the second depth image, rotated six times, contained in the M5 group are as unique as possible.

[0138] Step S104: Extract feature points from the image to be processed.

[0139] The feature points include 2D-2D matching points, first 2D-3D matching points, and 3D-3D matching points. A 2D-2D matching point refers to a 2D corner point on multiple different images corresponding to a real point in the same world coordinate system; the relationship between these 2D corner points is called a 2D-2D matching point. A first 2D-3D matching point refers to a 2D corner point on multiple different images and a 3D corner point converted from a 2D corner point on another image, both corresponding to real points in the same world coordinate system; the relationship between these 2D corner points and the 3D corner points converted from 2D corner points on another image is called a first 2D-3D matching point. A 3D-3D matching point refers to a 3D corner point converted from a 2D corner point on multiple different images and a 3D corner point converted from a 2D corner point on another image, both corresponding to real points in the same world coordinate system; the relationship between these 3D corner points and the 3D corner points converted from 2D corner points on multiple different images is called a 3D-3D point.

[0140] In one possible implementation, feature points of the image to be processed are extracted, including:

[0141] Extract the first 2D corner point of the first RGB image.

[0142] Based on the first 2D corner point, the corresponding 2D-2D matching point of the dome camera is determined.

[0143] In practical applications, the principle for extracting the first 2D corner points of the first RGB image is as follows: The image histogram is equalized using a local average adaptive thresholding method, followed by binarization and image dilation to separate the connections between the various black quadrilateral blocks. Next, quadrilateral detection is performed to identify the adjacent quadrilaterals of each quadrilateral and record the number of adjacent quadrilaterals. All quadrilaterals are classified according to whether they have four adjacent quadrilaterals; quadrilaterals with four adjacent quadrilaterals are the required chessboard squares. The number of required chessboard squares after classification is checked against the known number of corner points. If they are not the same, a loop detection is performed until the number of chessboard squares matches the known number of corner points. The sequence number of each required chessboard square can be sorted by proximity, and then the midpoint of the line connecting the two opposite points of the diagonal quadrilaterals is taken as the corner point.

[0144] In one possible implementation, feature points of the image to be processed are extracted, including:

[0145] Extract the second 2D corner points of the second RGB image and the third 2D corner points of the first intensity image, wherein there is a one-to-one correspondence between the second 2D corner points and the third 2D corner points.

[0146] Based on the pixel correspondence between the first intensity image and the first depth image, the first position point of the third 2D corner point on the first depth image is found, and the third 2D corner point is converted into the first 3D corner point.

[0147] Based on the second 2D corner points of the second RGB image and the first 2D corner points of the first intensity image, the first 2D-3D matching points of the dome camera with the first laser camera, the second laser camera and the third laser camera are determined respectively.

[0148] In practical applications, the principle of extracting the second 2D corner point of the second RGB image and the third 2D corner point of the first intensity image is the same as the principle of extracting the first 2D corner point of the first RGB image, and will not be repeated here.

[0149] In one possible implementation, feature points of the image to be processed are extracted, including:

[0150] Extract the fourth 2D corner point of the second intensity image.

[0151] Based on the pixel correspondence between the second intensity image and the second depth image, the second position point of the fourth 2D corner point on the second depth image is found, and the fourth 2D corner point is converted into the second 3D corner point.

[0152] Based on the fourth 2D corner point and the second 3D corner point of the second intensity image, the 3D-3D matching points between the first laser camera and the second laser camera, and between the second laser camera and the third laser camera, are determined.

[0153] In practical applications, the principle of extracting the fourth 2D corner point of the second intensity image is the same as the principle of extracting the first 2D corner point of the first RGB image, and will not be repeated here.

[0154] Step S106: calibrate the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points.

[0155] The relative positions of the laser camera and the dome camera are fixed.

[0156] The intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera are determined based on feature points, including:

[0157] The first cost function is determined based on the 2D-2D matching points.

[0158] The second cost function is determined based on the first 2D-3D matching point.

[0159] By combining the first cost function and the second cost function, the first intrinsic parameter of the dome camera, the first extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the third extrinsic parameter between the third laser camera and the dome camera are optimized.

[0160] In practical applications, the first cost function is:

[0161]

[0162] in, Let x be the x-coordinate of the i-th point in the current frame projected onto the reference frame. The ordinate of the i-th point in the current frame projected onto the reference frame; The x-coordinate of the reference frame projected onto the current frame; The vertical coordinate of the reference frame projected onto the current frame.

[0163] The second cost function is:

[0164]

[0165] Where x refers to the x-coordinate of the 2D pixel, y refers to the y-coordinate of the 2D pixel, and p 3d_up_reproject This refers to the 2D pixels projected from the first laser camera onto the dome camera, p up_cam It refers to the 2D pixels of the second RGB image; p 3d_min_reproject This refers to the 2D points projected from the second laser camera onto the dome camera, p min_cam It refers to the 2D points of the second RGB image; p 3d_down_reproject This refers to the 2D pixels projected from the third laser camera onto the dome camera, p down_camM1 refers to the 2D pixels of the second RGB image, M2 refers to the number of first 2D-3D matching points between the first laser camera and the dome camera and between the third laser camera and the dome camera, and M3 refers to the number of first 2D-3D matching points between the second laser camera and the dome camera.

[0166] It is understood that, in the embodiments of this application, the first intrinsic parameter of the dome camera, the first extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the third extrinsic parameter between the third laser camera and the dome camera are determined by combining the first cost function and the second cost function.

[0167] Preferably, after calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on feature points, the method further includes:

[0168] The first intrinsic parameter of the dome camera and the second extrinsic parameter between the second laser camera and the dome camera are set to fixed values.

[0169] Initialize the first extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera.

[0170] The third cost function is determined based on the 3D-3D matching points.

[0171] Based on the third cost function, and on the first extrinsic parameter between the first laser camera and the dome camera and the third extrinsic parameter between the third laser camera and the dome camera after initialization, the fourth extrinsic parameter between the first laser camera and the dome camera and the fifth extrinsic parameter between the third laser camera and the dome camera are optimized.

[0172] In practical applications, the third cost function is:

[0173]

[0174] Where, p 3d_up_mid_reproject x up This refers to the abscissa of the 3D-3D matching points between the first and second laser cameras projected onto the second intensity map corresponding to the first laser camera. 3d_up_mid_reproject x mid This refers to the abscissa of the 3D-3D matching point between the first and second laser cameras projected onto the second intensity map corresponding to the second laser camera. 3d_up_mid_reproject y up This refers to the ordinate of the 3D-3D matching points between the first and second laser cameras, projected back onto the second intensity map corresponding to the first laser camera. 3d_up_mid_reproject y midThis refers to the ordinate of the 3D-3D matching point between the first and second laser cameras, projected back onto the second intensity map corresponding to the second laser camera. 3d_mid_down_reproject x mid This refers to the abscissa of the 3D-3D matching points between the second and third laser cameras projected onto the second intensity map corresponding to the second laser camera. 3d_up_mid_reproject x mid This refers to the abscissa of the 3D-3D matching points between the second and third laser cameras projected onto the second intensity map corresponding to the second laser camera. 3d_mid_down_reproject y mid This refers to the ordinate of the 3D-3D matching points between the second and third laser cameras, projected back onto the second intensity map corresponding to the second laser camera. 3d_up_mid_reproject y mid It refers to the vertical coordinate of the 3D-3D matching point between the first laser camera and the second laser camera back-projected onto the second intensity map corresponding to the second laser camera.

[0175] It is understood that, based on the first calibration of the first internal parameter of the dome camera, the first external parameter between the first laser camera and the dome camera, the second external parameter between the second laser camera and the dome camera, and the third external parameter between the third laser camera and the dome camera, this application may also calibrate the fourth external parameter between the first laser camera and the dome camera and the fifth external parameter between the third laser camera and the dome camera.

[0176] In an optional implementation, after optimizing the fourth and fifth extrinsic parameters between the first laser camera and the dome camera based on the initialized first extrinsic parameters between the first laser camera and the dome camera and the third extrinsic parameters between the third laser camera and the dome camera according to the third cost function, the method further includes:

[0177] Extract 2D line features from the third RGB image.

[0178] Extract the 3D line features corresponding to the third depth image.

[0179] Initialize the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera.

[0180] Based on 2D and 3D line features, the second 2D-3D matching points of the dome camera with the first, second, and third laser cameras are determined respectively.

[0181] The fourth cost function is determined based on the second 2D-3D matching point.

[0182] Based on the fourth cost function, the second intrinsic parameter of the dome camera, the sixth extrinsic parameter between the first laser camera and the dome camera, the seventh extrinsic parameter between the second laser camera and the dome camera, and the eighth extrinsic parameter between the third laser camera and the dome camera are calibrated, based on the initial first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera.

[0183] The fourth cost function is:

[0184]

[0185] Where x refers to the x-coordinate of the 2D pixel, y refers to the y-coordinate of the 2D pixel, and p 3d_up_reproject This refers to the 2D pixels projected from the first laser camera onto the dome camera, p up_cam It refers to the 2D pixels of the second RGB image; p 3d_min_reproject This refers to the 2D points projected from the second laser camera onto the dome camera, p min_cam It refers to the 2D points of the second RGB image; p 3d_down_reproject This refers to the 2D pixels projected from the third laser camera onto the dome camera, p down_cam M5 refers to the number of 2D pixels in the second RGB image, M6 refers to the number of second 2D-3D matching points between the first laser camera and the dome camera and between the third laser camera and the dome camera, and M6 refers to the number of second 2D-3D matching points between the second laser camera and the dome camera.

[0186] In practical applications, the specific implementation method for extracting 2D line features from the third RGB image is to use the Line Segment Detector (LSD) algorithm, and the specific implementation method for extracting 3D line features from the third depth image is to directly extract line features from the restored 3D point cloud through clustering.

[0187] It is understood that, based on the second calibration of the fourth external parameter between the first laser camera and the dome camera and the fifth external parameter between the third laser camera and the dome camera, this application can further refine the second internal parameter of the dome camera, the sixth external parameter between the first laser camera and the dome camera, the seventh external parameter between the second laser camera and the dome camera, and the eighth external parameter between the third laser camera and the dome camera.

[0188] In this embodiment, the process involves acquiring the image to be processed corresponding to the calibration object; extracting feature points from the image to be processed; and calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points, wherein the relative positions between the laser camera and the dome camera are fixed. It is evident that this application calibrates the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera by processing the feature points of the calibration object, thereby achieving automated calibration of the laser camera and the dome camera. Furthermore, by combining the characteristics of both the laser camera and the dome camera, the calibration accuracy of their intrinsic and extrinsic parameters is improved. The calibration process is time-efficient and highly effective.

[0189] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0190] Example 5

[0191] Corresponding to the method described in the above embodiments, Figure 5 shows a structural block image of the automatic calibration device for laser cameras and dome cameras provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0192] Referring to image 2, the device includes:

[0193] The acquisition module 21 is used to acquire the image to be processed corresponding to the calibration object;

[0194] Extraction module 22 is used to extract feature points of the image to be processed;

[0195] The calibration module 23 is used to calibrate the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points, wherein the relative positions between the laser camera and the dome camera are fixed.

[0196] In one possible implementation, the calibration object includes a first structural component and a second structural component;

[0197] The image to be processed includes a first RGB image, a second RGB image, a first intensity image, and a first depth image;

[0198] The acquisition module includes:

[0199] The first acquisition module is used to acquire the first RGB image corresponding to the first structural component;

[0200] Wherein, the first RGB image is the image corresponding to the first structural component obtained by the dome camera at a preset distance from the first structural component;

[0201] The acquisition module includes:

[0202] The second acquisition module is used to acquire the second RGB image, the first intensity image, and the first depth image corresponding to the second structural component;

[0203] The second RGB image, the first intensity image, and the first depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos of the second structural component from a preset distance from the dome camera.

[0204] In one possible implementation, the feature points include 2D-2D matching points and a first 2D-3D matching point;

[0205] The extraction module includes:

[0206] The first extraction submodule is used to extract the first 2D corner points of the first RGB image;

[0207] The first determining submodule is used to determine the 2D-2D matching point corresponding to the dome camera based on the first 2D corner point;

[0208] The extraction module includes:

[0209] The second extraction submodule is used to extract the second 2D corner points of the second RGB image and the third 2D corner points of the first intensity image, wherein there is a one-to-one correspondence between the second 2D corner points and the third 2D corner points;

[0210] The first conversion submodule is used to find the first position point of the third 2D corner point on the first depth image according to the pixel correspondence between the first intensity image and the first depth image, and convert the third 2D corner point into a first 3D corner point.

[0211] The second determining submodule is used to determine the first 2D-3D matching points of the dome camera with the first laser camera, the second laser camera, and the third laser camera respectively, based on the second 2D corner points of the second RGB image and the first 2D corner points of the first intensity image.

[0212] In one possible implementation, the calibration module includes:

[0213] The third determining submodule is used to determine the first cost function based on the 2D-2D matching points;

[0214] The fourth determining submodule is used to determine the second cost function based on the first 2D-3D matching points;

[0215] The joint calibration submodule is used to combine the first cost function and the second cost function to optimize the first intrinsic parameter of the dome camera, the first extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the third extrinsic parameter between the third laser camera and the dome camera.

[0216] In one possible implementation, the image to be processed includes a second intensity image and a second depth image;

[0217] The acquisition module includes:

[0218] The third acquisition module is used to acquire the second intensity image and the second depth image corresponding to the calibration object. The second intensity image and the second depth image are images obtained by the first laser camera and the second laser camera, and the second laser camera and the third camera taking photos together.

[0219] The extraction module includes:

[0220] The third extraction submodule is used to extract the fourth 2D corner point of the second intensity image;

[0221] The second conversion submodule is used to find the second position point of the fourth 2D corner point on the second depth image according to the pixel correspondence between the second intensity image and the second depth image, and convert the fourth 2D corner point into a second 3D corner point;

[0222] The fifth determining submodule is used to determine the 3D-3D matching points between the first laser camera and the second laser camera, and between the second laser camera and the third laser camera, based on the fourth 2D corner point and the second 3D corner point of the second intensity image;

[0223] The device further includes:

[0224] The setting module is used to set the first intrinsic parameter of the dome camera and the second extrinsic parameter between the second laser camera and the dome camera to fixed values;

[0225] An initialization module is used to initialize the first extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera;

[0226] The second determining module is used to determine the third cost function based on the 3D-3D matching points;

[0227] The second calibration module is used to optimize the fourth extrinsic parameter between the first laser camera and the dome camera and the fifth extrinsic parameter between the third laser camera and the dome camera based on the third cost function and the first extrinsic parameter between the first laser camera and the dome camera after initialization, as well as the third extrinsic parameter between the third laser camera and the dome camera.

[0228] In one possible implementation, the image to be processed includes a third RGB image, a third intensity image, and a third depth image;

[0229] The acquisition module includes:

[0230] The third acquisition submodule is used to acquire the third RGB image, the third intensity image and the third depth image corresponding to the second structural component;

[0231] The third RGB image, the third intensity image, and the third depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos together with the dome camera during rotation at a preset position away from the second structural component.

[0232] The device further includes:

[0233] The second extraction submodule is used to extract 2D line features from the third RGB image;

[0234] The third extraction submodule is used to extract the 3D line features corresponding to the third depth image;

[0235] An initialization module is used to initialize the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera;

[0236] The third determining module is used to determine the second 2D-3D matching points of the dome camera with the first laser camera, the second laser camera and the third laser camera respectively, based on the 2D line features and the 3D line features;

[0237] The fourth determining module is used to determine the fourth cost function based on the second 2D-3D matching point;

[0238] The third calibration module is used to calibrate the second intrinsic parameter of the dome camera, the sixth extrinsic parameter between the first laser camera and the dome camera, the seventh extrinsic parameter between the second laser camera and the dome camera, and the eighth extrinsic parameter between the third laser camera and the dome camera, based on the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera, according to the fourth cost function.

[0239] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0240] Figure 3 This is a schematic diagram of the server structure provided in an embodiment of this application. Figure 3 As shown, the server 3 in this embodiment includes: at least one processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30. When the processor 30 executes the computer program 32, it implements the steps in any of the above-described method embodiments.

[0241] The server 3 may be a computing device such as a cloud server. This server may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of server 3 and does not constitute a limitation on server 3. It may include more or fewer components than shown in the image, or a combination of certain components, or different components, such as input / output devices, network access devices, etc.

[0242] The processor 30 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0243] In some embodiments, the memory 31 may be an internal storage unit of the server 3, such as a hard disk or memory of the server 3. In other embodiments, the memory 31 may be an external storage device of the server 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the server 3. Furthermore, the memory 31 may include both internal storage units and external storage devices of the server 3. The memory 31 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0244] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0245] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0246] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0247] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0248] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0249] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0250] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0251] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A calibration method for a laser camera and a dome camera, characterized in that, include: Obtain the image to be processed corresponding to the calibration object; Extract feature points from the image to be processed; the feature points include 2D-2D matching points and a first 2D-3D matching point; the laser camera includes a first laser camera, a second laser camera, and a third laser camera with fixed relative positions; The intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera are determined based on the feature points, including: determining a first cost function based on 2D-2D matching points; determining a second cost function based on the first 2D-3D matching points; and optimizing the first intrinsic parameters of the dome camera, the first extrinsic parameters between the first laser camera and the dome camera, the second extrinsic parameters between the second laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera by combining the first and second cost functions; wherein the relative positions between the laser camera and the dome camera are fixed. The image to be processed includes a second intensity image and a second depth image. Obtaining the image to be processed corresponding to the calibration object includes: obtaining the second intensity image and the second depth image corresponding to the calibration object, wherein the second intensity image and the second depth image are images obtained by co-viewing the first laser camera and the second laser camera, or the second laser camera and the third camera; feature points include 3D-3D matching points; extracting feature points from the image to be processed includes: extracting the fourth 2D corner point of the second intensity image; finding the second position point of the fourth 2D corner point on the second depth image based on the pixel correspondence between the second intensity image and the second depth image, and converting the fourth 2D corner point into a second 3D corner point; determining the location of the first laser camera and the second laser camera based on the fourth 2D corner point and the second 3D corner point of the second intensity image. The process includes: establishing 3D-3D matching points between laser cameras and between the second and third laser cameras; calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser cameras and the dome camera based on feature points; setting the first intrinsic parameter of the dome camera and the second extrinsic parameter between the second laser camera and the dome camera to fixed values; initializing the first extrinsic parameter between the first laser camera and the dome camera and the third extrinsic parameter between the third laser camera and the dome camera; determining the third cost function based on the 3D-3D matching points; and optimizing the fourth extrinsic parameter between the first laser camera and the dome camera and the fifth extrinsic parameter between the third laser camera and the dome camera based on the third cost function and the initialized first extrinsic parameter between the first laser camera and the dome camera.

2. The calibration method for laser cameras and dome cameras as described in claim 1, characterized in that, The calibration objects include a first structural component and a second structural component; The image to be processed includes a first RGB image, a second RGB image, a first intensity image, and a first depth image; Obtain the image to be processed corresponding to the calibration object, including: Obtain the first RGB image corresponding to the first structural component; Wherein, the first RGB image is an image of the first structural component obtained by the dome camera at a preset distance from the first structural component; Obtain the image to be processed corresponding to the calibration object, including: Obtain the second RGB image, the first intensity image, and the first depth image corresponding to the second structural component; The second RGB image, the first intensity image, and the first depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos of the second structural component from a preset distance from the dome camera.

3. The calibration method for laser cameras and dome cameras as described in claim 2, characterized in that, Extracting feature points from the image to be processed includes: Extract the first 2D corner points of the first RGB image; Based on the first 2D corner point, the corresponding 2D-2D matching point of the dome camera is determined; Extracting feature points from the image to be processed includes: Extract the second 2D corner points of the second RGB image and the third 2D corner points of the first intensity image, wherein there is a one-to-one correspondence between the second 2D corner points and the third 2D corner points; Based on the pixel correspondence between the first intensity image and the first depth image, the first position point of the third 2D corner point on the first depth image is found, and the third 2D corner point is converted into a first 3D corner point. Based on the second 2D corner point of the second RGB image and the first 2D corner point of the first intensity image, the first 2D-3D matching points of the dome camera with the first laser camera, the second laser camera and the third laser camera are determined respectively.

4. The calibration method for laser cameras and dome cameras as described in claim 2, characterized in that, The image to be processed includes a third RGB image, a third intensity image, and a third depth image; Obtain the image to be processed corresponding to the calibration object, including: Obtain the third RGB image, third intensity image, and third depth image corresponding to the second structural component; The third RGB image, the third intensity image, and the third depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos together with the dome camera during rotation at a preset position away from the second structural component. Based on the third cost function, and after optimizing the fourth and fifth extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera, according to the initialized first extrinsic parameters between the first laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera, the following further steps are taken: Extract 2D line features from the third RGB image; Extract 3D line features corresponding to the third depth image; Initialize the first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera; Based on the 2D line features and the 3D line features, the second 2D-3D matching points between the dome camera and the first laser camera, the second laser camera, and the third laser camera are determined respectively. The fourth cost function is determined based on the second 2D-3D matching point; Based on the fourth cost function, and on the initial first intrinsic parameter of the dome camera, the fourth extrinsic parameter between the first laser camera and the dome camera, the second extrinsic parameter between the second laser camera and the dome camera, and the fifth extrinsic parameter between the third laser camera and the dome camera, the second intrinsic parameter of the dome camera, the sixth extrinsic parameter between the first laser camera and the dome camera, the seventh extrinsic parameter between the second laser camera and the dome camera, and the eighth extrinsic parameter between the third laser camera and the dome camera are calibrated.

5. A calibration device for a laser camera and a dome camera, characterized in that, include: The acquisition module is used to acquire the image to be processed corresponding to the calibration object; An extraction module is used to extract feature points from the image to be processed; the feature points include 2D-2D matching points and a first 2D-3D matching point; the laser camera includes a first laser camera, a second laser camera, and a third laser camera with fixed relative positions; A calibration module is used to calibrate the intrinsic parameters of the dome camera and the extrinsic parameters between the laser camera and the dome camera based on the feature points. This includes: determining a first cost function based on 2D-2D matching points; determining a second cost function based on the first 2D-3D matching points; and combining the first and second cost functions to optimize the first intrinsic parameters of the dome camera, the first extrinsic parameters between the first laser camera and the dome camera, the second extrinsic parameters between the second laser camera and the dome camera, and the third extrinsic parameters between the third laser camera and the dome camera; wherein the relative positions between the laser camera and the dome camera are fixed. The image to be processed includes a second intensity image and a second depth image. Obtaining the image to be processed corresponding to the calibration object includes: obtaining the second intensity image and the second depth image corresponding to the calibration object, wherein the second intensity image and the second depth image are images obtained by co-viewing the first laser camera and the second laser camera, or the second laser camera and the third camera; feature points include 3D-3D matching points; extracting feature points from the image to be processed includes: extracting the fourth 2D corner point of the second intensity image; finding the second position point of the fourth 2D corner point on the second depth image based on the pixel correspondence between the second intensity image and the second depth image, and converting the fourth 2D corner point into a second 3D corner point; determining the location of the first laser camera and the second laser camera based on the fourth 2D corner point and the second 3D corner point of the second intensity image. The process includes: establishing 3D-3D matching points between laser cameras and between the second and third laser cameras; calibrating the intrinsic parameters of the dome camera and the extrinsic parameters between the laser cameras and the dome camera based on feature points; setting the first intrinsic parameter of the dome camera and the second extrinsic parameter between the second laser camera and the dome camera to fixed values; initializing the first extrinsic parameter between the first laser camera and the dome camera and the third extrinsic parameter between the third laser camera and the dome camera; determining the third cost function based on the 3D-3D matching points; and optimizing the fourth extrinsic parameter between the first laser camera and the dome camera and the fifth extrinsic parameter between the third laser camera and the dome camera based on the third cost function and the initialized first extrinsic parameter between the first laser camera and the dome camera.

6. The calibration device for laser cameras and dome cameras as described in claim 5, characterized in that, The calibration objects include a first structural component and a second structural component; The image to be processed includes a first RGB image, a second RGB image, a first intensity image, and a first depth image; The acquisition module includes: The first acquisition module is used to acquire the first RGB image corresponding to the first structural component; Wherein, the first RGB image is an image of the first structural component obtained by the dome camera at a preset distance from the first structural component; The acquisition module includes: Obtain the second RGB image, the first intensity image, and the first depth image corresponding to the second structural component; The second RGB image, the first intensity image, and the first depth image are images of the second structural component obtained by the first laser camera, the second laser camera, and the third laser camera taking photos of the second structural component from a preset distance from the dome camera.

7. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 4.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images