A method, device, electronic device, medium and system for calibrating external parameters of a camera

Abstract

Embodiments of the present disclosure disclose a method, apparatus, electronic device, medium, and system for calibrating external parameters of a camera. The method includes: determining reference position information according to reference camera data transmitted by a reference depth camera; determining target position information according to target camera data transmitted by the depth camera to be calibrated; selecting, from each of the reference position information and each of the target position information, the reference position information and the target position information corresponding to the same spatial point to form a position pair; and determining the external parameters of the depth camera to be calibrated according to each of the selected position pairs. By using the above method, through the provided double-sided calibration object, the position pairs formed by the corresponding points in space between the reference depth camera and the depth camera to be calibrated are determined, and the calibration of the external parameters of the depth camera to be calibrated is realized.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of camera calibration, and in particular, to a method, device, electronic device, medium, and system for calibrating external parameters of a camera. Background Art

[0002] Traditional camera calibration methods generally use a single-sided checkerboard or other patterns with similar principles as calibration plates. After extracting 2D feature points by collecting multiple images of the calibration plate, the internal and external parameters of the camera are estimated, and then re-projected, the error is calculated, and iterative optimization is performed.

[0003] However, with the further maturity of AR and VR technologies, 3D volumetric videos have emerged. The acquisition of 3D volumetric videos often uses multiple depth cameras with overlapping viewpoints, such as RGBD cameras. Multiple depth cameras are arranged in a circle and shoot inward. There is often no situation where a calibration plate can be placed at a position that can be captured by all depth cameras. If the traditional camera calibration method is used for external parameter calibration, there will be technical problems such as difficult camera calibration and difficult registration tasks, especially it is very difficult to calibrate the external parameters between two cameras with a large difference in directions.

[0004] Therefore, how to calibrate the external parameters of two depth cameras with a large difference in directions is an urgent problem to be solved currently. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method, device, electronic device, medium, and system for calibrating external parameters of a camera to calibrate the external parameters of a depth camera.

[0006] In a first aspect, embodiments of the present disclosure provide a method for calibrating external parameters of a camera, including:

[0007] Determine reference position information according to reference camera data transmitted by a reference depth camera, where the reference camera data is data output after the reference depth camera captures a calibration object in an acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located;

[0008] Determine target position information according to target camera data transmitted by a depth camera to be calibrated, where the target camera data is data output after the depth camera to be calibrated captures a calibration object in an acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located;

[0009] Select reference position information and target position information corresponding to the same spatial point from each of the reference position information and each of the target position information to form a position pair;

[0010] Determine the external parameters of the depth camera to be calibrated according to each selected position pair;

[0011] Among them, the calibration object captured by the reference depth camera is the same as the calibration object captured by the depth camera to be calibrated. The calibration object is a double-sided calibration object, and different identifiers are corresponding to each side of the double-sided calibration object to distinguish each side of the double-sided calibration object.

[0012] In a second aspect, an embodiment of the present disclosure further provides a device for calibrating the external parameters of a camera, including:

[0013] A first determination module, configured to determine reference position information according to the reference camera data transmitted by the reference depth camera. The reference camera data is the data output after the reference depth camera captures the calibration object in the acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located;

[0014] A second determination module, configured to determine target position information according to the target camera data transmitted by the depth camera to be calibrated. The target camera data is the data output after the depth camera to be calibrated captures the calibration object in the acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located;

[0015] A selection module, configured to select the reference position information and the target position information corresponding to the same spatial point from each of the reference position information and each of the target position information to form a position pair;

[0016] A calibration module, configured to determine the external parameters of the depth camera to be calibrated according to each selected position pair;

[0017] Among them, the calibration object captured by the reference depth camera is the same as the calibration object captured by the depth camera to be calibrated. The calibration object is a double-sided calibration object, and different identifiers are corresponding to each side of the double-sided calibration object to distinguish each side of the double-sided calibration object.

[0018] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:

[0019] One or more processing devices;

[0020] A storage device, configured to store one or more programs;

[0021] The one or more programs are executed by the one or more processing devices, so that the one or more processing devices implement the camera external parameter calibration method provided in any embodiment of the present disclosure.

[0022] Fourthly, an embodiment of the present disclosure further provides a computer-readable medium, on which a computer program is stored. When the program is executed by a processing device, it implements the camera extrinsic parameter calibration method provided by any embodiment of the present disclosure.

[0023] Fifthly, an embodiment of the present disclosure further provides a camera extrinsic parameter calibration system, including: an electronic device, a reference depth camera, at least one depth camera to be calibrated, and at least one calibration object;

[0024] The reference depth camera is configured to capture the calibration object within the acquisition area of the reference depth camera, obtain reference camera data, and transmit the reference camera data to the electronic device;

[0025] The depth camera to be calibrated is configured to capture the calibration object within the acquisition area of the depth camera to be calibrated, obtain target camera data, and transmit the target camera data to the electronic device;

[0026] The electronic device is configured to execute the method described in any one of the first aspect;

[0027] The calibration object is a double-sided calibration object, and each face of the double-sided calibration object corresponds to a different identifier for distinguishing each face of the double-sided calibration object.

[0028] An embodiment of the present disclosure provides a camera extrinsic parameter calibration method, apparatus, electronic device, medium, and system. The method determines reference position information according to the reference camera data transmitted by the reference depth camera. The reference camera data is the data output after the reference depth camera captures the calibration object within the acquisition area. The reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located; determines target position information according to the target camera data transmitted by the depth camera to be calibrated. The target camera data is the data output after the depth camera to be calibrated captures the calibration object within the acquisition area. The target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located; selects the reference position information and the target position information corresponding to the same spatial point from each of the reference position information and each of the target position information to form a position pair; determines the extrinsic parameters of the depth camera to be calibrated according to each of the selected position pairs; wherein, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, and each face of the double-sided calibration object corresponds to a different identifier for distinguishing each face of the double-sided calibration object. Using the above technical solution, by setting the double-sided calibration object, the position pairs formed by the corresponding points in space between the reference depth camera and the depth camera to be calibrated are determined, and the calibration of the extrinsic parameters of the depth camera to be calibrated is realized. Description of the Drawings

[0029] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the original elements and elements are not necessarily drawn to scale.

[0030] Figure 1 It is a schematic flowchart of a method for calibrating the external parameters of a camera provided in the first embodiment of the present disclosure;

[0031] Figure 1a It is a schematic diagram of a double-sided calibration object provided in the first embodiment of the present disclosure;

[0032] Figure 1b It is another schematic diagram of a double-sided calibration object provided in the first embodiment of the present disclosure;

[0033] Figure 1c It is a schematic diagram of the sorting of a double-sided calibration object provided in the first embodiment of the present disclosure;

[0034] Figure 2 It is a schematic flowchart of a method for calibrating the external parameters of a camera provided in the second embodiment of the present disclosure;

[0035] Figure 3 It is a schematic structural diagram of a device for calibrating the external parameters of a camera provided in the third embodiment of the present disclosure;

[0036] Figure 4 It is a schematic structural diagram of an electronic device provided in the fourth embodiment of the present disclosure;

[0037] Figure 5 It is a schematic structural diagram of a system for calibrating the external parameters of a camera provided in the fifth embodiment of the present disclosure. Specific Embodiments

[0038] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0039] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0040] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0041] It should be noted that the concepts such as "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0042] It should be noted that the modifications of "one" and "a plurality of" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly stated otherwise in the context, it should be understood as "one or more".

[0043] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0044] In the following embodiments, optional features and examples are provided in each embodiment. The various features recorded in the embodiments can be combined to form multiple alternative solutions. Each numbered embodiment should not be regarded as only one technical solution. In addition, without conflict, the embodiments in this disclosure and the features in the embodiments can be combined with each other.

[0045] Embodiment 1

[0046] Figure 1 FIG. is a schematic flowchart of a method for calibrating the external parameters of a camera provided in Embodiment 1 of this disclosure. This method is applicable to the situation of calibrating the external parameters of a camera. This method can be executed by a camera external parameter calibration device, where the device can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes but is not limited to devices such as computers and mobile phones.

[0047] The embodiments of this disclosure provide a method for calibrating the external parameters of a camera, which enables two depth cameras with a large difference in direction to also perform external parameter calibration.

[0048] It can be understood that multiple depth cameras are usually used for video acquisition. The multiple depth cameras are arranged in a circle and shoot inward. Generally, one depth camera needs to be selected as the reference depth camera, and the remaining cameras are used as the depth cameras to be calibrated. The coordinate systems of all the depth cameras to be calibrated need to be unified to the coordinate system where the reference depth camera is located. When performing camera calibration, first, the external camera parameters between the reference depth camera and a selected depth camera to be calibrated are determined. Then, the external camera parameters between the reference depth camera and the remaining depth cameras to be calibrated are determined using the same method to complete the calibration of the external parameters for all depth cameras.

[0049] The following S110 to S140 are the processes for calibrating the external camera parameters between the reference depth camera and a selected depth camera to be calibrated. As Figure 1 shown, a method for calibrating external camera parameters provided in Embodiment 1 of the present disclosure includes the following steps:

[0050] S110. Determine the reference position information according to the reference camera data transmitted by the reference depth camera.

[0051] When performing camera calibration, it can be considered that the calibration is carried out based on the reference depth camera. The reference camera data may refer to the data output after the reference depth camera shoots the calibration object in the acquisition area. The reference position information may be the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located.

[0052] Among them, the specific data included in the reference camera data is not limited. For example, it can be red-green-blue-depth (RGBD) data or image data. The acquisition area can be considered as the area that the depth camera can shoot. A calibration object is set in the acquisition area, and the calibration object is a double-sided calibration object.

[0053] Specifically, in this step, multiple corner points of the calibration object captured by the reference depth camera can be determined according to the reference camera data transmitted by the reference depth camera, and then the reference position information of the corner points in the coordinate system where the reference depth camera is located can be determined.

[0054] In this embodiment, the number of double-sided calibration objects is at least one, for example, it can be one or multiple. The placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera and the depth cameras to be calibrated. The double-sided calibration object is formed by one or more groups of single-sided calibration objects. Each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size. The two single-sided calibration objects are respectively fixed on opposite faces of the bracket. The thickness of the bracket is less than a set threshold, and the edges of the two single-sided calibration objects coincide when fixed on the bracket. The color of the double-sided calibration object is gray to ensure sufficient reflectivity to better obtain the reference camera data.

[0055] Figure 1a Schematic diagram of a double-sided calibration object provided in the first embodiment of the present disclosure, as Figure 1a shown, the double-sided calibration object is formed by a group of single-sided calibration objects, namely single-sided calibration object 1 and single-sided calibration object 2, and single-sided calibration object 1 and single-sided calibration object 2 are respectively fixed on opposite faces of the bracket 3.

[0056] Figure 1b Schematic diagram of another double-sided calibration object provided in the first embodiment of the present disclosure, as Figure 1b shown, the double-sided calibration object is formed by two groups of single-sided calibration objects. One group of single-sided calibration objects is single-sided calibration object 4 and single-sided calibration object 5, and the other group of single-sided calibration objects is single-sided calibration object 6 and single-sided calibration object 7. The two groups of single-sided calibration objects are respectively fixed on opposite faces of the bracket 8.

[0057] It should be noted that when calibrating the camera, one or more double-sided calibration objects can be placed in the acquisition areas of the reference depth camera and the depth camera to be calibrated, and the double-sided calibration object can also be moved to obtain sufficient reference camera data.

[0058] Among them, each face of the double-sided calibration object corresponds to a different identifier for distinguishing each face of the double-sided calibration object. For example, the identifiers of each face of the double-sided calibration object can be: the faces of the double-sided calibration object are encoded based on numbers. Specifically, one face of the double-sided calibration object is encoded with odd numbers, and the other face of the double-sided calibration object is encoded with even numbers; the identifiers of each face of the double-sided calibration object can also be: different shapes are set on each face of the double-sided calibration object.

[0059] Specifically, when each face calibration object of the double-sided calibration object is obtained by encoding numbers, the parity of the encoded digital numbers corresponding to each face is different. For example, when each face of the double-sided calibration object is formed by a group of single-sided calibration objects, one face of the double-sided calibration object can be encoded with 1, and the other face of the double-sided calibration object can be encoded with 2; when each face of the double-sided calibration object is formed by four groups of single-sided calibration objects, one face of the double-sided calibration object can be encoded with 1, 3, 5, 7, and the other face of the double-sided calibration object can be encoded with 2, 4, 6, 8.

[0060] Exemplarily, the manufacturing process of the double-sided calibration object can be: select two different ARUCO diagrams of the same size, one of which has an odd ID and the other has an even ID; print the two diagrams on paper of the same size, and the color is neutral gray; stick the two pieces of paper back to back so that the four corners coincide precisely; fix the calibration paper on the bracket.

[0061] S120. Determine the target position information according to the target camera data transmitted by the depth camera to be calibrated.

[0062] The depth camera to be calibrated may refer to the depth camera to be calibrated. For example, when collecting volumetric videos with multiple depth cameras, any depth camera can be selected as the reference depth camera, and the remaining depth cameras can be used as the depth cameras to be calibrated.

[0063] The target camera data may refer to the data output after the depth camera to be calibrated captures the calibration object within the acquisition area, and the target position information may be the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located.

[0064] Among them, the specific data included in the target camera data is not limited. For example, it can be red-green-blue-depth (RGBD) data, or it can also be image data; the calibration object captured by the depth camera to be calibrated within the acquisition area is the same as the calibration object captured by the reference depth camera, and it can also be considered as a double-sided calibration object, which can be the same side of the calibration object or different sides of the calibration object.

[0065] The placement positions of the double-sided calibration object evenly cover the acquisition areas of the reference depth camera and the depth camera to be calibrated. The placement positions of the double-sided calibration object can be determined according to the positional relationship between the reference depth camera and the depth camera to be calibrated. For example, it can be determined according to the angular position between the reference depth camera and the depth camera to be calibrated, or it can also be determined according to the distance between the reference depth camera and the depth camera to be calibrated.

[0066] Specifically, when the placement positions of the double-sided calibration object are determined according to the angular position between the reference depth camera and the depth camera to be calibrated, when the angular direction between the reference depth camera and the depth camera to be calibrated is less than the set value, the same side of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; when the angular direction between the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different sides of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

[0067] Among them, the set value can be pre-configured by relevant configuration personnel or automatically set by the system, which is not limited here. Exemplarily, in this embodiment, the set value can be selected as 90 degrees. Specifically: if the angular direction between the depth camera to be calibrated and the reference depth camera is less than 90 degrees, then let the same side of the calibration object face these two cameras; if the angular direction between the depth camera to be calibrated and the reference depth camera is greater than or equal to 90 degrees, then let the two sides of the calibration object face one camera each.

[0068] As can be seen from the above steps, in this embodiment, the placement position of the double-sided calibration object is determined according to the included angle between the reference depth camera and the depth camera to be calibrated, that is, by better placing the double-sided calibration object, the external parameter calibration can be performed between two cameras with a large subsequent direction difference. Among them, the included angle can be considered as the included angle between the optical axis direction of the reference depth camera and the optical axis direction of the depth camera to be calibrated.

[0069] This embodiment can determine multiple corner points of the calibration object captured by the depth camera to be calibrated according to the target camera data transmitted by the depth camera to be calibrated, and then determine the target position information of each of the corner points in the coordinate system where the depth camera to be calibrated is located.

[0070] S130. Select the reference position information and the target position information corresponding to the same spatial point from each of the reference position information and each of the target position information to form a position pair.

[0071] Among them, the position pair can be considered as a 3D corresponding point, which can be formed by the reference position information and the target position information under the same spatial point.

[0072] According to the above steps, each of the reference position information and each of the target position information can be obtained, and then the reference position information and the target position information under the same spatial point are selected to form each position pair.

[0073] Among them, for example, when selecting the position pair, after sorting each of the reference position information and each of the target position information, a reference position information and a target position information are read in sequence to form a position pair; for example, when selecting the position pair, after sorting each of the reference position information and each of the target position information, a reference position information and a target position information are selected at a certain interval to form a position pair. This embodiment does not limit this.

[0074] S140. Determine the external parameters of the depth camera to be calibrated according to each of the selected position pairs.

[0075] As can be seen from the above steps, there is a one-to-one correspondence between the reference position information and the target position information between the reference depth camera and the depth camera to be calibrated, that is, each position pair is formed. In this step, the external parameters of the depth camera to be calibrated can be determined according to each of the selected position pairs.

[0076] Exemplarily, according to each of the selected position pairs, the RT matrix between the reference depth camera and the depth camera to be calibrated can be calculated by the least squares method.

[0077] A method for calibrating the extrinsic parameters of a camera provided in Embodiment 1 of the present disclosure includes: First, determine reference position information according to the reference camera data transmitted by a reference depth camera, where the reference camera data is the data output after the reference depth camera captures a calibration object in the acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located; Second, determine target position information according to the target camera data transmitted by the depth camera to be calibrated, where the target camera data is the data output after the depth camera to be calibrated captures a calibration object in the acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located; Then, select the reference position information and the target position information corresponding to the same spatial point from each of the reference position information and each of the target position information to form a position pair; Finally, determine the extrinsic parameters of the depth camera to be calibrated according to each of the selected position pairs. Among them, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, and different identifiers are corresponding to each face of the double-sided calibration object to be used to distinguish each face of the double-sided calibration object. Using the above method, by setting the double-sided calibration object, the position pairs formed by the corresponding points in space of the reference depth camera and the depth camera to be calibrated are determined, and the extrinsic parameters of multiple depth cameras to be calibrated are calibrated.

[0078] On the basis of the above embodiment, a variant embodiment of the above embodiment is proposed. Here, it should be noted that for the sake of brevity of description, only the differences from the above embodiment are described in the variant embodiment.

[0079] In one embodiment, the determining the reference position information according to the reference camera data transmitted by the reference depth camera includes:

[0080] Determine the reference identification information and the reference position information of the calibration object captured by the reference depth camera according to the reference camera data transmitted by the reference depth camera, and the reference arrangement order of the reference position information is determined according to the reference identification information.

[0081] Among them, the reference identification information may refer to the identification information of the captured calibration object, which can be obtained by identifying the reference camera data transmitted by the reference depth camera; the reference position information may be the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located.

[0082] As an implementable method, the steps for determining the reference position information may be as follows: when each surface of the double-sided calibration object is formed by a group of single-sided calibration objects, each single-sided calibration object has a reference identification information. According to the reference camera data transmitted by the reference depth camera, the reference identification information of the calibration object captured by the reference depth camera can be determined, and then the multiple corner points of the calibration object captured by the reference depth camera are sorted according to the identified reference identification information. Among them, the sorting steps may be, for example, when the code of the calibration object is odd, the corner points are sorted in the counterclockwise direction; when the code of the calibration object is even, the corner points are sorted in the clockwise direction.

[0083] Exemplarily, according to the obtained RGBD data, perform ARUCO code detection on the RGB part to find 4 corner points. Combine the depth values and perform back-projection to obtain 3D points in the camera coordinate system. According to the detected Aruco ID, rearrange the order of the corner points: if the code number of one side of the double-sided calibration object is odd, the order of the 4 corner points is top left - top right - bottom right - bottom left; if the code number of one side of the double-sided calibration object is even, the order of the 4 corner points is top right - top left - bottom left - bottom right.

[0084] As an implementable method, the steps for determining the reference position information may also be as follows: when each surface of the double-sided calibration object is formed by multiple groups of single-sided calibration objects, each single-sided calibration object has a reference identification information. According to the reference camera data transmitted by the reference depth camera, the reference identification information of the calibration object captured by the reference depth camera can be determined. The captured calibration object may be at least one. If there are multiple calibration objects, the reference identification information may be sorted first, and then the multiple corner points of each calibration object are sorted.

[0085] Among them, the sorting steps may be, for example, first sort the multiple groups of single-sided calibration objects. The code numbers of one side of the double-sided calibration object are encoded as 1, 3, 5, 7 in the order of top left - top right - bottom right - bottom left, and the code numbers of the other side of the double-sided calibration object are encoded as 2, 4, 6, 8 in the order of top left - top right - bottom right - bottom left. Then sort the multiple corner points of the multiple groups of single-sided calibration objects. For example, when the code number is 3, the order of the 4 corner points is top left - top right - bottom right - bottom left; when the code number is 2, the order of the 4 corner points is top right - top left - bottom left - bottom right.

[0086] Figure 1c Schematic diagram of double-sided calibration object sorting provided by Embodiment 1 of the present disclosure, as Figure 1cAs shown, each side of the double-sided calibration object is formed by four groups of single-sided calibration objects, and the four groups of single-sided calibration objects are respectively fixed on the opposite sides of the bracket 9. Among them, 1 and 4 are a group of single-sided calibration objects, 3 and 2 are a group of single-sided calibration objects, 5 and 8 are a group of single-sided calibration objects, and 7 and 6 are a group of single-sided calibration objects; then, the multiple corner points of multiple groups of single-sided calibration objects are sorted. For example, the multiple corner points of the group of single-sided calibration objects with 3 and 2 are sorted, as Figure 1c As shown, the 4 corner points of the single-sided calibration object with the coding number 3 are ①②③④ in the order of upper left - upper right - lower right - lower left, and the 4 corner points of the single-sided calibration object with the coding number 2 are ①②③④ in the order of upper right - upper left - lower left - lower right.

[0087] In one embodiment, determining the target position information according to the target camera data transmitted by the depth camera to be calibrated includes:

[0088] According to the target camera data transmitted by the depth camera to be calibrated, determining the target identification information and target position information of the calibration object photographed by the depth camera to be calibrated, and the target arrangement order of the target position information is determined according to the target identification information;

[0089] Among them, the position information at the same queue position in the arranged reference position information and the arranged target position information corresponds to the same spatial point.

[0090] In this embodiment, the target identification information may refer to the identification information of the photographed calibration object, and the target identification information can be obtained by identifying the reference camera data transmitted by the reference depth camera; the target position information may be the position information of multiple corner points of the calibration object photographed by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located.

[0091] For the technical means of sorting the target position information in this embodiment, reference can be made to the technical means of sorting the reference position information, which will not be elaborated here.

[0092] Among them, the queue position can be considered as each arranged position. The position information at the same queue position in the arranged reference position information and the arranged target position information corresponds to the same spatial point. Exemplarily, as Figure 1c As shown, the position information at the second position after sorting the 4 corner points of the single-sided calibration object with the coding number 3 and the position information at the second position after sorting the 4 corner points of the single-sided calibration object with the coding number 2 correspond to the same spatial point.

[0093] Embodiment 2

[0094] Figure 2The flowchart of a method for calibrating the extrinsic parameters of a camera provided in Embodiment 2 of the present disclosure. Embodiment 2 is specified based on each optional solution in the above embodiments. In this embodiment, selecting the reference position information and the target position information corresponding to the same spatial point from each of the reference position information and each of the target position information is further specified as follows: sequentially obtaining a piece of reference position information from each of the reference position information, and obtaining a piece of target position information from each of the target position information to form a position pair until a corresponding position pair is formed for each piece of reference position information in the reference position information and each piece of target position information in the target position information.

[0095] For the content not detailed in this embodiment, please refer to the above embodiments.

[0096] As Figure 2 shown, a method for calibrating the extrinsic parameters of a camera provided in Embodiment 2 of the present disclosure specifically includes the following steps:

[0097] S210. Determine the reference position information according to the reference camera data transmitted by the reference depth camera.

[0098] S220. Determine the target position information according to the target camera data transmitted by the depth camera to be calibrated.

[0099] S230. Sequentially obtain a piece of reference position information from each of the reference position information, and obtain a piece of target position information from each of the target position information to form a position pair until a corresponding position pair is formed for each piece of reference position information in the reference position information and each piece of target position information in the target position information.

[0100] After obtaining the sorted reference position information and target position information according to the above steps, then sequentially obtain a piece of reference position information from each of the reference position information, and obtain a piece of target position information from each of the target position information to form a position pair until a corresponding position pair is formed for each piece of reference position information in the reference position information and each piece of target position information in the target position information. Among them, a piece of reference position information sequentially obtained from each of the reference position information and a piece of target position information obtained from each of the target position information are for the same spatial point.

[0101] Exemplarily, the sorted reference position information can be obtained according to step S210: the encoded number of the calibration object is 1, and the four corner points are 1, 2, 3, and 4 in the order of top left - top right - bottom right - bottom left; the sorted target position information can be obtained according to step S220: the encoded number of the calibration object is 2, and the four corner points are 1, 2, 3, and 4 in the order of top right - top left - bottom left - bottom right; sequentially obtain the reference position information 1 from each of the reference position information, obtain a target position information 1 from each of the target position information, form a position pair (1, 1), obtain the reference position information 2 from each of the reference position information, obtain a target position information 2 from each of the target position information, form a position pair (2, 2) …… until the reference position information in each of the reference position information and the target position information in each of the target position information form corresponding position pairs.

[0102] S240. Determine the external parameters of the depth camera to be calibrated according to the selected position pairs.

[0103] A camera external parameter calibration method provided in Embodiment 2 of the present disclosure, the method includes: first, determine reference position information according to the reference camera data transmitted by the reference depth camera, where the reference camera data is the data output after the reference depth camera captures the calibration object in the acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located; secondly, determine target position information according to the target camera data transmitted by the depth camera to be calibrated, where the target camera data is the data output after the depth camera to be calibrated captures the calibration object in the acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located; then sequentially obtain a reference position information from each of the reference position information, obtain a target position information from each of the target position information, form a position pair until the reference position information in each of the reference position information and the target position information in each of the target position information form corresponding position pairs; finally, determine the external parameters of the depth camera to be calibrated according to the selected position pairs, where the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, and different identifiers are corresponding to each side of the double-sided calibration object to be used to distinguish each side of the double-sided calibration object. Using the above method, through the set double-sided calibration object, sequentially obtain a reference position information and a target position information from each of the arranged reference position information and target position information to form a position pair, realize forming a position pair between the reference position information and the target position information of the same spatial point, and further realize calibrating the external parameters of multiple depth cameras to be calibrated.

[0104] Embodiment 3

[0105] Figure 3 FIG. is a schematic structural diagram of an external parameter calibration device for a camera provided in Embodiment 3 of the present disclosure. This device can be applied to the situation of calibrating the external parameters of a camera. The device can be implemented by software and / or hardware and is generally integrated on an electronic device. As Figure 3 shown, the device includes:

[0106] A first determination module 310, configured to determine reference position information according to reference camera data transmitted by a reference depth camera, where the reference camera data is data output after the reference depth camera captures a calibration object in an acquisition area, and the reference position information is position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located;

[0107] A second determination module 320, configured to determine target position information according to target camera data transmitted by the depth camera to be calibrated, where the target camera data is data output after the depth camera to be calibrated captures a calibration object in an acquisition area, and the target position information is position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located;

[0108] A selection module 330, configured to select reference position information and target position information corresponding to the same spatial point from the respective reference position information and the respective target position information to form a position pair;

[0109] A calibration module 340, configured to determine the external parameters of the depth camera to be calibrated according to the selected position pairs; wherein, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, and the calibration object is a double-sided calibration object, and different identifiers are corresponding to each face of the double-sided calibration object to be used to distinguish each face of the double-sided calibration object.

[0110] Embodiment 3 of the present disclosure provides an external parameter calibration device for a camera. The device determines reference position information through a first determination module 310 based on reference camera data transmitted by a reference depth camera. The reference camera data is data output after the reference depth camera captures a calibration object in the acquisition area. The reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located. The device determines target position information through a second determination module 320 based on target camera data transmitted by the depth camera to be calibrated. The target camera data is data output after the depth camera to be calibrated captures a calibration object in the acquisition area. The target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located. The device selects, through a selection module 330, reference position information and target position information corresponding to the same spatial point from each piece of the reference position information and each piece of the target position information to form a position pair. The device determines the external parameters of the depth camera to be calibrated through a calibration module 340 based on the selected position pairs. Among them, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same. The calibration object is a double-sided calibration object, and different identifiers are corresponding to each face of the double-sided calibration object to distinguish each face of the double-sided calibration object. The device determines the position pairs formed by the corresponding points in space between the reference depth camera and the depth camera to be calibrated through the provided double-sided calibration object, and realizes the calibration of the external parameters of multiple depth cameras to be calibrated.

[0111] Further, the first determination module 310 is specifically configured to:

[0112] Determine the reference identification information and reference position information of the calibration object captured by the reference depth camera based on the reference camera data transmitted by the reference depth camera. The reference arrangement order of the reference position information is determined according to the reference identification information.

[0113] Further, the second determination module 320 is specifically configured to:

[0114] Determine the target identification information and target position information of the calibration object captured by the depth camera to be calibrated based on the target camera data transmitted by the depth camera to be calibrated. The target arrangement order of the target position information is determined according to the target identification information;

[0115] Among them, the position information at the same queue position in the arranged reference position information and the arranged target position information corresponds to the same spatial point.

[0116] Further, the selection module 330 is specifically configured to:

[0117] Obtain a reference position information from each of the reference position information in sequence, and obtain a target position information from each of the target position information to form a position pair until the reference position information in each of the reference position information and the target position information in each of the target position information form corresponding position pairs.

[0118] Further, the number of the double-sided calibration objects is at least one, and the placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera and the depth camera to be calibrated. The double-sided calibration object is formed by one or more groups of single-sided calibration objects. Each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size, and the two single-sided calibration objects are respectively fixed on opposite surfaces of the bracket. The thickness of the bracket is less than a set threshold, and the edges of the two single-sided calibration objects coincide when fixed on the bracket.

[0119] Further, the color of the double-sided calibration object is gray.

[0120] Further, each surface calibration object of the double-sided calibration object is obtained after digital encoding, and the parities of the encoded numbers corresponding to each surface are different.

[0121] Further, when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is less than a set value, the same surface of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different surfaces of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

[0122] The above camera external parameter calibration device can execute the camera external parameter calibration method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0123] Embodiment IV

[0124] Figure 4 It is a schematic structural diagram of an electronic device provided in Embodiment IV of the present disclosure. Figure 4 A schematic structural diagram of an electronic device 400 suitable for implementing the embodiments of the present disclosure is shown. The electronic device 400 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.Figure 4 The illustrated electronic device 400 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0125] As Figure 4 shown, the electronic device 400 may include one or more processing devices (such as a central processing unit, a graphics processing unit, etc.) 401, which may perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) 402 or the programs loaded from the storage device 408 into the random access memory (RAM) 403. One or more processing devices 401 implement the methods provided by the present disclosure. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0126] Generally, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc., and the storage device 408 is used to store one or more programs; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 the electronic device 400 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be alternatively implemented or had.

[0127] Particularly, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

[0128] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0129] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as the Hyper Text Transfer Protocol (HTTP), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0130] The above computer-readable medium can be included in the above electronic device 400; or it can exist separately and not be assembled into the electronic device 400.

[0131] The above computer-readable medium stores one or more computer programs which, when executed by a processing device, implement the following method: determining reference position information based on reference camera data transmitted by a reference depth camera, where the reference camera data is data output after the reference depth camera captures a calibration object within an acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located;

[0132] determining target position information based on target camera data transmitted by the depth camera to be calibrated, where the target camera data is data output after the depth camera to be calibrated captures a calibration object within the acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located;

[0133] selecting, from the respective reference position information and the respective target position information, the reference position information and the target position information corresponding to the same spatial point to form position pairs;

[0134] determining the external parameters of the depth camera to be calibrated based on the selected position pairs;

[0135] wherein, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, and different identifiers are corresponding to each face of the double-sided calibration object for distinguishing each face of the double-sided calibration object.

[0136] The above computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device 400 to: be able to write computer program code for performing the operations of the present disclosure in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. Each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0138] The modules described in the embodiments of the present disclosure can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the module itself in some cases.

[0139] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programming Logic Device (CPLD), and so on.

[0140] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] According to one or more embodiments of the present disclosure, Example 1 provides a method for calibrating the extrinsic parameters of a camera, including:

[0142] Determining reference position information according to reference camera data transmitted by a reference depth camera, where the reference camera data is data output after the reference depth camera captures a calibration object within an acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located;

[0143] Determining target position information according to target camera data transmitted by a depth camera to be calibrated, where the target camera data is data output after the depth camera to be calibrated captures a calibration object within an acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located;

[0144] Selecting, from each of the reference position information and each of the target position information, the reference position information and the target position information corresponding to the same spatial point to form a position pair;

[0145] Determining the extrinsic parameters of the depth camera to be calibrated according to each of the selected position pairs;

[0146] Wherein, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, and different identifiers are corresponding to each face of the double-sided calibration object to be used to distinguish each face of the double-sided calibration object.

[0147] According to one or more embodiments of the present disclosure, Example 2 is based on the method described in Example 1,

[0148] The determining the reference position information according to the reference camera data transmitted by the reference depth camera includes:

[0149] Determine the reference identification information and reference position information of the calibration object captured by the reference depth camera according to the reference camera data transmitted by the reference depth camera, wherein the reference arrangement order of the reference position information is determined according to the reference identification information;

[0150] According to one or more embodiments of the present disclosure, Example 3 is based on the method described in Example 2,

[0151] Determining the target position information according to the target camera data transmitted by the depth camera to be calibrated includes:

[0152] Determine the target identification information and target position information of the calibration object captured by the depth camera to be calibrated according to the target camera data transmitted by the depth camera to be calibrated, wherein the target arrangement order of the target position information is determined according to the target identification information;

[0153] Wherein, the position information at the same queue position in the arranged reference position information and the arranged target position information corresponds to the same spatial point.

[0154] According to one or more embodiments of the present disclosure, Example 4 is based on the method described in Example 2,

[0155] Selecting the reference position information and the target position information corresponding to the same spatial point from the respective reference position information and the respective target position information to form a position pair includes:

[0156] Sequentially obtain a reference position information from each of the reference position information, and obtain a target position information from each of the target position information to form a position pair until the reference position information in each of the reference position information and the target position information in each of the target position information form corresponding position pairs.

[0157] According to one or more embodiments of the present disclosure, Example 5 is based on the method described in Example 1,

[0158] The number of the double-sided calibration objects is at least one, and the placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera and the depth camera to be calibrated. The double-sided calibration object is formed by one or more groups of single-sided calibration objects. Each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size. The two single-sided calibration objects are respectively fixed on opposite surfaces of the bracket. The thickness of the bracket is less than a set threshold value, and the edges of the two single-sided calibration objects coincide when fixed on the bracket.

[0159] According to one or more embodiments of the present disclosure, Example 6 is based on the method described in Example 4,

[0160] The color of the double-sided calibration object is gray.

[0161] According to one or more embodiments of the present disclosure, Example 7 is based on the method described in Example 4.

[0162] Each calibration object on the two sides of the double-sided calibration object is obtained by digitally encoding, and the parity of the encoded numbers corresponding to each side is different.

[0163] According to one or more embodiments of the present disclosure, Example 8 is based on the method described in Example 1.

[0164] When the included angle between the directions of the reference depth camera and the depth camera to be calibrated is less than a set value, the same side of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different sides of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

[0165] According to one or more embodiments of the present disclosure, Example 9 provides an external camera parameter calibration device, including:

[0166] A first determination module, configured to determine reference position information according to reference camera data transmitted by a reference depth camera, where the reference camera data is data output after the reference depth camera captures a calibration object in an acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located;

[0167] A second determination module, configured to determine target position information according to target camera data transmitted by the depth camera to be calibrated, where the target camera data is data output after the depth camera to be calibrated captures a calibration object in an acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located;

[0168] A selection module, configured to select reference position information and target position information corresponding to the same spatial point from the respective reference position information and the respective target position information to form a position pair;

[0169] A calibration module, configured to determine the external parameters of the depth camera to be calibrated according to the selected position pairs;

[0170] Wherein, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, and different identifiers are corresponding to each side of the double-sided calibration object to be used to distinguish each side of the double-sided calibration object.

[0171] According to one or more embodiments of the present disclosure, Example 10 provides an electronic device, including:

[0172] One or more processing devices;

[0173] A storage device for storing one or more programs;

[0174] When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the method according to any one of claims 1-8.

[0175] According to one or more embodiments of the present disclosure, Example 11 provides a computer-readable medium having a computer program stored thereon, characterized in that when the program is executed by a processing device, the method according to any one of claims 1-8 is implemented.

[0176] According to one or more embodiments of the present disclosure, Example 12 provides an external camera calibration system, including an electronic device, a reference depth camera, at least one depth camera to be calibrated, and at least one calibration object;

[0177] The reference depth camera is configured to photograph the calibration object within the acquisition area of the reference depth camera, obtain reference camera data, and transmit the reference camera data to the electronic device;

[0178] The depth camera to be calibrated is configured to photograph the calibration object within the acquisition area of the depth camera to be calibrated, obtain target camera data, and transmit the target camera data to the electronic device;

[0179] The electronic device is configured to execute the method according to any one of claims 1-8;

[0180] The calibration object is a double-sided calibration object, and each face of the double-sided calibration object corresponds to a different identifier for distinguishing each face of the double-sided calibration object.

[0181] According to one or more embodiments of the present disclosure, Example 13 is based on the system described in Example 12,

[0182] The placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera and the corresponding depth cameras to be calibrated.

[0183] According to one or more embodiments of the present disclosure, Example 14 is based on the system described in Example 12,

[0184] The double-sided calibration object is formed by one or more groups of single-sided calibration objects. Each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size. The two single-sided calibration objects are respectively fixed on opposite faces of the bracket. The thickness of the bracket is less than a set threshold value, and the edges of the two single-sided calibration objects coincide when fixed on the bracket.

[0185] According to one or more embodiments of the present disclosure, Example 15 The system according to Example 14,

[0186] The color of the double-sided calibration object is gray.

[0187] According to one or more embodiments of the present disclosure, Example 16 The system according to Example 14,

[0188] Each calibration object on each side of the double-sided calibration object is obtained by digitally encoding, and the parity of the encoded numbers corresponding to each side is different.

[0189] According to one or more embodiments of the present disclosure, Example 17 The system according to Example 12,

[0190] For any depth camera to be calibrated, when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is less than a set value, the same side of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different sides of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

[0191] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0192] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0193] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms of implementing the claims.

[0194] Example 5

[0195] Figure 5 As shown in the structural schematic diagram of an external camera parameter calibration system provided in Example 5 of the present disclosure, Figure 5 as shown, the external camera parameter calibration system includes an electronic device 10, a reference depth camera 11, a depth camera to be calibrated 12, a depth camera to be calibrated 13, a calibration object 14, and a calibration object 15;

[0196] The reference depth camera 11 is configured to capture the calibration object 14 and the calibration object 15 within the acquisition area of the reference depth camera, obtain reference camera data, and transmit the reference camera data to the electronic device 10;

[0197] The depth cameras to be calibrated 12 and 13 are configured to capture the calibration object 14 and the calibration object 15 within the acquisition area of the depth cameras to be calibrated, obtain target camera data, and transmit the target camera data to the electronic device 10;

[0198] The electronic device 10 is configured to execute any of the methods described in Example 1;

[0199] The calibration object 14 and the calibration object 15 are double-sided calibration objects, and each side of the double-sided calibration object corresponds to a different identifier for distinguishing each side of the double-sided calibration object.

[0200] Specifically, the placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera 11 and the corresponding depth camera to be calibrated 12 or 13; the double-sided calibration object is formed by one or more groups of single-sided calibration objects, and each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size, and the two single-sided calibration objects are respectively fixed on opposite faces of the bracket. The thickness of the bracket is less than a set threshold, and the edges of the two single-sided calibration objects coincide when fixed on the bracket; the color of the double-sided calibration plate is gray; the calibration objects on each side of the double-sided calibration object are obtained by digitally encoding, and the parity of the encoded numbers corresponding to each side is different.

[0201] For any depth camera to be calibrated, when the included angle between the reference depth camera and the depth camera to be calibrated is less than a set value, the same side of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; when the included angle between the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different sides of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

[0202] Exemplarily, as Figure 5As shown, the placement positions of the calibration object 14 evenly cover the acquisition areas of the reference depth camera 11 and the depth camera to be calibrated 12, and the placement positions of the calibration object 15 evenly cover the acquisition areas of the reference depth camera 11 and the depth camera to be calibrated 13. For example, for the depth camera to be calibrated 12, when the included angle between the directions of the reference depth camera 11 and the depth camera to be calibrated 12 is less than 90 degrees, the same surface of the calibration object 14 faces the reference depth camera 11 and the depth camera to be calibrated 12; when the included angle between the directions of the reference depth camera 11 and the depth camera to be calibrated 12 is greater than or equal to 90 degrees, different surfaces of the calibration object 14 face the reference depth camera 11 and the depth camera to be calibrated 12 respectively.

Claims

1. A method for calibrating the external parameters of a camera, characterized in that Including: Determine reference position information according to reference camera data transmitted by a reference depth camera, where the reference camera data is data output after the reference depth camera captures a calibration object in an acquisition area, and the reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located; Determine target position information according to target camera data transmitted by a depth camera to be calibrated, where the target camera data is data output after the depth camera to be calibrated captures a calibration object in an acquisition area, and the target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located; Select, from each of the reference position information and each of the target position information, the reference position information and the target position information corresponding to the same spatial point to form a position pair; Determine the external parameters of the depth camera to be calibrated according to each of the selected position pairs; Wherein, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same, the calibration object is a double-sided calibration object, the double-sided calibration object is formed by one or more groups of single-sided calibration objects, each group of single-sided calibration objects includes two surfaces, and different surfaces of the double-sided calibration object correspond to different identifiers for distinguishing each surface of the double-sided calibration object; when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is less than a set value, the same surface of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; When the included angle between the directions of the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different surfaces of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

2. The method according to claim 1, characterized in that The determining the reference position information according to the reference camera data transmitted by the reference depth camera includes: Determine the reference identifier information and the reference position information of the calibration object captured by the reference depth camera according to the reference camera data transmitted by the reference depth camera, and the reference arrangement order of the reference position information is determined according to the reference identifier information.

3. The method according to claim 2, wherein The determining the target position information according to the target camera data transmitted by the depth camera to be calibrated includes: Determine the target identifier information and the target position information of the calibration object captured by the depth camera to be calibrated according to the target camera data transmitted by the depth camera to be calibrated, and the target arrangement order of the target position information is determined according to the target identifier information; Wherein, the position information at the same queue position in the arranged reference position information and the arranged target position information corresponds to the same spatial point.

4. The method according to claim 2, wherein The selecting, from each of the reference position information and each of the target position information, the reference position information and the target position information corresponding to the same spatial point to form a position pair includes: Sequentially obtain a piece of reference position information from each of the reference position information, and obtain a piece of target position information from each of the target position information to form a position pair until the reference position information in each of the reference position information and the target position information in each of the target position information have all formed corresponding position pairs.

5. The method according to claim 1, characterized in that The number of the double-sided calibration objects is at least one, and the placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera and the depth camera to be calibrated. Each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size. The two single-sided calibration objects are respectively fixed on opposite faces of the bracket. The thickness of the bracket is less than a set threshold value, and the edges of the two single-sided calibration objects coincide when fixed on the bracket.

6. The method according to claim 4, wherein The color of the double-sided calibration object is gray.

7. The method according to claim 4, wherein Each surface calibration object of the double-sided calibration object is obtained by encoding numbers. The parities of the encoded numbers corresponding to each surface are different.

8. An external camera parameter calibration device, characterized in that, It includes: A first determination module, configured to determine reference position information according to the reference camera data transmitted by the reference depth camera. The reference camera data is the data output after the reference depth camera captures the calibration object in the acquisition area. The reference position information is the position information of multiple corner points of the calibration object captured by the reference depth camera in the coordinate system where the reference depth camera is located. A second determination module, configured to determine target position information according to the target camera data transmitted by the depth camera to be calibrated. The target camera data is the data output after the depth camera to be calibrated captures the calibration object in the acquisition area. The target position information is the position information of multiple corner points of the calibration object captured by the depth camera to be calibrated in the coordinate system where the depth camera to be calibrated is located. A selection module, configured to select the reference position information and the target position information corresponding to the same spatial point from each of the reference position information and each of the target position information to form a position pair. A calibration module, configured to determine the external parameters of the depth camera to be calibrated according to each selected position pair. Among them, the calibration object captured by the reference depth camera and the calibration object captured by the depth camera to be calibrated are the same. The calibration object is a double-sided calibration object. The double-sided calibration object is formed by one or more groups of single-sided calibration objects. Each group of single-sided calibration objects includes two surfaces. Different identifiers correspond to each surface of the double-sided calibration object to be used to distinguish each surface of the double-sided calibration object. When the included angle between the directions of the reference depth camera and the depth camera to be calibrated is less than a set value, the same surface of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated. When the included angle between the directions of the reference depth camera and the depth camera to be calibrated is greater than or equal to the set value, different surfaces of the double-sided calibration object face the reference depth camera and the depth camera to be calibrated respectively.

9. An electronic device, characterized in that, It includes: One or more processing devices; A storage device, configured to store one or more programs; When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the method according to any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processing device, the method according to any one of claims 1-7 is implemented.

11. An external camera calibration system, characterized in that, It includes an electronic device, a reference depth camera, at least one depth camera to be calibrated, and at least one calibration object; The reference depth camera is used to photograph a calibration object within the acquisition area of the reference depth camera, obtain reference camera data, and transmit the reference camera data to the electronic device; The depth camera to be calibrated is used to photograph a calibration object within the acquisition area of the depth camera to be calibrated, obtain target camera data, and transmit the target camera data to the electronic device; The electronic device is used to execute the method described in any one of claims 1-7; The calibration object is a double-sided calibration object, and different identifiers are corresponding to each face of the double-sided calibration object to be used to distinguish each face of the double-sided calibration object.

12. The system according to claim 11, characterized in that, The placement positions of the double-sided calibration objects evenly cover the acquisition areas of the reference depth camera and the corresponding depth camera to be calibrated.

13. The system according to claim 11, wherein The double-sided calibration object is formed by one or more groups of single-sided calibration objects. Each group of single-sided calibration objects is formed by two single-sided calibration objects fixed on a bracket. The two single-sided calibration objects have the same size. The two single-sided calibration objects are respectively fixed on opposite faces of the bracket. The thickness of the bracket is less than a set threshold value. When the two single-sided calibration objects are fixed on the bracket, their respective edges coincide.

14. The system according to claim 13, wherein, The color of the double-sided calibration object is gray.

15. The system according to claim 13, wherein, Each face calibration object of the double-sided calibration object is obtained after digital encoding, and the parity of the encoded numbers corresponding to each face is different.

16. The system according to claim 11, wherein For any depth camera to be calibrated, when the included angle between the directions of the reference depth camera and the depth camera to be calibrated is less than a set value, the same face of the double-sided calibration object faces the reference depth camera and the depth camera to be calibrated; When the direction angles of the reference depth camera and the depth camera to be calibrated are greater than or equal to the set value, different faces of the double-sided calibration object respectively face the reference depth camera and the depth camera to be calibrated.

Images