Calibration method of scanning equipment and related equipment

Abstract

The invention discloses a calibration method of scanning equipment and related equipment. The method comprises the following steps: acquiring a calibration plate image of a calibration plate by using a camera; determining a first conversion relation between the calibration plate coordinate system and the camera coordinate system by using the coordinate of the first flag bit in the calibration plate coordinate system and the coordinate of the first flag bit in the camera coordinate system; determining coordinates of a second flag bit in the calibration plate in two camera coordinate systems of the same group of dual-target fixed cameras by using the first conversion relationship, and further determining a second conversion relationship between the two camera coordinate systems of the same group of dual-target fixed cameras based on the coordinates; and when the second camera and the third camera are the same group of double-target fixed cameras and a second conversion relation or a third conversion relation exists between the camera coordinate system of the third camera and the camera coordinate system of the first camera, determining a third conversion relation between the camera coordinate systems of the first camera and the second camera. Therefore, the method can accurately calibrate the conversion relation between the cameras.

Description

Technical Field

[0001] This application belongs to the technical field of non-contact measurement, and particularly relates to a calibration method for a scanning device and related devices. Background Art

[0002] When performing non-contact measurement on a target object, for a target object with regular dimensions and shape, a high-precision measurement result can be obtained by scanning with a scanning device.

[0003] However, when scanning a target object with irregular dimensions and shape, for example, when scanning a target object with a large curvature on its surface, since the current scanning device often scans through a single camera, it is often difficult to obtain an accurate measurement result.

[0004] If multiple cameras at different angles are simply assembled in the scanning device, since it is difficult to accurately calibrate the conversion relationship between the camera coordinate systems of each camera, it is impossible to accurately measure a target object with a large curvature on its surface through multiple cameras. Summary of the Invention

[0005] Embodiments of this application provide a calibration method for a scanning device and related devices, which can solve the problem of difficultly and accurately calibrating the conversion relationship between the camera coordinate systems of each camera when the scanning device includes multiple cameras at different angles.

[0006] In a first aspect, embodiments of this application provide a calibration method for a scanning device, the method including:

[0007] Using each camera to respectively collect calibration board images of a calibration board within the corresponding camera's field of view, where the calibration board images include a first marker and a second marker on the calibration board;

[0008] For each camera, using the coordinates of the first marker on the calibration board in the calibration board coordinate system and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system, to determine a first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system;

[0009] For each group of binocular calibration cameras, using the first conversion relationship, to determine the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras respectively, where the overlapping range of the calibration board images within the field of view of each group of binocular calibration cameras is greater than a preset range threshold;

[0010] Using the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras, to determine a second conversion relationship between the two camera coordinate systems of the same group of binocular calibration cameras;

[0011] Using the second conversion relationship between the two camera coordinates in the same-group binocular calibrated cameras, determine the third conversion relationship between the camera coordinate system of the preset first camera and the camera coordinate system of the second camera in the same-group binocular calibrated cameras, where the same-group binocular calibrated cameras include the second camera and the third camera, and there is a second conversion relationship or a third conversion relationship between the camera coordinate system of the third camera and the camera coordinate system of the first camera.

[0012] Among them, the calibration board coordinate system includes the calibration board coordinate systems when the calibration board is respectively in multiple different planes, and the multiple planes are parallel to each other.

[0013] Furthermore, the calibration board images include the calibration board images collected when the calibration board is in different planes; for each camera, use the coordinates of the first marker in the calibration board coordinate system and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system to determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system, including:

[0014] For each camera, use the coordinates of the first marker in each calibration board coordinate system and the coordinates of the first marker in the calibration board image corresponding to each calibration board coordinate system in the camera coordinate system to respectively determine the first conversion relationship between each calibration board coordinate system and the camera coordinate system;

[0015] Using the first conversion relationship, determine the coordinates of the second marker in the calibration board in the two camera coordinate systems of the same-group binocular calibrated cameras respectively, including:

[0016] Using the first conversion relationship between any calibration board coordinate system and each camera coordinate system, determine the coordinates of the second marker in the calibration board in the camera coordinate systems of the two binocular calibrated cameras when the calibration board is in the corresponding plane.

[0017] Among them, the calibration board images include the light projection images when the calibration board is in multiple planes, and the light projection images are the images when straight light rays are projected on the calibration board; in the case where the multiple planes include the first plane and the second plane, after using the coordinates of the first marker in the calibration board coordinate system and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system to determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system, the method further includes:

[0018] For each camera, use the first conversion relationship between the first calibration board coordinate system and each camera coordinate system, and the coordinates of the first straight light ray projected on the calibration board in the first calibration board coordinate system, to determine the coordinates of the first straight light ray in the first straight light image in the corresponding camera coordinate system;

[0019] Determine the coordinates of the first straight light ray in the first light projection image in the second calibration plate coordinate system by using the first conversion relationship between the second calibration plate coordinate system and each camera coordinate system, and the coordinates of the first straight light ray in the corresponding camera coordinate system within the first light projection image;

[0020] Determine the common plane where the coordinates of the first straight light ray in the second calibration plate coordinate system and the coordinates of the second straight light ray projected on the calibration plate in the second plane are located in the second calibration plate coordinate system;

[0021] Calibrate the angle between the common plane and the second plane into the corresponding camera, so that the corresponding camera can use the angle and any first conversion relationship to determine the coordinates of any position in any plane in the camera coordinate system of the corresponding camera.

[0022] Wherein, when the second camera is in a circular arrangement formed by a plurality of cameras in a predetermined circular order, it is the next camera adjacent to the third camera and not adjacent to the first camera.

[0023] Furthermore, use the second conversion relationship between the two camera coordinates in the same group of binocular calibration cameras to determine the third conversion relationship between the camera coordinate system of the preset first camera and the camera coordinate system of the second camera in the same group of binocular calibration cameras, including:

[0024] In each round of calculation of the third conversion relationship, use the second conversion relationship or the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round to determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round;

[0025] Take the second camera in the current round as the third camera in the next round of calculation, and perform the next round of calculation of the third conversion relationship until the third conversion relationship between the camera coordinate system of the first camera and the camera coordinate systems of each non-adjacent camera is determined.

[0026] Furthermore, use the second conversion relationship or the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round to determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round, including:

[0027] In the case where the third camera is adjacent to the first camera;

[0028] Determine the third transformation relationship between the two camera coordinate systems of the first camera and the second camera in the current round by using the second transformation relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second transformation relationship between the two camera coordinate systems of the second camera and the third camera in the current round.

[0029] Further, determine the third transformation relationship between the two camera coordinate systems of the first camera and the second camera in the current round by using the second or third transformation relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second transformation relationship between the two camera coordinate systems of the second camera and the third camera in the current round, including:

[0030] In the case where the third camera is not adjacent to the first camera;

[0031] Determine the third transformation relationship between the two camera coordinate systems of the first camera and the second camera in the current round by using the third transformation relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second transformation relationship between the two camera coordinate systems of the second camera and the third camera in the current round.

[0032] In a second aspect, an embodiment of the present application provides a calibration device for a scanning device, and the device includes:

[0033] An acquisition module, configured to use each camera to respectively acquire a calibration board image of the calibration board within the corresponding camera's field of view, where the calibration board image includes a first marker and a second marker on the calibration board;

[0034] A first transformation relationship determination module, configured to, for each camera, determine the first transformation relationship between the calibration board coordinate system and the corresponding camera coordinate system by using the coordinates of the first marker on the calibration board in the calibration board coordinate system and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system;

[0035] A coordinate determination module, configured to, for each group of binocular calibration cameras, determine the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras by using the first transformation relationship, where the overlapping range of the calibration board images within the fields of view of each group of binocular calibration cameras is greater than a preset range threshold;

[0036] A second transformation relationship determination module, configured to determine the second transformation relationship between the two camera coordinate systems of the same group of binocular calibration cameras by using the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras;

[0037] The third conversion relationship determination module is configured to determine the third conversion relationship between the camera coordinate system of the preset first camera and the camera coordinate system of the second camera in the same group of binocular calibrated cameras by using the second conversion relationship between the two camera coordinates in the same group of binocular calibrated cameras. Among them, the same group of binocular calibrated cameras includes a second camera and a third camera, and there is a second conversion relationship or a third conversion relationship between the camera coordinate system of the third camera and the camera coordinate system of the first camera.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, which includes:

[0039] a processor and a memory storing computer program instructions;

[0040] When the processor executes the computer program instructions, the calibration method of the scanning device as described in any one of the previous items is implemented.

[0041] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the calibration method of the scanning device as described in any one of the previous items is implemented.

[0042] In a fifth aspect, when the instructions in a computer program product are executed by the processor of an electronic device, the electronic device is caused to execute the calibration method of the scanning device as described in any one of the previous items.

[0043] In a sixth aspect, an embodiment of the present application provides a scanning device, where the scanning device includes an electronic device, and the electronic device is configured to execute the calibration method of the scanning device as described in any one of the previous items.

[0044] The calibration method and related equipment of the scanning device according to the embodiments of the present application set multiple cameras in the scanning device, so that when scanning the target part, the images of the target part can be collected from multiple angles by the multiple cameras respectively. Before scanning, each camera is calibrated by using a calibration board, so that a single target camera can accurately convert the images collected by each camera into the camera coordinate system of the target camera; when calibrating the camera, by first calibrating the second conversion relationship between the camera coordinate system of each camera and the calibration board coordinate system, when each camera scans the target part, an accurate image can be obtained based on the accurate second conversion relationship, reducing the distortion of the image; further, when calibrating the conversion relationship between cameras, by first calibrating the second conversion relationship of the camera coordinate system between each adjacent camera, since the camera field of view overlap range between two adjacent cameras is much larger than that between two non-adjacent cameras, more landmark positions can be selected in a larger overlap range to calculate the second conversion relationship between the camera coordinate systems of two adjacent cameras, and thus the obtained second conversion relationship is more accurate; when there is a camera to be solved in two adjacent cameras, the third conversion relationship between the target camera and the non-adjacent camera to be solved can be established through the second conversion relationship of the camera coordinate systems of these two adjacent cameras, realizing the calculation of the third conversion relationship between the target camera and the camera to be solved by transmitting a relatively accurate second conversion relationship, making the calculated third conversion relationship more accurate. Therefore, when scanning the target part, after the target camera converts the images of other non-adjacent cameras into the camera coordinate system of the target camera, a more accurate conversion result can be obtained, and thus the size of the target part can be accurately captured, avoiding inaccurate third conversion relationships obtained when calculating the third conversion relationship between two non-adjacent cameras using too few landmark positions in the overlap range due to the small camera field of view overlap range between two non-adjacent cameras and fewer selectable landmark positions in the overlap range, which further causes the target camera to be unable to accurately convert the images of other non-adjacent cameras into its own camera coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0046] Figure 1 is a three-dimensional view of a scanning device provided by an embodiment of the present application;

[0047] Figure 2 is a first side view of a scanning device provided by an embodiment of the present application;

[0048] Figure 3It is a second side view of a scanning device provided by an embodiment of the present application;

[0049] Figure 4 It is a schematic flowchart of a calibration method for a scanning device provided by an embodiment of the present application;

[0050] Figure 5 It is a schematic structural diagram of a calibration device for a scanning device provided by an embodiment of the present application;

[0051] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0052] The features and exemplary embodiments of various aspects of the present application will be described in detail below. To make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0053] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, elements defined by the statement "including..." do not exclude the presence of additional identical elements in the process, method, article or device including the elements.

[0054] As described in the background art section, the related calibration technology of scanning devices is still difficult to meet the needs in actual work.

[0055] In the process of implementing the present application, it is found that in the process of non-contact measurement of a target part, when scanning a target part with irregular dimensions and shapes, for example, when scanning a target part with a large curvature surface shape, since the current scanning device often scans through a single camera, it is often difficult to obtain accurate measurement results.

[0056] If multiple cameras at different angles are simply assembled in a scanning device, in the calibration technology of the related scanning device, since it is difficult to accurately calibrate the conversion relationship between the camera coordinate systems of each camera, it is impossible to accurately measure a target part with a large curvature surface shape through multiple cameras.

[0057] To solve the problems of the prior art, an embodiment of the present application provides a calibration method for a scanning device and related devices.

[0058] In an embodiment of the present application, the scanning device includes a fixed bracket and multiple optical components.

[0059] Among them, the fixed bracket can rotate within a plane, and the rotation trajectory of the fixed bracket is parallel to the plane where the target part is located.

[0060] Multiple optical components are arranged on the fixed bracket, and each optical component is arranged on the fixed bracket at a fixed connection angle for collecting a target part image of the target part to be tested from a fixed angle.

[0061] Each optical component includes a camera and a laser. The laser is used to project a laser plane onto the target part to be measured at a fixed projection angle, and the laser plane forms a laser line when projected on the target part. The camera is used to collect the target part image of the target part with the laser line at a fixed collection angle, and each camera has its own camera coordinate system.

[0062] When collecting the target image, since the camera field of view of each camera is limited and the collection is at a fixed collection angle, only a part of the target part image can be included in the collected target part image.

[0063] In this embodiment, the cameras on the fixed bracket are arranged in a predetermined order.

[0064] For example, the cameras can be arranged in a circular arrangement in a preset circular order; or in a linear arrangement in a preset linear order.

[0065] Figure 1 A specific example of a scanning device is shown.

[0066] As Figure 1 shown, the scanning includes 4 optical components, namely optical component 101, optical component 102, optical component 103, and optical component 104, and includes a fixed bracket 105.

[0067] Among them, the optical component can specifically be, for example, a line structured light component, or other types of optical components.

[0068] In Figure 1In the example, the optical components 101, 102, 103, and 104 are respectively connected to the four corners of the fixed bracket 105 at fixed connection angles, and respectively acquire target images of the target part 106.

[0069] Among them, a laser and a camera are arranged in each optical component.

[0070] Such as Figure 1 As shown, based on the lasers and cameras respectively arranged in each optical component, each optical component projects its own laser plane onto the target part 106 with a large curvature surface shape, and acquires the target image of the target part 106 from the corresponding acquisition angle through its own camera.

[0071] Figure 2 Fig. shows a side view of the scanning device, which includes two adjacent optical components 101 and 102.

[0072] Taking Figure 2 the optical components 101 and 102 shown in as an example, a laser 1011 and a camera 1012 are arranged in the optical component 101, and a laser 1021 and a camera 1022 are arranged in the optical component 102.

[0073] Such as Figure 3 As shown, the optical components 101 and 102 are respectively connected to the fixed bracket 105 at fixed connection angles. Among them, the laser 1011 of the optical component 101 and the laser 1021 of the optical component 102 project their own laser planes onto the target part 106 at fixed projection angles respectively, and the camera 1012 of the optical component 101 and the camera 1022 of the optical component 102 acquire the target images of the target part 106 at fixed acquisition angles respectively.

[0074] Figure 3 Fig. shows another side view of the scanning device, which includes two adjacent optical components 102 and 103.

[0075] Such as Figure 3 As shown, the optical components 102 and 103 are respectively connected to the fixed bracket 105 at fixed connection angles, and acquire the target images of the target part 106 at fixed acquisition angles.

[0076] Based on the scanning device in this embodiment, before using the scanning device to scan the target object, for each camera in the scanning device, the conversion relationship between the camera coordinate system and the world coordinate system can be calibrated, and the conversion relationship between the camera coordinate systems of each camera in the scanning device can be calibrated. When using the calibrated cameras for scanning, each camera can collect accurate and distortion-free images of the target object, and the images of the target object collected by each camera can be accurately and distortion-free converted into the camera coordinate system of another camera.

[0077] Next, based on the foregoing scanning device and calibration board, and in combination with the accompanying drawings, the calibration method of the scanning device provided by the embodiments of the present application will be described in detail.

[0078] Figure 4 FIG. shows a schematic flowchart of a calibration method of a scanning device provided by an embodiment of the present application.

[0079] Refer to Figure 4 , the calibration method of the scanning device according to an embodiment of the present application includes the following steps: 401-405:

[0080] S401. Use each camera to respectively collect a calibration board image of the calibration board within the corresponding camera's field of view. The calibration board image includes a first marker and a second marker on the calibration board.

[0081] In this embodiment, during the calibration of each camera, the calibration board images of the calibration board can be collected by each camera in the scanning device first.

[0082] When calibrating each camera, a preset calibration board can be used to replace the target object, placed at the position of the target object, and the calibration board images of the calibration board are collected by each camera, so as to calibrate each camera using the calibration board images.

[0083] Among them, the calibration board can be, for example, a checkerboard pattern with coordinate scales. In any position of the calibration board, the calibration board represents a calibration board coordinate system corresponding to that position and represents a world coordinate system. When the calibration board is in the second position, the calibration board represents another calibration board coordinate system corresponding to the second position and represents another world coordinate system.

[0084] Before collecting the calibration board image, in the area of the calibration board corresponding to the field of view of each camera, a first marker corresponding to the camera is marked, so as to mark a plurality of first markers on the calibration board.

[0085] Among them, each first marker can be a group of marker points composed of multiple marker points or a single marker point.

[0086] Before collecting the calibration board images, multiple second fiducial marks can also be marked on the calibration board.

[0087] Each of the second fiducial marks is simultaneously within the camera fields of view of two cameras in a set of binocular calibration cameras.

[0088] In this embodiment, before marking the second fiducial marks, the sets of binocular calibration cameras in the scanning device can be determined first.

[0089] Specifically, for two cameras in the scanning device, when the overlapping range of the camera fields of view of the two cameras is greater than a preset range threshold, these two cameras can be used as a set of binocular calibration cameras.

[0090] Based on this, a second fiducial mark corresponding to the two cameras in each set of binocular calibration cameras can be selected within the overlapping range of the camera fields of view of each set of binocular calibration cameras.

[0091] Each of the second fiducial marks can be a set of fiducial points composed of multiple fiducial points or a single fiducial point.

[0092] Based on the marked first fiducial marks and second fiducial marks, each camera collects the calibration board images within its camera field of view, so that the calibration board images of each camera include the first fiducial marks and second fiducial marks corresponding to that camera.

[0093] S402. For each camera, use the coordinates of the first fiducial mark in the calibration board coordinate system in the calibration board and the coordinates of the first fiducial mark in the corresponding calibration board image in the camera coordinate system to determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system.

[0094] In this embodiment, based on the coordinate scale in the calibration board, the coordinates of each first fiducial mark in the calibration board coordinate system can be determined.

[0095] Based on the pixel coordinates of the first fiducial mark in the pixel coordinate system in the calibration board image, the pixel coordinates of the first fiducial mark in the pixel coordinate system can be converted into the coordinates of the first fiducial mark in the camera coordinate system. Among them, the pixel coordinates can be determined according to the focal length and the measured pixel length. In this embodiment, the coordinates of the first fiducial mark in the camera coordinate system in the calibration board image can also be determined by other methods.

[0096] Furthermore, based on the calibration board images collected in the foregoing step 401, the coordinates of the corresponding first fiducial marks in each calibration board image in the camera coordinate system can be determined.

[0097] Based on this, the first conversion relationship for converting the coordinates of the first fiducial mark in the calibration board coordinate system to the coordinates of the same first fiducial mark in the corresponding camera coordinate system can be calculated.

[0098] In this embodiment, in the process of calculating the first conversion relationship for converting the coordinates of the first fiducial mark in the calibration board coordinate system to the coordinates of the same first fiducial mark in the corresponding camera coordinate system, the first conversion function between the calibration board coordinate system and the camera coordinate system can be used for calculation.

[0099] Among them, the first conversion function specifically represents the operation logic when converting the coordinates in the calibration board coordinate system to the coordinates in the camera coordinate system.

[0100] In this embodiment, the first conversion function can be specifically expressed as the following formula (1):

[0101] P' = K[R T]P (1)

[0102] Among them, P' represents the coordinates of the first fiducial mark in the calibration board in the calibration board coordinate system, P represents the coordinates of the first fiducial mark in the calibration board image in the camera coordinate system, R represents the rotation matrix when converting between the coordinates P' and P, T represents the translation vector when converting between the coordinates P' and P, and K represents the camera internal parameters of the camera, and the camera internal parameters can be the camera internal parameter matrix in matrix form.

[0103] Accordingly, based on the pre-known camera internal parameters, using the coordinates of the first fiducial mark in the calibration board determined in advance and the coordinates of the first fiducial mark in the calibration board image in the camera coordinate system, the rotation matrix R and the translation vector T required for converting the coordinates in the calibration board coordinate system to the coordinates in the camera coordinate system can be calculated, so as to determine the operation logic in the first conversion function, that is, the first conversion relationship.

[0104] S403. For each group of binocular calibration cameras, use the first conversion relationship to determine the coordinates of the second fiducial mark in the calibration board in the camera coordinate systems of the two binocular calibration cameras respectively; wherein, the overlapping range of the calibration board images within the field of view of each group of binocular calibration cameras is greater than the preset range threshold.

[0105] In this embodiment, based on the pre-determined groups of binocular calibration cameras, before determining the second conversion relationship between the two camera coordinate systems, the first conversion relationship determined in the foregoing step 402 can be used to calculate the coordinates of each second fiducial mark in the two camera coordinate systems of the corresponding group of binocular calibration cameras, so that the second conversion relationship for converting between these two camera coordinate systems can be determined using the same second fiducial mark.

[0106] Specifically, for each group of binocular calibrated cameras, the coordinates of the second marker corresponding to this group of binocular calibrated cameras in the calibration board coordinate system can be determined first based on the coordinate scale in the calibration board.

[0107] Furthermore, use the formula (1) in the aforementioned step 402 to calculate the coordinates of the second marker in the calibration board coordinate system after being respectively transformed into the coordinate systems of the two cameras.

[0108] Specifically, for each camera in a group of binocular calibrated cameras, based on the known camera internal parameter K, and based on the rotation matrix R and translation vector T determined in the aforementioned step 402, using the coordinates of the determined second marker in the calibration board coordinate system, the coordinates of the second marker in the camera coordinate system of this camera can be calculated.

[0109] S404. Use the coordinates of the second marker in the calibration board in the coordinate systems of the two cameras in the same group of binocular calibrated cameras to determine the second conversion relationship between the coordinate systems of the two cameras in the same group of binocular calibrated cameras.

[0110] In this embodiment, for the two cameras in each group of binocular calibrated cameras, use the coordinates of the second marker in the calibration board in the calibration board coordinate system and the coordinates of the second marker in the corresponding calibration board image in the camera coordinate system to determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system.

[0111] Among them, for the coordinates of the second marker in the calibration board in the calibration board coordinate system, it can be determined through the coordinate scale in the calibration board.

[0112] Based on the coordinates of the corresponding same second marker respectively in the two camera coordinate systems determined in the aforementioned step 403, the second conversion relationship for coordinate conversion between the two camera coordinate systems can be determined using the second conversion function.

[0113] Among them, the second conversion function specifically represents the operation logic when performing coordinate conversion between the two camera coordinate systems.

[0114] In this embodiment, the second conversion function can specifically be expressed as the following formula (2):

[0115] P CAMx =[R CxinCy T CxinCy P CAMy (2)

[0116] Among them, P CAMx represents the coordinates of the second marker in the camera coordinate system of camera x, and P CAMy represents the coordinates of the same second marker in the camera coordinate system of camera y, and R CxinCyRepresents the rotation matrix when the second flag bit performs coordinate transformation between two camera coordinate systems, and specifically represents the coordinate P CAMx and the coordinate P CAMy The rotation matrix T during the coordinate transformation between them CxinCy Represents the translation vector when performing coordinate transformation between two camera coordinate systems, and specifically represents the coordinate P CAMx and the coordinate P CAMy The translation vector during the coordinate transformation between them.

[0117] Accordingly, by using the coordinates of the same second flag bit pre-determined in the two camera coordinate systems of a group of binocular calibrated cameras respectively, the rotation matrix R and the translation vector T required for coordinate transformation between the two camera coordinate systems of a group of binocular calibrated cameras can be calculated, thereby determining the operation logic in the second transformation function, that is, the second transformation relationship.

[0118] S405. Use the second transformation relationship between the two camera coordinates in the same group of binocular calibrated cameras to determine the third transformation relationship between the camera coordinate system of the preset first camera and the camera coordinate system of the second camera in the same group of binocular calibrated cameras, where the same group of binocular calibrated cameras includes a second camera and a third camera, and there is a second transformation relationship or a third transformation relationship between the camera coordinate system of the third camera and the camera coordinate system of the first camera.

[0119] Among them, the first camera can be any one camera selected arbitrarily from all the cameras of the scanning device. When the scanning device is applied, during the process of scanning the target part, the first camera can be used to obtain the target part images collected by the other cameras, and the first camera can convert the target part images obtained in each camera coordinate system into the camera coordinate system of the first camera, or be used to obtain the coordinates of the target part in the corresponding camera coordinate system from the target part images collected by the other cameras respectively, and convert the coordinates of the target part in each camera coordinate system into the first camera coordinate system, so that the first camera can accurately construct the three-dimensional information of the target part by using the target part images collected by each camera at different acquisition angles in the first camera coordinate system.

[0120] That is, when scanning the target part, all the other cameras except the selected first camera target the first camera, and send the collected target part images to the first camera, or send the coordinates of the target part in the corresponding camera coordinate system in the collected target part images to the first camera.

[0121] Therefore, before scanning the target part, during the calibration process of each camera of the scanning device, it is necessary to calibrate the transformation relationship between the camera coordinate system of the first camera and the camera coordinate systems of all the other cameras respectively.

[0122] Among them, each camera other than the first camera can be divided into the same binocular calibration camera of the first camera and the non - same - group binocular calibration camera.

[0123] In this embodiment, the second conversion relationship of the camera coordinate system between the first camera and its same - group binocular calibration camera has been obtained through the foregoing step 404. In this step, the third conversion relationship of the camera coordinate system between the first camera and the non - same - group binocular calibration camera can be completed through multiple calibrations. And in each calibration, the third conversion relationship between the first camera and a non - same - group binocular calibration camera is calibrated.

[0124] Among them, in each calibration, the non - same - group binocular calibration camera to be calibrated is used as the second camera.

[0125] In this embodiment, during the process of calibrating the third conversion relationship of the two camera coordinate systems between the first camera and the second camera each time, the second flag bit often represents a group of multiple feature points. Since the overlapping range of the camera fields of view of the first camera and the second camera is less than or equal to the range threshold, furthermore, there will be a situation where there are only a small number of feature points that are simultaneously within the camera fields of view of the first camera and the second camera respectively. Therefore, if a small number of feature points that are simultaneously within the camera fields of view of the first camera and the second camera respectively are used to calibrate the third conversion relationship, there will be a problem that the calibrated third conversion relationship is inaccurate, or there will also be a situation where there are no feature points that are simultaneously within the camera fields of view of the first camera and the second camera respectively. In this situation, it is impossible to use the flag bits that are simultaneously within the camera fields of view of the first camera and the second camera respectively to calibrate the third conversion relationship.

[0126] Based on the above problems, in this step, during the process of calibrating the third conversion relationship of the two camera coordinate systems of the first camera and the second camera each time, another camera that has a conversion relationship with the camera coordinate systems of both the first camera and the second camera can be determined, and this camera is used to calibrate the third conversion relationship of the two camera coordinate systems of the first camera and the second camera.

[0127] Among them, another camera that has a conversion relationship with the camera coordinate systems of both the first camera and the second camera can be used as the third camera in this embodiment.

[0128] In some cases, the third camera can be both the same - group binocular calibration camera of the first camera and the same - group binocular calibration camera of the second camera, that is, there is a second conversion relationship of the two camera coordinate systems between the third camera and the first camera, and at the same time, there is also a second conversion relationship of the two camera coordinate systems between the third camera and the second camera.

[0129] In some other cases, the third camera can also be only the binocular calibration camera of the same group as the second camera. At the same time, the third camera is the non - same - group binocular calibration camera of the first camera, but there is a third conversion relationship between the two camera coordinate systems pre - determined between the third camera and the first camera.

[0130] Based on this, when there is a second conversion relationship or a third conversion relationship between the camera coordinate systems of the first camera and the third camera, and there is a second conversion relationship between the camera coordinate systems of the third camera and the second camera, the third conversion relationship between the camera coordinate systems of the first camera and the second camera can be calibrated through the third camera.

[0131] In a specific example of this embodiment, the second conversion relationship or the third conversion relationship between the camera coordinate systems of the first camera and the third camera can be considered as follows: by rotating and / or translating the coordinates of the second fiducial mark in the camera coordinate system of the first camera, the coordinates of the second fiducial mark in the camera coordinate system of the third camera are obtained; or, by rotating and / or translating the coordinates of the second fiducial mark in the camera coordinate system of the third camera, the coordinates of the second fiducial mark in the camera coordinate system of the first camera are obtained.

[0132] Based on this, the second conversion relationship or the third conversion relationship between the camera coordinate systems of the first camera and the third camera can be expressed as the following formula (3):

[0133] P CAM3 =[R C3inC1 T C3inC1 P CAM1 (3)

[0134] Wherein, P CAM1 represents the coordinates of the second fiducial mark in the first camera coordinate system, P CAM3 represents the coordinates of the same second fiducial mark in the third camera coordinate system, R C3inC1 represents the rotation matrix for coordinate conversion between the two camera coordinate systems of the second fiducial mark, and specifically represents the rotation matrix for coordinate conversion between the coordinates P CAM1 and the coordinates P CAM3 , T C3inC1 represents the translation vector for coordinate conversion between the two camera coordinate systems of the second fiducial mark, and specifically represents the translation vector for coordinate conversion between the coordinates P CAM1 and the coordinates P CAM3 .

[0135] The second transformation relationship of the camera coordinate system existing between the third camera and the second camera can be considered as follows: by rotating and / or translating the coordinates of the second flag bit in the camera coordinate system of the third camera, the coordinates of the second flag bit in the camera coordinate system of the second camera are obtained; or, by rotating and / or translating the coordinates of the second flag bit in the camera coordinate system of the third camera, the coordinates of the second flag bit in the camera coordinate system of the first camera are obtained.

[0136] Based on this, the second transformation relationship of the camera coordinate system existing between the third camera and the second camera can be expressed as the following formula (4):

[0137] P CAM2 =[R C2inC3 T C2inC3 P CAM3 (4)

[0138] Among them, P CAM2 represents the coordinates of the same second flag bit in the second camera coordinate system as in formula (3), R C2inC3 represents the rotation matrix for coordinate transformation between the camera coordinate systems of the second camera and the third camera for the same second flag bit, and specifically represents the rotation matrix for coordinate transformation between coordinate P CAM2 and coordinate P CAM1 , T C2inC3 represents the translation vector for coordinate transformation between the camera coordinate systems of the second camera and the third camera for the second flag bit, and specifically represents the translation vector for coordinate transformation between coordinate P CAM2 and coordinate P CAM1 .

[0139] Furthermore, based on the above formula (3) and formula (4), the part on the right side of the equal sign in formula (3) representing P CAM3 can be substituted into P CAM3 in formula (4) to obtain the following formula (5):

[0140] P CAM2 =[R C2inC3 T C2inC3 [R C3inC1 T C3inC1 P CAM1 (5)

[0141] Furthermore, the product of [R C2inC3 T C2inC3 and [R C3inC1 T C3inC1 can be expressed as [R C2inC1 T C2inC1 , thus obtaining the following formula (6):

[0142] P CAM2 = [R C2inC1 T C2inC1 P CAM1 (6)

[0143] wherein, R C2inC1 represents the rotation matrix during coordinate transformation of the same second flag bit as in formulas (3) and (4) between the camera coordinate systems of the second camera and the first camera, and specifically represents the rotation matrix during coordinate transformation between coordinate P CAM1 and coordinate P CAM2 , and T C2inC1 represents the translation vector during coordinate transformation of the same second flag bit between the camera coordinate systems of the second camera and the first camera, and specifically represents the translation vector during coordinate transformation between coordinate P CAM1 and coordinate P CAM2 .

[0144] Based on this, in this embodiment, by setting multiple cameras in the scanning device, when scanning the target part, images of the target part can be collected from multiple angles by the multiple cameras respectively. Before scanning, each camera is calibrated using a calibration board, so that a single target camera can accurately convert the images collected by each camera into the camera coordinate system of the target camera. When calibrating the cameras, by first calibrating the second conversion relationship between the camera coordinate system of each camera and the calibration board coordinate system, when each camera scans the target part, accurate images can be obtained based on the accurate second conversion relationship, reducing the distortion of the images. Further, when calibrating the conversion relationship between the cameras, by first calibrating the second conversion relationship of the camera coordinate systems between each adjacent camera, since the overlapping range of the camera fields of view between two adjacent cameras is much larger than that between two non-adjacent cameras, more fiducial marks can be selected within a larger overlapping range to calculate the second conversion relationship between the camera coordinate systems of two adjacent cameras, and thus the obtained second conversion relationship is more accurate. When there is a camera to be solved among two adjacent cameras, the third conversion relationship between the target camera and the non-adjacent camera to be solved can be established through the second conversion relationship of the camera coordinate systems of these two adjacent cameras, realizing the calculation of the third conversion relationship between the target camera and the camera to be solved by transmitting a relatively accurate second conversion relationship, making the calculated third conversion relationship more accurate. Therefore, when scanning the target part, after the target camera converts the images of other non-adjacent cameras into the camera coordinate system of the target camera, a more accurate conversion result can be obtained, and thus the size of the target part can be accurately captured, avoiding the inaccurate third conversion relationship obtained when calculating the third conversion relationship between two non-adjacent cameras using too few fiducial marks within the overlapping range due to the smaller overlapping range of the camera fields of view between two non-adjacent cameras and fewer selectable fiducial marks within the overlapping range, which may cause the target camera to be unable to accurately convert the images of other non-adjacent cameras into its own camera coordinate system.

[0145] In another embodiment of the present application, the internal parameters of each camera of the scanning device can affect the size and / or proportion of the calibration board or the target part in the collected images, and the internal parameters of each camera are often uniformly preset during production. However, the uniformly preset internal parameters of the cameras are often inaccurate, and inaccurate internal parameters of the cameras will cause the size and / or proportion of the calibration board or the target part in the images to be inaccurate, and thus the images collected by the cameras will be distorted.

[0146] Based on this, before respectively collecting calibration board images by each camera of the scanning device, the internal parameters of each camera can also be calibrated respectively.

[0147] Among them, the camera internal parameters are parameters describing the internal characteristics of the camera and are represented in matrix form, which are used to map points in three-dimensional space to a two-dimensional image plane. In this embodiment, the three-dimensional space can be the calibration board coordinate system representing the world coordinate system, and the two-dimensional image plane can be the pixel coordinate system corresponding to the image collected by the camera, such as the pixel coordinate system of the calibration board image and / or the target part image.

[0148] In this embodiment, during the process of calibrating the camera internal parameters of each camera, the image of the calibration board can be collected by the camera first and used as the internal parameter calibration image.

[0149] Among them, the internal parameter flag bits are pre-identified on the calibration board.

[0150] Based on this, for each camera, the coordinates of the internal parameter flag bits in the calibration board coordinate system in the calibration board and the pixel coordinates of the internal parameter flag bits in the corresponding internal parameter calibration image in the pixel coordinate system can be used to determine the camera internal parameters required when mapping points in the calibration board coordinate system to the pixel coordinate system.

[0151] Specifically, for each camera, the camera internal parameters can be calibrated according to the camera internal parameter mapping formula, that is, formula (7) shown below:

[0152]

[0153] Among them, u represents the pixel coordinate of the internal parameter flag bit in the internal parameter calibration image on the x-axis of the pixel coordinate system, v represents the pixel coordinate of the internal parameter flag bit in the internal parameter calibration image on the y-axis of the pixel coordinate system, f x represents the pixel length of the focal length of the camera in the x-axis direction of the pixel coordinate system, f y represents the pixel length of the focal length of the camera in the y-axis direction of the pixel coordinate system, u0 represents the pixel coordinate of the center point of the internal parameter calibration image on the x-axis of the pixel coordinate system, v0 represents the pixel coordinate of the center point of the internal parameter calibration image on the y-axis of the pixel coordinate system, X, Y, and Z represent the coordinates of the internal parameter flag in the calibration board in the calibration board coordinate system, and λ represents the scaling factor when mapping points in three-dimensional space to the two-dimensional image plane.

[0154] The matrix in the above formula (7) is the camera internal parameter matrix, that is, the camera internal parameters, and can be used as the camera internal parameter K in the above formula (1). In some other cases, the matrix dimension of the camera internal parameter K in formula (1) can also be different. For example, the camera internal parameter K can also be expressed as the matrix

[0155] In this embodiment, the pixel coordinates of the internal reference flag bits in the internal reference calibration image in the pixel coordinate system can be determined based on the focal length and the measured pixel length. The coordinates of the internal reference flags in the calibration board in the calibration board coordinate system can be determined through the coordinate scale in the calibration board. Accordingly, when determining the pixel coordinates u0 and v0 of the center point of the internal reference calibration image on the x-axis of the pixel coordinate system, the pixel length f in the camera internal parameter matrix can be calculated using the above formula (7). x and f y to determine the complete camera internal parameter matrix.

[0156] Furthermore, the determined camera internal parameter matrix can be calibrated into the camera to replace the camera internal parameters pre-uniformly set in the camera, thereby solving the problem that the size and / or proportion of the calibration board or the target object in the image are inaccurate due to the pre-uniformly set camera internal parameters.

[0157] Based on this, in this embodiment, based on the camera internal parameter mapping formula, when the coordinates of the internal reference flag bits in the internal reference calibration image in the pixel coordinate system and the coordinates of the internal reference flag bits in the calibration board in the calibration board coordinate system are determined, the camera internal parameters can be accurately calculated. By using the calculated camera internal parameters to replace the pre-set camera internal parameters in the camera for calibration, the camera can use accurate camera internal parameters to collect calibration board images or target object images, thereby ensuring that the images collected by the camera will not be distorted. That is, it is ensured that the size and proportion of the calibration board in the calibration board image collected by the camera are accurate, and the size and / or proportion of the target object in the collected target object image are accurate. Furthermore, the accurate and non-distorted calibration board image can ensure the accuracy of the conversion relationship between the calibrated camera coordinate system and the calibration board coordinate system.

[0158] In another embodiment of the present application, during the process of determining the first conversion relationship between the calibration coordinate system and the camera coordinate system, since the corresponding calibration board coordinate systems are different when the calibration board is in different planes, therefore, when the calibration board is in different planes respectively, the first conversion relationship between the camera coordinate and the corresponding calibration board can be determined respectively, so that when the calibration board is in different planes, the coordinates of the second flag bits in the calibration board in the camera coordinate system can be determined according to the corresponding first conversion relationship.

[0159] Among them, the multiple planes can be parallel to each other. In some other cases, the multiple planes can also be non-parallel to each other.

[0160] In this embodiment, when the calibration board is in different planes, the coordinates of the first flag bits in the calibration board coordinate systems corresponding to each plane can be determined according to the corresponding calibration board coordinate systems and the coordinate scales in the calibration board.

[0161] Based on calibration plates in different planes, for the corresponding calibration plate images acquired by the camera, the coordinates of the first fiducial mark in the corresponding calibration plate image within the camera coordinate system can be determined.

[0162] Based on this, when the calibration plate is in different planes, using the coordinates of the first fiducial mark in the corresponding calibration plate coordinate system and the coordinates within the camera coordinate system, according to formula (1) in the foregoing embodiment, the first conversion relationship between the camera coordinate system and the calibration plate coordinate systems of each plane can be calculated.

[0163] Furthermore, for each camera of the scanning device, after determining the first conversion relationship between the camera and each calibration plate coordinate system respectively, it is necessary to determine the second conversion relationship between the two camera coordinate systems of the stereo-calibration cameras in the same group based on the calibration plate in the same plane.

[0164] Specifically, in the process of determining the second conversion relationship between the two camera coordinate systems of the stereo-calibration cameras in the same group based on the calibration plate in the same plane, for any one of the multiple planes, when the calibration plate is in this plane, using the coordinates of the second fiducial mark in the calibration plate coordinate system of this plane and the first conversion relationship, according to formula (1) in the foregoing embodiment, the coordinates of the second fiducial mark in the calibration plate of this plane within each camera coordinate system can be calculated.

[0165] Furthermore, based on the coordinates of the second fiducial mark in the calibration plate of this plane within the camera coordinate systems of the stereo-calibration cameras in the same group, the second conversion relationship between the two camera coordinate systems of the stereo-calibration cameras in the same group can be calculated using formula (2) in the foregoing embodiment.

[0166] It should be noted that in the process of calculating the second conversion relationship between the two camera coordinate systems of the stereo-calibration cameras in the same group, if the two cameras of the stereo-calibration cameras in the same group acquire calibration plate images when the calibration plate is in different planes, since the second fiducial marks in different planes do not belong to the same calibration plate coordinate system, after converting the second fiducial marks in different calibration plate coordinate systems respectively, the coordinates of the second fiducial marks in the two obtained calibration plate images within the corresponding camera coordinate systems do not conform to the correlation relationship of formula (2), and the second conversion relationship between the two camera coordinate systems in the stereo-calibration cameras in the same group cannot be accurately calibrated.

[0167] Based on this, in this embodiment, since the second fiducial marks in the same plane must belong to the same calibration board coordinate system, during the process of calibrating the second conversion relationship between the two camera coordinate systems in the same group of binocular calibration cameras, the same second fiducial mark within the same calibration board coordinate system is used to convert to the coordinates in the corresponding camera coordinate system, so that the coordinates of the two camera coordinate systems in the same group of binocular calibration cameras can satisfy the correlation relationship of formula (2), thereby ensuring the accuracy of the determined second conversion relationship between the two camera coordinate systems in the same group of binocular calibration cameras.

[0168] In another embodiment of the present application, during the process of using each camera to separately collect the calibration board images of the calibration board within the corresponding camera field of view, when the calibration board is in each plane and there is no straight-line light projected on the calibration board, the corresponding calibration board images can be collected and used as non-light-projected images.

[0169] In this embodiment, in some cases, it is necessary to project a straight-line light on the calibration board. In this case, the brightness of the straight-line light projected on the calibration board is often very high. Therefore, at this time, the overly bright straight-line light on the calibration board will cause the collected calibration board image to be overexposed. The overexposed calibration board image will make it difficult to see other positions except the straight-line light, thus making it difficult to accurately determine the first fiducial mark and the second fiducial mark from the calibration board image.

[0170] Based on this, for the calibration board image used to calculate the first conversion relationship and the calibration board image used to calculate the coordinates of the second fiducial mark on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras respectively, it is necessary to collect the calibration board image when there is no straight-line light projected on the calibration board, so as to avoid the overly bright straight-line light in the calibration board image from affecting the positioning of the first fiducial mark and the second fiducial mark.

[0171] In this embodiment, when the calibration board is in each different plane and there is no straight-line light projected on the calibration board, the corresponding calibration board images are collected respectively and used as non-light-projected images.

[0172] Based on this, in this embodiment, by collecting the calibration board image, that is, the non-light-projected image, when there is no straight-line light projected on the calibration board, it is possible to avoid the overly bright straight-line light in the calibration board image, thereby overcoming the problem of overexposure of the calibration board caused by the overly bright straight-line light, and enabling the first fiducial mark and the second fiducial mark to be clearly and accurately identified when it is necessary to determine the first fiducial mark and the second fiducial mark from the calibration board image.

[0173] In another embodiment of the present application, for each camera, since any one of the first conversion relationships corresponding to the camera only represents the conversion relationship between the camera coordinate system of the camera and the calibration plate coordinate system of a corresponding plane, when the calibration plate is in another plane, the first flag bit and the second flag bit therein must utilize the first conversion relationship corresponding to this plane in order to determine the correct coordinates of the first flag bit and the second flag bit in the camera coordinate system, and it is impossible to accurately determine the coordinates in the camera coordinate system through the first conversion relationships corresponding to other planes.

[0174] Therefore, based on the first conversion relationships corresponding to multiple planes calculated in advance, when actually using the camera of the scanning device to scan and measure the target part, it is necessary to ensure that the plane where the target part is located is one of the multiple planes, and then the first conversion relationship corresponding to this plane can be used to convert the coordinates of the target part in the calibration plate coordinate system into the coordinates in the camera coordinate system.

[0175] However, the number of planes is infinite, and during the process of calibrating the first conversion relationship, it is only possible to calibrate the first conversion relationship for a limited number of planes, so that when actually using the camera of the scanning device to scan and measure the target part, the applicable situation of the calibrated camera is limited to: the target part is placed on the plane for which the first conversion relationship has been calibrated.

[0176] But in some cases, it is often difficult to accurately place the target part on the plane for which the first conversion relationship has been pre-calibrated. For example, the plane where the target part is placed may be higher or lower than the plane for which the first conversion relationship has been calibrated, resulting in the inability to accurately convert the coordinates of the target part in this plane into the camera coordinate system.

[0177] In this embodiment, to solve this problem, for each camera, after calibrating the first conversion relationships between the camera coordinate system and the calibration plate coordinate systems of each plane, when the calibration plate is in any two planes respectively, a straight line light can be projected onto the calibration plate, so that the common plane where two straight line lights belonging to two different planes are located can be determined by using the straight line lights projected when the calibration plate is in the two planes respectively. Thus, when using the camera to scan the target part, for the target part in any plane, the coordinates of any position in this plane in the camera coordinate system can be determined by using the common plane.

[0178] Among them, the straight line light projected onto the calibration plate can be, for example, a laser line.

[0179] During the process of projecting the laser line, specifically, a laser plane can be projected onto the calibration plate by the laser in the optical component, so that the laser plane is projected as a straight line light on the calibration plate.

[0180] In this embodiment, when projecting the laser plane, any two planes can be selected from multiple planes as the first plane and the second plane respectively, and the laser plane is projected in a fixed angle towards the direction where the calibration plate is placed. Thus, when the calibration plate is in the first plane, a laser line formed by the projection of the laser plane can be obtained and used as the first straight-line light ray. When the calibration plate is in the second plane, a laser line formed by the projection of the same laser plane can be obtained and used as the first straight-line light ray.

[0181] Based on this, each camera can be used to collect the first light projection image when the calibration plate is in the first plane.

[0182] Further, based on the obtained first light projection image, after determining the first conversion relationships between the camera coordinate system and the first calibration plate coordinate system of the first plane and the second calibration plate coordinate system of the second plane respectively, the coordinates of the first straight-line light ray in the first calibration plate coordinate system can be converted into the second calibration plate coordinate system by using the two first conversion relationships. Thus, the coordinates of the first straight-line light ray and the second straight-line light ray in the second calibration plate coordinate system can be determined, and then the common plane where the first straight-line light ray and the second straight-line light ray are located can be determined.

[0183] Specifically, by using the coordinates of the first straight-line light ray in the first calibration plate coordinate system of the first plane and the first conversion relationship, according to formula (1) in the foregoing embodiment, the coordinates of the first straight-line light ray in the first light projection image in the camera coordinate system can be calculated.

[0184] In a specific example, the coordinates of the first straight-line light ray in the first calibration plate coordinate system of the first plane can be substituted into formula (1) to calculate the coordinates of the first straight-line light ray in the first light projection image in the camera coordinate system.

[0185] Further, by using the coordinates of the first straight-line light ray in the camera coordinate system and the first conversion relationship, according to formula (1) in the foregoing embodiment, the coordinates of the first straight-line light ray in the second calibration plate coordinate system of the second plane can be calculated.

[0186] In a specific example, the coordinates of the first straight-line light ray in the camera coordinate system can be substituted into formula (1) to calculate the coordinates of the first straight-line light ray in the second calibration plate coordinate system of the second plane.

[0187] Further, based on the obtained coordinates of the first straight-line light ray in the second calibration plate coordinate system of the second plane and combined with the coordinates of the second straight-line light ray in the second calibration plate coordinate system of the second plane, the plane that contains both the first straight-line light ray and the second straight-line light ray can be determined in the same second calibration plate coordinate system and used as the common plane of the two straight-line light rays.

[0188] In some scenarios, the common plane can be the laser plane projected by the laser.

[0189] In some embodiments, the determined common plane can be calibrated to the corresponding camera, so that when the camera is actually used for scanning and measurement, based on the common plane, the coordinates of any position in the plane or other planes in the camera coordinate system of the corresponding camera can be determined using the first conversion relationship corresponding to any plane.

[0190] In some other embodiments, the relative pose between the common plane and the second plane can also be specifically determined, which can be, for example, the angle between the common plane and the second plane, and this angle is calibrated to the corresponding camera, so that when the camera is actually used for scanning and measurement, based on the common plane, the coordinates of any position in the plane or other planes in the camera coordinate system of the corresponding camera can be determined using the first conversion relationship corresponding to any plane.

[0191] Based on this, in this embodiment, by projecting a laser plane on the calibration board, when the calibration board is respectively in the first plane and the second plane, laser lines projected by the same laser plane can be obtained on the calibration board, so that the projected first straight light ray and the second straight light ray both belong to the same common plane, that is, the laser plane. During the process of calibrating the common plane, using the pre-determined first conversion relationship, the coordinates of the first straight light ray in the first calibration board coordinate system can be converted into the coordinates in the second calibration board coordinate system through two coordinate conversions, so that the coordinates of the first straight light ray and the second straight light ray can be obtained in the same second calibration board coordinate system. Thus, it is possible to determine the common plane of these two straight lines, that is, the common plane of the first straight light ray and the second straight light ray, in the calibration board coordinate system using the coordinates of the two straight lines in the same calibration board coordinate system.

[0192] Based on the calibrated common plane, for example, the angle between the calibrated common plane and the second plane, during the process of actually using the camera to scan and measure the target object, it can play a role in converting the coordinates of the position in any plane in the corresponding calibration board coordinate system to the camera coordinate system using the first conversion relationship corresponding to the common plane and the calibrated angle. For example, through the calibrated angle, the coordinates of the position in any plane in the corresponding calibration board coordinate system can be converted to the calibration board coordinate system corresponding to the second plane, so as to achieve converting the coordinates of the target object in the calibration board coordinate system corresponding to any plane to the camera coordinate system only using the first conversion relationship corresponding to one plane, without having to repeatedly obtain the first conversion relationship between the calibration point coordinate systems of multiple different planes and the camera coordinate system.

[0193] In another embodiment of the present application, in the process of determining the third transformation relationship between the camera coordinate systems of two non-adjacent cameras by using the second transformation relationship between the same group of binocular calibration cameras, the transformation relationship between the camera coordinate systems of each camera and the same camera coordinate system can be obtained through multiple rounds of calculations. In each round of calculation, the cameras participating in this round of calculation can be classified into a first camera, a second camera, and a third camera. Among them, there is a second transformation relationship or a third transformation relationship between the camera coordinate systems of the first camera and the third camera, and there is a third transformation relationship between the camera coordinate systems of the third camera and the second camera. Accordingly, the third transformation relationship between the camera coordinate systems of the first camera and the second camera can be determined by using the second transformation relationship or the third transformation relationship between the camera coordinate systems of the first camera and the third camera, and the second transformation relationship between the camera coordinate systems of the second camera and the third camera.

[0194] Among them, one camera is used to obtain the camera images collected by all the other cameras during actual application. Therefore, in this embodiment, it is necessary to determine the transformation relationship between the first camera and each of the other cameras with respect to the camera coordinate system.

[0195] The first camera and the third camera can be adjacent cameras of the same group of binocular calibration cameras and have a second transformation relationship between the camera coordinate systems. Alternatively, the first camera and the third camera can also be non-adjacent cameras, and the third transformation relationship between the camera coordinate systems has been previously determined.

[0196] The second camera is an adjacent camera of the same group of binocular calibration cameras as the third camera, has a second transformation relationship between the camera coordinate systems with the third camera, and is not adjacent to the first camera.

[0197] That is to say, the third camera can be used to transfer the transformation relationship between the first camera and the second camera. Specifically, the transformation relationship between the camera coordinate systems of the first camera and the third camera can be substituted into the transformation relationship between the camera coordinate systems of the third camera and the second camera, so as to determine the third transformation relationship between the camera coordinate systems of the first camera and the second camera.

[0198] In this embodiment, multiple cameras can form a circular arrangement in a predetermined circular order. For example, when the number of cameras is 4, each camera can be used as a corner of a rectangle, or each camera is located on the edge circumference of the same circle, so as to form a circular arrangement.

[0199] In the circular arrangement, any one of the cameras can be used as the first camera.

[0200] Based on this, when the second transformation relationship of the camera coordinate system has been determined between the first camera and two adjacent binocular calibrated cameras in the same group, the third transformation relationship of the camera coordinate system between the first camera and each adjacent camera can be obtained respectively through multiple rounds of calculations in a predetermined circular order.

[0201] In this embodiment, in the first round of calculation, the next adjacent camera of the first camera in the circular order can be used as the third camera. Accordingly, there is a second transformation relationship of the camera coordinate system between the first camera and the third camera, which is expressed as formula (3) in the foregoing embodiment.

[0202] And the next adjacent camera of the third camera in the circular order can be used as the second camera. Accordingly, there is a second transformation relationship of the camera coordinate system between the third camera and the second camera, which is expressed as formula (4) in the foregoing embodiment.

[0203] Based on this, the second transformation relationship of the camera coordinate system between the first camera and the third camera can be substituted into the second transformation relationship of the camera coordinate system between the third camera and the second camera to obtain formula (5) or formula (6) in the foregoing embodiment, thereby determining the third transformation relationship of the camera coordinate system between the first camera and the second camera.

[0204] Further, based on the first round of calculation, in the next round of calculation, the next adjacent camera of the second camera in the first round in the circular order can be used as the second camera in the next round of calculation, and the second camera in the first round can be used as the third camera for transmitting the transformation relationship in the next round of calculation.

[0205] Based on this, in the next round of calculation, based on the first round of calculation, there is a third transformation relationship of the camera coordinate system between the first camera and the third camera in the next round; since the third camera and the second camera in the next round are adjacent binocular calibrated cameras in the same group, there is a second transformation relationship of the camera coordinate system between the third camera and the second camera in the next round.

[0206] Further, in the next round of calculation, the third transformation relationship of the camera coordinate system between the first camera and the third camera can be substituted into the second transformation relationship of the camera coordinate system between the third camera and the second camera, so as to obtain the third transformation relationship of the camera coordinate system between the first camera and the second camera in the next round of calculation.

[0207] In this embodiment, taking Figure 1 、 Figure 2 and Figure 3 the specific scenarios shown as a specific example to specifically illustrate the calibration method of the scanning device.

[0208] When calibrating Figure 1Before calibrating each camera in it, for each camera fixed on the scanning device, the cameras 1012, 1022, 1032 and 1042 can be arranged in a circular arrangement in a preset circular order.

[0209] For example, in Figure 1 In the scene shown, the camera 1012 in the scanning device can be used as the first camera in the preset circular order. In Figure 1 In the circular arrangement shown, the circular order of each camera is successively: camera 1012, camera 1022, camera 1032 and camera 1042.

[0210] When fixing each camera on the scanning device, by presetting the acquisition angles of two adjacent cameras, the overlapping range of the camera fields of view of two adjacent cameras can be made greater than the range threshold, so that every two adjacent cameras are called a group of binocular calibration cameras.

[0211] Based on this, in Figure 1 , Figure 2 and Figure 3 In the scenes shown, there are a total of the following 4 groups of binocular calibration cameras in the same group: the camera 1012 of the optical component 101 and the camera 1022 of the optical component 102 are a group of binocular calibration cameras, the camera 1022 of the optical component 102 and the camera 1032 of the optical component 103 are a group of binocular calibration cameras, the camera 1032 of the optical component 103 and the camera 1042 of the optical component 104 are a group of binocular calibration cameras, and the camera 1042 of the optical component 104 and the camera 1012 of the optical component 101 are a group of binocular calibration cameras.

[0212] Based on the determined groups of binocular calibration cameras above, before calibrating each camera, corresponding second flag bits can be marked in the calibration board for the four groups of binocular calibration cameras respectively. That is, a common second flag bit is marked for the camera 1012 and the camera 1022, a common second flag bit is marked for the camera 1022 and the camera 1032, a common second flag bit is marked for the camera 1032 and the camera 1042, and a common second flag bit is marked for the camera 1042 and the camera 1012, and a total of 4 second flag bits are marked. Accordingly, the calibration board images collected by each camera include the second flag bit corresponding to this camera.

[0213] Based on the marked second flag bits, for each group of binocular calibration cameras, the coordinates of the second flag bit in the calibration board image of each camera in the corresponding camera coordinate system can be calculated, so that the second conversion relationship between the camera coordinate systems of the two cameras in each group of binocular calibration cameras can be determined.

[0214] Specifically, taking cameras 1012 and 1022 as an example, cameras 1012 and 1022 are a pair of binocular calibration cameras in the same group. When the calibration board is in the same plane, using the first conversion relationship between the camera coordinate system of camera 1012 and the calibration board coordinate system, that is, formula (1), the coordinates of the second marker position of this pair of binocular calibration in the calibration board image of camera 1012 in the camera coordinate system can be calculated. And using the first conversion relationship between the camera coordinate system of camera 1022 and the calibration board coordinate system, that is, formula (1), the coordinates of the second marker position of this pair of binocular calibration in the calibration board image of camera 1022 in the camera coordinate system can be calculated.

[0215] Further, the coordinates in the camera coordinate system of camera 1012 and the coordinates in the camera coordinate system of camera 1022 can be substituted into the above formula (2), so as to calculate the rotation matrix and translation vector therein, that is, obtain the second conversion relationship between the camera coordinate system of camera 1012 and the camera coordinate system of camera 1022.

[0216] For another pair of binocular calibration cameras: cameras 1012 and 1042, using the first conversion relationship between the camera coordinate system of camera 1012 and the calibration board coordinate system, that is, formula (1), the coordinates of the second marker position of this pair of binocular calibration in the calibration board image of camera 1012 in the camera coordinate system can be calculated. And using the first conversion relationship between the camera coordinate system of camera 1042 and the calibration board coordinate system, that is, formula (1), the coordinates of the second marker position of this pair of binocular calibration in the calibration board image of camera 1042 in the camera coordinate system can be calculated.

[0217] Further, the coordinates in the camera coordinate system of camera 1012 and the coordinates in the camera coordinate system of camera 1042 can be substituted into the above formula (2), so as to calculate the rotation matrix and translation vector therein, that is, obtain the second conversion relationship between the camera coordinate system of camera 1012 and the camera coordinate system of camera 1042.

[0218] Based on this, the second conversion relationship between the camera coordinate systems of the two cameras in each pair of binocular calibration cameras can be determined.

[0219] Among them, the second conversion relationship between the camera coordinate system of camera 1012 and the camera coordinate system of camera 1022 is expressed as the following formula (8):

[0220] P CAM1022 =[R C1022inC1012 T C1022inC1012 P CAM1012 (8)

[0221] And represent the second conversion relationship between the camera coordinate system of camera 1022 and the camera coordinate system of camera 1032 as the following formula (9):

[0222] P CAM1032 =[R C1032inC1022 T C1032inC1022 P CAM1022 (9)

[0223] Wherein, P CAM1022 represents the coordinates of the second flag bit in the camera coordinate system of camera 1022, P CAM1012 represents the coordinates of the same second flag bit in the camera coordinate system of camera 1012, R C1022inC1012 represents the rotation matrix when the coordinates of the second flag bit are converted between the two camera coordinate systems, and specifically represents the rotation matrix when converting the coordinates P CAM1022 and the coordinates P CAM1012 ; T C1022inC1012 represents the translation vector when the coordinates of the second flag bit are converted between the two camera coordinate systems, and specifically represents the translation vector when converting the coordinates P CAM1022 and the coordinates P CAM1012 ; P CAM1032 represents the coordinates of the second flag bit in the camera coordinate system of camera 1032, R C1032inC1022 represents the rotation matrix when the coordinates of the second flag bit are converted between the two camera coordinate systems, and specifically represents the rotation matrix when converting the coordinates P CAM1032 and the coordinates P CAM1022 ; T C1032inC1022 represents the translation vector when the coordinates of the second flag bit are converted between the two camera coordinate systems, and specifically represents the translation vector when converting the coordinates P CAM1032 and the coordinates P CAM1022 .

[0224] In the example of Figure 1 , based on the determined circular order, the first camera 1012 can be used as the first camera, and according to the circular order, determine the third conversion relationship between the camera coordinate system of camera 1012 and the camera coordinate systems of each non-adjacent camera.

[0225] Specifically, in the process of calculating the third conversion relationship between the camera coordinate system of camera 1012 and the camera coordinate system of non-adjacent camera 1032, use camera 1022 as the third camera and camera 1032 as the second camera.

[0226] Wherein, there is a second conversion relationship between the camera coordinate systems of camera 1012 and camera 1022, and there is also a second conversion relationship between the camera coordinate systems of camera 1022 and camera 1032.

[0227] Based on this, formula (8) can be substituted into formula (9) to obtain formula (10) as shown below:

[0228] P CAM1032 =[R C1032inC1022 T C1032inC1022 [R C1022inC1012 T C1022inC1012 P CAM1012 (10)

[0229] Among them, formula (10) is used to represent the third conversion relationship between the camera coordinate systems of camera 1012 and camera 1032.

[0230] Based on this, the conversion relationships between the camera coordinate system of camera 1012 and the camera coordinate systems of the remaining cameras 1022, 1032, and 1042 can be obtained.

[0231] Based on this, in this embodiment, by using the camera that has conversion relationships with both the first camera and the second camera as the third camera, the third camera can be used as the camera for transmitting the conversion relationship between the first camera and the second camera, so as to substitute the second or third conversion relationship between the camera coordinate systems of the first camera and the third camera into the second conversion relationship between the second camera and the third camera, thereby completing the transmission of the conversion relationship between the first camera and the second camera, and further establishing the third conversion relationship between the first camera and the second camera.

[0232] In another embodiment of the present application, after determining the conversion relationships between the camera coordinate systems of the determined first camera and each of the remaining cameras, since the third conversion relationships between the camera coordinate systems of the first camera and each non-adjacent camera are obtained by transmitting the conversion relationship through the third camera in each round of calculation, the third conversion relationships between the camera coordinate systems of the first camera and each non-adjacent camera may have errors due to the transmission of the conversion relationship. Accordingly, the circular order of the foregoing embodiment can be used as the positive order, and the third conversion relationships between the camera coordinate systems of the first camera and each non-adjacent camera are calculated again in the reverse order opposite to the positive order, and based on the difference between the corresponding third conversion relationship in the reverse order case and the corresponding third conversion relationship in the positive order case, it is determined whether there is a possibility of error in the corresponding third conversion relationship in the positive order case.

[0233] Among them, multiple cameras can be regarded as forming a circular arrangement in a predetermined reverse order.

[0234] Further, in the case where the second conversion relationship of the camera coordinate system has been determined between the first camera and two adjacent binocular calibration cameras in the same group, the third conversion relationship of the camera coordinate system between the first camera and each adjacent camera can be obtained respectively through multiple rounds of calculations in a predetermined reverse order.

[0235] In this embodiment, in the first round of calculation, the next adjacent camera to the first camera in the reverse order can be used as the third camera. Accordingly, there is a second conversion relationship of the camera coordinate system between the first camera and the third camera, which is expressed as formula (3) in the foregoing embodiment.

[0236] And the next adjacent camera to the third camera in the reverse order can be used as the second camera. Accordingly, there is a second conversion relationship of the camera coordinate system between the third camera and the second camera, which is expressed as formula (4) in the foregoing embodiment.

[0237] Based on this, the second conversion relationship of the camera coordinate system between the first camera and the third camera can be substituted into the second conversion relationship of the camera coordinate system between the third camera and the second camera to obtain formula (5) or formula (6) in the foregoing embodiment, thereby determining the third conversion relationship of the camera coordinate system between the first camera and the second camera.

[0238] Further, based on the first round of calculation, in the next round of calculation, the next adjacent camera to the second camera in the first round in the reverse order can be used as the second camera in the next round of calculation, and the second camera in the first round can be used as the third camera for transmitting the conversion relationship in the next round of calculation.

[0239] Based on this, in the next round of calculation, based on the first round of calculation, there is a third conversion relationship of the camera coordinate system between the first camera and the third camera in the next round; since the third camera and the second camera in the next round are adjacent binocular calibration cameras in the same group, there is a second conversion relationship of the camera coordinate system between the third camera and the second camera in the next round.

[0240] Further, in the next round of calculation, the third conversion relationship of the camera coordinate system between the first camera and the third camera can be substituted into the second conversion relationship of the camera coordinate system between the third camera and the second camera, so as to obtain the third conversion relationship of the camera coordinate system between the first camera and the second camera in the next round of calculation.

[0241] After determining the third conversion relationship of the camera coordinate system between the first camera and each non-adjacent and non-close camera in the reverse order, for the third conversion relationship of the camera coordinate system between the first camera and any one camera, the difference between the third conversion relationship corresponding to the forward order, that is, the circular order, and the third conversion relationship corresponding to the reverse order can be compared.

[0242] Among them, in the process of comparing the differences, the differences between the rotation matrices and / or translation vectors of the two can be compared, or the differences between the rotation matrices and translation vectors in all the third transformation relationships in the forward order are compared with the differences between the rotation matrices and translation vectors in all the third transformation relationships in the reverse order, so as to obtain the differences between the two.

[0243] Furthermore, when the compared differences do not meet the preset error conditions, it is considered that the difference between the third transformation relationship corresponding to the forward order and the third transformation relationship corresponding to the reverse order is small, and it is determined that there is no error or the error is within a reasonable range between the third transformation relationship corresponding to the forward order and the third transformation relationship corresponding to the reverse order.

[0244] When the compared differences meet the preset error conditions, it is considered that the difference between the third transformation relationship corresponding to the forward order and the third transformation relationship corresponding to the reverse order is large, and it is determined that among the third transformation relationship corresponding to the forward order and the third transformation relationship corresponding to the reverse order, the error of at least one third transformation relationship exceeds the reasonable range.

[0245] Among them, the situation where the differences meet the preset error conditions can be, for example: the difference is less than or equal to the preset verification difference threshold, and / or the difference is less than or equal to the preset verification difference threshold.

[0246] In an example of this embodiment, in Figure 1 、 Figure 2 and Figure 3 In the specific scenarios shown, for the third transformation relationship between the camera coordinate systems of the camera 1012 and the camera 1032 determined in the foregoing embodiment, there is a possibility that there is an error in this third transformation relationship. Accordingly, the third transformation relationship between the camera coordinate systems of the camera 1012 and the camera 1032 is calculated again. When the difference between the third transformation relationship corresponding to the reverse order and the third transformation relationship corresponding to the forward order is too large, it can be determined that there is an error in at least one of the third transformation relationship corresponding to the reverse order and the third transformation relationship corresponding to the forward order.

[0247] Specifically, taking the first camera 1012 as the first camera, when calculating the transformation relationship between the camera coordinate system of the camera 1012 and the camera coordinate systems of each of the remaining cameras, in accordance with the above reverse order, the camera 1042 is taken as the third camera, and the camera 1032 is taken as the second camera.

[0248] Based on this, using the second transformation relationship between the camera coordinate systems of the camera 1012 and the camera 1042, and the second transformation relationship between the camera coordinate systems of the camera 1042 and the camera 1032, the third transformation relationship between the camera coordinate systems of the camera 1012 and the camera 1032 can be calculated.

[0249] Among them, the second conversion relationship between the camera coordinate system of camera 1012 and the camera coordinate system of camera 1042 is expressed as formula (11) shown below:

[0250] P CAM1042 =[R C1042inC1012 T C1042inC1012 P CAM1012 (11)

[0251] The second conversion relationship between the camera coordinate system of camera 1042 and the camera coordinate system of camera 1032 is expressed as formula (12) shown below:

[0252] P CAM1032 =[R C1032inC1042 T C1032inC1042 P CAM1042 (12)

[0253] Among them, P CAM1042 represents the coordinates of the second flag bit in the camera coordinate system of camera 1042, R C1042inC1012 represents the rotation matrix when the coordinates of the second flag bit are converted between two camera coordinate systems, and specifically represents the rotation matrix between coordinate P CAM1042 and coordinate P CAM1012 when performing coordinate conversion, T C1042inC1012 represents the translation vector when the coordinates of the second flag bit are converted between two camera coordinate systems, and specifically represents the translation vector between coordinate P CAM1042 and coordinate P CAM1012 when performing coordinate conversion, R C1032inC1042 represents the rotation matrix when the coordinates of the second flag bit are converted between two camera coordinate systems, and specifically represents the rotation matrix between coordinate P CAM1032 and coordinate P CAM1042 when performing coordinate conversion, T C1032inC1042 represents the translation vector when the coordinates of the second flag bit are converted between two camera coordinate systems, and specifically represents the translation vector between coordinate P CAM1032 and coordinate P CAM1042 when performing coordinate conversion.

[0254] Furthermore, formula (11) can be substituted into formula (12) to obtain formula (13) shown below:

[0255] P CAM1032 =[R C1032inC1042 T C1032inC1042 [R C1042inC1012 T C1042inC1012 P CAM1012 (13)

[0256] Among them, formula (13) is used to represent the third conversion relationship between the camera coordinate systems of camera 1012 and camera 1032 determined in reverse order.

[0257] Based on this, in the process of comparing the differences between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order, the [R C1032inC1022 T C1032inC1022 [R C1022inC1012 T C1022inC1012 in formula (10) corresponding to the forward order can be compared with the [R C1032inC1042 T C1032inC1042 [R C1042inC1012 T C1042inC1012 in formula (13) corresponding to the reverse order.

[0258] Among them, when the above difference is less than or equal to the preset verification difference threshold, it can be considered that the difference between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order is small, and it is determined that there is no error or the error is within a reasonable range between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the forward order.

[0259] When the above difference is greater than the preset verification difference threshold, it can be considered that the difference between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order is large, and it is determined that in the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order, the error of at least one third conversion relationship exceeds the reasonable range.

[0260] In some other cases, in the process of comparing the differences between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the forward order, the differences of each third conversion relationship corresponding to the forward order can also be calculated, as well as the differences of each third conversion relationship corresponding to the reverse order, and it is determined whether there may be an error between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order according to the difference between the difference corresponding to the forward order and the difference corresponding to the reverse order.

[0261] Among them, when determining the difference of the third conversion relationship corresponding to the forward order or the reverse order, taking the forward order as an example, based on the rotation matrix and translation vector in the third conversion relationship between the first camera and each of the remaining non-adjacent cameras, in the forward order, the matrix difference between the matrix composed of the rotation matrix and translation vector in each third conversion relationship and the matrix composed of the rotation matrix and translation vector in the previous third conversion relationship is calculated in turn, and the obtained matrix differences in sequence are used as the conversion relationship differences corresponding to the forward order.

[0262] When the above difference is less than or equal to a preset verification difference threshold, it can be considered that the difference between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order is small, and it is determined that there is no error or the error is within a reasonable range between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the forward order.

[0263] When the above difference is greater than the preset verification difference threshold, it can be considered that the difference between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order is large, and it is determined that among the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order, the error of at least one third conversion relationship exceeds a reasonable range.

[0264] Based on this, in this embodiment, for the third conversion relationship of the camera coordinate system between the first camera and each non-adjacent camera obtained, since the process of calculating this third conversion relationship is essentially a process of transmitting the conversion relationship through the third camera in the forward order, that is, the circular order, therefore, in the case of the reverse order opposite to the circular order, when calculating the third conversion relationship of the camera coordinate system between the first camera and each non-adjacent camera again, the third conversion relationship is calculated through the transmission process of another conversion relationship. Therefore, by comparing the difference between the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order, it is possible to determine that there is an error in at least one of the third conversion relationship corresponding to the forward order and the third conversion relationship corresponding to the reverse order when the difference is large.

[0265] Based on the same inventive concept, corresponding to the method of any of the above embodiments, an embodiment of the present application further provides a calibration device for a scanning device.

[0266] Reference Figure 5 , the calibration device of the scanning device includes:

[0267] An acquisition module 501, configured to use each camera to respectively acquire a calibration board image of the calibration board within the corresponding camera's field of view, where the calibration board image includes a first marker and a second marker on the calibration board;

[0268] A first conversion relationship determination module 502, configured to, for each camera, use the coordinates of the first marker on the calibration board in the calibration board coordinate system and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system to determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system;

[0269] A coordinate determination module 503, configured to, for each group of binocular calibration cameras, use the first conversion relationship to determine the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras respectively, where the overlapping range of the calibration board images within the fields of view of each group of binocular calibration cameras is greater than a preset range threshold;

[0270] A second conversion relationship determination module 504, configured to determine a second conversion relationship between two camera coordinate systems of the same-group binocular calibrated cameras by using the coordinates of the second flag bit in the calibration board in the two camera coordinate systems of the same-group binocular calibrated cameras;

[0271] A third conversion relationship determination module 505, configured to determine a third conversion relationship between the camera coordinate system of a preset first camera and the camera coordinate system of a second camera in the same-group binocular calibrated cameras by using the second conversion relationship between the two camera coordinates in the same-group binocular calibrated cameras, where the same-group binocular calibrated cameras include a second camera and a third camera, and there is a second conversion relationship or a third conversion relationship between the camera coordinate system of the third camera and the camera coordinate system of the first camera.

[0272] Wherein, in one embodiment, the acquisition module 501 is specifically configured to:

[0273] In the case where a straight-line light is not projected on the calibration board, use each camera to respectively acquire a first non-light projection image when the calibration board is in the first plane and a second non-light projection image when the calibration board is in the second plane.

[0274] Wherein, the multiple planes include a first plane and a second plane.

[0275] In another embodiment, the acquisition module 501 may also be specifically configured to:

[0276] In the case where a straight-line light is projected on the calibration board, use each camera to acquire a first light projection image of a first straight-line light projected on the calibration board in the first plane.

[0277] In another embodiment, the first conversion relationship determination module 502 is specifically configured to: for each camera, determine the first conversion relationship between each calibration board coordinate system and the camera coordinate system by using the coordinates of the first flag bit in the calibration board in each calibration board coordinate system and the coordinates of the first flag bit in the corresponding calibration board image in the camera coordinate system;

[0278] Using the first conversion relationship to determine the coordinates of the second flag bit in the calibration board in the two camera coordinate systems of the same-group binocular calibrated cameras respectively includes:

[0279] Using the first conversion relationship between any calibration board coordinate system and each camera coordinate system to determine the coordinates of the second flag bit in the calibration board in the camera coordinate systems of the two binocular calibrated cameras respectively when the calibration board is in the corresponding plane.

[0280] Among them, the calibration board coordinate system includes the calibration board coordinate systems when the calibration board is in multiple different planes respectively, and the multiple planes are parallel to each other; the calibration board images include the calibration board images collected when the calibration board is in different planes respectively.

[0281] In another embodiment, the first conversion relationship determination module 502 may specifically be configured to:

[0282] After determining the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system by using the coordinates of the first marker in the calibration board coordinate system of the calibration board and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system, the first conversion relationship determination module 502 performs:

[0283] For each camera, by using the first conversion relationship between the first calibration board coordinate system and each camera coordinate system, and the coordinates of the first straight-line light projected on the calibration board in the first calibration board coordinate system, determine the coordinates of the first straight-line light in the first straight-line light image in the corresponding camera coordinate system;

[0284] By using the first conversion relationship between the second calibration board coordinate system and each camera coordinate system, and the coordinates of the first straight-line light in the first light projection image in the corresponding camera coordinate system, determine the coordinates of the first straight-line light in the first light projection image in the second calibration board coordinate system;

[0285] Determine the common plane where the coordinates of the first straight-line light in the second calibration board coordinate system and the coordinates of the second straight-line light projected on the calibration board in the second plane in the second calibration board coordinate system are located;

[0286] Calibrate the angle between the common plane and the second plane into the corresponding camera, so that the corresponding camera can use the angle and any first conversion relationship to determine the coordinates of any position in any plane in the camera coordinate system of the corresponding camera.

[0287] In another embodiment, the third conversion relationship determination module 505 is specifically configured to:

[0288] In each round of calculation of the third conversion relationship, by using the second conversion relationship or the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round, determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round;

[0289] Take the second camera in the current round as the third camera in the next round of calculation, and perform the next round of calculation of the third conversion relationship until the third conversion relationships between the camera coordinate system of the first camera and the camera coordinate systems of each non-adjacent camera are determined.

[0290] Wherein, when the second camera is formed in a circular arrangement by a plurality of cameras in a predetermined circular order, the second camera is the next camera adjacent to the third camera and is not adjacent to the first camera.

[0291] Wherein, the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round is determined by using the second conversion relationship or the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round, including:

[0292] When the third camera is adjacent to the first camera;

[0293] Using the second conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round, to determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round.

[0294] When the third camera is not adjacent to the first camera;

[0295] Using the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round, to determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round.

[0296] For the convenience of description, when describing the above device, it is described separately as various modules according to functions. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0297] The device in the above embodiment is used to implement the calibration method of the corresponding scanning device in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0298] Based on the same inventive concept, corresponding to the method in any of the above embodiments, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the calibration method of the scanning device in any of the above embodiments.

[0299] Figure 6 The schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present application is shown.

[0300] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0301] Specifically, the above-mentioned processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present application.

[0302] The memory 602 may include a mass memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 602 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 602 may be internal or external to the electronic device. In a specific embodiment, the memory 602 is a non-volatile solid state memory.

[0303] The memory 602 may include a read only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present disclosure.

[0304] The processor 601 reads and executes the computer program instructions stored in the memory 602 to implement any one of the calibration methods of the scanning device in the above embodiments.

[0305] In one example, the electronic device may further include a communication interface 603 and a bus 610. Among them, as Figure 6 shown, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 610 to complete the communication with each other.

[0306] The communication interface 603 is mainly used to implement the communication between the modules, devices, units, and / or devices in the embodiments of the present application.

[0307] Bus 610 includes hardware, software, or both, and couples components of the electronic device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, bus 610 may include one or more buses. Although embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0308] The electronic device can perform the transfer of the conversion relationship based on the same group of binocular calibration cameras, so as to execute the calibration method of the scanning device in the embodiments of the present application, thereby implementing the combination Figure 4 The calibration method of the scanning device described.

[0309] In addition, in combination with the calibration method of the scanning device in the above embodiments, an embodiment of the present application can be implemented by providing a computer storage medium. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, any one of the calibration methods of the scanning device in the above embodiments is implemented.

[0310] An embodiment of the present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, any one of the calibration methods of the scanning device in the above embodiments is implemented.

[0311] It should be clear that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0312] The functional blocks shown in the above structural block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, and so on. When implemented in software, the elements of the present application are programs or code segments for performing the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted over a transmission medium or a communication link via a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0313] Based on the same inventive concept, correspondingly to the method of any of the above embodiments, the present application further provides a scanning device, which includes an electronic device calibrated by the calibration method of the scanning device of any of the previous embodiments.

[0314] It should also be noted that in the exemplary embodiments mentioned in the present application, some methods or systems are described based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, can be different from the order in the embodiments, or several steps can be executed simultaneously.

[0315] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware for performing the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0316] The above are only the specific implementation manners of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A calibration method for a scanning device, characterized in that, The scanning device includes a plurality of cameras; the method includes: Using each camera to separately collect calibration board images of the calibration board within the corresponding camera's field of view, where the calibration board images include a first fiducial and a second fiducial on the calibration board; For each camera, using the coordinates of the first fiducial on the calibration board in the calibration board coordinate system and the coordinates of the first fiducial in the corresponding calibration board image in the camera coordinate system, determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system; For each group of binocular calibration cameras, using the first conversion relationship, determine the coordinates of the second fiducial on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras respectively, where the overlapping range of the calibration board images within the field of view of each group of binocular calibration cameras is greater than a preset range threshold; Using the coordinates of the second fiducial on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras, determine the second conversion relationship between the two camera coordinate systems of the same group of binocular calibration cameras; Using the second conversion relationship between the two camera coordinates in the same group of binocular calibration cameras, determine the third conversion relationship between the camera coordinate system of a preset first camera and the camera coordinate system of the second camera in the same group of binocular calibration cameras, where the same group of binocular calibration cameras includes the second camera and a third camera, and there is a second conversion relationship or a third conversion relationship between the camera coordinate system of the third camera and the camera coordinate system of the first camera.

2. The calibration method of the scanning device according to claim 1, wherein The calibration board coordinate system includes calibration board coordinate systems when the calibration board is respectively in a plurality of different planes, and the plurality of planes are parallel to each other; the calibration board images include calibration board images collected when the calibration board is in different planes; The step of, for each camera, using the coordinates of the first fiducial on the calibration board in the calibration board coordinate system and the coordinates of the first fiducial in the corresponding calibration board image in the camera coordinate system to determine the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system includes: For each camera, using the coordinates of the first fiducial on the calibration board in each calibration board coordinate system and the coordinates of the first fiducial in the calibration board image corresponding to each calibration board coordinate system in the camera coordinate system, respectively determine the first conversion relationship between each calibration board coordinate system and the camera coordinate system; The step of using the first conversion relationship to determine the coordinates of the second fiducial on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras respectively includes: Using the first conversion relationship between any calibration board coordinate system and each camera coordinate system, determine the coordinates of the second fiducial on the calibration board in the camera coordinate systems of the two binocular calibration cameras respectively when the calibration board is in the corresponding plane.

3. The calibration method of the scanning device according to claim 1, characterized in that, The calibration board image includes the light projection images when the calibration board is in multiple planes, and the light projection image is the image when a straight light ray is projected onto the calibration board; in the case where the multiple planes include a first plane and a second plane, after determining the first conversion relationship between the calibration board coordinate system and the corresponding camera coordinate system by using the coordinates of the first fiducial mark in the calibration board coordinate system in the calibration board and the coordinates of the first fiducial mark in the corresponding calibration board image in the camera coordinate system, the method further includes: For each camera, using the first conversion relationship between the first calibration board coordinate system and each camera coordinate system, and the coordinates of the first straight light ray projected on the calibration board in the first calibration board coordinate system, to determine the coordinates of the first straight light ray in the corresponding camera coordinate system within the first straight light image; Using the first conversion relationship between the second calibration board coordinate system and each camera coordinate system, and the coordinates of the first straight light ray in the corresponding camera coordinate system within the first light projection image, to determine the coordinates of the first straight light ray in the first light projection image in the second calibration board coordinate system; Determine the common plane where the coordinates of the first straight light ray in the second calibration board coordinate system and the coordinates of the second straight light ray projected on the calibration board in the second plane in the second calibration board coordinate system are located; Calibrate the angle between the common plane and the second plane into the corresponding camera, so that the corresponding camera can use the angle and any first conversion relationship to determine the coordinates of any position in any plane in the camera coordinate system of the corresponding camera.

4. The calibration method of the scanning device according to claim 1, characterized in that, When the second camera is the next camera adjacent to the third camera in the case where the multiple cameras are arranged in a ring in a predetermined circular order and is not adjacent to the first camera; the method of determining the third conversion relationship between the camera coordinate system of the preset first camera and the camera coordinate system of the second camera in the same group of binocular calibration cameras by using the second conversion relationship between the camera coordinates of two cameras in the same group of binocular calibration cameras includes: In each round of calculation of the third conversion relationship, using the second conversion relationship or the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round, to determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round; Take the second camera in the current round as the third camera in the next round of calculation, and perform the next round of calculation of the third conversion relationship until the third conversion relationships between the camera coordinate system of the first camera and the camera coordinate systems of each non - adjacent camera are determined.

5. The calibration method of the scanning device according to claim 4, characterized in that, The method of using the second conversion relationship or the third conversion relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second conversion relationship between the two camera coordinate systems of the second camera and the third camera in the current round, to determine the third conversion relationship between the two camera coordinate systems of the first camera and the second camera in the current round includes: In the case where the third camera is adjacent to the first camera; Determine the third transformation relationship between the two camera coordinate systems of the first camera and the second camera in the current round by using the second transformation relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second transformation relationship between the two camera coordinate systems of the second camera and the third camera in the current round.

6. The calibration method of the scanning device according to claim 4, characterized in that The method of determining the third transformation relationship between the two camera coordinate systems of the first camera and the second camera in the current round by using the second transformation relationship or the third transformation relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second transformation relationship between the two camera coordinate systems of the second camera and the third camera in the current round, includes: In the case where the third camera is not adjacent to the first camera; Determine the third transformation relationship between the two camera coordinate systems of the first camera and the second camera in the current round by using the third transformation relationship between the two camera coordinate systems of the first camera and the third camera in the current round, and the second transformation relationship between the two camera coordinate systems of the second camera and the third camera in the current round.

7. A calibration device for a scanning device, characterized in that, The device includes: An acquisition module, configured to respectively acquire a calibration board image of the calibration board within the corresponding camera's field of view by using each camera, where the calibration board image includes a first marker and a second marker on the calibration board; A first transformation relationship determination module, configured to, for each camera, determine the first transformation relationship between the calibration board coordinate system and the corresponding camera coordinate system by using the coordinates of the first marker on the calibration board in the calibration board coordinate system and the coordinates of the first marker in the corresponding calibration board image in the camera coordinate system; A coordinate determination module, configured to, for each group of binocular calibration cameras, determine the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras respectively by using the first transformation relationship, where the overlapping range of the calibration board images within the field of view of each group of binocular calibration cameras is greater than a preset range threshold; A second transformation relationship determination module, configured to determine the second transformation relationship between the two camera coordinate systems of the same group of binocular calibration cameras by using the coordinates of the second marker on the calibration board in the two camera coordinate systems of the same group of binocular calibration cameras; A third transformation relationship determination module, configured to determine the third transformation relationship between the camera coordinate system of the preset first camera and the camera coordinate system of the second camera in the same group of binocular calibration cameras by using the second transformation relationship between the two camera coordinates in the same group of binocular calibration cameras, where the same group of binocular calibration cameras includes the second camera and the third camera, and there is a second transformation relationship or a third transformation relationship between the camera coordinate system of the third camera and the camera coordinate system of the first camera.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the calibration method of the scanning device as described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by the processor, the calibration method of the scanning device as described in any one of claims 1-6 is implemented.

10. A scanning device, characterized in that, An electronic device is included, and the electronic device is calibrated by the calibration method of the scanning device according to any one of claims 1-6.

Images