Stitching calibration system, method and device for multi-array three-dimensional measurement system

Abstract

The present invention relates to a splicing calibration system of a multi-array three-dimensional measurement system, which includes an image acquisition device, a calibration parameter acquisition device and a measurement reference unification device, wherein the image acquisition device is used to scan and acquire multiple three-dimensional point cloud images of a calibration plate, and transmit the multiple three-dimensional point cloud images to the calibration parameter acquisition device; the calibration parameter acquisition device is used to acquire corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images, and transmit the calibration parameters to the measurement reference unification device; the measurement reference unification device is used to complete the measurement reference unification of each image acquisition device in the multi-array three-dimensional measurement system according to the calibration parameters. The present invention also discloses a splicing calibration method of a multi-array three-dimensional measurement system and a splicing calibration device of a multi-array three-dimensional measurement system.

Description

Technical Field

[0001] The present application relates to the field of optical measurement technology, and in particular to a splicing and calibration system for a multi-array three-dimensional measurement system, a splicing and calibration method for a multi-array three-dimensional measurement system, and a splicing and calibration device for a multi-array three-dimensional measurement system. Background Art

[0002] With the continuous iteration of industrial development, many three-dimensional measurement technologies are becoming more and more mature. Among them, common non-contact three-dimensional measurement technologies include: confocal microscopy, white light phase-shift interferometry, and line structured light measurement. Among them, confocal microscopy and white light phase-shift interferometry have high measurement accuracy, but their measurement cost is expensive, the measurement format is small, the detection speed is slow, and the measurement efficiency is low, which makes it difficult to meet the real-time measurement needs of various micro-components. Compared with the first two measurement methods, line structured light measurement technology has the advantages of high detection efficiency, good real-time performance, strong anti-interference, simple system structure, strong scalability and integration.

[0003] At present, line structured light measurement technology plays an increasingly important role in industrial automation and intelligent manufacturing, and has been widely used in the semiconductor industry, mobile phone industry and other fields. With the rapid development of the semiconductor industry and the mobile phone industry, semiconductor and mobile phone manufacturers are increasingly in need of large-format, high-precision 3D measurement equipment. However, for multi-array 3D measurement systems, due to factors such as hardware processing and installation errors, the fixed postures of each 3D camera in the multi-array 3D measurement system are different, making it impossible to unify the measurement benchmark, which seriously restricts the measurement efficiency, measurement accuracy and measurement speed of the multi-array 3D measurement system. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a stitching and calibration system for a multi-array three-dimensional measurement system, aiming to solve the problems in the existing detection methods, such as the differences in the fixed postures of each three-dimensional camera in the multi-array three-dimensional measurement system due to factors such as hardware processing and installation errors, making it impossible to unify the measurement benchmark, and so on.

[0005] A stitching calibration system for a multi-array three-dimensional measurement system, comprising an image acquisition device, a calibration parameter acquisition device and a measurement reference unification device, wherein the measurement reference unification device is electrically connected to the image acquisition device and the calibration parameter acquisition device, wherein the image acquisition device is used to scan and acquire multiple three-dimensional point cloud images of a calibration plate, and transmit the multiple three-dimensional point cloud images to the calibration parameter acquisition device; the calibration parameter acquisition device is used to acquire corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images, and transmit the calibration parameters to the measurement reference unification device; the measurement reference unification device is used to complete the measurement reference unification of each image acquisition device in the multi-array three-dimensional measurement system according to the calibration parameters.

[0006] Optionally, the image acquisition device includes multiple camera units, each of which is a three-dimensional camera.

[0007] Optionally, the calibration parameter acquisition device includes a first calibration parameter acquisition chip and a second calibration parameter acquisition chip, wherein the first calibration parameter acquisition chip is electrically connected to the image acquisition device and the measurement reference unified device, and the first calibration parameter acquisition chip is used to calculate the first calibration parameter of the camera unit relative to the calibration plate coordinate system based on the three-dimensional point cloud image; the second calibration parameter acquisition chip is electrically connected to the image acquisition device and the measurement reference unified device, and the second calibration parameter acquisition chip is used to calculate the second calibration parameter of the camera unit relative to the calibration plate coordinate system based on the three-dimensional point cloud image.

[0008] Optionally, the first calibration parameter acquisition chip includes a first angle calculation circuit, a height image correction circuit and a first offset calculation circuit, wherein the first angle calculation circuit is electrically connected to the image acquisition device, and the first angle calculation circuit is used to calculate the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system according to the first original height map of the calibration plate in the three-dimensional point cloud image, and transmit the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system to the height image correction circuit; the height image correction circuit is electrically connected to the first angle calculation circuit, and the height image correction circuit is used to correct the first original height map according to the received roll angle and pitch angle to obtain a first corrected height map, and transmit the first corrected height map, the roll angle and the pitch angle to the first offset calculation circuit; the first offset calculation circuit is electrically connected to the height image correction circuit and the measurement reference unification device, and the first offset calculation circuit is used to calculate the offset of each camera unit in a first direction relative to the calibration plate coordinate system according to the first corrected height map, wherein the first calibration parameter includes the roll angle, the pitch angle and the offset in the first direction.

[0009] Optionally, the first offset calculation circuit is further used to correct the first corrected height map according to the offset in the first direction to obtain a second corrected height map.

[0010] Optionally, the second calibration parameter acquisition chip includes a second angle calculation circuit and a second offset calculation circuit, wherein the second angle calculation circuit is electrically connected to the image acquisition device, and is used to calculate the yaw angle and the offset in the third direction of each camera unit relative to the calibration plate coordinate system based on the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud image; the second offset calculation circuit is electrically connected to the image acquisition device and the measurement reference unification device, and is used to calculate the offset of the camera unit in the second direction relative to the calibration plate coordinate system based on the second original height map and the third original height map in the three-dimensional point cloud image, wherein the second calibration parameters include the yaw angle and the offset in the second direction and the offset in the third direction.

[0011] Optionally, the second calibration parameter acquisition chip also includes a height image stitching circuit, wherein the height image stitching circuit is electrically connected to the second angle calculation circuit and the second offset calculation circuit, and the height image stitching circuit is used to stitch and correct the second original height map according to the offset in the second direction, the yaw angle and the offset in the third direction to obtain a corresponding stitching correction image.

[0012] Optionally, the first direction is a Z-axis direction, the second direction may be an X-axis direction, and the third direction may be a Y-axis direction.

[0013] To summarize, the stitching and calibration system of the multi-array three-dimensional measurement system described in the present application obtains multiple three-dimensional point cloud images of a calibration plate and calculates the corresponding calibration parameters of the calibration plate, thereby realizing on-site stitching and calibration of the multi-array three-dimensional measurement system, thereby effectively improving the measurement efficiency, measurement accuracy and measurement speed of the multi-array three-dimensional measurement system.

[0014] Based on the same inventive concept, the present application also provides a stitching and calibration method for a multi-array three-dimensional measurement system, which is executed by the stitching and calibration system of the multi-array three-dimensional measurement system. The stitching and calibration method of the multi-array three-dimensional measurement system includes: obtaining multiple three-dimensional point cloud images of a calibration plate; obtaining corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images of the calibration plate; and completing the unification of the measurement benchmark of each image acquisition device in the multi-array three-dimensional measurement system according to the calibration parameters.

[0015] Optionally, the second calibration parameters include a yaw angle and an offset in a second direction and an offset in a third direction, including: a first calibration parameter of the camera unit relative to a calibration plate coordinate system calculated based on the three-dimensional point cloud image; and a second calibration parameter of the camera unit relative to the calibration plate coordinate system calculated based on the three-dimensional point cloud image.

[0016] Optionally, the first calibration parameters of the camera units relative to the calibration plate coordinate system are calculated based on the three-dimensional point cloud image, including: calculating the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system based on the first original height map of the calibration plate in the three-dimensional point cloud image; correcting the first original height map based on the roll angle and the pitch angle to obtain a first corrected height map; calculating the offset of each camera unit in a first direction relative to the calibration plate coordinate system based on the first corrected height map, wherein the first calibration parameters include the roll angle, the pitch angle and the offset in the first direction; and correcting the first corrected height map based on the offset in the first direction to obtain a second corrected height map.

[0017] Optionally, the second calibration parameters of the camera unit relative to the calibration plate coordinate system are calculated based on the three-dimensional point cloud image, including: calculating the yaw angle and the offset in the third direction of each camera unit relative to the calibration plate coordinate system based on the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud image; calculating the offset in the second direction of the camera unit relative to the calibration plate coordinate system based on the second original height map and the third original height map in the three-dimensional point cloud image, wherein the second calibration parameters include the yaw angle and the offset in the second direction and the offset in the third direction; and performing splicing correction on the second original height map based on the offset in the second direction and the yaw angle and the offset in the third direction to obtain a corresponding splicing correction image.

[0018] To summarize, the stitching and calibration method of the multi-array three-dimensional measurement system described in the present application obtains multiple three-dimensional point cloud images of a calibration plate and calculates the corresponding calibration parameters of the calibration plate, thereby realizing on-site stitching and calibration of the multi-array three-dimensional measurement system, thereby effectively improving the measurement efficiency, measurement accuracy and measurement speed of the multi-array three-dimensional measurement system.

[0019] Based on the same inventive concept, the present application also provides a stitching and calibration device for a multi-array three-dimensional measurement system, which includes: at least one processor and a storage device, at least one of the processors executes computer execution instructions stored in the storage device, and at least one of the processors executes the above-mentioned stitching and calibration method for the multi-array three-dimensional measurement system.

[0020] In summary, the splicing and calibration device of the multi-array three-dimensional measurement system provided in the present application can realize on-site splicing and calibration of the multi-array three-dimensional measurement system, thereby effectively improving the measurement efficiency, measurement accuracy and measurement speed of the multi-array three-dimensional measurement system, and improving the market competitiveness of the product. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of the structure of a splicing and calibration system for a multi-array three-dimensional measurement system disclosed in an embodiment of the present application;

[0023] Figure 2 Schematic diagram of the calibration model for a multi-array 3D measurement system;

[0024] Figure 3 for Figure 1 A schematic structural diagram of a first calibration parameter acquisition chip of a stitching calibration system of a multi-array three-dimensional measurement system shown;

[0025] Figure 4 for Figure 1 A schematic structural diagram of a second calibration parameter acquisition chip of a stitching calibration system of a multi-array three-dimensional measurement system;

[0026] Figure 5 A schematic diagram of a flow chart of a stitching calibration method for a multi-array three-dimensional measurement system disclosed in an embodiment of the present application;

[0027] Figure 6 for Figure 5 A schematic flow chart of step S20 in the stitching calibration method of the multi-array three-dimensional measurement system shown;

[0028] Figure 7 for Figure 6 A schematic flow chart of step S21 in the stitching calibration method of the multi-array three-dimensional measurement system shown;

[0029] Figure 8 for Figure 6 A schematic flow chart of step S22 in the stitching calibration method of the multi-array three-dimensional measurement system shown;

[0030] Fig. 9 A schematic diagram of the hardware structure of a stitching calibration device for a multi-array three-dimensional measurement system disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present application are given in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thoroughly and comprehensively understood.

[0032] The following descriptions of the various embodiments are with reference to the attached diagrams to illustrate specific embodiments that the present application can be used to implement. The serial numbers for the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in the present application, unless otherwise specified, include direct and indirect connections (couplings). The directional terms mentioned in the present application, such as "upper", "lower", "front", "back", "left", "right", "inside", "outside", "side", etc., are only with reference to the directions of the attached diagrams. Therefore, the directional terms used are for better and clearer explanation and understanding of the present application, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to the present application.

[0033] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be fixedly connected, detachably connected, or integrally connected; it can be mechanically connected; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances. It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including", "may include", "include", or "may include" used in this application indicate the existence of the corresponding functions, operations, elements, etc. disclosed, and do not limit one or more other functions, operations, elements, etc. In addition, the terms "including" or "include" indicate the existence of the corresponding features, numbers, steps, operations, elements, components, or combinations thereof disclosed in the specification, and do not exclude the existence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, and are intended to cover non-exclusive inclusions.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0035] With the continuous iteration of industrial development, many three-dimensional measurement technologies are becoming more and more mature. Among them, common non-contact three-dimensional measurement technologies include: confocal microscopy, white light phase-shift interferometry, and line structured light measurement. Among them, confocal microscopy and white light phase-shift interferometry have high measurement accuracy, but their measurement cost is expensive, the measurement format is small, the detection speed is slow, and the measurement efficiency is low, which makes it difficult to meet the real-time measurement needs of various micro-components. Compared with the first two measurement methods, line structured light measurement technology has the advantages of high detection efficiency, good real-time performance, strong anti-interference, simple system structure, strong scalability and integration. At present, line structured light measurement technology plays an increasingly important role in industrial automation and intelligent manufacturing, and has been widely used in the semiconductor industry, mobile phone industry and other fields. With the rapid development of the semiconductor industry and the mobile phone industry, semiconductor and mobile phone manufacturers are increasingly in urgent need of large-format, high-precision three-dimensional measurement equipment. However, for multi-array 3D measurement systems, due to factors such as hardware processing and installation errors, there are differences in the fixed postures of each 3D camera in the multi-array 3D measurement system, making it impossible to unify the measurement benchmark, which seriously restricts the measurement efficiency, measurement accuracy and measurement speed of the multi-array 3D measurement system.

[0036] Based on this, the present application hopes to provide a solution that can solve the above-mentioned technical problems, which can realize on-site splicing and calibration of the multi-array three-dimensional measurement system, thereby effectively improving the measurement efficiency, measurement accuracy and measurement speed of the multi-array three-dimensional measurement system. The details will be explained in subsequent embodiments.

[0037] It should be noted that structured light is a system structure composed of a projector and a camera. After the projector projects specific light information onto the surface of an object and the background, the camera collects it and calculates the position and depth of the object based on the changes in the light signal caused by the object, thereby restoring the entire three-dimensional space. Line structured light three-dimensional (3D) measurement technology has been widely used in semiconductor industries such as PCB board inspection, Mini LED inspection, chip wafer inspection, and mobile phone glass cover 3D curved surface inspection, screen thickness inspection, and other application scenarios in the mobile phone industry. In the existing multi-array three-dimensional measurement system, due to factors such as hardware processing and installation errors, the fixed posture of each 3D camera in the multi-array three-dimensional measurement system is different, and its own coordinate system has changed to a certain extent. At this time, if the multi-array three-dimensional measurement system is not calibrated and corrected, the measurement results of each 3D camera block in the multi-array three-dimensional measurement system will not be unified.

[0038] See also Figure 1 , which is a schematic diagram of the structure of a splicing calibration system for a multi-array three-dimensional measurement system disclosed in an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a stitching calibration system 100 for a multi-array three-dimensional measurement system, which at least includes an image acquisition device 110, a calibration parameter acquisition device 120, and a measurement reference unification device 130. The image acquisition device 110 is electrically connected to the calibration parameter acquisition device 120, and the calibration parameter acquisition device 120 is electrically connected to the measurement reference unification device 130, that is, the image acquisition device 110, the calibration parameter acquisition device 120, and the measurement reference unification device 130 are electrically connected in sequence.

[0039] The image acquisition device 110 is used to scan and acquire multiple three-dimensional point cloud images of the calibration plate, and transmit the obtained multiple three-dimensional point cloud images to the calibration parameter acquisition device 120 respectively.

[0040] In an embodiment of the present application, the image acquisition device 110 may include a plurality of camera units, each of which may be a three-dimensional (3D) camera. In applications such as machine vision, image measurement, photogrammetry, and three-dimensional reconstruction, a calibration target is required to establish a geometric model of camera imaging in order to correct lens distortion, determine the conversion relationship between physical size and pixels, and determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image. By shooting a flat plate with a fixed-pitch pattern array with a camera and calculating it through a calibration algorithm, the geometric model of the camera can be obtained, thereby obtaining high-precision measurement and reconstruction results. The flat plate with a fixed-pitch pattern array is the calibration plate. In an embodiment of the present application, the calibration plate may be a steel ruler, a chessboard, or a PCB circular hole calibration plate.

[0041] Please also read Figure 2 Specifically, by using a steel ruler as a calibration plate, a coordinate system conversion model of the coordinate system Oc-XcYcZc of each camera unit in the multi-array three-dimensional measurement system relative to the coordinate system Ow-XwYwZw of the calibration plate is established, and its formula is as follows:

[0042]

[0043] Among them, the calibration plate coordinate system Ow-XwYwZw: the long axis of the calibration plate is the Xw axis, the short axis of the calibration plate is the Yw axis, and the perpendicular to the XwYw plane is the Zw axis; the camera unit's own coordinate system Oc-XcYcZc: the line laser direction of the 3D camera is the Xc axis, the scanning direction of the 3D camera is the Yc axis, and the perpendicular to the XcYc direction is the Zc axis. In the above formula (1), θ is the pitch angle around the Y axis, γ is the roll angle around the X axis, and ψ is the roll yaw angle around the Z axis; △X, △Y, and △Z are the offsets in the XYZ directions respectively.

[0044] The calibration parameter acquisition device 120 is used to calculate corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images of the calibration plate acquired by the image acquisition device 110, and transmit the obtained calibration parameters to the measurement reference unification device 130.

[0045] The measurement reference unification device 130 is used to complete the measurement reference unification of each image acquisition device 110 in the multi-array three-dimensional measurement system according to the calibration parameters obtained by the calibration parameter acquisition device 120.

[0046] In the embodiment of the present application, the calibration parameter acquisition device 120 includes a first calibration parameter acquisition chip 121 and a second calibration parameter acquisition chip 122. The first calibration parameter acquisition chip 121 is electrically connected to the image acquisition device 110 and the measurement reference unification device 130, and the second calibration parameter acquisition chip 122 is electrically connected to the image acquisition device 110 and the measurement reference unification device 130.

[0047] The first calibration parameter acquisition chip 121 is used to calculate the first calibration parameters of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image transmitted by the image acquisition device 110. The first calibration parameters include a roll angle, a pitch angle, and an offset in a first direction.

[0048] The second calibration parameter acquisition chip 122 is used to calculate the second calibration parameters of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image transmitted by the image acquisition device 110. The second calibration parameters include the yaw angle and the offset in the second direction and the offset in the third direction.

[0049] Specifically, in the embodiment of the present application, the first direction is the Z-axis direction, the second direction may be the X-axis direction, and the third direction may be the Y-axis direction.

[0050] like Figure 2 As shown, the calibration formula is simplified according to the coordinate system conversion model of each camera unit relative to the calibration plate coordinate system combined with the actual situation of the three-dimensional device (in this embodiment, the three-dimensional device can be a three-dimensional camera, which realizes the acquisition and simultaneous output of three-dimensional data and two-dimensional data). The simplification process is as follows: Considering the limitations of optical principles, the measurement range of the three-dimensional camera in the Z direction is very small, much smaller than the measurement range in the XY direction, and the installation deviation angles of the pitch angle and roll angle are small. Then:

[0051]

[0052]

[0053] Substituting formula (2) and formula (3) into the coordinate system conversion formula (1), the simplified calibration formula is as follows:

[0054]

[0055] According to the simplified calibration formula (4), the calibration process can be divided into two steps. In the first step, the pitch angle θ, roll angle γ and offset △Z in the first direction of each 3D camera relative to the calibration plate coordinate system Ow-XwYwZw are solved in turn, and the Z coordinate benchmark of each 3D camera in the multi-array 3D measurement system can be unified. In the second step, the yaw angle ψ, offset △X in the second direction, and offset △Y in the third direction of each 3D camera relative to the calibration plate coordinate system Ow-XwYwZw are solved in turn, and the XY coordinate benchmark of each 3D camera in the multi-array 3D measurement system can be unified.

[0056] Optionally, the number of 3D cameras is limited by the acquisition card, the computer host interface, etc. For example, 4 acquisition cards can be connected to a single host, each acquisition card has 4 camera interfaces, and a maximum of 16 cameras can be used. It is understandable that according to the actual maximum scanning area requirements, 15 cameras can achieve the maximum area scanning, and a total of 15 cameras are used.

[0057] See also Figure 3 , which is Figure 1 The schematic diagram of the structure of the first calibration parameter acquisition chip 121 of the splicing calibration system of the multi-array three-dimensional measurement system is shown in FIG. Figure 3 As shown, the first calibration parameter acquisition chip 121 includes a first angle calculation circuit 1211, a height image correction circuit 1212, and a first offset calculation circuit 1214. The first angle calculation circuit 1211 is electrically connected to the height image correction circuit 1212, the height image correction circuit 1212 is electrically connected to the first offset calculation circuit 1214, the first offset calculation circuit 1214 is also electrically connected to the measurement reference unification device 130, and the first angle calculation circuit 1211 is also electrically connected to the image acquisition device 110.

[0058] In this embodiment, the first angle calculation circuit 1211 is used to calculate the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system based on the first original height map of the calibration plate in the three-dimensional point cloud image transmitted by the image acquisition device 110, and transmit the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system to the height image correction circuit 1212.

[0059] Specifically, in an embodiment of the present application, the first angle calculation circuit 1211 sequentially solves the plane coefficients in the corresponding blocks of each camera unit (i.e., three-dimensional camera) in the first original height map of the calibration plate in the three-dimensional point cloud map transmitted by the image acquisition device 110, and obtains the roll angle θ and pitch angle γ of each camera unit relative to the calibration plate coordinate system.

[0060] The height image correction circuit 1212 is used to correct the first original height map according to the received roll angle and pitch angle to obtain a first corrected height map, and transmit the first corrected height map, the roll angle and the pitch angle to the first offset calculation circuit 1214.

[0061] The first offset calculation circuit 1214 is used to calculate the offset of each camera unit relative to the calibration plate coordinate system in the first direction according to the first corrected height map transmitted by the height image correction circuit 1212, and transmit the offset in the first direction and the roll angle and the pitch angle to the measurement reference unification device 130. In the embodiment of the present application, the first direction is the Z-axis direction.

[0062] The first offset calculation circuit 1214 is further configured to correct the first corrected height map according to the offset in the first direction to obtain a second corrected height map.

[0063] See also Figure 4 , which is Figure 1 The schematic diagram of the structure of the second calibration parameter acquisition chip 122 of the stitching calibration system of the multi-array three-dimensional measurement system is shown in FIG. Figure 4 As shown, the second calibration parameter acquisition chip 122 includes a second angle calculation circuit 1221 and a second offset calculation circuit 1222. The second angle calculation circuit 1221 is electrically connected to the second offset calculation circuit 1222, the second angle calculation circuit 1221 and the second offset calculation circuit 1222 are both electrically connected to the image acquisition device 110, and the second offset calculation circuit 1222 is also electrically connected to the measurement reference unification device 130.

[0064] In the embodiment of the present application, the second angle calculation circuit 1221 is used to calculate the yaw angle and the offset of the third direction of each camera unit relative to the calibration plate coordinate system according to the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud image transmitted by the image acquisition device 110, and transmit the yaw angle and the offset of the third direction to the second offset calculation circuit 1222. In the embodiment of the present application, the third direction can be the Y-axis direction.

[0065] Specifically, in an embodiment of the present application, the second angle calculation circuit 1221 is used to solve the straight line equations and four preset vertex coordinates of the upper and lower boundaries of the calibration plate in the block corresponding to each camera unit in the second original height map in the three-dimensional point cloud map transmitted by the image acquisition device 110 in turn according to the feature extraction algorithm, and perform corresponding calculations based on the actual tilt placement angle of the calibration plate and the four preset vertex coordinates to obtain the yaw angle ψ of the line laser from the first camera unit to the last camera unit relative to the calibration plate and the offset △Y in the third direction in turn.

[0066] The second offset calculation circuit 1222 is used to calculate the offset of the camera unit in the second direction relative to the calibration plate coordinate system according to the second original height map and the third original height map in the three-dimensional point cloud image transmitted by the image acquisition device 110, and transmit the obtained offset in the second direction and the yaw angle and the offset in the third direction transmitted by the second angle calculation circuit 1221 to the measurement reference unification device 130. In the embodiment of the present application, the second direction can be the X-axis direction.

[0067] Specifically, in the embodiment of the present application, the second offset calculation circuit 1222 is used to compare and calculate the difference in the same scale of the calibration plate between the second original height map and the third original height map in the three-dimensional point cloud map transmitted by the image acquisition device 110 according to the feature extraction algorithm, and the deviation value at the X starting point position according to the actual preset number of times (for example, twice) of the camera unit, and sequentially calculate the offset △X of the line laser from the first camera unit to the last camera unit in the multi-array three-dimensional measurement system relative to the calibration plate in the second direction.

[0068] The second calibration parameter acquisition chip 122 also includes a height image stitching circuit 1224, which is electrically connected to the second angle calculation circuit 1221 and the second offset calculation circuit 1222, and is used to stitch and correct the second original height map according to the offset in the second direction transmitted by the second offset calculation circuit 1222 and the yaw angle and the offset in the third direction transmitted by the second angle calculation circuit 1221 to obtain a corresponding stitching correction image.

[0069] See also Figure 5 , which is a flow chart of a method for stitching and calibrating a multi-array three-dimensional measurement system disclosed in an embodiment of the present application, the above Figure 1-Figure 4 The stitching calibration system of the multi-array three-dimensional measurement system in the embodiment shown uses the following stitching calibration method of the multi-array three-dimensional measurement system to perform on-site stitching calibration on the multi-array three-dimensional measurement system, thereby effectively improving the measurement efficiency, measurement accuracy and measurement speed of the multi-array three-dimensional measurement system. Figure 5As shown, the stitching calibration method of the multi-array three-dimensional measurement system includes at least the following steps.

[0070] S10, obtaining multiple three-dimensional point cloud images of the calibration plate.

[0071] In this embodiment, please combine Figure 1 , the image acquisition device 110 scans and acquires a plurality of three-dimensional point cloud images of the calibration plate, and transmits the obtained plurality of three-dimensional point cloud images to the calibration parameter acquisition device 120 respectively.

[0072] In an embodiment of the present application, the image acquisition device 110 may include a plurality of camera units, each of which may be a three-dimensional camera. In applications such as machine vision, image measurement, photogrammetry, and three-dimensional reconstruction, a calibration plate is required to establish a geometric model of camera imaging in order to correct lens distortion; determine the conversion relationship between physical size and pixels; and determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image. By photographing a flat plate with a fixed-pitch pattern array by a camera and calculating through a calibration algorithm, the geometric model of the camera can be obtained, thereby obtaining high-precision measurement and reconstruction results. The flat plate with a fixed-pitch pattern array is the calibration plate. In an embodiment of the present application, the calibration plate may be a steel ruler, a chessboard, or a PCB circular hole calibration plate.

[0073] Please also read Figure 2 Specifically, by using a steel ruler as a calibration plate, a coordinate system conversion model of the coordinate system Oc-XcYcZc of each camera unit in the multi-array three-dimensional measurement system relative to the coordinate system Ow-XwYwZw of the calibration plate is established, and its formula is as follows:

[0074]

[0075] Among them, the calibration plate coordinate system Ow-XwYwZw: the long axis of the calibration plate is the Xw axis, the short axis of the calibration plate is the Yw axis, and the perpendicular to the XwYw plane is the Zw axis; the camera unit's own coordinate system Oc-XcYcZc: the line laser direction of the 3D camera is the Xc axis, the scanning direction of the 3D camera is the Yc axis, and the perpendicular to the XcYc direction is the Zc axis. In the above formula (1), θ is the pitch angle around the Y axis, γ is the roll angle around the X axis, and ψ is the roll yaw angle around the Z axis; △X, △Y, and △Z are the offsets in the XYZ directions respectively.

[0076] S20: Calculate and obtain corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images of the calibration plate.

[0077] In this embodiment, please combine Figure 1 and Figure 2The calibration parameter acquisition device 120 calculates corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images of the calibration plate acquired by the image acquisition device 110, and transmits the obtained calibration parameters to the measurement reference unification device 130.

[0078] In the embodiments of this application, please refer to Figure 6 Combined with Figure 1 , the step S20 at least includes the following steps.

[0079] S21. Calculate, according to the three-dimensional point cloud image, a first calibration parameter of the camera unit relative to a calibration plate coordinate system.

[0080] Specifically, the first calibration parameter acquisition chip 121 calculates the first calibration parameter of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image transmitted by the image acquisition device 110. The first calibration parameter includes a roll angle, a pitch angle, and an offset in a first direction.

[0081] In the embodiments of this application, please refer to Figure 7 Combined with Figure 3 , the step S21 at least includes the following steps.

[0082] S211. Calculate the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system according to the first original height map of the calibration plate in the three-dimensional point cloud map.

[0083] Specifically, the first angle calculation circuit 1211 calculates the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system according to the first original height map of the calibration plate in the three-dimensional point cloud image transmitted by the image acquisition device 110, and transmits the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system to the height image correction circuit 1212.

[0084] In an embodiment of the present application, the first angle calculation circuit 1211 sequentially solves the plane coefficients in the corresponding blocks of each camera unit (i.e., three-dimensional camera) in the first original height map of the calibration plate in the three-dimensional point cloud map transmitted by the image acquisition device 110, and obtains the roll angle θ and pitch angle γ of each camera unit relative to the coordinate system of the calibration plate.

[0085] S212: Correct the first original height map according to the roll angle and the pitch angle to obtain a first corrected height map.

[0086] Specifically, the height image correction circuit 1212 corrects the first original height map according to the received roll angle and pitch angle to obtain a first corrected height map, and transmits the first corrected height map, the roll angle, and the pitch angle to the first offset calculation circuit 1214.

[0087] S213. Calculate the offset of each camera unit in the first direction relative to the calibration plate coordinate system according to the first corrected height map.

[0088] Specifically, the first offset calculation circuit 1214 calculates the offset of each camera unit relative to the calibration plate coordinate system in the first direction according to the first corrected height map transmitted by the height image correction circuit 1212, and transmits the offset in the first direction and the roll angle and the pitch angle to the measurement reference unification device 130. In the embodiment of the present application, the first direction is the Z-axis direction.

[0089] S214: Correct the first corrected height map according to the offset in the first direction to obtain a second corrected height map.

[0090] Specifically, the first offset calculation circuit 1214 is further configured to correct the first corrected height map according to the offset in the first direction to obtain a second corrected height map.

[0091] S22. Calculate a second calibration parameter of the camera unit relative to a calibration plate coordinate system according to the three-dimensional point cloud image.

[0092] Specifically, the second calibration parameter acquisition chip 122 calculates the second calibration parameters of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image transmitted by the image acquisition device 110. The second calibration parameters include the yaw angle and the offset in the second direction and the offset in the third direction.

[0093] Specifically, in the embodiment of the present application, the first direction is the Z-axis direction, the second direction may be the X-axis direction, and the third direction may be the Y-axis direction.

[0094] like Figure 2 As shown, the calibration formula is simplified according to the coordinate system conversion model of each camera unit relative to the calibration plate coordinate system combined with the actual situation of the three-dimensional device (in this embodiment, the three-dimensional device can be a three-dimensional camera, which realizes the acquisition and simultaneous output of three-dimensional data and two-dimensional data). The simplification process is as follows: Considering the limitations of optical principles, the measurement range of the three-dimensional camera in the Z direction is very small, much smaller than the measurement range in the XY direction, and the installation deviation angles of the pitch angle and roll angle are small. Then:

[0095]

[0096]

[0097] Substituting formula (2) and formula (3) into the coordinate system conversion formula (1), the simplified calibration formula is as follows:

[0098]

[0099] According to the simplified calibration formula (4), the calibration process can be divided into two steps. In the first step, the pitch angle θ, roll angle γ and offset △Z in the first direction of each 3D camera relative to the calibration plate coordinate system Ow-XwYwZw are solved in turn, and the Z coordinate benchmark of each 3D camera in the multi-array 3D measurement system can be unified. In the second step, the yaw angle ψ, offset △X in the second direction, and offset △Y in the third direction of each 3D camera relative to the calibration plate coordinate system Ow-XwYwZw are solved in turn, and the XY coordinate benchmark of each 3D camera in the multi-array 3D measurement system can be unified.

[0100] In the embodiments of this application, please refer to Figure 8 Combined with Figure 4 , the step S22 at least includes the following steps.

[0101] S221. Calculate the yaw angle and the offset in the third direction of each camera unit relative to the calibration plate coordinate system according to the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud map.

[0102] Specifically, the second angle calculation circuit 1221 calculates the yaw angle and the offset of the third direction of each camera unit relative to the calibration plate coordinate system according to the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud image transmitted by the image acquisition device 110, and transmits the yaw angle and the offset of the third direction to the second offset calculation circuit 1222. In the embodiment of the present application, the third direction may be the Y-axis direction.

[0103] Specifically, in an embodiment of the present application, the second angle calculation circuit 1221 solves the straight line equations and four preset vertex coordinates of the upper and lower boundaries of the calibration plate in the block corresponding to each camera unit in the second original height map in the three-dimensional point cloud map transmitted by the image acquisition device 110 in turn according to the feature extraction algorithm, and performs corresponding calculations based on the actual tilt placement angle of the calibration plate and the four preset vertex coordinates, and obtains the yaw angle ψ of the line laser from the first camera unit to the last camera unit relative to the calibration plate and the offset △Y in the third direction in turn.

[0104] S222. Calculate an offset of the camera unit in a second direction relative to the calibration plate coordinate system according to the second original height map and the third original height map in the three-dimensional point cloud map.

[0105] Specifically, the second offset calculation circuit 1222 calculates the offset of the camera unit in the second direction relative to the calibration plate coordinate system according to the second original height map and the third original height map in the three-dimensional point cloud image transmitted by the image acquisition device 110, and transmits the obtained offset in the second direction and the yaw angle and the offset in the third direction transmitted by the second angle calculation circuit 1221 to the measurement reference unification device 130. In the embodiment of the present application, the second direction may be the X-axis direction.

[0106] Specifically, in the embodiment of the present application, the second offset calculation circuit 1222 compares and calculates the difference in the same scale of the calibration plate between the second original height map and the third original height map in the three-dimensional point cloud map transmitted by the image acquisition device 110 according to the feature extraction algorithm, and the deviation value at the X starting point position according to the actual preset number of times (for example, twice) of the camera unit, and sequentially calculates and obtains the offset △X of the line laser from the first camera unit to the last camera unit in the multi-array three-dimensional measurement system relative to the calibration plate in the second direction.

[0107] S223: Perform stitching correction on the second original height map according to the offset in the second direction, the yaw angle, and the offset in the third direction to obtain a corresponding stitching correction map.

[0108] Specifically, the height image stitching circuit 1224 stitches and corrects the second original height map according to the offset in the second direction transmitted by the second offset calculation circuit 1222 and the yaw angle and the offset in the third direction transmitted by the second angle calculation circuit 1221 to obtain a corresponding stitching correction image.

[0109] S30, completing the unification of the measurement reference of each image acquisition device 110 in the multi-array three-dimensional measurement system according to the calibration parameters.

[0110] In this example, see Figure 1 The measurement reference unification device 130 completes the measurement reference unification of each image acquisition device 110 in the multi-array three-dimensional measurement system according to the calibration parameters obtained by the calibration parameter acquisition device 120.

[0111] See also Fig. 9 , which is a hardware structure diagram of a splicing calibration device for a multi-array three-dimensional measurement system disclosed in an embodiment of the present application. Fig. 9As shown, the stitching calibration device 200 of the multi-array three-dimensional measurement system provided in the embodiment of the present application includes at least one processor 201 and a memory 202. The stitching calibration device 200 of the multi-array three-dimensional measurement system also includes at least one bus 203. The processor 201 and the memory 202 are electrically connected through the bus 203. The stitching calibration device 200 of the multi-array three-dimensional measurement system can be a computer or a server, which is not particularly limited in the present application.

[0112] The stitching calibration device 200 of the multi-array three-dimensional measurement system may also include the above Figures 1 to 4 In the embodiment shown, the stitching and calibration system of the multi-array three-dimensional measurement system. In the specific implementation process, at least one processor 201 executes the computer-executable instructions stored in the memory 202, so that at least one processor 201 performs the following through the stitching and calibration system of the multi-array three-dimensional measurement system. Figure 5-Figure 8 The embodiment provides a stitching and calibration method for a multi-array three-dimensional measurement system.

[0113] The specific implementation process of the processor 201 provided in the embodiment of the present application can be referred to above Figure 4-Figure 6 The implementation principle and technical effect of the stitching calibration method embodiment of the multi-array three-dimensional measurement system in the above-mentioned embodiment are similar, and will not be described in detail in this embodiment.

[0114] It can be understood that the processor 201 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method provided in the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0115] The memory 202 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM).

[0116] The bus 203 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. For ease of representation, the bus 203 in the drawings of the present application is not limited to only one bus or one type of bus.

[0117] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

Claims

1. A splicing calibration system for a multi-array three-dimensional measurement system. It is characterized in that It includes an image acquisition device, a calibration parameter acquisition device and a measurement reference unification device, wherein the measurement reference unification device is electrically connected to the image acquisition device and the calibration parameter acquisition device, wherein: The image acquisition device is used to scan and acquire multiple three-dimensional point cloud images of the calibration plate, and transmit the multiple three-dimensional point cloud images to the calibration parameter acquisition device; The calibration parameter acquisition device is used to acquire the corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images, and transmit the calibration parameters to the measurement reference unified device; the calibration parameter acquisition device includes a first calibration parameter acquisition chip and a second calibration parameter acquisition chip, wherein the first calibration parameter acquisition chip is electrically connected to the image acquisition device and the measurement reference unified device, and the first calibration parameter acquisition chip is used to calculate the first calibration parameter of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image; the second calibration parameter acquisition chip is electrically connected to the image acquisition device and the measurement reference unified device, and the second calibration parameter acquisition chip is used to calculate the second calibration parameter of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image; The measurement reference unification device is used to complete the measurement reference unification of each image acquisition device in the multi-array three-dimensional measurement system according to the calibration parameters.

2. The stitching calibration system of the multi-array three-dimensional measurement system according to claim 1, It is characterized in that The image acquisition device includes a plurality of camera units, each of which is a three-dimensional camera.

3. The stitching calibration system of the multi-array three-dimensional measurement system according to claim 2, It is characterized in that The first calibration parameter acquisition chip includes a first angle calculation circuit, a height image correction circuit and a first offset calculation circuit, wherein: The first angle calculation circuit is electrically connected to the image acquisition device, and is used to calculate the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system according to the first original height map of the calibration plate in the three-dimensional point cloud image, and transmit the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system to the height image correction circuit; The height image correction circuit is electrically connected to the first angle calculation circuit, and the height image correction circuit is used to correct the first original height map according to the received roll angle and pitch angle to obtain a first corrected height map, and transmit the first corrected height map, the roll angle and the pitch angle to the first offset calculation circuit; The first offset calculation circuit is electrically connected to the height image correction circuit and the measurement reference unification device, and the first offset calculation circuit is used to calculate the offset of each camera unit in a first direction relative to the calibration plate coordinate system based on the first corrected height map, wherein the first calibration parameter includes the roll angle, the pitch angle and the offset in the first direction.

4. The stitching calibration system of the multi-array three-dimensional measurement system according to claim 3, It is characterized in that The first offset calculation circuit is further configured to correct the first corrected height map according to the offset in the first direction to obtain a second corrected height map.

5. The stitching calibration system of the multi-array three-dimensional measurement system according to claim 3, It is characterized in that The second calibration parameter acquisition chip includes a second angle calculation circuit and a second offset calculation circuit, wherein: The second angle calculation circuit is electrically connected to the image acquisition device, and is used to calculate the yaw angle and the offset in the third direction of each camera unit relative to the calibration plate coordinate system according to the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud image; The second offset calculation circuit is electrically connected to the image acquisition device and the measurement reference unification device, and is used to calculate the offset of the camera unit in the second direction relative to the calibration plate coordinate system based on the second original height map and the third original height map in the three-dimensional point cloud map, wherein the second calibration parameters include the yaw angle and the offset in the second direction and the offset in the third direction.

6. The stitching calibration system of the multi-array three-dimensional measurement system according to claim 5, It is characterized in that The second calibration parameter acquisition chip also includes a height image stitching circuit, wherein the height image stitching circuit is electrically connected to the second angle calculation circuit and the second offset calculation circuit, and the height image stitching circuit is used to stitch and correct the second original height map according to the offset in the second direction, the yaw angle and the offset in the third direction to obtain a corresponding stitching correction image.

7. The stitching calibration system of the multi-array three-dimensional measurement system according to claim 5, It is characterized in that The first direction is a Z-axis direction, the second direction may be an X-axis direction, and the third direction may be a Y-axis direction.

8. A method for stitching and calibrating a multi-array three-dimensional measurement system, performed by the stitching and calibration system for the multi-array three-dimensional measurement system according to any one of claims 1 to 7. It is characterized in that The stitching calibration method of the multi-array three-dimensional measurement system comprises: Obtain multiple three-dimensional point cloud images of the calibration plate; Acquire corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images of the calibration plate; The measurement reference of each image acquisition device in the multi-array three-dimensional measurement system is unified according to the calibration parameters.

9. The stitching calibration method of a multi-array three-dimensional measurement system according to claim 8, It is characterized in that The step of obtaining corresponding calibration parameters of the calibration plate according to the multiple three-dimensional point cloud images of the calibration plate includes: Calculate a first calibration parameter of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image; A second calibration parameter of the camera unit relative to the calibration plate coordinate system is calculated according to the three-dimensional point cloud image.

10. The stitching calibration method of a multi-array three-dimensional measurement system according to claim 9, It is characterized in that The first calibration parameter of the camera unit relative to the calibration plate coordinate system is calculated based on the three-dimensional point cloud image, including: Calculate the roll angle and pitch angle of each camera unit relative to the calibration plate coordinate system according to the first original height map of the calibration plate in the three-dimensional point cloud map; Correcting the first original height map according to the roll angle and the pitch angle to obtain a first corrected height map; Calculating an offset of each camera unit relative to a first direction of the calibration plate coordinate system according to the first corrected height map, wherein the first calibration parameter includes the roll angle, the pitch angle, and the offset in the first direction; The first corrected height map is corrected according to the offset in the first direction to obtain a second corrected height map.

11. The method for stitching and calibrating a multi-array three-dimensional measurement system according to claim 10, It is characterized in that The step of calculating the second calibration parameter of the camera unit relative to the calibration plate coordinate system according to the three-dimensional point cloud image includes: Calculate the yaw angle and the offset in the third direction of each camera unit relative to the calibration plate coordinate system according to the second original height map and the third original height map of the calibration plate in the three-dimensional point cloud image; Calculating an offset of the camera unit in a second direction relative to the calibration plate coordinate system according to the second original height map and the third original height map in the three-dimensional point cloud map, wherein the second calibration parameter includes the yaw angle and the offset in the second direction and the offset in the third direction; The second original height map is stitched and corrected according to the offset in the second direction, the yaw angle, and the offset in the third direction to obtain a corresponding stitched and corrected map.

12. A splicing calibration device for a multi-array three-dimensional measurement system, It is characterized in that include: At least one processor and a memory, at least one of the processors executes computer-executable instructions stored in the memory, and at least one of the processors executes the stitching calibration method for a multi-array three-dimensional measurement system as described in any one of claims 8 to 11.

Images