Three-dimensional image reconstruction method and system based on monocular structured light and rotating platform

Abstract

The three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform of the present invention include: projecting a Gray code image on a checkerboard calibration board, and obtaining calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera; projecting the Gray code image on a calibration sphere, the calibration sphere is located on the central axis of the rotating platform, and according to the principle of Gray code structured light, using the calibration parameters between the projector and the camera, obtaining first point cloud data of one side of the calibration sphere; calculating the coordinates of the center of the calibration sphere by spherical fitting according to the first point cloud data, and obtaining the relationship between the camera coordinate system and the rotating platform coordinate system; projecting the Gray code image on an object, rotating the object according to a preset angle and measuring, and obtaining second point cloud data of multiple viewing angles of the measured object, the object being located on the rotating platform and rotating; roughly splicing all the second point cloud data according to the preset angle to obtain third point cloud data; finely splicing the third point cloud data according to the ICP algorithm to obtain fourth point cloud data; and performing surface fitting according to the fourth point cloud data to generate a three-dimensional model.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform. Background Art

[0002] In recent years, computer vision technology has developed rapidly and has become an emerging discipline, playing a great role in real life. Among them, 3D reconstruction is one of the most important research contents and hot research directions of computer vision, and has been widely used in cultural relics protection, reverse engineering, precision measurement, art design and other fields.

[0003] Three-dimensional reconstruction based on vision generally obtains data images of scene objects through cameras, analyzes and processes the images, and then derives the three-dimensional information of objects in the real environment in combination with computer vision knowledge. Three-dimensional reconstruction technology based on vision can generally be divided into passive vision method and active vision method. The former is to perform three-dimensional reconstruction by obtaining the reflected light of external light sources on the surface of objects, such as binocular stereo vision; while the latter requires projecting light sources to objects and receiving return signals to complete three-dimensional reconstruction, such as TOF (Time of Flight) technology and structured light technology. Binocular stereo vision depends on the texture information of the object being measured, is not suitable for scenes with monotonous lack of texture, and has high algorithm complexity; although TOF technology is simple in principle, it depends on the quality of hardware and has low measurement resolution. Structured light technology adds texture features to the surface of the object being measured by projecting a known coded pattern, improves the effect of feature matching, has great advantages in depth resolution and depth measurement accuracy, and has the characteristics of simple structure, low cost, low calculation, high resolution and high accuracy.

[0004] In the prior art three-dimensional image reconstruction method based on monocular structured light and a rotating platform, there is a technical problem that when obtaining complete point cloud data of an object, the solution of pasting markers may cause damage to the surface of the object. Summary of the invention

[0005] The present invention provides a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform, which are used to solve the technical problem that in the prior art three-dimensional image reconstruction method based on monocular structured light and a rotating platform, the solution of using pasting marks to obtain complete point cloud data of an object may cause damage to the surface of the object.

[0006] The present invention provides a three-dimensional image reconstruction method based on monocular structured light and a rotating platform, comprising the following steps:

[0007] Project the Gray code image onto the checkerboard calibration board, and obtain the calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera;

[0008] Projecting a Gray code image onto a calibration sphere, and acquiring first point cloud data of a side surface of the calibration sphere using calibration parameters between a projector and a camera according to the principle of Gray code structured light, wherein the calibration sphere is located on the central axis of the rotating platform;

[0009] According to the first point cloud data, the coordinates of the center of the calibration sphere are calculated by spherical fitting to obtain the relationship between the camera coordinate system and the rotating platform coordinate system;

[0010] Projecting a Gray code image onto an object, rotating the object according to a preset angle and measuring it, and acquiring second point cloud data of multiple viewing angles of the measured object, wherein the object is located on the rotating platform and rotates;

[0011] Roughly stitching all the second point cloud data according to the preset angle to obtain third point cloud data;

[0012] The third point cloud data is precisely spliced ​​according to the ICP algorithm to obtain the fourth point cloud data.

[0013] Surface reconstruction is performed based on the fourth point cloud data to generate a three-dimensional model.

[0014] In some embodiments, the Gray code image is projected onto a checkerboard calibration board, and according to the calibration principle of the projector and the camera, the calibration parameters between the projector and the camera are obtained as follows:

[0015] Projecting a Gray code image onto a checkerboard calibration plate to obtain at least three sets of images;

[0016] According to the at least three groups of images, calibration parameters between the projector and the camera are obtained, wherein the calibration parameters between the projector and the camera include the projector internal parameter K p , camera intrinsic parameter K c , the rotation matrix R between the projector and the camera c←p and the translation vector T between the projector and the camera c←p .

[0017] In some embodiments, the projector internal parameter K p Specifically:

[0018]

[0019] Camera intrinsic parameter K c Specifically:

[0020]

[0021] Projector coordinate system O p and the camera coordinate system O c The relationship function between them is:

[0022]

[0023] In some embodiments, the projecting of the Gray code image onto the calibration sphere to obtain the first point cloud data of one side of the calibration sphere is specifically:

[0024] Projecting the Gray code image onto the calibration sphere to obtain a coded image modulated by the image surface, wherein the code value corresponding to each pixel on the Gray code image is unique;

[0025] Decoding the compiled image, and mapping relationship between projector pixels and camera pixels;

[0026] The calibration parameters and the mapping relationship are calculated according to the triangulation principle to obtain the three-dimensional coordinates of the point corresponding to each pixel of the object on the Gray code image in the camera coordinate system;

[0027] According to the three-dimensional coordinates, first point cloud data of a side surface of the calibration sphere is obtained.

[0028] In some embodiments, the relationship between the camera coordinate system and the rotating platform coordinate system is obtained according to the first point cloud data as follows:

[0029] Fitting the first point cloud data according to the least square method to obtain the coordinates of the center of the calibration sphere in the camera coordinate system;

[0030] According to the coordinates of the center of the calibration sphere in the camera coordinate system, the relationship between the camera coordinate system and the rotating platform coordinate system is obtained, wherein the camera coordinate system and the rotating platform coordinate system are set to maintain the same orientation.

[0031] In some embodiments, the first point cloud data is fitted according to the least squares method to obtain the coordinates of the center of the calibration sphere in the camera coordinate system:

[0032] The coordinates of the first point cloud data in the camera coordinate system are point sets:

[0033] {(x i ,y i , z i )}i=1,2,3...n;

[0034] According to the residual and formula:

[0035]

[0036] The residuals and are used to obtain partial derivatives of the parameters, where the coordinates of the center of the calibration sphere in the camera coordinate system are (a, b, c), and the radius of the calibration sphere is r:

[0037]

[0038] Substitute the coordinates of the first point cloud data in the camera coordinate system into each residual and partial derivative, set the value to 0, and construct the equation group:

[0039]

[0040] Solve the partial derivative equations to obtain the center coordinates (a, b, c) and radius r of the calibration sphere.

[0041] In some embodiments, the relationship between the camera coordinate system and the rotating platform coordinate system is obtained according to the coordinates of the center of the calibration ball in the camera coordinate system as follows:

[0042] According to the coordinates (a, b, c) of the center of the calibration ball in the camera coordinate system, obtain the camera coordinate system O c and the rotating platform coordinate system O t The relationship is:

[0043] An embodiment of the present invention further provides a three-dimensional image reconstruction system based on monocular structured light and a rotating platform, the system comprising:

[0044] A first acquisition module, which is used to project a Gray code image onto a checkerboard calibration board and acquire calibration parameters between the projector and the camera according to a calibration principle of the projector and the camera;

[0045] A second acquisition module, wherein the second acquisition module is used to project a Gray code image onto a calibration sphere, the calibration sphere being located on the central axis of the rotating platform, and obtaining first point cloud data of a side surface of the calibration sphere by using calibration parameters between the projector and the camera according to the principle of Gray code structured light;

[0046] A third acquisition module, the third acquisition module is used to calculate the coordinates of the center of the calibration sphere through spherical fitting according to the first point cloud data, and obtain the relationship between the camera coordinate system and the rotating platform coordinate system;

[0047] a fourth acquisition module, the fourth acquisition module being used to project a Gray code image onto an object, rotate the object according to a preset angle and measure it, and acquire second point cloud data of multiple viewing angles of the measured object, the object being located on the rotating platform and rotating;

[0048] a fifth acquisition module, the fifth acquisition module being used to roughly stitch all the second point cloud data according to the preset angle to acquire third point cloud data;

[0049] A sixth acquisition module, the sixth acquisition module is used to perform fine stitching of the third point cloud data according to the ICP algorithm to obtain fourth point cloud data,

[0050] A generation module is used to perform surface reconstruction and generate a three-dimensional model based on the fourth point cloud data.

[0051] An embodiment of the present invention provides a three-dimensional image reconstruction device based on monocular structured light and a rotating platform, including a processor and a memory;

[0052] The memory is used to store program code and transmit the program code to the processor;

[0053] The processor is used to execute the above-mentioned three-dimensional image reconstruction method based on monocular structured light and a rotating platform according to the instructions in the program code.

[0054] It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

[0055] The embodiment of the present invention provides a three-dimensional image reconstruction method, system and device based on monocular structured light and a rotating platform, the method comprising the following steps: projecting a Gray code image onto a checkerboard calibration board, and obtaining calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera; projecting the Gray code image onto a calibration sphere, and obtaining first point cloud data of a side of the calibration sphere by using the calibration parameters between the projector and the camera according to the principle of Gray code structured light, wherein the calibration sphere is located on the central axis of the rotating platform; calculating the coordinates of the center of the calibration sphere by spherical fitting according to the first point cloud data, and obtaining the relationship between the camera coordinate system and the rotating platform coordinate system; projecting the Gray code image onto an object, rotating the object according to a preset angle and measuring it, and obtaining second point cloud data of multiple perspectives of the measured object, wherein the object is located on the rotating platform and rotates; splicing all the second point cloud data according to the preset angle to obtain third point cloud data; splicing the third point cloud data according to an ICP algorithm to obtain fourth point cloud data; and generating a three-dimensional model according to the fourth point cloud data. The three-dimensional image reconstruction method based on monocular structured light and a rotating platform of the present invention uses a calibration sphere to calibrate the camera and the rotating platform. The calibration process is simple and convenient, and when there is no relative movement between the structured light system and the rotating platform, there is no need to calibrate again. This effectively solves the technical problem in the prior art of the three-dimensional image reconstruction method based on monocular structured light and a rotating platform that the solution of pasting marks to obtain complete point cloud data of the object may cause damage to the surface of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0057] Figure 1 A method flow chart of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0058] Figure 2 A hardware structure diagram of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0059] Figure 3 A Gray code pattern projected by a projector of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0060] Figure 4 A schematic diagram of a Gray code pattern projected onto a checkerboard calibration plate in a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0061] Figure 5 A diagram showing the relationship between the camera coordinate system and the rotating platform coordinate system of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0062] Figure 6 A schematic diagram of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention, in which a Gray code pattern is projected onto a calibration sphere, a point cloud of one side of the calibration sphere is obtained, and a spherical fitting is performed on the point cloud.

[0063] Figure 7 A schematic diagram of a Gray code pattern projected onto an object to be measured by a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0064] Figure 8 A schematic diagram of a point cloud of a measured object according to a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0065] Fig. 9 A schematic diagram of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform for multi-angle measurement of an object under test provided in an embodiment of the present invention.

[0066] Fig.10A schematic diagram of point cloud stitching and surface reconstruction of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0067] Fig.11 A printed structure diagram of a staircase in a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0068] Fig.12 A printed structure diagram of multiple cylinders of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0069] Fig.13 A schematic diagram of three-dimensional reconstruction of various rotating objects by a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0070] Fig.14 A system structure diagram of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention.

[0071] Fig.15 A device framework diagram of a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The embodiments of the present invention provide a three-dimensional image reconstruction method and system based on monocular structured light and a rotating platform, which are used to solve the technical problem that the prior art does not provide a high-concurrency read-write lock strategy when performing data annotation, resulting in high-concurrency read-write in the data annotation service.

[0073] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meanings as those commonly understood by those skilled in the art of the technical field of the embodiments of the present invention. The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0075] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described first. The nouns and terms involved in the embodiments of the present invention are subject to the following explanations.

[0076] There are many commercial measurement systems for 3D reconstruction, such as the ZED camera based on binocular stereo vision, the Kinect camera based on TOF technology, and the Realsense camera based on structured light. However, these products can only obtain point cloud data of one side of the object. How to obtain complete point cloud data on the surface of the measured object is a challenge. To solve this problem, it is generally necessary to measure the object from multiple angles to obtain multiple sets of point cloud data, and finally use point cloud stitching technology to obtain complete point cloud data of the object. In order to achieve point cloud stitching, there are many solutions, such as using a robotic arm to drive the measurement system to measure the object from multiple angles, and relying on the coordinate system of the robotic arm to complete the stitching; you can also stick some marks on the measured object and rely on the marks to complete the point cloud stitching; in addition, you can use a rotating platform to calibrate it to complete the point cloud stitching.

[0077] Reference Figure 1-14 As shown, the present invention provides a three-dimensional image reconstruction method based on monocular structured light and a rotating platform.

[0078] Among them, reference Figure 1 , in one embodiment, the method includes the following steps: projecting a Gray code image onto a checkerboard calibration board, and obtaining calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera; projecting a Gray code image onto a calibration sphere, and obtaining first point cloud data of a side of the calibration sphere according to the principle of Gray code structured light and using the calibration parameters between the projector and the camera, wherein the calibration sphere is located on the central axis of the rotating platform; calculating the coordinates of the center of the calibration sphere by spherical fitting according to the first point cloud data, and obtaining the relationship between the camera coordinate system and the rotating platform coordinate system; projecting the Gray code image onto an object, rotating the object according to a preset angle and measuring it, and obtaining second point cloud data of multiple perspectives of the measured object, wherein the object is located on the rotating platform and rotates; splicing all the second point cloud data according to the preset angle to obtain third point cloud data; splicing the third point cloud data according to the ICP algorithm to obtain fourth point cloud data, and generating a three-dimensional model according to the fourth point cloud data.

[0079] Among them Figure 2 As shown, the hardware device used in this embodiment is composed of a monocular structured light system and a rotating platform. The monocular structured light system is composed of a DLP projector 1 and a monocular industrial camera 2. The object is placed on the rotating platform 3 and is driven by the rotating platform to rotate according to a preset angle. The corresponding multiple groups of images are obtained by shooting with the monocular industrial camera.

[0080] The Gray code image is projected onto the checkerboard calibration board, and the calibration parameters between the projector and the camera are obtained according to the calibration principle of the projector and the camera.

[0081] The present invention adopts Gray code structured light, and the Gray code image is as follows Figure 3 As shown, the first and second images are all-white images and all-black images, the 3rd to 22nd images (total 20 images) are longitudinal Gray code images, and the 23rd to 42nd images (total 20 images) are transverse Gray code images.

[0082] like Figure 4 As shown, the Gray code image is projected onto a checkerboard calibration board. According to the calibration principle of the projector and the camera, the calibration parameters between the projector and the camera are obtained as follows:

[0083] Projecting a Gray code image onto a checkerboard calibration plate to obtain at least three sets of images;

[0084] The projector projects the Gray code image onto the checkerboard calibration plate, and the camera captures at least three groups of Gray code images by changing the placement of the checkerboard at least three times.

[0085] According to the at least three groups of images, calibration parameters between the projector and the camera are obtained, wherein the calibration parameters between the projector and the camera include the projector internal parameter K p , camera intrinsic parameter K c , the rotation matrix R between the projector and the camera c←p and the translation vector T between the projector and the camera c←p .

[0086] The projector internal parameter K p Specifically:

[0087]

[0088] Camera intrinsic parameter K c Specifically:

[0089]

[0090] Projector coordinate system O p and the camera coordinate system O c The relationship function between them is:

[0091]

[0092] Place a 3D printed calibration ball 4 in the center of the rotating platform 3, ensuring that the center of the calibration ball 4 is on the central axis of the rotating platform 3. Figure 5 As shown; a Gray code image is projected onto the calibration ball 4, and the first point cloud data is obtained by forming point cloud data of one side of the object in the above manner.

[0093] The first point cloud data of one side of the calibration sphere 4 is obtained as follows:

[0094] Projecting a Gray code image onto the calibration sphere 4 to obtain a coded image modulated by the image surface, wherein the code value corresponding to each pixel on the Gray code image is unique;

[0095] Decoding the compiled image, wherein the decoding process only requires restoring the Gray code to decimal numbers to obtain a mapping relationship between projector pixels and camera pixels;

[0096] The calibration parameters and the mapping relationship are calculated according to the triangulation principle to obtain the three-dimensional coordinates of the point corresponding to each pixel of the object on the Gray code image in the camera coordinate system;

[0097] According to the three-dimensional coordinates, first point cloud data of one side of the calibration sphere 4 is formed.

[0098] like Figure 6 As shown, the first point cloud data is fitted according to the least squares method to obtain the coordinates of the center of the calibration sphere 4 in the camera coordinate system; according to the coordinates of the center of the calibration sphere 4 in the camera coordinate system, the relationship between the camera coordinate system and the coordinate system of the rotating platform 3 is obtained, wherein the camera coordinate system and the coordinate system of the rotating platform 3 maintain the same orientation. According to the first point cloud data, the coordinates of the center of the calibration sphere are calculated by spherical fitting, and the relationship between the camera coordinate system and the coordinate system of the rotating platform 3 is obtained.

[0099] The first point cloud data is fitted according to the least squares method to obtain the coordinates of the center of the calibration sphere 4 in the camera coordinate system:

[0100] The coordinates of the first point cloud data in the camera coordinate system are point sets:

[0101] {(x i ,y i , z i )}i=1,2,3...n;

[0102] According to the residual and formula, the coordinates of the center of the calibration sphere in the camera coordinate system are (a, b, c), and the radius of the calibration sphere is r:

[0103]

[0104] The residuals and the partial derivatives of the parameters are calculated:

[0105]

[0106] Substitute the coordinates of the first point cloud data in the camera coordinate system into each residual and partial derivative, set the value to 0, and construct the equation group:

[0107]

[0108] Solving the partial derivative equations, we can obtain the center coordinates (a, b, c) and radius r of the calibration sphere 4.

[0109] The relationship between the camera coordinate system and the rotating platform 3 coordinate system is obtained according to the coordinates of the center of the calibration ball 4 in the camera coordinate system:

[0110] According to the coordinates (a, b, c) of the center of the calibration ball 4 in the camera coordinate system, the camera coordinate system O is obtained. c and the rotating platform 3 coordinate system O t The relationship is:

[0111] like Figure 7 As shown, a Gray code image is projected onto an object, the object is rotated and measured according to a preset angle, and second point cloud data of multiple viewing angles of the object are obtained, and the object is located on the rotating platform 3 and rotates;

[0112] By controlling the rotating platform 3 to rotate and shooting with the camera at the same time, images with multiple perspectives are obtained. According to the method of calibrating the first point cloud data of one side of the sphere 4, the first point cloud data corresponding to each image can be obtained, such as Figure 8 As shown;

[0113] Among them, the object is rotated by a preset angle, such as Fig. 9 As shown, for example, the point cloud can be obtained by rotating 30° each time and a rotation of 360° can obtain 12 groups of point clouds. Of course, it can also be 6 groups of 60 degrees or 4 groups of 90 degrees.

[0114] In one embodiment, all the second point cloud data are spliced ​​according to the preset angle to obtain the third point cloud data; according to the conversion relationship between the camera coordinate system and the rotating platform 3 coordinate system, the 12 groups of second point cloud data can be converted from the camera coordinate system to the rotating platform 3 coordinate system.

[0115] According to the rotation angle, the second end cloud data point cloud of any group is used as the reference point cloud data, and the other groups are reversely rotated according to the corresponding preset angles and merged into the reference point cloud data to complete the rough splicing of the point cloud data and obtain the third point cloud data, where the reverse rotation matrix of the point cloud from view j to view 1 is for:

[0116]

[0117] Among them, j=2, 3, 4...12.

[0118] The third point cloud data is spliced ​​according to the ICP algorithm to obtain the fourth point cloud data; after the rough splicing, the third point cloud data obtained can be globally finely spliced ​​using the ICP algorithm to obtain the fourth point cloud data of the object. According to the fourth point cloud data, surface fitting is performed to generate a three-dimensional model. Point cloud splicing and surface reconstruction can refer to Fig.10 shown.

[0119] The three-dimensional image reconstruction method based on monocular structured light and a rotating platform in this embodiment can adopt a relatively simple structure in hardware, and can be completed by only one projector, one industrial camera and one rotating platform. It is low-cost and has good practicality.

[0120] The three-dimensional image reconstruction method based on monocular structured light and a rotating platform in this embodiment performs three-dimensional reconstruction of objects based on a structured light method based on active vision. It does not rely on the texture features of the object's surface and can perform three-dimensional reconstruction of objects with regular and complex surface shapes with high reconstruction accuracy.

[0121] The 3D image reconstruction method based on monocular structured light and rotating platform of this embodiment has a simple and convenient calibration process, and no recalibration is required when there is no relative movement between the structured light system and the rotating platform. This effectively solves the technical problem that the method of pasting markers to obtain complete point cloud data of an object in the prior art 3D image reconstruction method based on monocular structured light and rotating platform may cause damage to the surface of the object.

[0122] Moreover, the above method completes the rough stitching of the second point cloud data according to the preset angle, and then uses the ICP algorithm for fine stitching, and finally realizes the three-dimensional reconstruction of the entire object under test. Therefore, its accuracy is relatively high. After reconstructing each staircase and column through the three-dimensional image reconstruction method based on monocular structured light and rotating platform, the following test accuracy table is obtained:

[0123]

[0124] l1, l2, l3, l4, l5, l6, l7, q1, q2, q3, f1, f2, f3, h1, h2 and h3 are given by Fig.11 and Fig.12 It is knowable.

[0125] It can be seen from the above measurement accuracy table that the three-dimensional image reconstruction method based on monocular structured light and rotating platform can obtain a three-dimensional model with high generation accuracy. In addition, the present invention is applicable to rotatable objects of various shapes, such as Fig.13 shown.

[0126] like Fig.14As shown, an embodiment of the present invention provides a three-dimensional image reconstruction system based on monocular structured light and a rotating platform, and the system includes the following modules:

[0127] A first acquisition module 201, which is used to project a Gray code image onto a checkerboard calibration board, and acquire calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera;

[0128] A second acquisition module 202, the second acquisition module 202 is used to project a Gray code image onto a calibration sphere to acquire first point cloud data of a side of the calibration sphere, wherein the calibration sphere is located on the central axis of the rotating platform;

[0129] A third acquisition module 203, the third acquisition module 203 is used to calculate the coordinates of the center of the calibration sphere through spherical fitting according to the first point cloud data, and obtain the relationship between the camera coordinate system and the rotating platform coordinate system;

[0130] A fourth acquisition module 204, the fourth acquisition module 204 is used to project a Gray code image onto an object, rotate the object according to a preset angle and measure it, and acquire second point cloud data of multiple viewing angles of the measured object, the object being located on the rotating platform and rotating;

[0131] A fifth acquisition module 205, the fifth acquisition module 205 is used to splice all the second point cloud data according to the preset angle to obtain third point cloud data;

[0132] A sixth acquisition module 206, the sixth acquisition module 206 is used to stitch the third point cloud data according to the ICP algorithm to acquire fourth point cloud data;

[0133] The generating module 207 is used to generate a three-dimensional model according to the fourth point cloud data.

[0134] In some embodiments, the step of projecting the Gray code image on the checkerboard calibration board and obtaining the calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera is specifically as follows: projecting the Gray code image on the checkerboard calibration board to obtain at least three sets of images; obtaining the calibration parameters between the projector and the camera according to the at least three sets of images, wherein the calibration parameters between the projector and the camera include the projector intrinsic parameter K p , camera intrinsic parameter K c , the rotation matrix R between the projector and the camera c←p and the translation vector T between the projector and the camera c←p .

[0135] In some embodiments, the projector internal parameter K p Specifically:

[0136]

[0137] Camera intrinsic parameter K c Specifically:

[0138]

[0139] Projector coordinate system O p and the camera coordinate system O c The relationship function between them is:

[0140]

[0141] In some embodiments, the step of projecting the Gray code image onto the calibration sphere and obtaining the first point cloud data of one side of the calibration sphere is specifically as follows: projecting the Gray code image onto the calibration sphere to obtain a coded image modulated on the image surface, wherein the code value corresponding to each pixel on the Gray code image is unique; decoding the compiled image, and obtaining a mapping relationship between projector pixels and camera pixels; then calculating the calibration parameters and the mapping relationship according to the triangulation principle, and obtaining the three-dimensional coordinates of the point corresponding to each pixel of the object on the Gray code image in the camera coordinate system; finally, obtaining the first point cloud data of one side of the calibration sphere according to the three-dimensional coordinates.

[0142] In some embodiments, the step of obtaining the relationship between the camera coordinate system and the rotating platform coordinate system based on the first point cloud data is specifically as follows: fitting the first point cloud data according to the least squares method to obtain the coordinates of the center of the calibration sphere in the camera coordinate system; obtaining the relationship between the camera coordinate system and the rotating platform coordinate system according to the coordinates of the center of the calibration sphere in the camera coordinate system, wherein the camera coordinate system and the rotating platform coordinate system maintain the same orientation.

[0143] In some embodiments, the step of fitting the first point cloud data according to the least squares method to obtain the coordinates of the center of the calibration sphere in the camera coordinate system is specifically:

[0144] Let the coordinates of the first point cloud data in the camera coordinate system be the point set

[0145] {(x i ,y i , z i )}i=1,2,3...n;

[0146] According to the residual and formula

[0147]

[0148] The residuals and are used to obtain partial derivatives of the parameters, where the coordinates of the center of the calibration sphere in the camera coordinate system are (a, b, c), and the radius of the calibration sphere is r:

[0149]

[0150] Substitute the coordinates of the first point cloud data in the camera coordinate system into each residual and partial derivative, set the value to 0, and construct the equation group:

[0151]

[0152] Solve the partial derivative equations to obtain the center coordinates (a, b, c) and radius r of the calibration sphere.

[0153] In some embodiments, the step of obtaining the relationship between the camera coordinate system and the rotating platform coordinate system according to the coordinates of the center of the calibration ball in the camera coordinate system is specifically:

[0154] According to the coordinates (a, b, c) of the center of the calibration ball in the camera coordinate system, obtain the camera coordinate system O c and the rotating platform coordinate system O t The relationship is:

[0155] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0156] like Fig.15 As shown, an embodiment of the present invention further provides a three-dimensional image reconstruction device based on monocular structured light and a rotating platform, the device comprising a processor 300 and a memory 301. The memory 301 is used to store a program code 302 and transmit the program code 302 to the processor; the processor 300 is used to execute the steps in the above-mentioned three-dimensional image reconstruction method based on monocular structured light and a rotating platform according to the instructions in the program code 302.

[0157] Exemplarily, the computer program 302 may be divided into one or more modules / units, which are stored in the memory 301 and executed by the processor 300 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 302 in the terminal device 30.

[0158] The terminal device 30 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 300 and a memory 301. Those skilled in the art will appreciate that Figure 5It is only an example of the terminal device 30 and does not constitute a limitation of the terminal device 30. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0159] The processor 300 may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0160] The memory 301 may be an internal storage unit of the terminal device 30, such as a hard disk or memory of the terminal device 30. The memory 301 may also be an external storage device of the terminal device 30, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal device 30. Further, the memory 301 may also include both an internal storage unit of the terminal device 30 and an external storage device. The memory 301 is used to store the computer program and other programs and data required by the terminal device. The memory 301 may also be used to temporarily store data that has been output or is to be output.

[0161] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0162] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0163] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0166] In the embodiment of the present invention, the client sends a request to the annotation platform server to obtain the annotation task image; after the annotation platform server receives the request, the distributed buffer registers the service with the coordination module;

[0167] After completing the registration service, the coordination module reads and writes the image index information with timestamps in the index library and sends the image index information to the distributed buffer; the distributed buffer feeds back the concurrent access volume to the coordination module;

[0168] The coordination module adjusts and distributes the image index information to the distributed buffer according to the concurrent access volume; the client reads the image index information in the distributed buffer and downloads the annotation task image according to the image index information;

[0169] The client submits the annotation information to the annotation platform server, and the annotation platform server updates the annotation status of the index information of the annotation task image. In the distributed storage of big data, the embodiment of the present invention uses the increase of memory buffer to solve the high-concurrency reading and writing problem, and provides a high-throughput concurrent annotation service method, which solves the technical problem that the prior art does not provide a high-concurrency read-write lock strategy when performing data annotation, resulting in high-concurrency reading and writing in the data annotation service.

[0170] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional image reconstruction method based on monocular structured light and a rotating platform, characterized in that: The method comprises the following steps: A. Project a Gray code image onto a checkerboard calibration board, and obtain calibration parameters between the projector and the camera according to the calibration principle of the projector and the camera; use Gray code structured light, the first and second images in the Gray code image are all-white images and all-black images, the 3rd to 22nd images (a total of 20 images) are longitudinal Gray code images, and the 23rd to 42nd images (a total of 20 images) are transverse Gray code images; Step A includes: Projecting a Gray code image onto a checkerboard calibration plate to obtain at least three groups of images, the projector projects the Gray code image onto the checkerboard calibration plate, and by changing the placement of the checkerboard at least three times, the camera captures at least three groups of Gray code images; According to the at least three groups of images, calibration parameters between the projector and the camera are obtained, wherein the calibration parameters between the projector and the camera include the projector internal parameter K p , camera intrinsic parameter K c , the rotation matrix R between the projector and the camera c←p and the translation vector T between the projector and the camera c←p ; The projector internal parameter K p for: Camera intrinsic parameter K c for: Projector coordinate system O p and the camera coordinate system O c The relationship function between them is: B. Projecting a Gray code image onto a calibration sphere, the calibration sphere being located on the central axis of the rotating platform, and obtaining first point cloud data of one side of the calibration sphere using calibration parameters between the projector and the camera based on the principle of Gray code structured light; Step B includes: Projecting a Gray code image onto a calibration sphere to obtain a coded image modulated by the surface of the calibration sphere, wherein the code value corresponding to each pixel on the Gray code image is unique; Decoding the coded image to obtain a mapping relationship between projector pixels and camera pixels, and then calculating the three-dimensional coordinates of a point corresponding to each pixel of the calibration sphere on the Gray code image in the camera coordinate system by using the triangulation principle according to the calibration parameters, and obtaining first point cloud data of one side of the calibration sphere; C. Calculate the coordinates of the center of the calibration sphere by spherical fitting based on the first point cloud data, and obtain the relationship between the camera coordinate system and the rotating platform coordinate system; Step C includes the following steps: C1. Fit the first point cloud data according to the least squares method to obtain the coordinates of the center of the calibration sphere in the camera coordinate system; C2. According to the coordinates of the center of the calibration ball in the camera coordinate system, obtain the relationship between the camera coordinate system and the rotating platform coordinate system, wherein the camera coordinate system and the rotating platform coordinate system are set to maintain the same orientation; Step C1 includes: Mark the coordinates of the first point cloud data in the camera coordinate system as a point set: {(x i ,y i ,z i )}i=1,2,3…n; According to the residual and formula, the coordinates of the center of the calibration sphere in the camera coordinate system are (a, b, c), and the radius of the calibration sphere is r: The residuals and the partial derivatives of the parameters are calculated: Substitute the coordinates of the first point cloud data in the camera coordinate system into each residual and partial derivative, set the value to 0, and construct the equation group: Solve the partial derivative equations to obtain the center coordinates (a, b, c) and radius r of the calibration sphere; Step C2 includes: According to the coordinates (a, b, c) of the center of the calibration ball in the camera coordinate system, obtain the camera coordinate system O c and the rotating platform coordinate system O t The relationship is: D. Projecting a Gray code image onto an object, rotating the object according to a preset angle and measuring the object, and acquiring second point cloud data of multiple viewing angles of the object, wherein the object is located on the rotating platform and rotates; By controlling the rotating platform to rotate and shooting with the camera at the same time, images with multiple perspectives are obtained, and second point cloud data corresponding to each image can be obtained according to the method of calibrating the first point cloud data of one side of the sphere, wherein the object is rotated by a preset angle, 30° each time, and 12 groups of point clouds can be obtained by rotating 360° in one circle; E. roughly stitching all the second point cloud data according to the preset angle to obtain third point cloud data; According to the conversion relationship between the camera coordinate system and the rotating platform coordinate system, the 12 groups of second point cloud data are converted from the camera coordinate system to the rotating platform coordinate system. According to the rotation angle, the second end cloud data point cloud of any group is used as the reference point cloud data, and the other groups are reversely rotated according to the corresponding preset angles and merged into the reference point cloud data to complete the rough splicing of the point cloud data and obtain the third point cloud data, where the reverse rotation matrix of the point cloud from view j to view 1 is for: Where, j = 2, 3, 4…12; F. Perform precise stitching of the third point cloud data according to the ICP algorithm to obtain fourth point cloud data; After rough stitching, the third point cloud data obtained is subjected to a global fine stitching using the ICP algorithm to obtain the fourth point cloud data of the object; G. Performing surface fitting based on the fourth point cloud data to generate a three-dimensional model, specifically, performing surface reconstruction based on the fourth point cloud data to generate a three-dimensional model.

2. A three-dimensional image reconstruction system based on monocular structured light and a rotating platform, used to implement the three-dimensional image reconstruction method based on monocular structured light and a rotating platform as claimed in claim 1, characterized in that: The system comprises: A first acquisition module, which is used to project a Gray code image onto a checkerboard calibration board and acquire calibration parameters between the projector and the camera according to a calibration principle of the projector and the camera; A second acquisition module, the second acquisition module is used to project a Gray code image onto a calibration sphere, the calibration sphere is located on the central axis of the rotating platform, and acquire first point cloud data of a side surface of the calibration sphere by using calibration parameters between the projector and the camera according to the principle of Gray code structured light; A third acquisition module, the third acquisition module is used to calculate the coordinates of the center of the calibration sphere through spherical fitting according to the first point cloud data, and obtain the relationship between the camera coordinate system and the rotating platform coordinate system; a fourth acquisition module, the fourth acquisition module being used to project a Gray code image onto an object, rotate the object according to a preset angle and measure it, and acquire second point cloud data of multiple viewing angles of the measured object, the object being located on the rotating platform and rotating; a fifth acquisition module, the fifth acquisition module being used to roughly stitch all the second point cloud data according to the preset angle to acquire third point cloud data; A sixth acquisition module, the sixth acquisition module is used to perform fine stitching of the third point cloud data according to the ICP algorithm to obtain fourth point cloud data, A generation module is used to perform surface reconstruction and generate a three-dimensional model based on the fourth point cloud data.

3. A three-dimensional image reconstruction device based on monocular structured light and a rotating platform, characterized in that: including a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the three-dimensional image reconstruction method based on monocular structured light and a rotating platform according to claim 1 according to the instructions in the program code.

Images