Projection Method and System of 3D Scanning Projection Lines Based on Structured Light

Abstract

Regarding the technical field of three-dimensional information reconstruction, in three-dimensional measurement, performing periodic switching or random switching of different sets of projection images so as to collect a plurality of sets of consecutive projection images (S101); projecting a plurality of images of a set of projection images onto the surface of an object to be measured (S102); imaging the projection images projected onto the surface of the object to be measured to obtain an imaged image (S103); and decrypting the imaged image and determining the correspondence between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the object to be measured (S104). A projection method of structured-light-based three-dimensional scanning projection lines including the above steps.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional information reconstruction technology, and more particularly to a method and system for projecting a three-dimensional scanning projection line based on structured light.

Background Art

[0002] In fields such as industrial control and medical treatment, there are many technologies for reconstructing three-dimensional information on the surface of an object. Among them, the lattice stripe projection method based on structured light has been widely studied and applied due to its advantages such as high precision, high efficiency, and high robustness.

[0003] Such a method mainly projects specific encoded lattice stripes as a projection image of structured light onto the surface of an object to be measured by a projection device, and then captures the structured light modulated and reflected by the object surface by an image sensor, such as a camera, to obtain an imaging image, decodes the imaging image, and finds the correspondence between each imaging position and the projection position in the decoded pattern. Since the position where the encoded pattern is projected onto the object is known, based on the above correspondence, the three-dimensional information of the surface of the object can be obtained by the triangulation method.

[0004] A commonly used structured light projection method is time encoding. The time encoding method projects a plurality of patterns into the measurement space in chronological order, and images are formed by a camera for each projection. By performing a certain decoding on these imaged stripe images, the three-dimensional information of the surface of the object to be measured projected can be obtained. A set of continuously projected images that can reconstruct the three-dimensional information of the surface of the object to be measured in this way is called a set of images.

[0005] A commonly used projection method designs a set of multiple projection images that can stably reconstruct the three-dimensional information of the surface of the object to be measured, and periodically projects the set of certain images designed using a projection device according to a certain time series.

[0006] Assume that as shown in FIG. 3, a set of projection images capable of stably reconstructing the three-dimensional information of the surface of the object to be measured is designed. There are a total of five images in this set, which are encoded as P1, P2, P3, P4, and P5 in chronological order. A common projection method is to periodically project the designed set of five fixed images. However, in such a commonly used projection method, since the projection images remain constant, differential design cannot be performed according to the scene to be measured, and the projection imaging quality of a specific scene cannot be optimized.

Summary of the Invention

[0007] An object of the present invention is to provide a projection method and system for three-dimensional scanning projection lines based on structured light to solve the above problems.

[0008] The technical solution according to the present invention is as follows.

[0009] The present invention is a projection method for three-dimensional scanning projection lines based on structured light, comprising: performing periodic switching or random switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images in three-dimensional measurement; projecting a plurality of images of the set of projection images onto the surface of the object to be measured; imaging the projection images projected onto the surface of the object to be measured to obtain an imaging image; decoding the imaging image so as to reconstruct the three-dimensional information of the surface of the object to be measured, and determining the correspondence between each imaging position and the projection position; A projection method for three-dimensional scanning projection lines based on structured light is provided, including the above steps.

[0010] In some embodiments, it further includes determining whether two sets of projection image sets are different sets of projection images. Specifically, based on the number of projection images, the position of the stripes, or the width of the stripes in the two sets of projection images, determining whether the two sets of projection images are the same; and when the number of projection images, the position of the stripes, or the width of the stripes in the two sets of projection images are different, determining that the two sets of projection images are different. The different sets of projection images include the minimum set of projection images necessary to achieve a three-dimensional reconstruction of one frame.

[0011] In some embodiments, in the three-dimensional measurement, periodically switching different sets of projection images to collect a plurality of consecutive sets of projection images further includes setting the time series of a plurality of different sets of projection images to a periodic sequence so as to switch between different sets of projection images.

[0012] In some embodiments, in the three-dimensional measurement, randomly switching different sets of projection images to collect a plurality of consecutive sets of projection images further includes setting the time series of a plurality of different sets of projection images to a random sequence so as to switch between different sets of projection images.

[0013] In some embodiments, decrypting the imaging image and determining the correspondence between each imaging position and the projection position includes: obtaining an initial correspondence between the imaging point and the projection position based on stripes having a first density; and obtaining a target correspondence between the imaging point and the projection position of stripes having a second density using the initial correspondence, where the first density is smaller than the second density.

[0014] A projection system for a structured light-based three-dimensional scanning projection line, comprising: In three-dimensional measurement, a projection device for periodically or randomly switching different sets of projection images to collect a plurality of consecutive sets of projection images and projecting a plurality of images of the sets of projection images onto the surface of an object to be measured; An image sensor for imaging the projection images projected onto the surface of the object to be measured and obtaining an imaging image; A reconstruction unit for decrypting the imaging image and determining the correspondence between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the object to be measured. A projection system for a structured light-based three-dimensional scanning projection line.

[0015] In some embodiments, the projection device is used to determine whether two sets of projected image sets are different sets of projected images. Specifically, Based on the number, stripe position, or stripe width of two sets of projected images, determine whether the two sets of projected images are the same, If the number, stripe position, or stripe width of the two sets of projected images are different, it is determined that the two sets of projected images are different, including The different sets of projected images include a minimum set of projected images necessary to realize the three-dimensional reconstruction of one frame.

[0016] In some embodiments, the projection device is used to set the time series of a plurality of different sets of projected images into a periodic sequence so as to switch between different sets of projected images.

[0017] In some embodiments, the projection device is used to set the time series of a plurality of different sets of projected images into a random sequence so as to switch between different sets of projected images.

[0018] In some embodiments, the reconstruction unit Obtain an initial correspondence relationship between the imaging point and the projection position based on the stripe having the first density, It is used to obtain the target correspondence relationship between the imaging point and the projection position of the stripe having the second density by using the initial correspondence relationship, The first density is smaller than the second density.

[0019] The three-dimensional scanning projection line projection method and system based on structured light provided by the present invention have at least the following beneficial effects.

[0020] 1. The present invention can design a plurality of different sets of projected images that can correspond to different scenes and can better reconstruct the three-dimensional information of the surface of the object to be measured in different scenes, so as to achieve the optimal design of the projection quality in different scenes and improve the three-dimensional reconstruction quality of the surface in different scenes.

[0021] 2. The present invention can quickly and automatically adapt to changes in the object to be measured by periodically or randomly switching the projection image set for reconstructing the three-dimensional information on the surface of the object to be measured, and can greatly improve the robustness of three-dimensional reconstruction.

[0022] Hereinafter, for the sake of clarity, preferred embodiments will be described with reference to the drawings, and the above characteristics, technical features, advantages and their implementation forms of the projection method and system of the three-dimensional scanning projection line based on structured light will be further described.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Modes for Carrying Out the Invention

[0024] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system configurations and technologies are presented to fully understand the embodiments of the present application. However, it should be understood by those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted so as not to impede the description of the present application.

[0025] As used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or assemblies, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, assemblies and / or sets.

[0026] To simplify the drawings, only the parts related to the present invention are schematically shown in each figure, and the actual configuration of the product is not shown. Also, to simplify and clarify the drawings, in some figures, members having the same configuration or function are schematically shown only for one of them, or only one of them is labeled. In the present text, "one" can represent not only "only this one" but also "more than one" case.

[0027] The term "and / or" used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and it should be further understood that these combinations are included.

[0028] Also, in the description of the present application, terms such as "first", "second", etc. are merely for distinction and are not understood to indicate or imply relative importance.

[0029] To more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the specific embodiments of the present invention will be described below with reference to the drawings. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained based on these drawings without creative labor, and other embodiments can be obtained.

[0030] In one embodiment, as shown in FIG. 1, the present invention provides a method for projecting a three-dimensional scanning projection line based on structured light, and the projection method includes: Step S101: In three-dimensional measurement, performing periodic switching or random switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images.

[0031] During the entire three-dimensional measurement process, different sets of projection images may be switched periodically or randomly.

[0032] 1. Periodic switching means that, for example, there are a total of three sets of projection images G1, G2, and G3, and the projection order is periodic: G1, G2, G3, G1, G2, G3, G1, G2, G3... 2. Random switching means that, for example, there are a total of three sets of projection images G1, G2, and G3, and the projection order is random: G1, G3, G1, G2, G3, G3, G1, G2, G3...

[0033] A set of projection images is a plurality of projection images that satisfy any of the following conditions. 1) They are consecutive projection images where the time interval between adjacent projection images within a certain time is relatively small. Hereinafter, different images projected as Px (x is a number) are represented, and - represents one time interval unit. A general projection interval can be as follows. P1 - P2 - P3 - P4----------P5 - P6 - P3 - P4----------P1 - P2 - P3 - P4---------- P1 - P2 - P3 - P4 may be defined as one set of projection images, and P5 - P6 - P3 - P4 may be defined as one set of images.

[0034] 2) The minimum set of projection images necessary to achieve a three-dimensional reconstruction of one frame. The minimum set of projection images necessary to achieve a three-dimensional reconstruction of one frame is A set of projected images includes a decoded image and a reconstructed image, but is not limited thereto, and includes the fact that in principle, the decoding of the stripe phase of the reconstructed image can be realized.

[0035] Hereinafter, different images projected as Px (x is a number) are represented, and - represents one time interval unit. One relatively extreme projection interval can be as follows. P1 - P2 - P3 - P4 - P5 - P6 - P3 - P4----------P1 - P2 - P3 - P4---------- According to the time interval, P1 - P2 - P3 - P4 is a set of images, and P1 - P2 - P3 - P4 - P5 - P6 - P3 - P4 is also a set of images. However, if P1 - P2 - P3 - P4 can complete a single 3D reconstruction alone, and P5 - P6 - P3 - P4 can also complete a single 3D reconstruction alone, then P1 - P2 - P3 - P4 may be defined as a set of images, and the position P5 - P6 - P3 - P4 may also be defined as a set of images.

[0036] Regarding the judgment criteria, by performing theoretical analysis and estimation according to the principles of structured light encoding and decoding, if 3D reconstruction is possible, it may be defined as a set of images.

[0037] S102 is to project a plurality of images in the set of projection images onto the surface of the object to be measured.

[0038] S103 is to image the projection image projected onto the surface of the object to be measured and obtain an imaged image.

[0039] S104 is to decode the imaged image so as to reconstruct the 3D information on the surface of the object to be measured and determine the correspondence between each imaged position and the projection position.

[0040] The present invention can adaptively convert projection fringes in different measurement scenes, reduce the dependence on the fringe quality of the measurement scene, obtain good 3D point reconstruction in various rapidly changing measurement scenes, and improve robustness.

[0041] In one embodiment, it further includes determining whether two sets of projection image sets are different projection image sets. Specifically, judging whether two sets of projection images are the same based on the number, position or width of the fringes of the two sets of projection images, and including, when the number, position or width of the fringes of the two sets of projection images are different, determining that the two sets of projection images are different. The different sets of projection images include the minimum set of projection images necessary to achieve a three-dimensional reconstruction of one frame.

[0042] For two sets of projection images to be the same means that the two sets of projection images simultaneously satisfy the following two conditions. 1). The number of projection images is the same. 2). Sort the two sets of projection images in chronological order of the projection images, and the positions of the stripes (bright stripes or dark stripes) of the projection images with corresponding numbers are the same, and the widths of the stripes are the same.

[0043] If the above two conditions are not met, it is considered that there is a difference between the two sets of projection images.

[0044] In one embodiment, in the three-dimensional measurement, performing a periodic switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images further includes setting a time series of a plurality of different sets of projection images to a periodic sequence so as to perform a switching of different sets of projection images.

[0045] Among a plurality of consecutive sets of projection images, there are differences between some two sets or a plurality of sets among them, but when viewed from a larger time series, such consecutive sets of projection images have a certain periodicity.

[0046] For two sets of projection images to be the same means that the two sets of projection images simultaneously satisfy the following two conditions. 1). The number of projection images is the same. 2). Sort the two sets of projection images in chronological order of the projection images, and the positions of the stripes (bright stripes or dark stripes) of the projection images with corresponding numbers are the same, and the widths of the stripes are the same.

[0047] If the above two conditions are not met, it is considered that there is a difference between the two sets of projection images.

[0048] In one embodiment, in the three-dimensional measurement, performing a random switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images Further including setting the time series of a plurality of different sets of projection images to a random sequence so as to perform switching between different sets of projection images.

[0049] Among a plurality of consecutive sets of projection images, there are differences between some two sets or multiple sets of them. However, looking from a larger time series, such consecutive sets of projection images are random.

[0050] In one embodiment, decoding the imaging image and determining the correspondence between each imaging position and the projection position includes: Obtaining an initial correspondence between the imaging points and the projection positions based on stripes having a first density; Using the initial correspondence to obtain a target correspondence between the imaging points and the projection positions of stripes having a second density, wherein the first density is smaller than the second density.

[0051] The decoding step is not limited thereto. First, a relatively sparse stripe can be used to obtain a relatively reliable and accurate correspondence between the imaging points and the projection positions, and using this as a reference condition, the correspondence between the imaging points and the projection positions of denser stripes can be obtained step by step.

[0052] In one embodiment, the present invention provides a projection system for a three-dimensional scanning projection line based on structured light. As shown in FIG. 2, the projection system includes: In three-dimensional measurement, a projection device 100 for performing periodic switching or random switching between different sets of projection images so as to collect a plurality of consecutive sets of projection images, and projecting a plurality of images of the set of projection images onto the surface of the object to be measured; An image sensor 200 for imaging the projection images projected onto the surface of the object to be measured and obtaining an imaging image; A reconstruction unit 300 for decoding the imaging image and determining the correspondence between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the object to be measured.

[0053] In one embodiment, the present invention provides a system for reconstructing three-dimensional information, comprising a projection device, an image sensor, and a reconstruction unit.

[0054] The projection device projects a plurality of images of the projection image set onto the surface of the object to be measured in chronological order by a projection method based on structured light for reconstructing the three-dimensional information.

[0055] The image sensor forms an image of the projection image projected onto the surface of the object to be measured and acquires an image of the formed image.

[0056] The reconstruction unit decrypts the image of the formed image, determines the correspondence between each position of the formed image and the projection position, and reconstructs the three-dimensional information of the surface of the object to be measured based on the correspondence.

[0057] The decryption step can first obtain the correspondence between the relatively reliable and accurate image formation points and the projection positions using relatively sparse stripes, and use it as a reference condition to gradually obtain the correspondence between the image formation points and the projection positions of denser stripes, but is not limited thereto.

[0058] Reconstruction step: 1. Calculate the correspondence between the position of the formed image and the projection position. 2. Calculate the three-dimensional coordinate position based on the preset parameters and the correspondence obtained in the first step.

[0059] In some embodiments, the projection device is used to determine whether two sets of projection image sets are different projection image sets. Specifically, Based on the number of two sets of projection images, the position of the stripes, or the width of the stripes, determine whether the two sets of projection images are the same. When the number of two sets of projection images, the position of the stripes, or the width of the stripes are different, determine that the two sets of projection images are different.

[0060] In one embodiment, the projection device is used to set the time series of a plurality of different sets of projection images into a periodic sequence so as to switch between different sets of projection images.

[0061] In one embodiment, the projection device is used to set the time series of a plurality of different sets of projection images into a random sequence so as to switch between different sets of projection images.

[0062] In one embodiment, the reconstruction unit is used to obtain an initial correspondence relationship between the imaging points and the projection positions based on the stripes having a first density, and to obtain a target correspondence relationship between the imaging points and the projection positions of the stripes having a second density by using the initial correspondence relationship, wherein the first density is smaller than the second density.

[0063] The present invention designs a plurality of different sets of projection images that can correspond to different scenes and can better reconstruct the three-dimensional information of the surface of the object to be measured in different scenes, thereby achieving an optimized design of the projection quality in different scenes and improving the three-dimensional reconstruction quality of the surface in different scenes.

[0064] The present invention can quickly and automatically adapt to changes in the object to be measured by periodically or randomly switching the set of projection images for reconstructing the three-dimensional information of the surface of the object to be measured, and greatly improves the robustness of three-dimensional reconstruction.

[0065] As can be understood by those skilled in the art, for the convenience and brevity of the description, only the division of each of the above program modules is taken as an example for description. In actual applications, the above functions can be assigned as needed and completed by different program modules. That is, the internal structure of the above device can be divided into different program units or modules to complete all or part of the functions described above. Each program module in the embodiments may be integrated into one processing unit, or each unit may physically exist independently, or two or more units may be integrated into one processing unit. The above integrated unit may be realized in the form of hardware or in the form of a software program unit. Also, the specific names of each program module are only for distinguishing from each other and not for limiting the protection scope of this application.

[0066] In the above embodiments, the descriptions of each embodiment have their respective focuses. For parts not described in detail or described in a certain embodiment, the related descriptions of other embodiments can be referred to.

[0067] Those skilled in the art can recognize that the units and algorithm steps of each example described in the embodiments disclosed in this specification can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Those skilled in the art can use different methods to realize the functions described for each specific application, but such realization should not be regarded as exceeding the scope of this application.

[0068] In the embodiments provided by this application, it should be understood that the disclosed devices and methods may be implemented in other ways. Exemplarily, the embodiments of the devices described above are merely schematic and exemplary, and the division of the above modules or units is only a logical function division, and there may be other division methods in actual implementation. Exemplarily, a plurality of units or assemblies may be combined, or integrated into another system, or some features may be ignored or not executed. In another aspect, the couplings, direct couplings, or communication connections shown or discussed among each other may be indirect couplings or communication connections through some interfaces, devices, or units, and may be in electrical, mechanical, or other forms.

[0069] The units described as the separation means may or may not be physically separated. The members shown as units may or may not be physical units, that is, they may be located in one place, or distributed among a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the solution means of this embodiment.

[0070] Also, each functional unit in each embodiment of this application may be integrated into one processing unit, or each unit may physically exist alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0071] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred embodiments of the present invention. For those skilled in the art, without departing from the principle of the present invention, some improvements and modifications can be made, and these improvements and modifications should also be regarded as within the protection scope of the present invention.

Claims

1. A method for projecting a three-dimensional scanning projection line based on structured light, comprising: in three-dimensional measurement, performing periodic switching or random switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images; projecting a plurality of images of the set of projection images onto the surface of the object to be measured; imaging the projection images projected onto the surface of the object to be measured to obtain an imaged image; decoding the imaged image so as to reconstruct three-dimensional information of the surface of the object to be measured, and determining a correspondence between each imaged position and the projection position.

2. further comprising determining whether two sets of projection image sets are different sets of projection images. Specifically, determining whether two sets of projection images are the same based on the number of projection images, the position of stripes, or the width of stripes in the two sets of projection images; when the number of projection images, the position of stripes, or the width of stripes in the two sets of projection images are different, determining that the two sets of projection images are different. The different sets of projection images include a minimum set of projection images required to achieve three-dimensional reconstruction of one frame.

3. In the three-dimensional measurement, performing periodic switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images further includes setting a time series of a plurality of different sets of projection images as a periodic sequence so as to perform switching between different sets of projection images.

4. In the three-dimensional measurement, performing random switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images further includes setting a time series of a plurality of different sets of projection images as a random sequence so as to perform switching between different sets of projection images.

5. Decoding the imaged image and determining the correspondence between each imaged position and the projection position includes: obtaining an initial correspondence between an imaged point and a projection position based on stripes having a first density; Obtaining a target correspondence relationship between the imaging points of stripes having a second density and the projection positions by using the initial correspondence relationship, wherein the first density is smaller than the second density, and the projection method of the structured light-based three-dimensional scanning projection line according to any one of claims 1 to 4 is characterized thereby.

6. A projection system of a structured light-based three-dimensional scanning projection line, comprising: In three-dimensional measurement, a projection device for performing periodic switching or random switching of different sets of projection images so as to collect a plurality of consecutive sets of projection images, and projecting a plurality of images of the set of projection images onto the surface of an object to be measured; An image sensor for imaging the projection images projected onto the surface of the object to be measured and acquiring an imaged image; A reconstruction unit for decoding the imaged image and determining a correspondence relationship between each imaging position and the projection position so as to reconstruct three-dimensional information on the surface of the object to be measured, and the projection system of the structured light-based three-dimensional scanning projection line is characterized thereby.

7. The projection device is used for determining whether two sets of projection images are different sets of projection images. Specifically, Determining whether two sets of projection images are the same based on the number of projection images, stripe positions, or stripe widths of the two sets of projection images; When the number of projection images, stripe positions, or stripe widths of the two sets of projection images are different, determining that the two sets of projection images are different, and different sets of the projection images include a minimum set of projection images necessary for realizing three-dimensional reconstruction of one frame, and the projection system of the structured light-based three-dimensional scanning projection line according to claim 6 is characterized thereby.

8. The projection device is used for setting a time series of a plurality of different sets of projection images into a periodic sequence so as to perform switching of different sets of projection images, and the projection system of the structured light-based three-dimensional scanning projection line according to claim 7 is characterized thereby.

9. The projection device is used for setting a time series of a plurality of different sets of projection images into a random sequence so as to perform switching of different sets of projection images, and the projection system of the structured light-based three-dimensional scanning projection line according to claim 8 is characterized thereby.

10. ​ The reconstruction unit is used to obtain an initial correspondence relationship between an imaging point and a projection position based on stripes having a first density, and to obtain a target correspondence relationship between an imaging point and a projection position of stripes having a second density by using the initial correspondence relationship, wherein the first density is smaller than the second density. A projection system for a three-dimensional scanning projection line based on structured light according to any one of claims 6 to 9, characterized by the above.

Images