A projection method and system for three-dimensional scanning projection lines based on structured light

Abstract

The present invention relates to the technical field of three-dimensional information reconstruction, and provides a projection method for three-dimensional scanning projection lines based on structured light, comprising: in three-dimensional measurement, periodically switching or randomly switching different projection image groups to collect multiple continuous projection image groups; projecting several images of the projection image groups onto the surface of the object to be measured; imaging the projection images projected onto the surface of the object to be measured to obtain imaging images; decoding the imaging images to determine the corresponding relationship between each imaging position and the projection position, so as to reconstruct the three-dimensional information of the surface of the object to be measured.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional information reconstruction, and in particular to a projection method and system for three-dimensional scanning projection lines based on structured light. Background Art

[0002] In the fields of industrial control, medical treatment, etc., many technologies have been used to reconstruct the three-dimensional information of the surface of objects. Among them, the grating fringe projection method based on structured light has been widely studied and applied due to its advantages such as high precision, high efficiency and high robustness.

[0003] This type of method mainly uses a projection device to project specific coded grating stripes as a projection image of structured light onto the surface of the object to be measured, and then uses an image sensor, such as a camera, to capture the structured light reflected after being modulated by the object surface to obtain an imaging image, decode the imaging image, and find the correspondence between each imaging position and the projection position in the decoded pattern; since the position where the coded pattern is projected onto the object is known, the three-dimensional information of the object surface can be obtained by using triangulation based on the aforementioned correspondence.

[0004] A commonly used structured light projection method is time encoding. The time encoding method projects multiple patterns into the measured space in time sequence, and each projection is imaged by a camera. By decoding these imaged fringe images, the three-dimensional surface information of the projected object can be obtained. The continuous projection of several images that can reconstruct the three-dimensional surface information of the object is called a group of images.

[0005] A commonly used projection method is to design a set of projection images that can stably reconstruct the three-dimensional information of the surface of the object being measured, and use a projection device to periodically project the designed set of fixed images in a fixed time sequence.

[0006] like Figure 3 As shown in the figure, it is assumed that a set of projection images that can stably reconstruct the three-dimensional information of the surface of the object under test is designed. There are 5 images in this set, which are encoded as P1, P2, P3, P4, and P5 in time sequence. The commonly used projection method is to periodically project this set of 5 fixed images. The limitation of this commonly used projection method is that since the projection image is fixed, it is impossible to perform differentiated design according to the scene under test, and it is impossible to optimize the projection imaging quality of a specific scene. Summary of the invention

[0007] The purpose of the present invention is to provide a projection method and system for three-dimensional scanning projection lines based on structured light to solve the above problems.

[0008] The technical solution provided by the present invention is as follows:

[0009] The present invention provides a projection method of three-dimensional scanning projection lines based on structured light, comprising:

[0010] In three-dimensional measurement, different projection image groups are periodically switched or randomly switched to acquire multiple continuous projection image groups;

[0011] Projecting a plurality of images of the projection image group onto the surface of the object to be measured;

[0012] Imaging the projection image projected onto the surface of the object to be measured to obtain an imaging image;

[0013] The imaging image is decoded to determine the corresponding relationship between each imaging position and the projection position, so as to reconstruct the three-dimensional information of the surface of the measured object.

[0014] In some embodiments, the method further includes: determining whether the two projection image groups are different projection image groups, specifically including:

[0015] Determining whether the two sets of projection images are the same based on the number or stripe positions or stripe widths of the two sets of projection images;

[0016] When the number or the stripe positions or the stripe widths of the two sets of projection images are different, it is determined that the two sets of projection images are different;

[0017] The different projection image groups include a minimum set of projection images required to achieve one frame of three-dimensional reconstruction.

[0018] In some embodiments, in the three-dimensional measurement, periodic switching of different projection image groups is performed to collect multiple continuous projection image groups, and further includes:

[0019] The time sequence of a plurality of different projection image groups is set as a periodic sequence to switch between different projection groups.

[0020] In some embodiments, in the three-dimensional measurement, performing random switching of different projection image groups to collect multiple continuous projection image groups includes:

[0021] The time sequence of a plurality of different projection image groups is set as a random sequence to switch between different projection groups.

[0022] In some embodiments, decoding the imaged image to determine the correspondence between each imaging position and the projection position includes:

[0023] Acquire an initial correspondence between an imaging point and a projection position based on first density fringes;

[0024] Using the initial correspondence relationship, a target correspondence relationship between the imaging point and the projection position of the second density fringes is obtained;

[0025] The first density is smaller than the second density.

[0026] A projection system for three-dimensional scanning projection lines based on structured light, comprising:

[0027] A projection device, used for periodically or randomly switching different projection image groups in three-dimensional measurement to collect multiple continuous projection image groups; projecting several images of the projection image group onto the surface of the object to be measured;

[0028] An image sensor is used to image the projection image projected onto the surface of the object to be measured to obtain an imaging image;

[0029] The reconstruction unit is used to decode the imaging image and determine the corresponding relationship between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the measured object.

[0030] In some embodiments, the projecting device is further configured to determine whether two projection image groups are different projection image groups, specifically comprising:

[0031] Determining whether the two sets of projection images are the same based on the number or stripe positions or stripe widths of the two sets of projection images;

[0032] When the number or the stripe positions or the stripe widths of the two sets of projection images are different, it is determined that the two sets of projection images are different;

[0033] The different projection image groups include a minimum set of projection images required to achieve one frame of three-dimensional reconstruction.

[0034] In some embodiments, the projection device is used to:

[0035] The time sequence of a plurality of different projection image groups is set as a periodic sequence to switch between different projection groups.

[0036] In some embodiments, the projection device is used to:

[0037] The time sequence of a plurality of different projection image groups is set as a random sequence to switch between different projection groups.

[0038] In some embodiments, the reconstruction unit is used to:

[0039] Acquire an initial correspondence between an imaging point and a projection position based on first density fringes;

[0040] Using the initial correspondence relationship, a target correspondence relationship between the imaging point and the projection position of the second density fringes is obtained;

[0041] The first density is smaller than the second density.

[0042] The present invention provides a three-dimensional scanning projection line projection method and system based on structured light, which has at least the following beneficial effects:

[0043] 1. The present invention can design multiple different projection image groups that can cope with different scenes and can be used to better reconstruct the three-dimensional information of the surface of the measured object in different scenes, thereby achieving the optimal design of the projection quality in different scenes, thereby improving the surface three-dimensional reconstruction quality of different scenes.

[0044] 2. The present invention can quickly and automatically adapt to changes in the object being measured by periodically or randomly switching the projection image group used to reconstruct the three-dimensional information of the surface of the object being measured, thereby greatly improving the robustness of the three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The following will explain the preferred implementation mode in a clear and understandable manner in combination with the accompanying drawings, and further explain the above-mentioned characteristics, technical features, advantages and implementation methods of a projection method and system for three-dimensional scanning projection lines based on structured light.

[0046] Figure 1 is a schematic diagram of an embodiment of a projection method of a three-dimensional scanning projection line based on structured light of the present invention;

[0047] Figure 2 is a schematic diagram of an embodiment of a projection system for three-dimensional scanning projection lines based on structured light of the present invention;

[0048] Figure 3 It is a logical diagram of an existing periodically projected fixed image group. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0050] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0051] In order to simplify the drawings, only the parts related to the present invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0052] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0055] In one embodiment, Figure 1 As shown, the present invention provides a projection method of three-dimensional scanning projection lines based on structured light, comprising the steps of:

[0056] S101 performs periodic or random switching of different projection image groups in three-dimensional measurement to collect multiple continuous projection image groups.

[0057] During the entire 3D measurement process, different projection image groups are switched periodically or randomly.

[0058] 1. Periodic switching. For example, there are three groups of projection images G1, G2 and G3. The projection order is periodic G1, G2, G3, G1, G2, G3, G1, G2, G3...

[0059] 2. Random switching. For example, there are three groups of projection images G1, G2 and G3; the projection order is random G1, G3, G1, G2, G3, G3, G1, G2, G3...

[0060] The so-called projection image group refers to a number of projection images that meet any of the following conditions:

[0061] 1) Continuous projection images with small time intervals between adjacent projection images within a period of time;

[0062] In the following, Px (x is a number) represents different projected images, and - represents a time interval unit.

[0063] The more common projection intervals may be as follows:

[0064] P1-P2-P3-P4----------P5–P6-P3-P4----------P1-P2-P3-P4----------

[0065] P1-P2-P3-P4 can be defined as a group of projected images, and P5–P6-P3-P4 can also be defined as a group of images.

[0066] 2) A minimum set of projection images required for one frame of three-dimensional reconstruction. The minimum set of projection images required for one frame of three-dimensional reconstruction includes:

[0067] The projected set of images includes but is not limited to a decoded image and a reconstructed image, and in principle, the decoding of the fringe phase of the reconstructed image can be achieved.

[0068] In the following, Px (x is a number) represents different projected images, and - represents a time interval unit.

[0069] A more extreme projection interval might be as follows:

[0070] P1-P2-P3-P4-P5–P6–P3-P4----------P1-P2-P3-P4----------

[0071] According to the time interval, P1-P2-P3-P4 is a group of images, and P1-P2-P3-P4-P5–P6–P3-P4 is also a group of images. However, if P1-P2-P3-P4 can complete a 3D reconstruction independently, and P5–P6-P3-P4 can also complete a 3D reconstruction independently, then P1-P2-P3-P4 can be defined as a group of images, and P5–P6-P3-P4 can also be defined as a group of images.

[0072] As for the judgment criteria, as long as theoretical analysis and inference are carried out according to the principles of structured light encoding and decoding, three-dimensional reconstruction can be completed and it is defined as a set of images.

[0073] S102 projects a plurality of images of the projection image group onto the surface of the object to be measured.

[0074] S103: Imaging the projection image projected onto the surface of the measured object to obtain an imaging image.

[0075] S104 decodes the imaging image and determines the corresponding relationship between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the measured object.

[0076] The present invention can adaptively transform the projected fringes in different measurement scenarios, reduce the dependence of the measured scene on the fringes quality, obtain better three-dimensional point reconstruction in various rapidly changing measured scenes, and improve robustness.

[0077] In one embodiment, the method further includes: determining whether the two projection image groups are different projection image groups, specifically including:

[0078] Determining whether the two sets of projection images are the same based on the number or stripe positions or stripe widths of the two sets of projection images;

[0079] When the number or the stripe positions or the stripe widths of the two sets of projection images are different, it is determined that the two sets of projection images are different;

[0080] The different projection image groups include a minimum set of projection images required to achieve one frame of three-dimensional reconstruction.

[0081] The two sets of projection images are the same, which means that the two sets of projection images meet the following two conditions at the same time:

[0082] 1). The number of projected images is the same;

[0083] 2). Sort the two groups of projection images according to the time sequence of the projection images. The stripes (bright stripes or dark stripes) of the projection images with corresponding serial numbers have the same position and the same stripe width.

[0084] If the above two conditions are not met, it is considered that the two sets of projection images are different.

[0085] In one embodiment, in the three-dimensional measurement, periodic switching of different projection image groups is performed to collect multiple continuous projection image groups, and further includes:

[0086] The time sequence of a plurality of different projection image groups is set as a periodic sequence to switch between different projection groups.

[0087] There are multiple continuous projection image groups, among which there are differences between two or more groups, but from a larger time series, such continuous projection image groups have a certain periodicity.

[0088] The two sets of projection images are the same, which means that the two sets of projection images meet the following two conditions at the same time:

[0089] 1). The number of projected images is the same;

[0090] 2). Sort the two groups of projection images according to the time sequence of the projection images. The stripes (bright stripes or dark stripes) of the projection images with corresponding serial numbers have the same position and the same stripe width.

[0091] If the above two conditions are not met, it is considered that the two sets of projection images are different.

[0092] In one embodiment, in the three-dimensional measurement, performing random switching of different projection image groups to collect multiple continuous projection image groups includes:

[0093] The time sequence of a plurality of different projection image groups is set as a random sequence to switch between different projection groups.

[0094] There are multiple groups of continuous projection image groups, among which there are differences between two or more groups. From the perspective of a larger time series, such continuous projection image groups are random.

[0095] In one embodiment, decoding the imaged image to determine the correspondence between each imaging position and the projection position includes:

[0096] Acquire an initial correspondence between an imaging point and a projection position based on first density fringes;

[0097] Using the initial correspondence relationship, a target correspondence relationship between the imaging point and the projection position of the second density fringes is obtained;

[0098] The first density is smaller than the second density.

[0099] Decoding steps: By, but not limited to, using relatively sparse stripes to first obtain a relatively reliable and accurate correspondence between imaging points and projection positions, using this as a reference condition, and gradually obtaining the correspondence between imaging points and projection positions of denser stripes.

[0100] In one embodiment, the present invention provides a projection system based on three-dimensional scanning projection lines of structured light, such as Figure 2 As shown, including:

[0101] The projection device 100 is used to periodically or randomly switch different projection image groups in three-dimensional measurement to collect multiple continuous projection image groups; and project several images of the projection image groups onto the surface of the object to be measured;

[0102] The image sensor 200 is used to image the projection image projected onto the surface of the object to be measured to obtain an imaging image;

[0103] The reconstruction unit 300 is used to decode the imaging image and determine the corresponding relationship between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the measured object.

[0104] In one embodiment, the present invention provides a system for three-dimensional information reconstruction, including a projection device, an image sensor, and a reconstruction unit;

[0105] The projection device projects the plurality of images of the projection image group onto the surface of the object to be measured in chronological order according to the structured light projection method for three-dimensional information reconstruction;

[0106] The image sensor images the projection image projected onto the surface of the object to be measured to obtain an imaging image;

[0107] The reconstruction unit decodes the imaging image, determines the correspondence between each imaging position and the projection position, and reconstructs the three-dimensional information of the surface of the measured object according to the correspondence.

[0108] Decoding steps: By, but not limited to, using relatively sparse stripes to first obtain a relatively reliable and accurate correspondence between imaging points and projection positions, using this as a reference condition, and gradually obtaining the correspondence between imaging points and projection positions of denser stripes.

[0109] Rebuild steps:

[0110] 1. Calculate the correspondence between the imaging position and the projection position.

[0111] 2. Calculate the three-dimensional coordinate position based on the pre-calibrated parameters and the corresponding relationship obtained in step 1.

[0112] In some embodiments, the projecting device is further configured to determine whether two projection image groups are different projection image groups, specifically comprising:

[0113] Determining whether the two sets of projection images are the same based on the number or stripe positions or stripe widths of the two sets of projection images;

[0114] When the number or the stripe positions or the stripe widths of the two sets of projection images are different, it is determined that the two sets of projection images are different.

[0115] In one embodiment, the projection device is used to:

[0116] The time sequence of a plurality of different projection image groups is set as a periodic sequence to switch between different projection groups.

[0117] In one embodiment, the projection device is used to:

[0118] The time sequence of a plurality of different projection image groups is set as a random sequence to switch between different projection groups.

[0119] In one embodiment, the reconstruction unit is used to:

[0120] Acquire an initial correspondence between an imaging point and a projection position based on first density fringes;

[0121] Using the initial correspondence relationship, a target correspondence relationship between the imaging point and the projection position of the second density fringes is obtained;

[0122] The first density is smaller than the second density.

[0123] The present invention can design multiple different projection image groups that can cope with different scenes and can be used to better reconstruct the three-dimensional information of the surface of the measured object in different scenes, thereby achieving the optimal design of projection quality in different scenes, thereby improving the surface three-dimensional reconstruction quality of different scenes.

[0124] The present invention can quickly and automatically adapt to changes in the measured object by periodically or randomly switching the projection image group used to reconstruct the three-dimensional information of the surface of the measured object, thereby greatly improving the robustness of the three-dimensional reconstruction.

[0125] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned program modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a processing unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software program units. In addition, the specific names of the program modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0126] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed with hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. Exemplarily, the device embodiments described above are merely schematic. Exemplarily, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation. Exemplarily, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A projection method of three-dimensional scanning projection line based on structured light, characterized in that: include: In three-dimensional measurement, different projection image groups are periodically switched or randomly switched to acquire multiple continuous projection image groups; Projecting a plurality of images of the projection image group onto the surface of the object to be measured; Imaging the projection image projected onto the surface of the object to be measured to obtain an imaging image; Decoding the imaging image to determine the corresponding relationship between each imaging position and the projection position so as to reconstruct the three-dimensional information of the surface of the measured object; The projection image group includes a minimum set of projection images required to achieve one frame of three-dimensional reconstruction; and the projection images of different projection image groups are different in number, stripe positions or stripe widths.

2. The projection method of three-dimensional scanning projection line based on structured light according to claim 1, characterized in that: Also includes: Determining whether two projection image groups are different projection image groups specifically includes: Determining whether the two sets of projection images are the same based on the number or stripe positions or stripe widths of the two sets of projection images; When the number or the stripe positions or the stripe widths of the two sets of projection images are different, it is determined that the two sets of projection images are different.

3. The projection method of three-dimensional scanning projection line based on structured light according to claim 2, characterized in that: In the three-dimensional measurement, periodic switching of different projection image groups is performed to collect multiple continuous projection image groups, and further includes: The time sequence of a plurality of different projection image groups is set as a periodic sequence to switch between different projection groups.

4. The projection method of three-dimensional scanning projection line based on structured light according to claim 3, characterized in that: In the three-dimensional measurement, random switching of different projection image groups is performed to collect a plurality of continuous projection image groups, including: The time sequence of a plurality of different projection image groups is set as a random sequence to switch between different projection groups.

5. The projection method of three-dimensional scanning projection line based on structured light according to any one of claims 1 to 4, characterized in that: The decoding of the imaging image to determine the corresponding relationship between each imaging position and the projection position includes: Acquire an initial correspondence between an imaging point and a projection position based on first density fringes; Using the initial correspondence relationship, a target correspondence relationship between the imaging point and the projection position of the second density fringes is obtained; The first density is smaller than the second density.

6. A projection system based on three-dimensional scanning projection lines of structured light, characterized in that: include: A projection device, used for periodically or randomly switching different projection image groups in three-dimensional measurement to collect multiple continuous projection image groups; Projecting a plurality of images of the projection image group onto the surface of the object to be measured; An image sensor is used to image the projection image projected onto the surface of the object to be measured to obtain an imaging image; A reconstruction unit, used to decode the imaging image and determine the corresponding relationship between each imaging position and the projection position, so as to reconstruct the three-dimensional information of the surface of the measured object; The projection image group includes a minimum set of projection images required to achieve one frame of three-dimensional reconstruction; and the projection images of different projection image groups are different in number, stripe positions or stripe widths.

7. The projection system based on three-dimensional scanning projection line of structured light according to claim 6, characterized in that: Also includes: The projection device is used to determine whether two projection image groups are different projection image groups, specifically including: Determining whether the two sets of projection images are the same based on the number or stripe positions or stripe widths of the two sets of projection images; When the number or the stripe positions or the stripe widths of the two sets of projection images are different, it is determined that the two sets of projection images are different.

8. The projection system based on three-dimensional scanning projection line of structured light according to claim 7, characterized in that: The projection device is used for: The time sequence of a plurality of different projection image groups is set as a periodic sequence to switch between different projection groups.

9. The projection system based on three-dimensional scanning projection line of structured light according to claim 8, characterized in that: The projection device is used for: The time sequence of a plurality of different projection image groups is set as a random sequence to switch between different projection groups.

10. The projection system based on three-dimensional scanning projection line of structured light according to any one of claims 6 to 9, characterized in that: The reconstruction unit is used for: Acquire an initial correspondence between an imaging point and a projection position based on first density fringes; Using the initial correspondence relationship, a target correspondence relationship between the imaging point and the projection position of the second density fringes is obtained; The first density is smaller than the second density.

Images