Projection method and system for three-dimensional scanning projection lines based on structured light
By switching between different projection image sets, the method adapts to varying scenes, optimizing projection quality and enhancing three-dimensional reconstruction robustness and accuracy.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- SHANGHAI ALLIEDSTAR MEDICAL TECH CO LTD
- Filing Date
- 2022-10-17
- Publication Date
- 2026-06-02
AI Technical Summary
Existing structured light projection methods for three-dimensional reconstruction are inflexible and fail to optimize projection image quality for varying scenes, leading to suboptimal reconstruction in different measurement scenarios.
The method involves periodically or randomly switching between different sets of projection images to collect multiple sequential images, using a projection device to project these images onto an object, and employing an image sensor and reconstruction unit to decode and determine correspondence, thereby adapting to different scenes.
This approach enhances the quality of three-dimensional reconstruction by optimizing projection design for varying scenes, improving robustness and adaptability to changes in the object being measured.
Smart Images
Figure 0007869406000001 
Figure 0007869406000002 
Figure 0007869406000003
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional information reconstruction technology, and particularly to a method and system for projecting a three-dimensional scanning projection line based on structured light.
Background Art
[0002] In fields such as industrial control and medical treatment, there are many technologies for reconstructing three-dimensional information on the surface of an object. Among them, the grating stripe projection method based on structured light has been widely studied and applied due to its advantages such as high precision, high efficiency, and high robustness.
[0003] Such a method mainly projects specific encoded grating stripes as a projection image of structured light onto the surface of the object to be measured by a projection device, and then captures the structured light modulated and reflected by the object surface by an image sensor, such as a camera, to obtain an imaging image, decodes the imaging image, and finds the correspondence between each imaging position and the projection position in the decoded pattern. Since the position where the encoded pattern is projected onto the object is known, based on the above correspondence, the three-dimensional information of the object surface can be obtained by the triangulation method.
[0004] A commonly used structured light projection method is time encoding. The time encoding method projects a plurality of patterns into the measurement space in chronological order, and images are formed by a camera for each projection. By performing a certain decoding on these imaged stripe images, the three-dimensional information of the surface of the object to be measured projected can be obtained. A set of images that can continuously project and reconstruct the three-dimensional information of the surface of the object to be measured in this way is called a set of images.
[0005] A commonly used projection method designs a set of multiple projection images that can stably reconstruct the three-dimensional information of the surface of the object to be measured, and periodically projects the set of certain images designed using a projection device according to a certain time series.
[0006] As shown in Figure 3, let's assume that a set of projection images capable of stably reconstructing the three-dimensional information of the surface of the object being measured has been designed. This set consists of a total of five images, which are encoded as P1, P2, P3, P4, and P5 in chronological order. A common projection method is to periodically project the five fixed images of the designed set. However, with this commonly used projection method, since the projected image remains constant, it is not possible to differentiate the design according to the scene being measured, and therefore it is not possible to optimize the projection image quality for a specific scene. [Overview of the project]
[0007] The objective of the present invention is to provide a method and system for projecting three-dimensional scanning projection lines based on structured light, thereby solving the above-mentioned problems.
[0008] The proposed technology according to the present invention is as follows:
[0009] The present invention is a method for projecting three-dimensional scanning projection lines based on structured light, In 3D measurement, to collect multiple sets of sequential projection images, periodic or random switching of different projection image sets is performed. Projecting multiple images from the aforementioned set of projected images onto the surface of the object to be measured, The projected image projected onto the surface of the object to be measured is formed, and the resulting image is acquired. The image is decoded to reconstruct the three-dimensional information of the surface of the object to be measured, and the correspondence between each image position and the projection position is determined. The present invention provides a method for projecting three-dimensional scanning projection lines based on structured light, including the following:
[0010] In some embodiments, the process further includes determining whether two sets of projected images are different sets of projected images, specifically determining whether two sets of projected images are the same based on the number of images, the position of the stripes, or the width of the stripes, and determining that two sets of projected images are different if the number of images, the position of the stripes, or the width of the stripes differs. Each of the aforementioned sets of projection images includes the minimum set of projection images necessary to achieve a three-dimensional reconstruction of a single frame.
[0011] In some embodiments, the periodic switching of different projection image sets in the three-dimensional measurement to collect multiple sequential projection image sets further includes setting the time series of multiple different projection image sets into a periodic sequence so as to switch between different projection image sets.
[0012] In some embodiments, the three-dimensional measurement involves randomly switching between different projection image sets to collect multiple sequential projection image sets, which further includes setting the time series of multiple different projection image sets into a random sequence to switch between different projection image sets.
[0013] In some embodiments, decoding the image and determining the correspondence between each image position and projection position is performed. Obtaining an initial correspondence between the image point and the projection position based on a fringe with a first density, This includes obtaining a target correspondence between the image point and projection position of a fringe having a second density, using the aforementioned initial correspondence relationship, wherein the first density is smaller than the second density.
[0014] A projection system for three-dimensional scanning projection lines based on structured light, In three-dimensional measurement, a projection device is used to collect multiple sets of sequential projection images by periodically or randomly switching between different sets of projection images, and to project multiple images of the set of projection images onto the surface of an object to be measured. An image sensor for forming an image of the projected image projected onto the surface of the object to be measured and acquiring the resulting image, A projection system for three-dimensional scanning lines based on structured light, comprising a reconstruction unit for decoding the image so as to reconstruct three-dimensional information of the surface of the object to be measured, and for determining the correspondence between each image position and the projection position.
[0015] In some embodiments, the projection device is used to determine whether two sets of projected images are different sets of projected images, specifically, Determining whether two sets of projected images are the same based on the number of stripes, the position of the stripes, or the width of the stripes, This includes determining that two sets of projected images are different if they differ in the number of images, the position of the stripes, or the width of the stripes, Each of the aforementioned sets of projection images includes the minimum set of projection images necessary to achieve a three-dimensional reconstruction of a single frame.
[0016] In some embodiments, the projection device is used to set the time series of multiple different projection image sets into a periodic sequence in order to switch between different projection image sets.
[0017] In some embodiments, the projection device is used to set the time series of multiple different projection image sets into a random sequence in order to switch between different projection image sets.
[0018] In some embodiments, the reconfiguration unit is Obtaining an initial correspondence between the image point and the projection position based on a fringe with a first density, This is used to obtain a target correspondence between the image point and projection position of a fringe having a second density, utilizing the aforementioned initial correspondence relationship. The first density is smaller than the second density.
[0019] The method and system for projecting three-dimensional scanning projection lines based on structured light provided by the present invention have at least the following beneficial effects.
[0020] 1. The present invention achieves optimized projection quality design in different scenes and improves the quality of three-dimensional surface reconstruction in different scenes by designing multiple sets of different projection images that can accommodate different scenes and better reconstruct the three-dimensional information of the surface of the object being measured in different scenes.
[0021] 2. The present invention can quickly and automatically adapt to changes in the object to be measured by periodically or randomly switching the projection image set for reconstructing the three-dimensional information on the surface of the object to be measured, and greatly improve the robustness of three-dimensional reconstruction.
[0022] Hereinafter, for the sake of clarity, preferred embodiments will be described with reference to the drawings, and the above characteristics, technical features, advantages and their implementation forms of the projection method and system of three-dimensional scanning projection lines based on structured light will be further described.
Brief Description of the Drawings
[0023] [Figure 1] It is a schematic diagram of an embodiment of the projection method of three-dimensional scanning projection lines based on structured light of the present invention. [Figure 2] It is a schematic diagram of an embodiment of the projection system of three-dimensional scanning projection lines based on structured light of the present invention. [Figure 3] It is a logical schematic diagram of periodically projecting a conventional fixed image set.
Modes for Carrying Out the Invention
[0024] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system configurations and technologies are presented to fully understand the embodiments of the present application. However, it should be understood by those skilled in the art that the present application can also be realized in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted so as not to impede the description of the present application.
[0025] As used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or assemblies, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, assemblies and / or collections.
[0026] To simplify the drawings, each figure schematically shows only the parts relevant to the present invention and does not show the actual configuration of the product. Furthermore, to make the drawings simpler and easier to understand, if several figures show components with the same configuration or function, only one of them is schematically shown, or only one of them is labeled. In this text, "one" can refer not only to "this one only" but also to "more than one."
[0027] The term "and / or" as used in the specification and appended claims of this application should be further understood to mean any combination of one or more of the items listed in relation and to include all possible combinations thereof.
[0028] Furthermore, in the description of this application, terms such as "First," "Second," etc., are used merely for distinction and should not be understood as indicating or implying relative importance.
[0029] To more clearly illustrate embodiments of the present invention or technical proposals in the prior art, specific embodiments of the present invention will be described below with reference to the drawings. As will be apparent, the drawings in the following description are only a few embodiments of the present invention, and those skilled in the art can obtain other drawings and other embodiments based on these drawings without any creative effort.
[0030] In one embodiment, as shown in Figure 1, the present invention provides a method for projecting three-dimensional scanning projection lines based on structured light, and the projection method is Step S101: The 3D measurement includes periodically or randomly switching between different sets of projection images in order to collect multiple sets of sequential projection images.
[0031] Throughout the 3D measurement process, different sets of projected images may switch periodically or randomly.
[0032] 1. Periodic switching is, for example, when there are a total of three sets of projection images G1, G2, and G3, and the projection order is periodic G1, G2, G3, G1, G2, G3, G1, G2, G3... 2. Random switching is, for example, when there are a total of three sets of projection images G1, G2, and G3, and the projection order is random: G1, G3, G1, G2, G3, G3, G1, G2, G3...
[0033] A set of projected images is a set of multiple projected images that satisfy one of the following conditions: 1) A series of projection images in which the time interval between adjacent projection images is relatively small within a given time period. Below, Px (where x is a number) represents a different projected image, and - represents a single time interval unit. Typical projection intervals can be as follows: P1-P2-P3-P4----------P5-P6-P3-P4----------P1-P2-P3-P4---------- P1-P2-P3-P4 may be defined as a set of projection images, and P5-P6-P3-P4 may be defined as a set of images.
[0034] 2) The minimum set of projection images required to achieve a 3D reconstruction of one frame. The minimum set of projection images required to achieve the aforementioned 3D reconstruction of one frame is: The projected set of images includes, but is not limited to, a decoded image and a reconstructed image, and in principle, it is possible to decode the fringe phase of the reconstructed image.
[0035] Below, Px (where x is a number) represents a different projected image, and - represents a single time interval unit. One relatively extreme projection interval can be as follows: P1-P2-P3-P4-P5-P6-P3-P4----------P1-P2-P3-P4---------- Depending on the time interval, P1-P2-P3-P4 is a set of images, and P1-P2-P3-P4-P5-P6-P3-P4 is also a set of images. However, if P1-P2-P3-P4 can complete a single 3D reconstruction on its own, and P5-P6-P3-P4 can also complete a single 3D reconstruction on its own, then P1-P2-P3-P4 may be defined as a set of images, and positions P5-P6-P3-P4 may also be defined as a set of images.
[0036] Regarding the criteria for judgment, if a three-dimensional reconstruction can be performed by theoretical analysis and estimation according to the principles of encoding and decoding of structured light, it may be defined as a set of images.
[0037] S102 is to project multiple images from the set of projected images onto the surface of the object to be measured.
[0038] S103 is to form an image of the projection image projected onto the surface of the object to be measured and to acquire the image.
[0039] S104 involves decoding the image so as to reconstruct the three-dimensional information of the surface of the object to be measured, and determining the correspondence between each image position and the projection position.
[0040] This invention enables self-adaptive transformation of projected fringes in different measurement scenes, reduces dependence on fringe quality of the measurement scene, and improves robustness by obtaining good 3D point reconstruction in various rapidly changing measurement scenes.
[0041] In one embodiment, it further includes determining whether two sets of projected images are different sets of projected images, and more specifically, Determining whether two sets of projected images are the same based on the number of stripes, the position of the stripes, or the width of the stripes, This includes determining that two sets of projected images are different if they differ in the number of images, the position of the stripes, or the width of the stripes. The aforementioned set of different projection images includes the minimum set of projection images necessary to achieve a three-dimensional reconstruction of one frame.
[0042] Two sets of projected images are considered identical if they simultaneously satisfy the following two conditions: 1) The number of projected images is the same. 2) Sort two sets of projected images in chronological order, and check that the positions of the fringes (light or dark fringes) in the corresponding numbered projected images are the same, and that the width of the fringes is the same.
[0043] If the above two conditions are not met, it is likely that there is a difference between the two sets of projected images.
[0044] In one embodiment, in the 3D measurement, periodically switching between different projection image sets is performed to collect multiple sets of sequential projection images. This further includes setting the time series of multiple different projection image sets into a periodic sequence in order to switch between different projection image sets.
[0045] Multiple consecutive sets of projected images may have differences between any two or more sets, but when viewed over a larger time series, such sets of projected images exhibit a certain periodicity.
[0046] Two sets of projected images are considered identical if they simultaneously satisfy the following two conditions: 1) The number of projected images is the same. 2) Sort two sets of projected images in chronological order, and check that the positions of the fringes (light or dark fringes) in the corresponding numbered projected images are the same, and that the width of the fringes is the same.
[0047] If the above two conditions are not met, it is likely that there is a difference between the two sets of projected images.
[0048] In one embodiment, in the 3D measurement, random switching between different projection image sets is performed to collect multiple sets of sequential projection images. This further includes setting the time series of multiple different projection image sets to a random sequence in order to switch between different projection image sets.
[0049] While there may be differences between two or more sets of consecutive projected images, when viewed over a larger time series, such sets of consecutive projected images appear random.
[0050] In one embodiment, decoding the image and determining the correspondence between each image position and projection position is: Obtaining an initial correspondence between the image point and the projection position based on a fringe with a first density, This includes obtaining a target correspondence between the image point and projection position of a fringe having a second density, using the aforementioned initial correspondence relationship. The first density is smaller than the second density.
[0051] The decoding step can first obtain a relatively reliable and accurate correspondence between image points and projection positions using relatively sparse fringes, and then, using this as a reference condition, obtain the correspondence between image points and projection positions of denser fringes stepwise, but is not limited to this.
[0052] In one embodiment, the present invention provides a projection system for three-dimensional scanning projection lines based on structured light. As shown in Figure 2, the projection system is In 3D measurement, a projection device 100 is used to collect multiple sets of sequential projection images by periodically or randomly switching between different sets of projection images, and to project multiple images of the projection image sets onto the surface of the object to be measured. An image sensor 200 for forming an image of the projected image projected onto the surface of the object to be measured and acquiring the image, The system includes a reconstruction unit 300 for decoding the image to reconstruct the three-dimensional surface information of the object to be measured and for determining the correspondence between each image position and the projection position.
[0053] In one embodiment, the present invention provides a system for reconstructing three-dimensional information, comprising a projection device, an image sensor, and a reconstruction unit.
[0054] The projection device uses a structured light-based projection method for reconstructing the three-dimensional information, projecting multiple images of the projection image set onto the surface of the object to be measured in chronological order.
[0055] The image sensor forms an image of the projected image projected onto the surface of the object to be measured, and acquires the resulting image.
[0056] The reconstruction unit decodes the image, determines the correspondence between each image position and the projection position, and reconstructs the three-dimensional surface information of the object being measured based on this correspondence.
[0057] The decoding step can first obtain a relatively reliable and accurate correspondence between image points and projection positions using relatively sparse fringes, and then, using this as a reference condition, obtain the correspondence between image points and projection positions of denser fringes stepwise, but is not limited to this.
[0058] Reconfiguration step: 1. Calculate the correspondence between the image formation position and the projection position. 2. The 3D coordinate position is calculated based on the pre-set parameters and the correspondence obtained in step 1.
[0059] In some embodiments, the projection device is used to determine whether two sets of projected images are different sets of projected images, specifically, Determining whether two sets of projected images are the same based on the number of stripes, the position of the stripes, or the width of the stripes, This includes determining that two sets of projected images are different if they differ in the number of images, the position of the stripes, or the width of the stripes.
[0060] In one embodiment, the projection device is used to set the time series of multiple different projection image sets into a periodic sequence in order to switch between different projection image sets.
[0061] In one embodiment, the projection device is used to set the time series of multiple different projection image sets into a random sequence in order to switch between different projection image sets.
[0062] In one embodiment, the reconstruction unit is Obtaining an initial correspondence between the image point and the projection position based on a fringe with a first density, This is used to obtain a target correspondence between the image point and projection position of a fringe having a second density, utilizing the aforementioned initial correspondence relationship. The first density is smaller than the second density.
[0063] This invention achieves optimized projection quality design in different scenes and improves the quality of 3D surface reconstruction in different scenes by designing multiple sets of different projection images that can accommodate different scenes and better reconstruct the 3D information of the surface of the object being measured in different scenes.
[0064] This invention significantly improves the robustness of 3D reconstruction by rapidly and automatically adapting to changes in the object being measured by periodically or randomly switching a set of projected images for reconstructing the 3D information of the object's surface.
[0065] For the convenience and brevity of explanation, and so that those skilled in the art can understand, only the division of each program module described above is given as an example. In actual applications, the above functions can be assigned and completed by different program modules as needed; that is, the internal structure of the device can be divided into different program units or modules to complete all or some of the functions described above. Each program module in the embodiment may be integrated into one processing unit, each unit may exist physically independently, or two or more units may be integrated into one processing unit. The integrated unit may be implemented in hardware form or in the form of a software program unit. Furthermore, the specific names of each program module are merely for distinguishing them from one another and are not intended to limit the scope of protection of this application.
[0066] In the above embodiments, each embodiment has its own emphasis, but for parts that are not described or explained in detail in one embodiment, you can refer to the relevant explanations in other embodiments.
[0067] A person skilled in the art will recognize, by referring to the units and algorithmic steps of each example described in the embodiments disclosed herein, that these can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the proposed technology. A person skilled in the art may implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.
[0068] In the embodiments provided in this application, the disclosed apparatus and methods should be understood to be implementable in other ways. Exemplarily, the embodiments of the apparatus described herein are schematic and illustrative, and the division of the modules or units is merely one logical functional division; other division methods may exist in actual implementation. Exemplarily, multiple units or assemblies may be combined or integrated into another system, or some features may be ignored or not performed. In other words, the combinations, direct combinations or communication connections between them that are shown or discussed may be indirect combinations or communication connections via some interfaces, devices or units, and may be electrical, mechanical or otherwise.
[0069] The units described as separation means may or may not be physically separated, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed among multiple network units. Depending on the actual needs, some or all of these units can be selected to achieve the objective of the solution of this embodiment.
[0070] Furthermore, each functional unit in each embodiment of the present application may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented in hardware form or in the form of a software functional unit.
[0071] Furthermore, the above embodiments can be freely combined as needed. The above are merely preferred embodiments of the present invention, and those skilled in the art can make several improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for projecting three-dimensional scanning projection lines based on structured light, In 3D measurement, to collect multiple sets of sequential projection images, periodic or random switching of different projection image sets is performed. Projecting multiple images from the aforementioned set of projected images onto the surface of the object to be measured, The projected image projected onto the surface of the object to be measured is formed, and the resulting image is acquired. This includes decoding the image so as to reconstruct the three-dimensional information of the surface of the object to be measured, and determining the correspondence between each image position and the projection position. A method for projecting three-dimensional scanning projection lines based on structured light, characterized in that the different sets of projection images include the minimum set of projection images necessary to achieve three-dimensional reconstruction of one frame, and the number of projection images, the position of the stripes, or the width of the stripes differ among the different sets of projection images.
2. This further includes determining whether two sets of projected images are different sets of projected images, and more specifically, Determining whether two sets of projected images are the same based on the number of images, the position of the stripes, or the width of the stripes, A method for projecting three-dimensional scanning projection lines based on structured light according to claim 1, characterized in that it includes determining that two sets of projection images are different if the number of projection images, the position of the stripes, or the width of the stripes differs between the two sets of projection images.
3. A method for projecting a three-dimensional scanning projection line based on structured light, according to claim 2, wherein the three-dimensional measurement further includes setting the time series of the multiple different projection image sets into a periodic sequence so as to switch between different projection image sets.
4. A method for projecting a three-dimensional scanning projection line based on structured light, according to claim 3, further comprising randomly switching between different projection image sets to collect a plurality of sequential projection image sets in the three-dimensional measurement, by setting the time series of a plurality of different projection image sets in a random sequence to switch between different projection image sets.
5. Decoding the aforementioned image and determining the correspondence between each image position and projection position is: To obtain an initial correspondence between the image point and the projection position based on a fringe having a first density, A method for projecting three-dimensional scanning projection lines based on structured light according to any one of claims 1 to 4, comprising obtaining a target correspondence between the image point and projection position of a fringe having a second density using the initial correspondence relationship, wherein the first density is smaller than the second density.
6. A projection system for three-dimensional scanning projection lines based on structured light, In three-dimensional measurement, a projection device is used to collect multiple sets of sequential projection images by periodically or randomly switching between different sets of projection images, and to project multiple images of the set of projection images onto the surface of an object to be measured. An image sensor for forming an image of the projected image projected onto the surface of the object to be measured and acquiring the resulting image, The system includes a reconstruction unit for decoding the image and determining the correspondence between each image position and projection position in order to reconstruct the three-dimensional information of the surface of the object to be measured. A projection system for three-dimensional scanning projection lines based on structured light, characterized in that the different sets of projection images include the minimum set of projection images necessary to achieve three-dimensional reconstruction of one frame, and the number of projection images, the position of the stripes, or the width of the stripes differ among the different sets of projection images.
7. The projection device is used to determine whether two sets of projected images are different sets of projected images, specifically, Determining whether two sets of projected images are the same based on the number of images, the position of the stripes, or the width of the stripes, A three-dimensional scanning projection line projection system based on structured light according to claim 6, characterized in that it includes determining that two sets of projected images are different if the number of projection images, the position of the stripes, or the width of the stripes differs between the two sets of projection images.
8. The projection system for three-dimensional scanning projection lines based on structured light according to claim 7, characterized in that the projection device is used to set the time series of a plurality of different projection image sets into a periodic sequence in order to switch between different projection image sets.
9. The projection system for three-dimensional scanning projection lines based on structured light according to claim 8, characterized in that the projection device is used to set the time series of a plurality of different projection image sets into a random sequence in order to switch between different projection image sets.
10. The reconstruction unit is used to obtain an initial correspondence between an image point and a projection position based on a fringe having a first density, and to obtain a target correspondence between an image point and a projection position of a fringe having a second density using the initial correspondence, wherein the first density is smaller than the second density, characterized in that a three-dimensional scanning projection line projection system based on structured light according to any one of claims 6 to 9.