Scanning method, image reconstruction method, 3D equipment and image reconstruction device

Abstract

The invention provides a scanning method, an image reconstruction method, 3D equipment and an image reconstruction device, and the method comprises the steps: emitting N laser rays through a laser, N being greater than 1; scanning K sections of areas through rotation of a galvanometer of the laser, wherein K is greater than 1; wherein when the galvanometer rotates to scan a section of area, the N laser lines correspond to N area images. According to the technical scheme, rapid scanning can be achieved, higher image contrast is obtained, the frequency of scanning the structured light is improved, and the collection efficiency is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a scanning method, an image reconstruction method, a 3D device and an image reconstruction apparatus. BACKGROUND

[0002] A 3D measurement device (such as a line laser profiler) is composed of a laser and a camera (video camera). The laser is used to project a laser beam onto the surface of a measured object (i.e. a measured target), and the camera is used to take a picture of the measured object to obtain a laser image reflected by the measured object. After obtaining the laser image, a three-dimensional reconstruction of the measured object can be performed based on the laser image to obtain a three-dimensional reconstruction image of the measured object.

[0003] For example, if the laser image is an image of line structured light (i.e. a line structured light image), the structured light pattern of the line structured light image can be obtained, and the structured light pattern can be converted according to pre-calibrated sensor parameters to obtain the spatial coordinates (i.e. three-dimensional coordinates) of the measured object at the current position. Based on the spatial coordinates of the measured object at the current position, a three-dimensional image reconstruction of the measured object can be realized.

[0004] In related technologies, the scanning efficiency of the laser is relatively low in the process of three-dimensional reconstruction of the measured object, the time consumption of the three-dimensional reconstruction process is relatively long, and the effect of the three-dimensional reconstruction is relatively poor. SUMMARY

[0005] The present application provides a scanning method, which comprises: emitting N laser lines by a laser, N being greater than 1; rotating and scanning K regions by a galvanometer of the laser, K being greater than 1; wherein, when rotating and scanning a region by the galvanometer, N laser lines correspond to N region images.

[0006] The present application provides an image reconstruction method based on the above-mentioned scanning method, which comprises: performing image reconstruction based on N region images corresponding to N laser lines.

[0007] For example, N region images corresponding to N laser lines are obtained; wherein, N laser lines are emitted by a laser, N being greater than 1; K regions are rotated and scanned by a galvanometer of the laser, K being greater than 1; wherein, when rotating and scanning a region by the galvanometer, N laser lines correspond to N region images; performing image reconstruction based on N region images corresponding to N laser lines.

[0008] The present application provides a 3D device, which comprises a laser, a galvanometer, a camera and a processor, wherein: the laser is used to emit N laser lines, N being greater than 1; The galvanometer is used to rotate and scan K sections, and K is greater than 1. The camera is used to collect region images; wherein, when the galvanometer rotates and scans a section, N laser lines correspond to N region images, and the camera collects the N region images. The processor is used to perform image reconstruction based on the N region images corresponding to the N laser lines.

[0009] The present application provides an image reconstruction device, which comprises: An acquisition module is used to acquire N region images corresponding to N laser lines; wherein, N laser lines are emitted by a laser, and N is greater than 1; K sections are rotated and scanned by a galvanometer of the laser, and K is greater than 1; wherein, when the galvanometer rotates and scans a section, N laser lines correspond to N region images. A reconstruction module is used to perform image reconstruction based on the N region images corresponding to the N laser lines.

[0010] As can be seen from the above technical solutions, in the embodiments of the present application, N laser lines are scanned in K sections, N is greater than 1, and K is greater than 1, so that faster acquisition speed and higher contrast can be obtained, fast scanning can be realized, higher image contrast can be obtained, the frequency of the scanning structured light can be improved, and the acquisition efficiency can be significantly improved. When three-dimensional reconstruction is performed on the object to be measured, the scanning efficiency of the laser is relatively high, the time consumption of the three-dimensional reconstruction process is relatively small, and the effect of the three-dimensional reconstruction is relatively good. For example, the contrast can be improved by N times, the background brightness can be reduced to 1 / N, and the scanning frequency of the laser galvanometer can be improved by N times. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of an image reconstruction method in an embodiment of the present application; Figure 2 is a flowchart of an image reconstruction method in an embodiment of the present application; Figure 3A is a schematic diagram of a scanning process in an embodiment of the present application; Figure 3B is a schematic diagram of a phase-shifted fringe in an embodiment of the present application; Figure 3C is a schematic diagram of a binary fringe pattern in an embodiment of the present application; Figure 4A is a schematic diagram of N laser lines in K sections for fringe structured light scanning in the present application; Figure 4B is a schematic diagram of N laser lines in K sections for fringe structured light scanning in the present application; Figure 4C is a schematic diagram of N laser lines in K sections for fringe structured light scanning in the present application; Figure 5 is a flowchart of an image reconstruction method in an embodiment of the present application; Figure 6 is a schematic diagram of a plurality of second pixel points in an embodiment of the present application; Figure 7 is a flowchart of an image reconstruction method in an embodiment of the present application; Figure 8 is a structural schematic diagram of an image reconstruction device in an embodiment of the present application; Figure 9 is a hardware structural diagram of a 3D device in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The scanning method includes: emitting N laser lines by a laser, N being greater than 1; and rotating and scanning K regions by a galvanometer of the laser, K being greater than 1; wherein, when the galvanometer rotates and scans a region, N laser lines correspond to N region images.

[0013] For example, the scanning range of the laser is divided into N scanning sections corresponding to N laser lines, and each scanning section is divided into K regions. For example, when N is 3 and K is 4, the scanning range of the laser is divided into scanning section a1 corresponding to laser line 1, scanning section a2 corresponding to laser line 2, and scanning section a3 corresponding to laser line 3. Scanning section a1 is divided into region a1_1, region a1_2, region a1_3, and region a1_4. Scanning section a2 is divided into region a2_1, region a2_2, region a2_3, and region a2_4. Scanning section a3 is divided into region a3_1, region a3_2, region a3_3, and region a3_4.

[0014] In the first scanning process, the first region is rotated and scanned by the galvanometer of the laser, that is, laser line 1, laser line 2, and laser line 3 are emitted by the laser, laser line 1 scans region a1_1, laser line 2 scans region a2_1, and laser line 3 scans region a3_1. When laser line 1 scans region a1_1, laser line 1 corresponds to region image b1_1. When laser line 2 scans region a2_1, laser line 2 corresponds to region image b2_1. When laser line 3 scans region a3_1, laser line 3 corresponds to region image b3_1.

[0015] In the second scanning process, the second section area is scanned by the mirror rotation of the laser, i.e., the laser emits laser line 1, laser line 2 and laser line 3, laser line 1 scans area a1_2, laser line 2 scans area a2_2, and laser line 3 scans area a3_2. When laser line 1 scans area a1_2, laser line 1 corresponds to area image b1_2. When laser line 2 scans area a2_2, laser line 2 corresponds to area image b2_2. When laser line 3 scans area a3_2, laser line 3 corresponds to area image b3_2.

[0016] Similarly, until the end of the fourth scanning process, the scanning process is ended.

[0017] For example, when the i-th mirror rotation is performed, i is in the range of 1 to K, and the laser line scans the i-th section area in the scanning section corresponding to the laser line; for example, when the second mirror rotation is performed, the laser line scans the second section area in the scanning section corresponding to the laser line, for example, the second section area scanned by laser line 1 is area a1_2, the second section area scanned by laser line 2 is area a2_2, and the second section area scanned by laser line 3 is area a3_2.

[0018] For example, when the laser line scans the i-th section area in the scanning section corresponding to the laser line, the laser line is used to control the laser pattern of the i-th section area, and the laser pattern of the remaining areas except the i-th section area is off. In this case, the laser pattern of the i-th section area is a normal laser pattern, which is a bright-dark changing laser pattern. The laser pattern of the remaining areas except the i-th section area is off (dark).

[0019] For example, when the laser line scans the i-th section area in the scanning section corresponding to the laser line, the laser line is used to control the laser pattern of the i-th section area, i.e., the laser pattern of the i-th section area is a normal laser pattern, which is a bright-dark changing laser pattern. In addition, the laser line does not scan the remaining areas except the i-th section area.

[0020] For example, the two scanning sections corresponding to the adjacent two laser lines can have overlapping areas; or the two scanning sections corresponding to the adjacent two laser lines can have no overlapping areas.

[0021] For example, when the i-th mirror rotation is performed, i is in the range of 1 to K, and on this basis, the laser line starts to scan from the starting position of the scanning section and ends at the ending position of the scanning section; wherein, in the scanning process, the laser line is off in the remaining areas except the i-th section area; in the i-th section area, the laser line is bright-dark changing, so that a bright-dark changing laser pattern is formed in the i-th section area. In the remaining areas except the i-th section area, since the laser line is off, no bright-dark changing laser pattern is formed.

[0022] Alternatively, the laser line starts scanning from a position before the start position of the i-th segment region of the scanning segment (i.e., the position is located in front of the start position of the i-th segment region and is spaced apart from the start position by a small distance, such as the distance between the two positions is less than a threshold value), and scans until the end position of the i-th segment region. In this way, in the i-th segment region, the laser line is bright-dark changed, and a bright-dark changed laser pattern is formed in the i-th segment region.

[0023] Alternatively, the laser line starts scanning from a position before the start position of the i-th segment region of the scanning segment (i.e., the position is located in front of the start position of the i-th segment region and is spaced apart from the start position by a small distance, such as the distance between the two positions is less than a threshold value), and scans until the end position of the i-th segment region. In this way, in the i-th segment region, the laser line is bright-dark changed, and a bright-dark changed laser pattern is formed in the i-th segment region.

[0024] An image reconstruction method is proposed in the embodiments of the present application, which can be applied to 3D devices (such as 3D measurement devices), such as line laser profilers, etc. In the image reconstruction process, N region images corresponding to N laser lines can be used for image reconstruction. For example, referring to FIG. 1, which is a flowchart of the image reconstruction method, the image reconstruction method can include the following steps: Figure 1 Step 101: Obtain N region images corresponding to N laser lines; wherein N laser lines are emitted by a laser, and N is greater than 1; a K-segment region is scanned by a galvanometer of the laser, and K is greater than 1; wherein when a segment region is scanned by the galvanometer, N region images corresponding to N laser lines are obtained; in this way, when a segment region is scanned by the galvanometer, N region images corresponding to N laser lines can be obtained.

[0025] For example, a segmented image corresponding to a measured object can be obtained, which can include N region images corresponding to N laser lines; wherein for a region image corresponding to each laser line, the region image includes a fringe structured light pattern generated when the laser line scans the measured object.

[0026] For example, in the first scanning process, a first segment region is scanned by the galvanometer of the laser, laser line 1 scans region a1_1, laser line 2 scans region a2_1, and laser line 3 scans region a3_1. In this way, the segmented image includes region image b1_1 when laser line 1 scans region a1_1, region image b2_1 when laser line 2 scans region a2_1, and region image b3_1 when laser line 3 scans region a3_1.

[0027] For example, when a segment region is scanned by the galvanometer of the laser each time, a segmented image can be obtained, and when K segment regions are scanned by the galvanometer of the laser, K segmented images can be obtained.

[0028] Step 102: Perform image reconstruction based on N region images corresponding to N laser lines. For example, a target three-dimensional reconstruction image of the measured object can be generated based on K segmented images.​

[0029] Exemplarily, the laser line corresponds to M scanning angles, the M scanning angles are scanning segments corresponding to the laser line, the scanning segments are divided into K segment regions, i.e., the scanning segments are divided into K scanning angle sets, and the K scanning angle sets are the K segment regions. For any scanning angle set (region), the laser is controlled to project laser lines to the object to be measured based on the plurality of scanning angles in the scanning angle set in turn, so as to obtain a region image corresponding to the scanning angle set. The region image includes a fringe structured light pattern corresponding to the M scanning angles.

[0030] Exemplarily, there is a repeated scanning angle in all scanning angles corresponding to the two adjacent laser lines; or there is no repeated scanning angle in all scanning angles corresponding to the two adjacent laser lines.

[0031] Exemplarily, the image reconstruction based on the N region images corresponding to the N laser lines can include but is not limited to: performing a fusion operation on K segmented images corresponding to K segment regions to obtain a fused image; wherein the segmented images can include the N region images; wherein for any pixel point in the fused image, a fusion pixel value corresponding to the pixel point in the fused image is determined based on pixel values corresponding to the pixel point in the K segmented images; and three-dimensional reconstruction is performed based on the fused image to obtain a target three-dimensional reconstruction image.

[0032] Exemplarily, the fusion pixel value corresponding to the pixel point in the fused image is determined based on the pixel values corresponding to the pixel point in the K segmented images, including: based on the pixel values corresponding to the pixel point in the K segmented images, the maximum pixel value is determined as the fusion pixel value corresponding to the pixel point in the fused image.

[0033] Exemplarily, the K segmented images include K first segmented images captured by the first camera and K second segmented images captured by the second camera. When the K first segmented images corresponding to the K regions are fused, a first fused image can be obtained; when the K second segmented images corresponding to the K regions are fused, a second fused image can be obtained. Based on this, the fused image includes the first fused image and the second fused image. Based on the fused image, three-dimensional reconstruction is performed to obtain a target three-dimensional reconstruction image, which can include: for a first pixel point in the first fused image, determining N second pixel points corresponding to the first pixel point in the second fused image; wherein the phase corresponding to the second pixel point is the same as the phase corresponding to the first pixel point, and the phase represents the scanning angle corresponding to the laser line; constructing a pixel point pair, the pixel point pair including the first pixel point and a target second pixel point corresponding to the first pixel point; wherein for any second pixel point, a depth value is determined based on the first pixel point and the second pixel point, if the depth value satisfies the distance range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; or, a disparity value is determined based on the first pixel point and the second pixel point, if the disparity value satisfies the disparity range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; performing three-dimensional reconstruction based on the pixel point pair to obtain the target three-dimensional reconstruction image.

[0034] Exemplarily, based on the N region images corresponding to the N laser lines, image reconstruction can include but is not limited to: based on the segmented image corresponding to a region, three-dimensional reconstruction is performed to obtain an initial three-dimensional reconstruction image; wherein the segmented image includes the N region images; the initial three-dimensional reconstruction images of the K segmented images corresponding to the K regions are fused to obtain a target three-dimensional reconstruction image; wherein for any pixel point in the target three-dimensional reconstruction image, a fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image is determined based on the depth value corresponding to the pixel point in the K initial three-dimensional reconstruction images.

[0035] Exemplarily, based on the segmented image corresponding to a region, three-dimensional reconstruction is performed to obtain an initial three-dimensional reconstruction image, which can include: after obtaining the segmented image corresponding to the region, during scanning the next region of the region, based on the segmented image corresponding to the region, three-dimensional reconstruction is performed to obtain an initial three-dimensional reconstruction image; or, after obtaining the K segmented images corresponding to the K regions, based on the segmented image corresponding to the region, three-dimensional reconstruction is performed to obtain an initial three-dimensional reconstruction image.

[0036] For example, the segmented image corresponding to a region includes a first segmented image captured by the first camera and a second segmented image captured by the second camera; based on the first segmented image and the second segmented image, three-dimensional reconstruction is performed on the measured object to obtain an initial three-dimensional reconstruction image.

[0037] For example, determining the fusion depth value of the pixel point in the target three-dimensional reconstruction image based on the depth values of the pixel point in the K initial three-dimensional reconstruction images can include: determining the maximum depth value as the fusion depth value of the pixel point in the target three-dimensional reconstruction image based on the depth values of the pixel point in the K initial three-dimensional reconstruction images; or determining the minimum depth value as the fusion depth value of the pixel point in the target three-dimensional reconstruction image based on the depth values of the pixel point in the K initial three-dimensional reconstruction images; or determining the average depth value as the fusion depth value of the pixel point in the target three-dimensional reconstruction image based on the depth values of the pixel point in the K initial three-dimensional reconstruction images.

[0038] For example, the three-dimensional reconstruction of the object to be measured based on the first segmented image and the second segmented image to obtain an initial three-dimensional reconstruction image can include but is not limited to: for a first pixel point in the first segmented image, determining N second pixel points corresponding to the first pixel point in the second segmented image; wherein each second pixel point corresponds to the same phase as the first pixel point, and the phase represents the scanning angle of the laser line; constructing a pixel point pair, the pixel point pair including the first pixel point and a target second pixel point corresponding to the first pixel point; wherein for any second pixel point, a depth value is determined based on the first pixel point and the second pixel point, if the depth value satisfies the distance range constraint, the second pixel point can be the target second pixel point, otherwise, the second pixel point is not the target second pixel point; or a disparity value is determined based on the first pixel point and the second pixel point, if the disparity value satisfies the disparity range constraint, the second pixel point can be the target second pixel point, otherwise, the second pixel point is not the target second pixel point. Then, the initial three-dimensional reconstruction image of the object to be measured is obtained by three-dimensional reconstruction based on the pixel point pair.

[0039] For example, for any segmented image, the segmented image can be an image formed by one scan of the laser; in the process of one scan of the laser, the scanning patterns controlled by the N laser lines are consistent.

[0040] As can be seen from the above technical solutions, in the embodiments of the present application, N laser lines can be divided into K segments for scanning, N is greater than 1, and K is greater than 1, which can obtain faster acquisition speed and higher contrast, realize fast scanning, obtain higher image contrast, improve the frequency of the scanning structured light, and significantly improve the acquisition efficiency. When three-dimensional reconstruction is performed on the object to be measured, the scanning efficiency of the laser is relatively high, the time consumption of the three-dimensional reconstruction process is relatively small, and the effect of the three-dimensional reconstruction is relatively good. For example, the contrast can be improved by N times, the background brightness is reduced to 1 / N, and the scanning frequency of the laser galvanometer can be improved by N times.

[0041] The above technical solutions of the embodiments of the present application are described in combination with specific application scenarios.

[0042] An image reconstruction method is provided in the embodiments of the present application, which can be applied to a 3D device (3D measurement device). The 3D device is constructed based on the principle of laser triangulation and includes a cooperative measurement system of a laser and a camera. The 3D device of the embodiments of the present application covers all measurement devices based on laser triangulation, which can include but is not limited to a line laser profiler or a monocular structured light imaging device or a binocular structured light imaging device, etc.

[0043] Taking a line laser profiler as an example, the line laser profiler can be composed of a laser generating unit (such as a laser) and an image acquisition unit (such as a camera). The laser generates a laser beam to project onto the surface of a measured object, and the camera synchronously acquires a laser image reflected by the surface of the measured object, and three-dimensional coordinate calculation is realized based on the laser image. For example, taking a laser image as an image of line structured light, the structured light pattern of the line structured light image can be obtained, and the structured light pattern is converted according to the pre-calibrated sensor parameters to obtain the spatial coordinates (i.e. three-dimensional coordinates) of the measured object at the current position. Based on the spatial coordinates of the measured object at the current position, the three-dimensional image reconstruction of the measured object can be realized.

[0044] If the image reconstruction method is applied to a 3D device, the 3D device can acquire a laser image and realize the image reconstruction method based on the laser image. Alternatively, the image reconstruction method can also be applied to a management device (such as a server, a personal computer, a notebook computer, a smart phone, etc.) of the 3D device. If the image reconstruction method is applied to the management device, the 3D device can acquire a laser image and send the laser image to the management device, and the management device realizes the image reconstruction method based on the laser image, which is not limited in this regard.

[0045] An image reconstruction method is provided in the embodiments of the present application, which can be applied to a 3D device or a management device. In subsequent embodiments, a 3D device is taken as an example for description. The 3D device can include a laser and at least two cameras. For the convenience of description, two cameras are taken as an example for description. When the number of cameras is more, the image reconstruction method is similar, which is not described herein again. The two cameras can be denoted as a first camera and a second camera. The first camera can be a left camera, and the second camera can be a right camera, or the first camera can be a right camera, and the second camera can be a left camera.

[0046] In the embodiments of the present application, the laser projects N laser lines to the object to be measured (such as the surface of the object to be measured) each time, where N can be a positive integer greater than 1, instead of the laser projecting only one laser line to the object to be measured each time. The N laser lines are multi-line lasers, that is, the point laser is collimated and then projects a plurality of laser lines at certain angles through the DOE. For example, the laser projects two laser lines to the object to be measured each time, or the laser projects three laser lines to the object to be measured each time, or the laser projects four laser lines to the object to be measured each time, which is not limited, and the following takes N as 2 for example, and the two laser lines are recorded as laser line a1 and laser line a2.

[0047] In the embodiments of the present application, the laser performs stripe structured light scanning in K segments, and the camera (such as the first camera and the second camera) collects images in K segments, that is, collects K segmented images. For example, the laser emits N laser lines, and the galvanometer of the laser rotates to scan K segments of regions, when the galvanometer rotates to scan a segment of regions, the N laser lines correspond to N region images, that is, one segmented image includes N region images corresponding to the N laser lines. When the galvanometer rotates to scan K segments of regions, K segmented images can be collected.

[0048] For each segmented image, the segmented image is an image formed by one-time scanning of the laser. For example, when the laser is scanned for the first time, the laser line a1 scans a plurality of scanning angles in turn, the laser line a2 scans a plurality of scanning angles in turn, and the camera collects a segmented image b1 formed by the first-time scanning. The segmented image b1 includes region images when the laser line a1 scans a plurality of scanning angles and region images when the laser line a2 scans a plurality of scanning angles.

[0049] When the laser is scanned for the second time, the laser line a1 scans a plurality of scanning angles in turn, the laser line a2 scans a plurality of scanning angles in turn, and the camera collects a segmented image b2 formed by the second-time scanning.

[0050] By analogy, when the laser is scanned for the Kth time, the laser line a1 scans a plurality of scanning angles in turn, the laser line a2 scans a plurality of scanning angles in turn, and the camera collects a segmented image bK formed by the Kth-time scanning.

[0051] It should be noted that K segmented images, such as the segmented image b1, the segmented image b2, …, and the segmented image bK, can be obtained through K times of scanning, so that a target three-dimensional reconstruction image of the object to be measured can be generated based on the K segmented images. Obviously, the K segmented images obtained through K times of scanning are not a target three-dimensional reconstruction image of the object to be measured generated by a laser image obtained through one-time scanning.

[0052] For the convenience of description, taking K times of scanning as twice of scanning as an example, the segmented images formed by twice of scanning are denoted as segmented image b1 and segmented image b2. Considering that the first camera can capture two segmented images and the second camera can capture two segmented images, the segmented images captured by the first camera can be denoted as segmented image b1 and segmented image b2, and the segmented images captured by the second camera can be denoted as segmented image b1' and segmented image b2'.

[0053] In summary, in the embodiment of the present application, the N laser lines (N>1) are divided into K segments (K>1) for stripe structured light scanning, so that faster acquisition speed and higher contrast can be obtained.

[0054] Referring to Figure 2 As shown in the figure, the image reconstruction method can include the following steps. In step 201, for each laser line, M scanning angles corresponding to the laser line are determined, which can also be referred to as scanning positions, i.e., M scanning positions corresponding to the laser line are determined, and M is a positive integer.

[0055] For example, assuming that the laser needs to scan from position 1 to position 2, the scanning angle range from position 1 to position 2 is C, and during the scanning process, the scanning angle range can be equally divided into 2048 parts, and the angle of each part is C / 2048. On this basis, 2048 scanning angles of the laser can be obtained, the first scanning angle corresponds to position 1, the second scanning angle corresponds to position 1+C / 2048, the third scanning angle corresponds to position 1+C / 2048+C / 2048, and so on, and the 2048th scanning angle corresponds to position 2.

[0056] On this basis, all scanning angles can be equally divided into all laser lines. Taking two laser lines as an example, all scanning angles (2048 scanning angles) are equally divided into two laser lines, each laser line corresponds to M scanning angles, and all laser lines correspond to M scanning angles covering all scanning angles. For example, the M scanning angles corresponding to the laser line a1 are referred to as the scanning segment of the laser line a1, and the scanning segment of the laser line a1 is divided into K regions. The M scanning angles corresponding to the laser line a2 are referred to as the scanning segment of the laser line a2, and the scanning segment of the laser line a2 is divided into K regions.

[0057] For example, in determining the M scanning angles corresponding to each laser line, there is no repeated scanning angle in all scanning angles corresponding to two adjacent laser lines, that is, one scanning angle can only correspond to one laser line. For example, laser line a1 corresponds to the 1st scanning angle to the 1024th scanning angle, and laser line a2 corresponds to the 1025th scanning angle to the 2048th scanning angle. Thus, the next scanning angle of the last scanning angle of laser line a1 is the first scanning angle of laser line a2.

[0058] For example, in determining the M scanning angles corresponding to each laser line, there is a repeated scanning angle in all scanning angles corresponding to two adjacent laser lines, that is, part of the scanning angles can correspond to two adjacent laser lines. For example, in order to avoid the appearance of a gap (the laser line is not a straight line), the last scanning of the previous laser line covers the scanning area of the next laser line. Thus, the number of repeated scanning angles can be pre-configured, which can be configured according to actual needs, and is not limited.

[0059] For example, in determining the M scanning angles corresponding to each laser line, there is a repeated scanning angle in all scanning angles corresponding to two adjacent laser lines, that is, part of the scanning angles can correspond to two adjacent laser lines. For example, in order to avoid the appearance of a gap (the laser line is not a straight line), the last scanning of the previous laser line covers the scanning area of the next laser line. Thus, the number of repeated scanning angles can be pre-configured, which can be configured according to actual needs, and is not limited.

[0060] In step 202, for each laser line, the M scanning angles corresponding to the laser line are divided into K scanning angle sets (i.e., K regions). For example, the M scanning angles can be evenly divided into K scanning angle sets, or the M scanning angles can be unevenly divided into K scanning angle sets.

[0061] For example, assuming that laser line a1 corresponds to the 1st scanning angle to the 1024th scanning angle, all scanning angles (1024 scanning angles) can be evenly divided into scanning angle set c1 and scanning angle set c2. Scanning angle set c1 is one region, and scanning angle set c2 is another region. Scanning angle set c1 can include the 1st scanning angle to the 512th scanning angle, and scanning angle set c2 can include the 513th scanning angle to the 1024th scanning angle.

[0062] For example, assuming that the laser line a2 corresponds to the 1025th scanning angle to the 2048th scanning angle, all scanning angles (1024 scanning angles) can be evenly divided into the scanning angle set c3 and the scanning angle set c4, the scanning angle set c3 can include the 1025th scanning angle to the 1536th scanning angle, and the scanning angle set c4 can include the 1537th scanning angle to the 2048th scanning angle.

[0063] Step 203, for each scanning angle set, controlling the laser to sequentially project a laser line to the object to be measured based on a plurality of scanning angles in the scanning angle set, to obtain a region image corresponding to the scanning angle set, the region image including a stripe structured light pattern corresponding to M scanning angles.

[0064] For example, in the first scanning process of the laser, that is, by rotating the galvanometer of the laser to scan the first region, N laser lines are emitted by the laser. In this way, the laser is controlled to sequentially project the laser line a1 to the object to be measured based on a plurality of scanning angles (such as 512 scanning angles) in the scanning angle set c1, to obtain a region image d1 corresponding to the scanning angle set c1, and the laser is controlled to sequentially project the laser line a2 to the object to be measured based on a plurality of scanning angles (such as 512 scanning angles) in the scanning angle set c3, to obtain a region image d3 corresponding to the scanning angle set c3. Obviously, the region image d1 is a laser image generated by projecting the laser line a1 to the object to be measured, also referred to as the region image d1 corresponding to the laser line a1, and the region image d3 is a laser image generated by projecting the laser line a2 to the object to be measured, also referred to as the region image d3 corresponding to the laser line a2. As described above, when the galvanometer rotates to scan a region, N laser lines correspond to N region images.

[0065] It should be noted that the laser line a1 and the laser line a2 are scanned synchronously, for example, when the laser line a1 is projected to the object to be measured based on the 1st scanning angle in the scanning angle set c1, the laser line a2 is synchronously projected to the object to be measured based on the 1st scanning angle in the scanning angle set c3, when the laser line a1 is projected to the object to be measured based on the 2nd scanning angle in the scanning angle set c1, the laser line a2 is synchronously projected to the object to be measured based on the 2nd scanning angle in the scanning angle set c3, and so on.

[0066] For example, for each region image, the region image includes the stripe structured light pattern corresponding to the M scanning angles of the laser line, instead of the stripe structured light pattern corresponding to the scanning angles in the scanning angle set. For example, since the laser line a1 corresponds to the 1st scanning angle to the 1024th scanning angle, the region image d1 includes the stripe structured light pattern corresponding to the 1st-1024th scanning angle. For the 1st-512th scanning angle, since the laser line a1 scans these regions, the stripe structured light pattern of these regions is controlled by the pattern of the laser line a1, for example, when the laser line a1 is on, the corresponding region in the region image d1 is on, and when the laser line a1 is off, the corresponding region in the region image d1 is off. For the 513th-1024th scanning angle, since the laser line a1 does not scan these regions, the stripe structured light pattern of these regions is not controlled by the pattern of the laser line a1, and is usually a black region, i.e., the corresponding region in the region image d1 is off.

[0067] Similarly, since the laser line a2 corresponds to the 1025th scanning angle to the 2048th scanning angle, the region image d3 includes the stripe structured light pattern corresponding to the 1025th-2048th scanning angle. For the 1025th-1536th scanning angle, since the laser line a2 scans these regions, the stripe structured light pattern of these regions is controlled by the pattern of the laser line a2, for example, when the laser line a2 is on, the corresponding region in the region image d3 is on, and when the laser line a2 is off, the corresponding region in the region image d3 is off. For the 1537th-2048th scanning angle, since the laser line a2 does not scan these regions, the stripe structured light pattern of these regions is not controlled by the pattern of the laser line a2, and is usually a black region.

[0068] In summary, when the mirror is rotated for the i-th time, i is in the range of 1 to K, the laser line scans the i-th region, and the laser line is used to control the laser pattern of the scanned region. For example, the laser line scans from the start position of the i-th region to the end position of the i-th region.

[0069] In addition, when the laser line scans the i-th region, the laser line can also be used to control the laser pattern of the i-th region. For example, the laser line scans from the start position of the i-th region to the end position of the i-th region, or the laser line scans from the front retreat position of the start position of the i-th region to the end position of the i-th region.

[0070] For example, in the second scanning process of the laser, the laser is controlled to project the laser line a1 to the object to be measured according to a plurality of scanning angles (e.g., 512 scanning angles) in the scanning angle set c2, and the laser is controlled to project the laser line a2 to the object to be measured according to a plurality of scanning angles (e.g., 512 scanning angles) in the scanning angle set c4, so as to obtain the region image d2 corresponding to the scanning angle set c2 and the region image d4 corresponding to the scanning angle set c4. The region image d2 is a laser image generated by projecting the laser line a1 to the object to be measured, and the region image d4 is a laser image generated by projecting the laser line a2 to the object to be measured.

[0071] It should be noted that the laser line a1 and the laser line a2 are scanned synchronously. For example, when the laser line a1 is projected to the object to be measured according to the first scanning angle in the scanning angle set c2, the laser line a2 is projected to the object to be measured according to the first scanning angle in the scanning angle set c4 synchronously, and so on.

[0072] For example, for each region image, the region image includes a stripe structured light pattern corresponding to M scanning angles corresponding to the laser line. For example, since the laser line a1 corresponds to the first scanning angle to the 1024th scanning angle, the region image d2 includes a stripe structured light pattern corresponding to the first-1024th scanning angle. For the first-512th scanning angle, since the laser line a1 does not scan these regions, the stripe structured light pattern of these regions is not controlled by the pattern of the laser line a1, and is usually a black region. For the 513th-1024th scanning angle, since the laser line a1 scans these regions, the stripe structured light pattern of these regions is controlled by the pattern of the laser line a1. For example, when the laser line a1 is on, the corresponding region in the region image d1 is bright, and when the laser line a1 is off, the corresponding region in the region image d1 is off.

[0073] Similarly, since the laser line a2 corresponds to the 1025th scanning angle to the 2048th scanning angle, the region image d4 can include a stripe structured light pattern corresponding to the 1025th-2048th scanning angle.

[0074] In step 204, a segmented image corresponding to the object to be measured is obtained, and the segmented image includes N region images corresponding to N laser lines. For example, the stripe structured light can be a structured light designed according to a stripe. Obviously, K segmented images can be obtained through K scanning processes.

[0075] For example, in the first scanning process of the laser, the segmented image b1 corresponding to the object to be measured is obtained, and the segmented image b1 includes the region image d1 corresponding to the laser line a1 and the region image d3 corresponding to the laser line a2. In the second scanning process of the laser, the segmented image b2 corresponding to the object to be measured is obtained, and the segmented image b2 includes the region image d2 corresponding to the laser line a1 and the region image d4 corresponding to the laser line a2.

[0076] Obviously, when the laser scans the stripe structured light in K segments and the camera collects images in K segments, if K is 2, two segmented images can be obtained, that is, the segmented image b1 and the segmented image b2.

[0077] For example, for any segmented image, the segmented image can be an image formed by one scanning of the laser; in the one scanning process of the laser, the scanning patterns controlled by the N laser lines are consistent.

[0078] For example, the segmented image b1 is an image formed by the first scanning process of the laser, and the segmented image b1 includes the region image d1 and the region image d3. In the one scanning process of the laser, the scanning patterns controlled by the N laser lines are consistent, that is, the scanning pattern controlled by the laser line a1 is consistent with the scanning pattern controlled by the laser line a2. The scanning pattern controlled by the laser line a1 is controlled by the on-off of the laser line a1, for example, the laser line a1 is on for 512 scanning angles, and the scanning pattern is the on-off of the 512 scanning angles. For example, the laser line a1 is on for the first 256 scanning angles, and the laser line a1 is off for the last 256 scanning angles. For example, the laser line a1 is on for the odd scanning angles, and the laser line a1 is off for the even scanning angles. For example, the laser line a1 is on for all scanning angles. For example, the laser line a1 is off for all scanning angles. For example, the laser line a1 is on for the first 500 scanning angles, and the laser line a1 is off for the last 12 scanning angles. The above are only a few examples, and the scanning pattern controlled by the laser line a1 is not limited, and can be configured according to actual needs.

[0079] On this basis, the scanning pattern controlled by the laser line a2 needs to be consistent with the scanning pattern controlled by the laser line a1. For example, when the laser line a1 is on for the first 256 scanning angles and off for the last 256 scanning angles, the laser line a2 is on for the first 256 scanning angles and off for the last 256 scanning angles. For example, when the laser line a1 is on for the odd scanning angles and off for the even scanning angles, the laser line a2 is on for the odd scanning angles and off for the even scanning angles.

[0080] For example, segmented image b2 is the image formed during the second scan of the laser. Segmented image b2 includes region image d2 and region image d4. During the second scan of the laser, the scanning patterns controlled by N laser lines are consistent, that is, the scanning pattern controlled by laser line a1 is consistent with the scanning pattern controlled by laser line a2. The scanning pattern controlled by laser line a1 is controlled by the on / off state of laser line a1. If laser line a1 corresponds to 512 scanning angles, then the scanning pattern is the on / off state of 512 scanning angles.

[0081] Step 205: Generate a target 3D reconstructed image of the object under test based on K segmented images.

[0082] For example, steps 201-204 can be applied to a 3D device. After obtaining K segmented images, the 3D device can generate a target 3D reconstructed image of the object under test based on the K segmented images. Alternatively, after obtaining K segmented images, the 3D device can also send the K segmented images to a management device, which will then generate the target 3D reconstructed image of the object under test based on the K segmented images. For details on how to generate the target 3D reconstructed image of the object under test based on K segmented images, please refer to subsequent embodiments.

[0083] In one possible implementation, see Figure 3A The diagram illustrates the scanning process. Assuming the laser needs to scan from position 1 to position 2, and the scanning angle range from position 1 to position 2 is C, then the scanning angle range can be equally divided into 2048 parts, each with an angle of C / 2048. Based on this, we can obtain 2048 scanning angles for the laser. The first scanning angle corresponds to position 1, the second scanning angle corresponds to position 1 + C / 2048, and so on, with the 2048th scanning angle corresponding to position 2.

[0084] Taking a single laser line as an example, the laser emits a laser line towards the object being measured. This laser line scans 2048 scanning angles (i.e., scanning positions) sequentially. During the scanning process, the brightness and darkness of the laser line at each scanning angle are controlled, and the laser image is captured through a single exposure of the camera. For example, the laser line can be controlled to be bright at the first scanning angle, and dark at the second scanning angle, and so on, allowing for arbitrary control of the laser line's scanning pattern. Figure 3A In the laser image on the right, the white area corresponds to the scanning angle when the laser line is on, and the black area corresponds to the scanning angle when the laser line is off.

[0085] After the laser completes one cycle of scanning, that is, the laser scans from position 1 to position 2, the laser returns to the initial position (position 1), and then scans from position 1 to position 2 again, and so on.

[0086] Referring to Figure 3B Fig. 4 shows a schematic diagram of a phase-shifted fringe, which can be obtained by controlling the laser line to be on or off at 2048 scanning angles, and the phase-shifted fringe includes 2048-position fringe structure light patterns. Then, the required binary fringe pattern can be obtained by encoding the 2048-position fringe structure light patterns. Referring to Figure 3C Fig. 5 shows a schematic diagram of a binary fringe pattern. In Figure 3C , taking a Gray code fringe as an example, three different Gray code fringe patterns are shown. In the Gray code fringe pattern, there are 32 control positions, and the corresponding fringe patterns can be obtained by different control codes (rows 1-3). In addition, if the speed control or current control (sinusoidal variation) is performed, a sinusoidal variation pattern (i.e., a phase-shifted fringe) can also be obtained, as shown in Figure 3C . In addition, a line-shift pattern can also be obtained by periodic on and off. Here, only a few examples of controlling the laser line are shown, and the examples are not limited.

[0087] In the above manner, the laser emits a laser line to the object to be measured, and the laser line needs to scan all scanning angles (e.g., 2048 scanning angles). That is, one scanning process of the laser needs to scan the entire field of view of the laser, and therefore, the exposure time of the laser is relatively long, i.e., the background exposure time is relatively long, the background appears relatively bright, the pattern contrast is relatively low, and the overall acquisition time is relatively long.

[0088] For example, assuming that the scanning angle range of the laser is C (i.e., the total scanning angle of the galvanometer of the laser is C, the galvanometer is a rotating mechanism that can quickly and accurately modulate the angle position, and the load is a mirror), the laser power of the laser is P, the exposure time is E, and the projection area is A, then the intensity per unit area may be: In addition, the intensity corresponding to the background area may be: F can represent the ambient light power. On this basis, the contrast d can be represented as: . Obviously, the contrast d can be determined based on the laser power P, the ambient light power F, and the projection area A.

[0089] In the embodiments of the present application, N laser lines (N>1) are divided into K segments (K>1) for fringe structure light scanning. Since the laser power of the laser is P, when the N laser lines are divided, the laser power of each laser line is The laser projects in N unit intervals in space, and each laser line is responsible for a unit, so the projection area of each laser line is In addition, considering that the scanning exposure process is divided into K segments (K segments scanning range covers one unit interval), and the exposure time is E, the single-segment exposure time is set as The single-segment projection area of each laser line is .

[0090] On this basis, the intensity per unit area may be: In addition, the intensity corresponding to the background area may be: Further, the contrast may be expressed as: Obviously, the contrast may be N times the contrast d.

[0091] Suppose m projection patterns need to be collected (each projection pattern is one reconstruction process, and one frame of target three-dimensional reconstruction image is obtained), for the single laser line scheme, the projection times of m projection patterns are m times, and the total exposure time is m E. For the N laser line (N>1) stripe structure light scanning mode divided into K segments (K>1), when divided into K segments for scanning, the projection times of m projection patterns are m K times, and the total exposure time is m =m E / N, that is, the total projection time is .

[0092] As described above, the collection efficiency can be significantly improved. For example, the total scanning angle of the laser can be It can be seen that the reduction of the total collection time comes from the reduction of the scanning angle of the laser, because the maximum scanning frequency is fixed, and the smaller the scanning angle, the smaller the collection time.

[0093] In addition, the pattern contrast is N times the original, because during the laser line scanning process, no matter how large the scanning range is, the time the laser line stays at a single pixel position is constant, that is, the generated laser brightness is constant, but the brightness of the background depends on the exposure time of the current scanning, and the K-segment interval scanning exposure makes the background brightness reduce to 1 / N of the original, so that the contrast of the laser image is higher.

[0094] As can be seen from the above, in the embodiment of the application, the scanning frequency of the laser can be increased by N times by the multi-laser line segmented scanning stripe structure light method, and N represents the number of laser lines.

[0095] In one possible implementation, refer to Figure 4AThe diagram illustrates a striped structured light scan using N laser lines (N>1) divided into K segments (K>1). Here, we take an example where N is 2 and K is 2, meaning two laser lines scan in two segments. In this case, each segment of each laser line approximately covers the entire field of view. .

[0096] In the first scan, laser line 1 and laser line 2 each scan approximately half of the designed pattern area. For example, see [link to example scan]. Figure 4A As shown, laser line 1 scans from red position 1 to red position 2, and laser line 2 scans from blue position 1 to blue position 2. In the second scan, laser lines 1 and 2 each scan approximately half of the remaining area of ​​the design pattern; for example, laser line 1 scans from red position 2 to red position 3, and laser line 2 scans from blue position 2 to blue position 3. To avoid gaps (laser lines are not perfectly straight), the last scan segment of the previous laser line will cover the scanning unit of the next laser line, as shown... Figure 4A In the middle, the red position 3 will scan past the blue position 1.

[0097] See Figure 4B The diagram illustrates a striped structured light scan using N laser lines (N>1) divided into K segments (K>1). For the first segment, laser line 1 scans from red position 1 to red position 2, and laser line 2 scans from blue position 1 to blue position 2. By controlling the on / off states of laser lines 1 (and 2), the following parameters are obtained: Figure 4B The segmented image on the right includes the region image corresponding to laser line 1 and the region image corresponding to laser line 2. For the second scan segment, laser line 1 scans from red position 2 to red position 3, and laser line 2 scans from blue position 2 to blue position 3. By controlling the changes in the on / off state of laser line 1 (laser line 2), the following is obtained: Figure 4B The segmented image on the lower right includes the region image corresponding to laser line 1 and the region image corresponding to laser line 2.

[0098] See Figure 4C The diagram illustrates a striped structured light scan using N laser lines (N>1) divided into K segments (K>1). For the first segment, laser line 1 scans from red position 1 to red position 2, and laser line 2 scans from blue position 1 to blue position 2. By controlling the on / off states of laser lines 1 (and 2), the following parameters are obtained: Figure 4C The segmented image on the right includes the region image corresponding to laser line 1 and the region image corresponding to laser line 2. Clearly, compared to... Figure 4B In contrast, the changes in the on / off states of laser line 1 (laser line 2) are not the same, that is, the segmented images are not the same.

[0099] For the second scan, laser line 1 scans from red position 2 to red position 3, and laser line 2 scans from blue position 2 to blue position 3. By controlling the changes in the on / off state of laser line 1 (laser line 2), the following can be obtained: Figure 4C The segmented image on the lower right includes the region image corresponding to laser line 1 and the region image corresponding to laser line 2. Clearly, compared to... Figure 4B In contrast, the changes in the on / off states of laser line 1 (laser line 2) are not the same, thus the segmented images are not the same.

[0100] For example, for a single-laser-line scanning scheme, when scanning 8th-order Gray code + 16-step line shift, assuming an exposure time of 10ms, the total acquisition time is 240ms. When using a multi-laser-line scanning method (e.g., 4 laser lines instead of N laser lines), due to distance constraints, 4 laser lines are allowed. In a K-segment scan (2 segments), when scanning 8th-order Gray code + 16-step line shift, the corresponding exposure time is... That is, the exposure time is 1.25ms, and the total acquisition time is m. E / N=60ms, greatly reducing the acquisition time.

[0101] In summary, the scanning time can be reduced from 240ms to 60ms, and by configuring the values ​​of K and N, the scanning time can be reduced even further while improving pattern contrast. This allows for faster scene scanning and the achievement of higher image contrast.

[0102] When using N laser lines (N>1) divided into K segments (K>1) for fringe structured light scanning, the fringe structured light scheme can include Gray code, phase shift, line shift, etc., without limitation. When the laser lines are divided into N lines, each line covers 1 / N of the field of view. Each field of view is then divided into K segments for scanning. Originally, a single laser line could produce m images (such as 8 Gray code + 8 phase shift / line shift, 8 Gray code + 16 phase shift / line shift, multi-frequency phase shift / line shift, etc.). With the scheme of N laser lines and K segments, K*m images can also be obtained (such as 8 Gray code + 8 phase shift / line shift, 8 Gray code + 16 phase shift / line shift, multi-frequency phase shift / line shift, etc.).

[0103] This application proposes an image reconstruction method that can generate a target 3D reconstructed image of an object under test based on K segmented images. For example, see... Figure 5 The diagram shown illustrates the process of this image reconstruction method, which can be applied to 3D devices or management devices. The method may include: Step 501: Obtain K segmented images corresponding to the object under test. The K segmented images may include K first segmented images acquired by the first camera and K second segmented images acquired by the second camera.

[0104] For example, for the K first segment images captured by the first camera, the method for obtaining the K first segment images is described in steps 201-205. The segment images captured by the first camera are recorded as the first segment images.

[0105] For example, for the K second segment images captured by the second camera, the method for obtaining the K second segment images is described in steps 201-205. The segment images captured by the second camera are recorded as second segment images.

[0106] Step 502: Perform a fusion operation on the K first segmented images to obtain a first fused image. Specifically, for any pixel in the first fused image, determine the fused pixel value corresponding to that pixel in the first fused image based on the pixel values ​​corresponding to that pixel in the K first segmented images.

[0107] For example, for any pixel in the first fused image, based on the pixel values ​​corresponding to that pixel in the K first segmented images, the maximum pixel value is determined as the fused pixel value corresponding to that pixel in the first fused image. For instance, the K first segmented images may include segmented image b1 and segmented image b2. Segmented image b1 may include region image d1 and region image d3, and segmented image b2 may include region image d2 and region image d4. Region image d1 includes the striped structured light pattern corresponding to the 1st to 1024th scanning angles (for the 1st to 512th scanning angles, laser line a1 sweeps through these regions; for the 513th to 1024th scanning angles, laser line a1 does not sweep through these regions), and region image d2 includes the striped structured light pattern corresponding to the 1st to 1024th scanning angles (for the 1st to 512th scanning angles, laser line a1 does not sweep through these regions; for the 513th to 1024th scanning angles, laser line a1 sweeps through these regions). Region image d3 includes the striped structured light pattern corresponding to the 1025-2048th scanning angle, and region image d4 includes the striped structured light pattern corresponding to the 1025-2048th scanning angle.

[0108] Based on this, the first fused image may include 2048 pixels. For the first pixel, the pixel value corresponding to this pixel in segmented image b1 is the pixel value of the striped structured light pattern corresponding to the first scanning angle (located in region image d1), and the pixel value corresponding to this pixel in segmented image b2 is the pixel value of the striped structured light pattern corresponding to the first scanning angle (located in region image d2). The larger of these two pixel values ​​is taken as the fused pixel value corresponding to this pixel in the first fused image. Similarly, for the 1025th pixel, the pixel value corresponding to this pixel in segmented image b1 is the pixel value of the striped structured light pattern corresponding to the 1025th scanning angle (located in region image d3), and the pixel value corresponding to this pixel in segmented image b2 is the pixel value of the striped structured light pattern corresponding to the 1025th scanning angle (located in region image d4). The larger of these two pixel values ​​can be taken as the fused pixel value corresponding to this pixel in the first fused image, and so on.

[0109] In summary, the first fused image can be obtained by performing a fusion operation on the K first segmented images.

[0110] Step 503: Perform a fusion operation on the K second segmented images to obtain a second fused image. For any pixel in the second fused image, determine the fused pixel value corresponding to that pixel in the second fused image based on the pixel values ​​corresponding to that pixel in the K second segmented images. For example, for any pixel in the second fused image, based on the pixel values ​​corresponding to that pixel in the K second segmented images, determine the maximum pixel value as the fused pixel value corresponding to that pixel in the second fused image.

[0111] Step 504: For the first pixel in the first fused image, determine N second pixels corresponding to the first pixel in the second fused image; wherein, the phase corresponding to each second pixel is the same as the phase corresponding to the first pixel, and the phase represents the scanning angle corresponding to the laser line.

[0112] For example, after obtaining the first fused image and the second fused image, a 3D reconstruction of the object under test can be performed based on the first fused image and the second fused image to obtain a target 3D reconstructed image of the object under test. In order to perform 3D reconstruction of the object under test, phase resolution needs to be performed first. Phase resolution refers to determining N second pixels in the second fused image corresponding to the first pixel in the first fused image (for ease of description, the pixel in the first fused image is denoted as the first pixel), and the phase corresponding to each second pixel is the same as the phase corresponding to the first pixel.

[0113] For example, for a first pixel in the first fused image, if this first pixel corresponds to the first scanning angle of laser line a1, then the phase of this first pixel can be 1, representing the first scanning angle corresponding to laser line a1. For any second pixel in the second fused image, the phase of the second pixel x1 corresponding to the first scanning angle of laser line a1 is 1, meaning the phase of the second pixel x1 is the same as the phase corresponding to the first pixel. Similarly, the phase of the second pixel x2 corresponding to the first scanning angle of laser line a2 is 1, meaning the phase of the second pixel x2 is the same as the phase corresponding to the first pixel. Thus, the first pixel corresponds to both second pixel x1 and second pixel x2 in the second fused image. Obviously, when there are N laser lines, there are N second pixels.

[0114] In summary, when there are N laser lines, if the phase corresponding to the first pixel is 1, then the N laser lines will correspond to N scanning angles with a phase of 1. That is, the N laser lines will correspond to N second pixels with a phase of 1. Thus, when performing phase calculation, N second pixels can be obtained.

[0115] Similarly, if the first pixel is the pixel corresponding to the 5th scanning angle of laser line a1, then the phase corresponding to the first pixel is 5, and N laser lines correspond to N second pixels with phase 5, thus obtaining N second pixels corresponding to the first pixel in the second fused image, and so on.

[0116] Clearly, the more values ​​of N there are, the more second pixels there are during phase calculation, making the process more complex. Therefore, N can be greater than 1, but it cannot be arbitrarily large. A larger N value results in N identical phases appearing in the same row of the image, further complicating the phase calculation. Based on this, N can be no greater than 6 (this is just an example), thus N can be 2, 3, 4, 5, 6, etc. Considering that laser lines are generally divided symmetrically in the middle, the number of laser lines can also be odd, meaning N can be an odd number, such as 3, 5, etc.

[0117] Step 505: For the first pixel in the first fused image, select one second pixel from the N second pixels corresponding to the first pixel as the target second pixel.

[0118] For example, see Figure 6 As shown, the left side is the first fused image. Figure 6The first image shows the first pixel within the first fused image, and the right image shows the second fused image. Taking three laser lines out of N as an example, the first pixel corresponds to three second pixels. Based on this, one second pixel needs to be selected from these three as the target second pixel. The laser line corresponding to the target second pixel is the same as the laser line corresponding to the first pixel. Assuming the first pixel corresponds to laser line a1, then the target second pixel corresponds to laser line a1. That is, the second pixel corresponding to laser line a1 is used as the target second pixel, and the second pixel corresponding to laser line a2 is not used. This embodiment does not limit how one second pixel is selected from the N second pixels as the target second pixel.

[0119] In one possible implementation, the measurement range can be constrained, i.e., a distance range constraint can be pre-configured. For example, the distance range constraint can be 0.5~4m, meaning that the camera can only photograph objects within a range of 0.5~4m. If the distance between the object and the camera is within the range of 0.5~4m, the distance range constraint is satisfied; if the distance between the object and the camera is not within the range of 0.5~4m, the distance range constraint is not satisfied.

[0120] Based on this, for any second pixel, a depth value (representing the distance between the object and the camera) is determined based on the first pixel and the second pixel. For example, the depth value can be determined by methods such as triangulation. There are no restrictions on the method of determination, as long as the depth value can be obtained.

[0121] If the depth value meets the distance range constraint (e.g., the depth value is within the range of 0.5~4m), then the second pixel is taken as the target second pixel. Otherwise, if the depth value does not meet the distance range constraint (e.g., the depth value is not within the range of 0.5~4m), then the second pixel is not taken as the target second pixel.

[0122] For example, see Figure 6 As shown, when determining the depth value based on the first pixel and the first second pixel on the right, the depth value is 1m. Therefore, the first second pixel on the right is selected as the target second pixel. When determining the depth value based on the first pixel and the second second pixel on the right, the depth value is 9m. Therefore, the second second pixel on the right is not selected as the target second pixel. When determining the depth value based on the first pixel and the third second pixel on the right, the depth value is 30m. Therefore, the third second pixel on the right is not selected as the target second pixel. In summary, by applying distance constraints, it is possible to distinguish which second pixel is the target second pixel, and clearly select the point corresponding to 1m as the correct matching result.

[0123] In one possible implementation, the disparity range can be constrained, i.e., a pre-configured disparity range constraint can be set. For example, the disparity range constraint can be 0 to W / N, meaning the disparity cannot be greater than W / N, where W represents the width of the first fused image (or the second fused image), and N represents the number of laser lines. For instance, if an image with a width of W is scanned by N laser lines, then the range of each laser line is approximately W / N. The disparity between the left-eye camera (e.g., the first camera) and the right-eye camera (e.g., the second camera) cannot be greater than W / N, because a point x under the left-eye camera might match a point x + W / N under the right-eye camera, thus preventing ambiguity in phase resolution.

[0124] Based on this, for any second pixel, the disparity value is determined based on the first pixel and the second pixel. That is, the disparity value between the two is determined based on the pixel coordinates of the first pixel in the first fused image and the pixel coordinates of the second pixel in the second fused image.

[0125] If the disparity value satisfies the disparity range constraint (e.g., the disparity value is not greater than W / N), then the second pixel can be used as the target second pixel. Otherwise, if the disparity value does not satisfy the disparity range constraint (e.g., the disparity value is greater than W / N), then the second pixel will not be used as the target second pixel.

[0126] Step 506: Construct pixel pairs, which may include a first pixel and a target second pixel corresponding to the first pixel. For example, multiple first pixels can be selected from the first fused image (the number of first pixels is not limited) to construct multiple pixel pairs. For any pixel pair, the pixel pair may include a first pixel and a target second pixel corresponding to the first pixel.

[0127] Step 507: Perform 3D reconstruction based on pixel pairs to obtain a target 3D reconstructed image of the object under test. For example, based on multiple pixel pairs, 3D reconstruction of the object under test can be performed to obtain a target 3D reconstructed image of the object under test. There are no restrictions on the 3D reconstruction process of the object under test.

[0128] This application proposes an image reconstruction method that can generate a target 3D reconstructed image of an object under test based on K segmented images. For example, see... Figure 7 The diagram shown illustrates the process of this image reconstruction method, which can be applied to 3D devices or management devices. The method may include: Step 701: Obtain K segmented images corresponding to the object under test. The K segmented images may include K first segmented images acquired by the first camera and K second segmented images acquired by the second camera.

[0129] For example, after obtaining a first segmented image and a second segmented image, 3D reconstruction can be performed during the process of obtaining the next first segmented image and the second segmented image, i.e., steps 702 and 705 are executed. Alternatively, 3D reconstruction can be performed based on the first segmented images and the second segmented images only after obtaining K first segmented images and K second segmented images, i.e., steps 702 and 705 are executed.

[0130] Step 702: For any first segment image, based on the first segment image and the corresponding second segment image, for the first pixel in the first segment image, determine N second pixels in the second segment image corresponding to the first pixel; wherein, the phase corresponding to each second pixel is the same as the phase corresponding to the first pixel, and the phase represents the scanning angle corresponding to the laser line.

[0131] For example, the K first segmented images include segmented image b1 and segmented image b2. Segmented image b1 can include region image d1 and region image d3, and segmented image b2 can include region image d2 and region image d4. The K second segmented images include segmented image b1' and segmented image b2'. Segmented image b1 can correspond to segmented image b1', and segmented image b2 can correspond to segmented image b2'.

[0132] For example, for the first pixel in segmented image b1, determine the N second pixels corresponding to the first pixel in segmented image b1'. Assume that the phase corresponding to the first pixel is 5, that is, the first pixel is the pixel corresponding to the 5th scanning angle of laser line a1. Then, N laser lines correspond to N second pixels with phase 5 (the pixels corresponding to the 5th scanning angle of N laser lines), resulting in N second pixels. The phase corresponding to each second pixel is the same as the phase corresponding to the first pixel.

[0133] Step 703: For the first pixel in the first segmented image, select one second pixel from the N corresponding second pixels as the target second pixel. For example, the laser line corresponding to the target second pixel can be the same as the laser line corresponding to the first pixel.

[0134] In one possible implementation, the measurement range can be constrained, i.e., a distance range constraint can be pre-configured. Based on this, for any second pixel, a depth value (representing the distance between the object and the camera) is determined based on the first pixel and the second pixel. If the depth value satisfies the distance range constraint, the second pixel is designated as the target second pixel; otherwise, if the depth value does not satisfy the distance range constraint, the second pixel is not designated as the target second pixel.

[0135] In one possible implementation, the disparity range can be constrained, i.e., a disparity range constraint can be pre-configured. For any second pixel, a disparity value is determined based on the first pixel and the second pixel. If the disparity value satisfies the disparity range constraint, then the second pixel is used as the target second pixel; otherwise, if the disparity value does not satisfy the disparity range constraint, then the second pixel is not used as the target second pixel.

[0136] Step 704: Construct pixel pairs, which may include a first pixel and a target second pixel corresponding to the first pixel. For example, multiple first pixels can be selected from the first segmented image (the number of first pixels is not limited) to construct multiple pixel pairs. For any pixel pair, the pixel pair may include a first pixel and a target second pixel corresponding to the first pixel.

[0137] Step 705: Perform 3D reconstruction based on pixel pairs to obtain an initial 3D reconstructed image of the object under test. For example, for any first segment image, multiple pixel pairs can be constructed based on the first segment image and the corresponding second segment image. Based on these multiple pixel pairs, 3D reconstruction of the object under test can be performed to obtain an initial 3D reconstructed image of the object under test. There are no restrictions on this reconstruction process.

[0138] Step 706: Perform a fusion operation on the K initial 3D reconstructed images (which can be obtained by fusing the K initial 3D reconstructed images corresponding to the K first segment images) to obtain the target 3D reconstructed image. Specifically, for any pixel in the target 3D reconstructed image, determine the fusion depth value of the pixel in the target 3D reconstructed image based on the depth value of the pixel in the K initial 3D reconstructed images (the pixel value in the initial 3D reconstructed images is called the depth value, which represents distance).

[0139] For example, for any pixel in the target 3D reconstructed image, based on the depth values ​​corresponding to that pixel in K initial 3D reconstructed images, the maximum depth value is determined as the fusion depth value corresponding to that pixel in the target 3D reconstructed image; or, based on the depth values ​​corresponding to that pixel in K initial 3D reconstructed images, the minimum depth value is determined as the fusion depth value corresponding to that pixel in the target 3D reconstructed image; or, based on the depth values ​​corresponding to that pixel in K initial 3D reconstructed images, the average depth value is determined as the fusion depth value corresponding to that pixel in the target 3D reconstructed image.

[0140] In one possible implementation, when determining the M scanning angles corresponding to each laser line, if there are no overlapping scanning angles among all the scanning angles corresponding to two adjacent laser lines, then... Figure 5 The image reconstruction method shown can also be used.Figure 7 The image reconstruction method shown is as follows. Alternatively, when determining the M scanning angles corresponding to each laser line, if there are overlapping scanning angles among all the scanning angles corresponding to two adjacent laser lines, then the following method can be used: Figure 7 The image reconstruction method shown.

[0141] In one possible implementation, during the actual scanning process, the first segment, the second segment, ..., the Kth segment of each image can be scanned, and the next image can be scanned only after the first scan is completed. Thus, when calculating the phase of each segment, the last image of that segment, which is also the last image of the entire projection scheme, needs to be scanned. Alternatively, during the actual scanning process, each segment can be scanned with m images completed before moving on to the next segment with m images. This allows for the complete phase calculation of each segment before moving on to the next segment.

[0142] In one possible implementation, for phase calculation, regardless of the projection scheme (e.g., 8 Gray code + 8 phase shift / line shift, 8 Gray code + 16 phase shift / line shift, etc.), the phase of each scan range is ultimately obtained. Within the K segments of the scan range of the same laser line, the phase is continuous. The phases of the N laser lines are the same, and the cycle can be determined based on the range constraint of the measurement distance. Therefore, the maximum number of laser lines that can be used is determined by the measurement range. Furthermore, since laser lines are not perfectly straight lines, and the scan range is not precisely controlled, there will be overlap at the boundaries of adjacent laser lines. At these overlapping locations, the larger phase is taken, which is the phase calculated for the next laser line range, and can also be calculated up to the phase of the previous laser line, without affecting subsequent matching and reconstruction. By stitching together the phase segments and fusing the overlapping areas, a complete phase map (i.e., the matching relationship between the first pixel and the target's second pixel) can be obtained, and then matching and reconstruction are performed based on the complete phase map.

[0143] As can be seen from the above technical solutions, the embodiments of this application propose a rapid 3D reconstruction scheme based on multi-line laser projection. In the field of 3D reconstruction, a multi-line laser scanning scheme is designed, using N laser lines divided into K segments for striped structured light scanning, where N is greater than 1 and K is greater than 1. This results in faster acquisition speed and higher contrast, achieving rapid scanning, higher image contrast, and increased scanning structured light frequency, significantly improving acquisition efficiency. When performing 3D reconstruction on the object under test, the laser scanning efficiency is relatively high, the 3D reconstruction process takes less time, and the 3D reconstruction effect is better. For example, the contrast can be improved by N times, the background brightness can be reduced to 1 / N, and the scanning frequency of the laser galvanometer can be increased by N times.

[0144] Based on the same concept as the methods described above, this application proposes an image reconstruction apparatus, see [link to relevant documentation]. Figure 8The diagram shown is a structural schematic of the image reconstruction device, which may include: The acquisition module 81 is used to acquire N region images corresponding to N laser lines; wherein, N laser lines are emitted by the laser, where N is greater than 1; and the galvanometer of the laser is rotated to scan K regions, where K is greater than 1; wherein, when the galvanometer rotates to scan a region, N laser lines correspond to N region images. The reconstruction module 82 is used to reconstruct images based on N regions corresponding to N laser lines.

[0145] For example, when the reconstruction module 82 performs image reconstruction based on N region images corresponding to N laser lines, it is specifically used to: perform a fusion operation on K segmented images corresponding to the K regions to obtain a fused image; wherein, the segmented images include the N region images; wherein, for any pixel in the fused image, the fused pixel value corresponding to the pixel in the fused image is determined based on the pixel value corresponding to the pixel in the K segmented images; and perform three-dimensional reconstruction based on the fused image to obtain a target three-dimensional reconstructed image.

[0146] For example, when the reconstruction module 82 determines the fused pixel value corresponding to the pixel in the fused image based on the pixel value corresponding to the pixel in the K segmented images, it specifically performs the following: based on the pixel value corresponding to the pixel in the K segmented images, the maximum pixel value is determined as the fused pixel value corresponding to the pixel in the fused image.

[0147] For example, the fused image includes a first fused image and a second fused image. When the reconstruction module 82 performs three-dimensional reconstruction based on the fused image to obtain the target three-dimensional reconstructed image, it specifically performs the following steps: For a first pixel in the first fused image, it determines N second pixels corresponding to the first pixel in the second fused image; wherein the phase corresponding to the second pixel is the same as the phase corresponding to the first pixel, and the phase represents the scanning angle corresponding to the laser line; it constructs pixel pairs, the pixel pairs including the first pixel and the target second pixel corresponding to the first pixel; wherein, for any second pixel, it determines a depth value based on the first pixel and the second pixel, and if the depth value satisfies the distance range constraint, then the second pixel is the target second pixel, otherwise, the second pixel is not the target second pixel; or, it determines a disparity value based on the first pixel and the second pixel, and if the disparity value satisfies the disparity range constraint, then the second pixel is the target second pixel, otherwise, the second pixel is not the target second pixel; it performs three-dimensional reconstruction based on the pixel pairs to obtain the target three-dimensional reconstructed image.

[0148] For example, when the reconstruction module 82 performs image reconstruction based on N region images corresponding to N laser lines, it is specifically used to: perform three-dimensional reconstruction based on segmented images corresponding to a region to obtain an initial three-dimensional reconstructed image; wherein, the segmented image includes the N region images; fuse the initial three-dimensional reconstructed images of the K segmented images corresponding to the K regions to obtain a target three-dimensional reconstructed image; wherein, for any pixel in the target three-dimensional reconstructed image, the fusion depth value corresponding to the pixel in the target three-dimensional reconstructed image is determined based on the depth value corresponding to the pixel in the K initial three-dimensional reconstructed images.

[0149] For example, when the reconstruction module 82 performs 3D reconstruction based on the segmented image corresponding to a region to obtain the initial 3D reconstructed image, it is specifically used to: after obtaining the segmented image corresponding to the region, perform 3D reconstruction based on the segmented image corresponding to the region during the scanning of the next region of the region to obtain the initial 3D reconstructed image; or, after obtaining K segmented images corresponding to the K regions, perform 3D reconstruction based on the segmented images corresponding to the region to obtain the initial 3D reconstructed image.

[0150] For example, when the reconstruction module 82 determines the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images, it specifically performs the following: determining the maximum depth value as the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images; or, determining the minimum depth value as the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images; or, determining the average depth value as the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images.

[0151] Based on the same concept as the above method, this application proposes a 3D device, see [link to relevant documentation]. Figure 9 The diagram shows the hardware structure of a 3D device. The 3D device includes a laser, a galvanometer, a camera, and a processor. Specifically: the laser emits N laser lines, where N is greater than 1; the galvanometer rotates to scan K segments of a region, where K is greater than 1; the camera acquires images of the regions; and when the galvanometer rotates to scan a region, the N laser lines correspond to N region images, which the camera then acquires. The processor performs image reconstruction based on the N region images corresponding to the N laser lines.

[0152] For example, the scanning range of the laser is divided into N scanning segments corresponding to the N laser lines, and the scanning segments are divided into K regions.

[0153] For example, when the galvanometer rotates for the i-th time, the value of i ranges from 1 to K, and the laser line scans the i-th segment of the scanning segment corresponding to the laser line; wherein, the laser line is used to control the laser pattern of the scanning segment, and the laser pattern of the remaining areas other than the laser pattern of the i-th segment is off; or, the laser line is used to control the laser pattern of the i-th segment.

[0154] For example, the two scan segments corresponding to two adjacent laser lines have overlapping regions; or, the two scan segments corresponding to two adjacent laser lines do not have overlapping regions.

[0155] For example, when the galvanometer rotates for the i-th time, where i ranges from 1 to K, the laser line scans from the beginning position of the scanning segment to the end position of the scanning segment; wherein, in the remaining areas except for the i-th segment region, the laser line is off; or, the laser line scans from the beginning position of the i-th segment region to the end position of the i-th segment region; or, the laser line scans from the front retraction position of the beginning position of the i-th segment region to the end position of the i-th segment region.

[0156] For example, when the processor performs image reconstruction based on N region images corresponding to N laser lines, it specifically performs the following steps: merging the K segmented images corresponding to the K regions to obtain a merged image; wherein the segmented images include the N region images; wherein, for any pixel in the merged image, the fused pixel value corresponding to the pixel in the merged image is determined based on the pixel value corresponding to the pixel in the K segmented images; and three-dimensional reconstruction is performed based on the merged image to obtain a target three-dimensional reconstructed image.

[0157] For example, when the processor determines the fused pixel value corresponding to the pixel in the fused image based on the pixel value corresponding to the pixel in the K segmented images, it specifically performs the following: based on the pixel value corresponding to the pixel in the K segmented images, the maximum pixel value is determined as the fused pixel value corresponding to the pixel in the fused image.

[0158] For example, the fused image includes a first fused image and a second fused image. When the processor performs 3D reconstruction based on the fused image to obtain a target 3D reconstructed image, it specifically performs the following steps: For a first pixel in the first fused image, it determines N second pixels corresponding to the first pixel in the second fused image; wherein the phase corresponding to the second pixel is the same as the phase corresponding to the first pixel, and the phase represents the scanning angle corresponding to the laser line; it constructs pixel pairs, the pixel pairs including the first pixel and the target second pixel corresponding to the first pixel; wherein, for any second pixel, it determines a depth value based on the first pixel and the second pixel, and if the depth value satisfies the distance range constraint, then the second pixel is a target second pixel, otherwise, the second pixel is not a target second pixel; or, it determines a disparity value based on the first pixel and the second pixel, and if the disparity value satisfies the disparity range constraint, then the second pixel is a target second pixel, otherwise, the second pixel is not a target second pixel; it performs 3D reconstruction based on the pixel pairs to obtain the target 3D reconstructed image.

[0159] For example, when the processor performs image reconstruction based on N region images corresponding to N laser lines, it specifically performs: three-dimensional reconstruction based on segmented images corresponding to a region to obtain an initial three-dimensional reconstructed image; wherein the segmented image includes the N region images; and fuses the initial three-dimensional reconstructed images of the K segmented images corresponding to the K regions to obtain a target three-dimensional reconstructed image; wherein, for any pixel in the target three-dimensional reconstructed image, the fusion depth value corresponding to the pixel in the target three-dimensional reconstructed image is determined based on the depth value corresponding to the pixel in the K initial three-dimensional reconstructed images.

[0160] For example, when the processor performs 3D reconstruction based on the segmented image corresponding to a region to obtain an initial 3D reconstructed image, it specifically performs 3D reconstruction based on the segmented image corresponding to the region after obtaining the segmented image corresponding to the region, during the scanning of the next region of the region, to obtain an initial 3D reconstructed image; or, after obtaining K segmented images corresponding to the K regions, it performs 3D reconstruction based on the segmented images corresponding to the K regions to obtain an initial 3D reconstructed image.

[0161] For example, when the processor determines the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images, it specifically performs the following: determining the maximum depth value as the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images; or, determining the minimum depth value as the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images; or, determining the average depth value as the fusion depth value of the pixel in the target 3D reconstructed image based on the depth values ​​of the pixel in the K initial 3D reconstructed images.

[0162] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A scanning method, characterized by, The method comprises: N laser lines are emitted by a laser, N is greater than 1; K regions are scanned by a galvanometer of the laser, K is greater than 1; wherein, when a region is scanned by the galvanometer, N laser lines correspond to N region images.

2. The method of claim 1, wherein, A scanning range of the laser is divided into N scanning sections corresponding to the N laser lines, and the scanning sections are divided into K regions.

3. The method of claim 2, wherein, when the i-th time is scanned by the galvanometer, i is in the range of 1 to K, the laser line scans the i-th region in the scanning section corresponding to the laser line; wherein, the laser line is used to control the laser pattern of the scanning section, and the laser pattern of the remaining regions except the i-th region is off; or, the laser line is used to control the laser pattern of the i-th region.

4. The method of claim 2, wherein, there is an overlapping region between two scanning sections corresponding to adjacent two laser lines; or, there is no overlapping region between two scanning sections corresponding to adjacent two laser lines.

5. The method of claim 2, wherein, when the i-th time is scanned by the galvanometer, i is in the range of 1 to K, the laser line starts scanning from the starting position of the scanning section and ends at the ending position of the scanning section; wherein, in the remaining regions except the i-th region, the laser line is off; or, the laser line starts scanning from the starting position of the i-th region of the scanning section and ends at the ending position of the i-th region; or, the laser line starts scanning from the front side retreat position of the starting position of the i-th region of the scanning section and ends at the ending position of the i-th region.

6. An image reconstruction method based on the scanning method according to any one of claims 1 to 5, characterized in that, The method comprises: image reconstruction based on N region images corresponding to N laser lines.

7. The method of claim 6, wherein, the image reconstruction based on N region images corresponding to N laser lines comprises: performing fusion operation on K segmented images corresponding to K regions to obtain a fused image; wherein, the segmented images include the N region images; wherein, for any pixel point in the fused image, a fusion pixel value corresponding to the pixel point in the fused image is determined based on pixel values corresponding to the pixel point in the K segmented images; performing three-dimensional reconstruction based on the fused image to obtain a target three-dimensional reconstruction image.

8. The method of claim 7, wherein, the determination of the fusion pixel value corresponding to the pixel point in the fused image based on the pixel values corresponding to the pixel point in the K segmented images specifically comprises: determining the maximum pixel value as the fusion pixel value corresponding to the pixel point in the fused image based on the pixel values corresponding to the pixel point in the K segmented images.

9. The method of claim 7, wherein, the fused image includes a first fused image and a second fused image, and the three-dimensional reconstruction based on the fused image to obtain a target three-dimensional reconstruction image comprises: For the first pixel point in the first fused image, determine the N second pixel points corresponding to the first pixel point in the second fused image; wherein the phase corresponding to the second pixel point is the same as the phase corresponding to the first pixel point, and the phase represents the scanning angle corresponding to the laser line; Construct a pixel point pair, which includes the first pixel point and the target second pixel point corresponding to the first pixel point; wherein for any second pixel point, based on the first pixel point and the second pixel point, determine the depth value, if the depth value meets the distance range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; or, based on the first pixel point and the second pixel point, determine the disparity value, if the disparity value meets the disparity range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; Based on the pixel point pair, three-dimensional reconstruction is performed to obtain the target three-dimensional reconstruction image.

10. The method of claim 6, wherein, the image reconstruction based on the N region images corresponding to the N laser lines comprises: performing three-dimensional reconstruction based on a segmented image corresponding to a segment region to obtain an initial three-dimensional reconstruction image; wherein the segmented image includes the N region images; fuse the initial three-dimensional reconstruction images of the K segmented images corresponding to the K segment regions to obtain a target three-dimensional reconstruction image; wherein for any pixel point in the target three-dimensional reconstruction image, based on the depth value corresponding to the pixel point in the K initial three-dimensional reconstruction images, determine the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image.

11. The method of claim 10, wherein, the three-dimensional reconstruction based on a segmented image corresponding to a segment region to obtain an initial three-dimensional reconstruction image comprises: after obtaining the segmented image corresponding to the segment region, during the scanning of the next segment region of the segment region, perform three-dimensional reconstruction based on the segmented image corresponding to the segment region to obtain an initial three-dimensional reconstruction image; or, after obtaining the K segmented images corresponding to the K segment regions, perform three-dimensional reconstruction based on the segmented image corresponding to the segment region to obtain an initial three-dimensional reconstruction image.

12. The method of claim 10, wherein, the determination of the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth value corresponding to the pixel point in the K initial three-dimensional reconstruction images specifically comprises: based on the depth value corresponding to the pixel point in the K initial three-dimensional reconstruction images, determine the maximum depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image; or, based on the depth value corresponding to the pixel point in the K initial three-dimensional reconstruction images, determine the minimum depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image; or, based on the depth value corresponding to the pixel point in the K initial three-dimensional reconstruction images, determine the average depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image.

13. A 3D device, characterized by the 3D device includes a laser, a galvanometer, a camera, and a processor, wherein: The laser is configured to emit N laser lines, where N is greater than 1; The galvanometer is configured to rotate and scan K regions, where K is greater than 1; The camera is configured to capture region images; wherein, when the galvanometer rotates and scans a region, N laser lines correspond to N region images, and the camera captures the N region images; The processor is configured to perform image reconstruction based on the N region images corresponding to the N laser lines.

14. The 3D device of claim 13, wherein, The scanning range of the laser is divided into N scanning sections corresponding to the N laser lines, and the scanning sections are divided into K regions; Alternatively, when the galvanometer is rotated for the i th time, i ranges from 1 to K, and the laser line scans the i th region in the scanning section corresponding to the laser line; wherein, the laser line is configured to control the laser pattern of the scanning section, and the laser pattern of the remaining regions except for the i th region is off; or, the laser line is configured to control the laser pattern of the i th region; Alternatively, the two scanning sections corresponding to the two adjacent laser lines have an overlapping region; or, the two scanning sections corresponding to the two adjacent laser lines do not have an overlapping region; Alternatively, when the galvanometer is rotated for the i th time, i ranges from 1 to K, and the laser line scans from the starting position of the scanning section to the ending position of the scanning section; wherein, in the remaining regions except for the i th region, the laser line is off; or, the laser line scans from the starting position of the i th region of the scanning section to the ending position of the i th region; or, the laser line scans from the front side retreat position of the starting position of the i th region of the scanning section to the ending position of the i th region; Alternatively, the processor is configured to perform image reconstruction based on the N region images corresponding to the N laser lines, and specifically configured to perform fusion operation on K segmented images corresponding to the K regions to obtain a fused image; wherein, the segmented images include the N region images; wherein, for any pixel point in the fused image, a fusion pixel value corresponding to the pixel point in the fused image is determined based on pixel values corresponding to the pixel point in the K segmented images; three-dimensional reconstruction is performed based on the fused image to obtain a target three-dimensional reconstruction image; Alternatively, the processor is configured to determine the fusion pixel value corresponding to the pixel point in the fused image based on the pixel values corresponding to the pixel point in the K segmented images, and specifically configured to determine the maximum pixel value as the fusion pixel value corresponding to the pixel point in the fused image based on the pixel values corresponding to the pixel point in the K segmented images. Or, the processor is specifically configured to perform the following steps when the processor performs three-dimensional reconstruction based on the post-fusion images to obtain a target three-dimensional reconstruction image: determining, for a first pixel point in the first post-fusion image, N second pixel points corresponding to the first pixel point in the second post-fusion image; wherein the phase corresponding to the second pixel point is the same as the phase corresponding to the first pixel point, and the phase represents a scanning angle corresponding to a laser line; constructing a pixel point pair, the pixel point pair comprising the first pixel point and a target second pixel point corresponding to the first pixel point; wherein, for any second pixel point, a depth value is determined based on the first pixel point and the second pixel point, if the depth value satisfies a distance range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; or, a disparity value is determined based on the first pixel point and the second pixel point, if the disparity value satisfies a disparity range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; performing three-dimensional reconstruction based on the pixel point pair to obtain the target three-dimensional reconstruction image; Or, the processor is specifically configured to perform the following steps when the processor performs image reconstruction based on the N regional images corresponding to the N laser lines: performing three-dimensional reconstruction based on a segmented image corresponding to a segment to obtain an initial three-dimensional reconstruction image; wherein the segmented image comprises the N regional images; fusing initial three-dimensional reconstruction images of K segmented images corresponding to K segments to obtain a target three-dimensional reconstruction image; wherein, for any pixel point in the target three-dimensional reconstruction image, a fusion depth value of the pixel point in the target three-dimensional reconstruction image is determined based on depth values of the pixel point in K initial three-dimensional reconstruction images; Or, the processor is specifically configured to perform the following steps when the processor performs three-dimensional reconstruction based on a segmented image corresponding to a segment to obtain an initial three-dimensional reconstruction image: after obtaining the segmented image corresponding to the segment, performing three-dimensional reconstruction based on the segmented image corresponding to the segment to obtain an initial three-dimensional reconstruction image during a process of scanning a next segment of the segment; or, after obtaining K segmented images corresponding to K segments, performing three-dimensional reconstruction based on the segmented image corresponding to the segment to obtain an initial three-dimensional reconstruction image. Or, the processor is specifically configured to determine the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images by: determining the maximum depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images; or determining the minimum depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images; or determining the average depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images.

15. An image reconstruction apparatus, characterized by comprising: The apparatus comprises: The acquisition module is configured to acquire N regional images corresponding to N laser lines; wherein the laser emits N laser lines, and N is greater than 1; the K regions are scanned by the vibration mirror of the laser, and K is greater than 1; wherein when the vibration mirror scans a region, N laser lines correspond to N regional images; The reconstruction module is configured to perform image reconstruction based on the N regional images corresponding to the N laser lines.

16. The apparatus of claim 15, wherein, The reconstruction module is specifically configured to perform image reconstruction based on the N regional images corresponding to the N laser lines by: performing a fusion operation on K segmented images corresponding to the K regions to obtain a fused image; wherein the segmented images include the N regional images; wherein for any pixel point in the fused image, a fusion pixel value corresponding to the pixel point in the fused image is determined based on pixel values corresponding to the pixel point in the K segmented images; and three-dimensional reconstruction is performed based on the fused image to obtain a target three-dimensional reconstruction image; Or, the reconstruction module is specifically configured to determine the fusion pixel value corresponding to the pixel point in the fused image based on the pixel values corresponding to the pixel point in the K segmented images by: determining the maximum pixel value as the fusion pixel value corresponding to the pixel point in the fused image based on the pixel values corresponding to the pixel point in the K segmented images. Or, the post-fusion image includes a first post-fusion image and a second post-fusion image, and the reconstruction module performs three-dimensional reconstruction based on the post-fusion image to obtain a target three-dimensional reconstruction image, specifically for: for a first pixel point in the first post-fusion image, determining N second pixel points corresponding to the first pixel point in the second post-fusion image; wherein the phase corresponding to the second pixel point is the same as the phase corresponding to the first pixel point, and the phase represents the scanning angle corresponding to the laser line; constructing a pixel point pair, the pixel point pair including the first pixel point and the target second pixel point corresponding to the first pixel point; wherein for any second pixel point, based on the first pixel point and the second pixel point, a depth value is determined, if the depth value satisfies the distance range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; or, based on the first pixel point and the second pixel point, a disparity value is determined, if the disparity value satisfies the disparity range constraint, the second pixel point is the target second pixel point, otherwise, the second pixel point is not the target second pixel point; performing three-dimensional reconstruction based on the pixel point pair to obtain the target three-dimensional reconstruction image; Or, when the reconstruction module performs image reconstruction based on N regional images corresponding to N laser lines, it is specifically used for: performing three-dimensional reconstruction based on a segmented image corresponding to a segment of regions to obtain an initial three-dimensional reconstruction image; wherein the segmented image includes the N regional images; fuse the initial three-dimensional reconstruction images of the K segmented images corresponding to the K segments of regions to obtain a target three-dimensional reconstruction image; wherein for any pixel point in the target three-dimensional reconstruction image, the fusion depth value of the pixel point in the target three-dimensional reconstruction image is determined based on the depth value of the pixel point in the K initial three-dimensional reconstruction images; Or, when the reconstruction module performs three-dimensional reconstruction based on a segmented image corresponding to a segment of regions to obtain an initial three-dimensional reconstruction image, it is specifically used for: after obtaining the segmented image corresponding to the segment of regions, in the process of scanning the next segment of regions of the segment of regions, performing three-dimensional reconstruction based on the segmented image corresponding to the segment of regions to obtain an initial three-dimensional reconstruction image; or, after obtaining the K segmented images corresponding to the K segments of regions, performing three-dimensional reconstruction based on the segmented image corresponding to the segment of regions to obtain an initial three-dimensional reconstruction image; Or, the reconstruction module determines the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images, and is specifically configured to: determine the maximum depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images; or determine the minimum depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images; or determine the average depth value as the fusion depth value corresponding to the pixel point in the target three-dimensional reconstruction image based on the depth values corresponding to the pixel point in the K initial three-dimensional reconstruction images.

Images