Image reconstruction method and apparatus and device

Abstract

An image reconstruction method, comprising: on the basis of the current line structured light image, determining initial three-dimensional coordinates of a measured object (101); on the basis of a target image interval between the current line structured light image and a first line structured light image, querying a compensation table to obtain a target slope and a target offset, the compensation table comprising mapping relationships between image intervals and slopes and between the image intervals and offsets (102); on the basis of the target slope, determining a rotation compensation quantity and, on the basis of the target offset, determining a height compensation quantity (103); on the basis of the rotation compensation quantity and the height compensation quantity, performing vibration compensation on the height coordinate value of the initial three-dimensional coordinates to obtain target three-dimensional coordinates (104); and, on the basis of the target three-dimensional coordinates, generating a three-dimensional reconstructed image of the measured object (105). The present application can implement vibration compensation for three-dimensional reconstruction. Also provided are an image reconstruction apparatus and an electronic device.

Description

An image reconstruction method, apparatus and device Technical Field

[0001] This application relates to the field of three-dimensional measurement technology, and in particular to an image reconstruction method, apparatus and device. Background Technology

[0002] A 3D imaging device can consist of a laser and a camera. The laser projects structured light onto the surface of the object being measured (i.e., the target object), and the camera captures an image of the object, resulting in an image containing the light stripes formed by the projection—a structured light image. The centerline of the light stripes in the structured light image can then be acquired and transformed according to pre-calibrated sensor parameters to obtain the spatial coordinates (i.e., 3D coordinates) of the object at its current position. Based on these 3D coordinates, 3D reconstruction of the object can be achieved.

[0003] When a 3D imaging device performs a scanning operation (the laser projects structured light onto the surface of the object being measured, and the camera takes a picture of the object), it is affected by vibrations in the surrounding environment, such as vibrations caused by the movement of the 3D imaging device or the inherent vibrations of the object being measured. These vibrations can cause changes in the distance or angle between the 3D imaging device and the object being measured at different points in time during the scanning process, resulting in inaccurate 3D coordinates of the object being measured. Consequently, it becomes impossible to obtain an accurate 3D reconstructed image based on these 3D coordinates. Summary of the Invention

[0004] This application provides an image reconstruction method applied to a three-dimensional imaging device, the three-dimensional imaging device including a camera and a laser, the method comprising:

[0005] When the laser projects line structured light onto the object under test, the camera captures the current line structured light image of the object under test; the initial three-dimensional coordinates of the object under test are determined based on the current line structured light image.

[0006] Based on the target image interval between the current line structured light image and the first line structured light image, the obtained compensation table is queried to obtain the target slope and target offset; wherein, the compensation table includes the mapping relationship between image interval and slope, and the mapping relationship between image interval and offset;

[0007] The rotation compensation amount is determined based on the target slope, and the height compensation amount is determined based on the target offset;

[0008] Vibration compensation is performed on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the measured object.

[0009] A three-dimensional reconstructed image of the object under test is generated based on the target's three-dimensional coordinates.

[0010] This application provides an image reconstruction apparatus applied to a three-dimensional imaging device, the three-dimensional imaging device including a camera and a laser, the apparatus comprising:

[0011] The acquisition module is used to acquire the current line structured light image of the object being measured by the camera when the laser projects line structured light onto the object being measured; determine the initial three-dimensional coordinates of the object being measured based on the current line structured light image; and query the acquired compensation table based on the target image interval between the current line structured light image and the first line structured light image to obtain the target slope and target offset; wherein, the compensation table includes the mapping relationship between the image interval and the slope, and the mapping relationship between the image interval and the offset;

[0012] The determination module is used to determine the rotation compensation amount based on the target slope and the height compensation amount based on the target offset; and to perform vibration compensation on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the object under test.

[0013] The reconstruction module is used to generate a three-dimensional reconstructed image of the object under test based on the target's three-dimensional coordinates.

[0014] This application provides an electronic device, including: a processor and a non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute the machine-executable instructions to implement the image reconstruction method of the example above in this application.

[0015] As can be seen from the above technical solutions, in the embodiments of this application, a compensation table can be pre-calibrated. The compensation table includes the mapping relationship between image interval and slope, and also includes the mapping relationship between image interval and offset. During 3D reconstruction, the compensation table is queried based on the target image interval between the current line structured light image and the first line structured light image to obtain the target slope and target offset. Then, based on the target slope and target offset, vibration compensation is performed on the initial 3D coordinates to obtain the target 3D coordinates. Image reconstruction is performed based on the target 3D coordinates to obtain the 3D reconstructed image of the measured object. The target slope and target offset can compensate for the error caused by vibration, reducing the impact of vibration on measurement accuracy. When obtaining the 3D coordinates of the measured object, accurate and reliable 3D coordinates can be obtained, and then an accurate 3D reconstructed image can be obtained based on these 3D coordinates. By evaluating the average vibration on the time axis and calculating the relative vibration amplitude at each time point, the average vibration of all frames is used as a reference, and all contours are aligned with the reference, effectively suppressing the impact of vibration on the contours of a single frame. Attached Figure Description

[0016] Figure 1 is a flowchart illustrating an image reconstruction method according to one embodiment of this application.

[0017] Figure 2 is a flowchart illustrating the compensation table calibration process in one embodiment of this application.

[0018] Figure 3A is a schematic diagram showing that the contour height and contour angle remain unchanged in one embodiment of this application.

[0019] Figure 3B is a schematic diagram of the changes in contour height and / or contour angle in one embodiment of this application.

[0020] Figure 3C is a schematic diagram of the ROI coverage area following mode in one embodiment of this application.

[0021] Figure 3D is a schematic diagram of a fixed pattern of ROI coverage area in one embodiment of this application.

[0022] Figure 4 is a schematic flowchart of the vibration compensation process in one embodiment of this application.

[0023] Figure 5 is a schematic diagram of the structure of an image reconstruction apparatus according to one embodiment of this application.

[0024] Figure 6 is a hardware structure diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0025] This application proposes an image reconstruction method that can be applied to a three-dimensional imaging device, which may include a camera and a laser. Referring to Figure 1, which is a flowchart of the image reconstruction method, the method may include the following steps:

[0026] Step 101: When the laser projects line structured light onto the object under test, acquire the current line structured light image of the object under test captured by the camera, and determine the initial three-dimensional coordinates of the object under test based on the current line structured light image.

[0027] Step 102: Based on the target image interval between the current line structured light image and the first line structured light image, query the obtained compensation table to obtain the target slope and target offset; wherein, the compensation table includes the mapping relationship between image interval and slope, and the compensation table includes the mapping relationship between image interval and offset.

[0028] Step 103: Determine the rotation compensation amount based on the target slope, and determine the height compensation amount based on the target offset.

[0029] Step 104: Based on the rotation compensation amount and the height compensation amount, perform vibration compensation on the height coordinate value of the initial three-dimensional coordinates to obtain the target three-dimensional coordinates of the object being measured.

[0030] Step 105: Generate a 3D reconstructed image of the object under test based on the target's 3D coordinates.

[0031] For example, before querying the acquired compensation table based on the target image interval between the current line structured light image and the first line structured light image, the process of acquiring the compensation table may include, but is not limited to: during the movement of the laser or the sample object, projecting multiple line structured lights onto the sample object through the laser to acquire multiple line structured light images of the sample object captured by the camera. For each line structured light image, determine the height coordinates corresponding to the line structured light image; determine the height difference based on the height coordinates corresponding to the line structured light image and the height coordinates corresponding to the previous line structured light image; determine the first slope corresponding to the line structured light image based on the height difference; determine the second slope of the line structured light image relative to the first line structured light image based on the first slope corresponding to the line structured light image and the first slope corresponding to each line structured light image preceding the line structured light image; record the mapping relationship between the image interval of the line structured light image and the second slope in the compensation table, where the image interval represents the number of images between the line structured light image and the first line structured light image.

[0032] For example, determining the first slope corresponding to the line structured light image based on the height difference may include, but is not limited to: selecting multiple pixels from the reference region of the line structured light image; constructing multiple coordinate points corresponding to the multiple pixels in a coordinate system; wherein, for each coordinate point corresponding to a pixel, the horizontal coordinate of the coordinate point may be the column identifier corresponding to the pixel in the line structured light image, and the vertical coordinate of the coordinate point may be the height difference between the height coordinate of the pixel corresponding to the line structured light image and the height coordinate of the pixel corresponding to the previous line structured light image. After obtaining the multiple coordinate points corresponding to the multiple pixels, a straight line is fitted based on the multiple coordinate points to obtain the first slope corresponding to the line structured light image.

[0033] For example, before querying the acquired compensation table based on the target image interval between the current line structured light image and the first line structured light image, the process of acquiring the compensation table may include, but is not limited to: for each line structured light image, performing rotation compensation on the height coordinates corresponding to the line structured light image based on the first slope corresponding to the line structured light image to obtain the rotated and compensated height coordinates; determining the rotated and compensated height difference based on the rotated and compensated height coordinates corresponding to the previous line structured light image; and determining the first offset corresponding to the line structured light image based on the rotated and compensated height difference. Then, based on the first offset corresponding to the line structured light image and the first offset corresponding to each line structured light image preceding the line structured light image, determining the second offset of the line structured light image relative to the first line structured light image; thus, the mapping relationship between the image interval of the line structured light image and the second offset can be recorded in the compensation table.

[0034] For example, determining the first offset corresponding to the line structured light image based on the height difference after rotation compensation may include, but is not limited to: selecting multiple pixels from the reference region of the line structured light image; after obtaining the multiple pixels, obtaining the height difference after rotation compensation corresponding to the multiple pixels, and determining the first offset corresponding to the line structured light image based on the average value of the height differences after rotation compensation corresponding to the multiple pixels; wherein, for each pixel, the height difference after rotation compensation corresponding to the pixel is the height difference between the height coordinate of the pixel after rotation compensation in the line structured light image and the height coordinate of the pixel after rotation compensation in the previous line structured light image.

[0035] For example, before selecting multiple pixels from the reference region of the line structured light image, the reference region of the line structured light image can be determined. For instance, the reference region of the line structured light image is determined based on the reference region of the previous line structured light image and an X-direction offset value; the X-direction offset value represents the X-direction offset between the reference region of the line structured light image and the reference region of the previous line structured light image.

[0036] For example, determining the rotation compensation amount based on the target slope and the height compensation amount based on the target offset may include, but is not limited to: for each pixel in the current line structured light image, determining the rotation compensation amount corresponding to the pixel based on the difference between the horizontal coordinate of the pixel and the horizontal coordinate of a specified pixel and the target slope; and determining the target offset as the height compensation amount corresponding to each pixel.

[0037] For example, vibration compensation is performed on the height coordinate value of the initial three-dimensional coordinates based on rotation compensation and height compensation to obtain the target three-dimensional coordinates of the object under test. This includes, but is not limited to: for each pixel in the current line structured light image, vibration compensation is performed on the height coordinate value corresponding to the pixel in the initial three-dimensional coordinates based on the rotation compensation and height compensation corresponding to the pixel to obtain the target three-dimensional coordinates of the object under test, where the target three-dimensional coordinates include the height coordinate value after vibration compensation.

[0038] As can be seen from the above technical solutions, in the embodiments of this application, a compensation table can be pre-calibrated. The compensation table includes the mapping relationship between image interval and slope, and also includes the mapping relationship between image interval and offset. During 3D reconstruction, the compensation table is queried based on the target image interval between the current line structured light image and the first line structured light image to obtain the target slope and target offset. Then, based on the target slope and target offset, vibration compensation is performed on the initial 3D coordinates to obtain the target 3D coordinates. Image reconstruction is performed based on the target 3D coordinates to obtain the 3D reconstructed image of the measured object. The target slope and target offset can compensate for the error caused by vibration, reducing the impact of vibration on measurement accuracy. When obtaining the 3D coordinates of the measured object, accurate and reliable 3D coordinates can be obtained, and then an accurate 3D reconstructed image can be obtained based on these 3D coordinates. By evaluating the average vibration on the time axis and calculating the relative vibration amplitude at each time point, the average vibration of all frames is used as a reference, and all contours are aligned with the reference, effectively suppressing the impact of vibration on the contours of a single frame.

[0039] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0040] A three-dimensional imaging device can consist of a laser and a camera. In addition to the laser and camera, the three-dimensional imaging device may also include a processor (for implementing the image reconstruction method of this embodiment), a galvanometer motor and a galvanometer drive. There are no restrictions on the structure of the three-dimensional imaging device. It can be any device with three-dimensional imaging function, such as any three-dimensional imaging device in the fields of machine vision or industrial automation.

[0041] In the 3D reconstruction process of a 3D imaging device, a laser projects line structured light onto the surface of the object being measured, and a camera captures an image of the object, obtaining a line structured light image. The center line of the light stripes in the line structured light image is acquired, and then transformed according to pre-calibrated sensor parameters (such as laser and camera parameters) to obtain the 3D coordinates of the object. Based on these 3D coordinates, 3D reconstruction (3D remodeling) of the object can be achieved, resulting in a 3D reconstructed image.

[0042] However, during the scanning process (laser projecting structured light onto the surface of the object being measured, and camera capturing an image of the object), 3D imaging equipment is affected by vibrations from the surrounding environment. These vibrations can be caused by the movement of the 3D imaging equipment itself or the inherent vibrations of the object being measured. Such vibrations can lead to changes in the distance or angle of the 3D imaging equipment relative to the object at different points in time during the scanning process. For example, when the 3D imaging equipment is scanning a plane, severe vibrations can cause the resulting point cloud to exhibit random jagged edges, making it difficult to guarantee measurement accuracy and stability. In summary, due to the influence of vibration, 3D imaging equipment cannot obtain accurate 3D reconstructed images based on 3D coordinates.

[0043] In response to the above findings, this application proposes an image reconstruction method. By pre-calibrating a compensation table, the error caused by vibration can be reduced or eliminated, thereby reducing the impact of vibration on measurement accuracy, improving the measurement stability of the laser, and obtaining an accurate three-dimensional reconstructed image.

[0044] This image reconstruction method can be applied to 3D imaging devices, which may include a camera and a laser. The laser can be a line laser sensor, also known as a profilometer. A line laser sensor is a sensor that reconstructs the contour of an object's surface by photographing the deformation of a laser line on the object's surface.

[0045] The image reconstruction method in this embodiment involves a compensation table calibration process and a vibration compensation process. During the compensation table calibration process, a compensation table (such as a vibration compensation table) needs to be calibrated. This compensation table can include the mapping relationship between image intervals and slopes, and it can also include the mapping relationship between image intervals and offsets. During the vibration compensation process, vibration compensation needs to be performed based on the compensation table to obtain a three-dimensional reconstructed image of the measured object. The compensation table calibration process and the vibration compensation process are described below.

[0046] First, the process of calibrating the compensation table.

[0047] Before the 3D imaging equipment leaves the factory, a sample object (i.e., the object under test in the compensation table calibration process) and the 3D imaging equipment can be deployed in the target scene (also known as the test scene or calibration scene). The 3D imaging equipment includes a laser and a camera. The laser projects line structured light onto the surface of the sample object, and the camera takes pictures of the surface of the sample object to obtain a line structured light image of the sample object.

[0048] Referring to Figure 2, which is a flowchart of the compensation table calibration process, the process may include:

[0049] Step 201: During the movement of the laser or the sample object, multiple line structured lights are projected onto the sample object by the laser, and multiple line structured light images of the sample object are acquired by the camera.

[0050] For example, a sample object and a 3D imaging device are deployed in the target scene. A laser is used to project line structured light onto the surface of the sample object, and a camera is used to capture images of the sample object's surface, resulting in a line structured light image a1. Then, the sample object is moved, and line structured light is projected onto its surface using a laser, and the camera is used to capture images of the sample object's surface, resulting in a line structured light image a2. This process can be repeated to obtain multiple line structured light images of the sample object.

[0051] For example, after obtaining the line structured light image a1, the laser is moved to project line structured light onto the surface of the sample object, and a camera is used to photograph the surface of the sample object to obtain the line structured light image a2 of the sample object. In this way, multiple line structured light images of the sample object can be obtained.

[0052] For example, after obtaining the line structured light image a1, the sample object and the laser are moved simultaneously. The laser is used to project line structured light onto the sample object, and a camera is used to take a picture of the sample object to obtain the line structured light image a2 of the sample object. In this way, multiple line structured light images of the sample object can be obtained.

[0053] Step 202: For each line structured light image, determine the corresponding height coordinates.

[0054] For example, after obtaining a line structured light image, the center line of the light stripes in the image can be acquired. This center line can then be transformed according to pre-calibrated parameters to obtain the three-dimensional coordinates of the sample object. This process is not restricted and is the working principle of a 3D imaging device. Clearly, the three-dimensional coordinates of the sample object include length, width, and height coordinates. The height coordinate is the height coordinate corresponding to the line structured light image, and it represents the distance between the sample object and the laser.

[0055] Step 203: For each line structured light image, determine the height difference based on the height coordinates of the line structured light image and the height coordinates of the previous line structured light image. This height difference is denoted as dz.

[0056] For example, line structured light image a1 is the first line structured light image and serves as a reference for subsequent line structured light images; therefore, it is not necessary to calculate the height difference corresponding to line structured light image a1. For line structured light image a2, the difference between the height coordinates corresponding to line structured light image a2 and the height coordinates corresponding to line structured light image a1 is calculated, and this difference is taken as the height difference corresponding to line structured light image a2. For line structured light image a3, the difference between the height coordinates corresponding to line structured light image a3 and the height coordinates corresponding to line structured light image a2 is calculated, and this difference is taken as the height difference corresponding to line structured light image a3, and so on, to obtain the height difference corresponding to each line structured light image. In this embodiment, the laser can acquire multiple line structured light images, and the "first" line structured light image represents the first acquired line structured light image.

[0057] Step 204: For each line structured light image, determine the first slope corresponding to the line structured light image based on the height difference corresponding to the line structured light image. The first slope represents the difference in rotational vibration between adjacent frames.

[0058] For example, a reference region can be defined in the line structured light image. The reference region is also called the ROI (Region of Interest). It means that the vibration assessment is performed using data located within the reference region in the line structured light image. The reference region can be set according to user needs, and there are no restrictions on it.

[0059] For example, multiple pixels can be selected from the reference area of ​​the line structured light image. For instance, all pixels can be selected from the reference area of ​​the line structured light image, or some pixels can be selected from the reference area of ​​the line structured light image. There is no limitation on this, as long as multiple pixels can be selected.

[0060] For example, multiple coordinate points corresponding to the selected multiple pixels can be constructed in a coordinate system. For each coordinate point corresponding to a pixel, the horizontal coordinate of the coordinate point can be the column identifier of the pixel in the line structured light image, and the vertical coordinate of the coordinate point can be the height difference between the height coordinate of the pixel in the line structured light image and the height coordinate of the pixel in the previous line structured light image.

[0061] For example, taking the line structured light image a2 as an example, in step 203, the height difference corresponding to the line structured light image a2 may include the height difference corresponding to each pixel of the line structured light image a2, that is, the height difference between the height coordinate of the pixel corresponding to the line structured light image a2 and the height coordinate of the pixel corresponding to the line structured light image a1, that is, the height difference corresponding to each pixel of the line structured light image a2.

[0062] In step 204, after selecting multiple pixels from the reference region of the line structured light image a2, for each selected pixel, a coordinate point can be constructed in the coordinate system. The horizontal coordinate of this coordinate point can be the column identifier (column id) of the pixel in the line structured light image a2, indicating which column the pixel is located in. The vertical coordinate of this coordinate point can be the height difference corresponding to the pixel, that is, the height difference between the height coordinate of the pixel in the line structured light image a2 and the height coordinate of the pixel in the line structured light image a1, representing the height difference between two adjacent frames.

[0063] In summary, multiple coordinate points corresponding to multiple pixels can be constructed in a coordinate system.

[0064] For example, after constructing multiple coordinate points corresponding to multiple pixels in the coordinate system, a straight line can be fitted based on the multiple coordinate points to obtain the first slope k corresponding to the structured light image. For instance, the least squares method can be used to fit a straight line to the multiple coordinate points to obtain the first slope k, or other fitting algorithms can be used to fit a straight line to the multiple coordinate points to obtain the first slope k; there are no restrictions on this.

[0065] Step 205: For each line structured light image, based on the first slope corresponding to the line structured light image and the first slope corresponding to each line structured light image preceding the line structured light image, determine the second slope of the line structured light image relative to the first line structured light image, that is, the rotational vibration difference relative to the first frame.

[0066] For example, the first slope corresponding to the line structured light image can be converted into an angle value. The first slope corresponding to each line structured light image preceding the line structured light image can also be converted into an angle value. Then, these angle values ​​are summed, and the summed angle values ​​are converted into a slope. This slope is the second slope.

[0067] For example, suppose the first slope of line structured light image a1 is k1 (k1 can be 0 for the first frame), the first slope of line structured light image a2 is k2, the first slope of line structured light image a3 is k3, the first slope of line structured light image a4 is k4, and so on.

[0068] For a line structured light image a1, based on the first slope k1 corresponding to the line structured light image a1, the second slope k1 of the line structured light image a1 relative to the line structured light image a1 is determined, and k1 can be 0.

[0069] For a line structured light image a2, based on the first slope k2 corresponding to line structured light image a2, and the first slope (i.e., k1) corresponding to each line structured light image preceding line structured light image a2, determine the angle value α2 corresponding to the first slope k2 and the angle value α1 corresponding to the first slope k1. Determine the sum of the angle values ​​α2 and α1, and determine the slope value corresponding to this sum of angles. This slope value is denoted as k2'. The second slope of line structured light image a2 relative to line structured light image a1 is determined as k2'. In some examples, the sum of the angle values ​​α2 and α1 is α1 + α2.

[0070] For the line structured light image a3, based on the first slope k3 corresponding to the line structured light image a3 and the first slopes (i.e., k1 and k2) corresponding to each line structured light image preceding the line structured light image a3, the angle value α3 corresponding to the first slope k3, the angle value α2 corresponding to the first slope k2, and the angle value α1 corresponding to the first slope k1 are determined. Then, the angle sum of the angle values ​​α3, α2, and α1 is determined, and the slope value corresponding to this angle sum is determined. This slope value is recorded as k3', and the second slope of the line structured light image a3 relative to the line structured light image a1 is determined to be k3'.

[0071] For the line structured light image a4, based on the first slope k4 corresponding to the line structured light image a4 and the first slopes (i.e., k1, k2, and k3) corresponding to each line structured light image preceding the line structured light image a4, the angle values ​​α4, α3, α2, and α1 corresponding to the first slope k4, α2, and k1 are determined. Then, the sum of the angle values ​​α4, α3, α2, and α1 is determined, and the slope value corresponding to this sum of angle values ​​is determined. This slope value is recorded as k4', and the second slope of the line structured light image a4 relative to the line structured light image a1 is determined to be k4'.

[0072] By analogy, a second slope can be obtained for each line structured light image relative to the first line structured light image.

[0073] In the above process, the relationship between the slope value and the angle value is tan(α) = k. Based on this relationship, the angle value corresponding to the first slope can be determined, and the sum of the angles can be converted into the second slope.

[0074] In summary, we can calculate the rotational vibration difference between two adjacent frames (i.e., the first slope corresponding to the line structured light image), and then accumulate the rotational vibration difference to obtain the rotational vibration difference of the line structured light image relative to the first line structured light image (i.e., the second slope corresponding to the line structured light image).

[0075] Step 206: For each line structured light image, record the mapping relationship between the image interval of the line structured light image and the second slope (i.e., the second slope of the line structured light image relative to the first line structured light image) in the compensation table. The image interval represents the number of images between the line structured light image and the first line structured light image.

[0076] Referring to Table 1, which serves as an example of a compensation table, the image interval corresponding to the first line structured light image a1 is 0, and the second slope is k1; the image interval corresponding to the line structured light image a2 is 1, and the second slope is k2', and so on. Here, image interval 1 indicates that the number of images between line structured light image a2 and line structured light image a1 is 1, and k2' represents the second slope of line structured light image a2 relative to line structured light image a1; image interval 2 indicates that the number of images between line structured light image a3 and line structured light image a1 is 2, and k3' represents the second slope of line structured light image a3 relative to line structured light image a1, and so on. Table 1

[0077] Step 207: For each line structured light image, perform rotation compensation on the height coordinates of the line structured light image based on the first slope corresponding to the line structured light image to obtain the rotated and compensated height coordinates.

[0078] For example, for each pixel in a line structured light image, the x-coordinate (x), y-coordinate (y), and height coordinate (z) of that pixel can be determined. x represents the x-coordinate of the pixel in the line structured light image, y represents the y-coordinate, and (x, y) represents the pixel coordinates in the line structured light image. z represents the height coordinate of the pixel, that is, the distance between the physical location (world coordinate system position) of the pixel on the sample object and the laser.

[0079] For each pixel in a line structured light image, the rotation compensation value can be determined based on the pixel's x-coordinate, the x-coordinate of the center pixel of the reference region (or other pixels in the reference region), and the first slope corresponding to the line structured light image. For example, the rotation compensation value (also called the rotation compensation amount) can be determined using the following formula (1). Of course, the following formula (1) is just an example, and there are no restrictions on how the rotation compensation value is determined. 补 =(xx) z中心 )·k 当前帧 Formula (1)

[0080] z 补 This represents the rotation compensation value corresponding to this pixel, where x represents the x-coordinate of this pixel. z中心k represents the x-coordinate of the center pixel of the reference region in a line structured light image. 当前帧 This represents the first slope corresponding to the line structured light image. Clearly, the rotation compensation value for each pixel in the line structured light image can be obtained.

[0081] For example, for each pixel in a line structured light image, the height coordinates of that pixel can be rotated based on the rotation compensation value corresponding to that pixel to obtain the rotated-compensated height coordinates. For instance, given the height coordinate z and the rotation compensation value z of the pixel... 补 It can calculate the height coordinate z and the rotation compensation value z. 补 The sum of these two values ​​is the height coordinate after rotation compensation. After performing the above processing on each pixel of the line structured light image, the rotation-compensated height coordinate of each pixel is obtained.

[0082] Step 208: For each line structured light image, determine the height difference after rotation compensation based on the rotation-compensated height coordinates of the line structured light image and the rotation-compensated height coordinates of the previous line structured light image.

[0083] For example, for a line structured light image a2, the difference between the rotationally compensated height coordinates of line structured light image a2 and the height coordinates of line structured light image a1 (without rotation compensation) can be calculated. This difference is taken as the rotationally compensated height difference for line structured light image a2. Similarly, for a line structured light image a3, the difference between the rotationally compensated height coordinates of line structured light image a3 and the rotationally compensated height coordinates of line structured light image a2 can be calculated. This difference is taken as the rotationally compensated height difference for line structured light image a3.

[0084] By analogy, the height difference after rotation compensation for each line structured light image can be obtained.

[0085] Step 209: For each line structured light image, determine the first offset corresponding to the line structured light image based on the height difference after rotation compensation. The first offset represents the height vibration difference between adjacent frames.

[0086] For example, a reference region of the line structured light image can be defined, and multiple pixels can be selected from this reference region. For instance, all pixels can be selected from the reference region, or only a portion of the pixels can be selected. After obtaining multiple pixels, the rotation-compensated height difference corresponding to these pixels can be obtained. For each pixel, the rotation-compensated height difference is the height difference between the pixel's rotation-compensated height coordinates in the current line structured light image and its rotation-compensated height coordinates in the previous line structured light image.

[0087] For example, taking the line structured light image a3 as an example, in step 208, the height difference corresponding to the rotation-compensated line structured light image a3 can include the height difference corresponding to each pixel of the line structured light image a3 after rotation compensation. That is, the height difference between the height coordinates of the pixel corresponding to the rotation-compensated line structured light image a3 and the height coordinates of the pixel corresponding to the rotation-compensated line structured light image a2. In other words, each pixel corresponds to a height difference after rotation compensation. In step 209, for each selected pixel, the height difference corresponding to that pixel after rotation compensation can be determined.

[0088] For example, after selecting multiple pixels from the reference region of the line structured light image, the first offset b corresponding to the line structured light image can be determined based on the average value of the height difference after rotation compensation corresponding to the multiple pixels. For example, the first offset b can be the average value of the height difference after rotation compensation corresponding to the multiple pixels.

[0089] Step 210: For each line structured light image, based on the first offset corresponding to the line structured light image and the first offset corresponding to each line structured light image preceding the line structured light image, determine the second offset of the line structured light image relative to the first line structured light image, that is, the height vibration difference relative to the first frame.

[0090] For example, based on the first offset corresponding to the line structured light image and the first offset corresponding to each line structured light image preceding the line structured light image, the total offset value of the line structured light image relative to the first line structured light image (i.e., the sum of the first offsets mentioned above) can be calculated. Then, the average value of the total offset values ​​of all line structured light images is calculated. Then, the difference between the first offset corresponding to the line structured light image and the average value of the total offset value is taken as the second offset of the line structured light image relative to the first line structured light image.

[0091] For example, suppose the first offset corresponding to line structured light image a1 is b1 (for the first frame, b1 can be 0), the first offset corresponding to line structured light image a2 is b2, the first offset corresponding to line structured light image a3 is b3, the first offset corresponding to line structured light image a4 is b4, and so on.

[0092] The total offset of line structured light image a1 relative to the first line structured light image is b1; the total offset of line structured light image a2 relative to the first line structured light image is b2+b1, and so on, denoted as b2'; the total offset of line structured light image a3 relative to the first line structured light image is b3+b2+b1, and so on, denoted as b3'; the total offset of line structured light image a4 relative to the first line structured light image is b4+b3+b2+b1, and so on, denoted as b4'. Calculate the average of the total offset values ​​of all line structured light images, such as the average of b1, b2', b3', and b4'.

[0093] For line structured light image a1, the second offset of line structured light image a1 relative to line structured light image a1 is b1 minus the average of the total offset values; for line structured light image a2, the second offset of line structured light image a2 relative to line structured light image a1 is b2 minus the average of the total offset values; for line structured light image a3, the second offset of line structured light image a3 relative to line structured light image a1 is b3 minus the average of the total offset values; for line structured light image a4, the second offset of line structured light image a4 relative to line structured light image a1 is b4 minus the average of the total offset values. And so on, the second offset corresponding to each line structured light image can be obtained.

[0094] In summary, we can calculate the height vibration difference between two adjacent frames (i.e., the first offset corresponding to the line structured light image), then take the average of the height vibration differences between the line structured light image and the previous frame to obtain the mean vibration, and then calculate the difference between the first offset of the line structured light image and the mean vibration, i.e., the height vibration difference of the line structured light image relative to the first line structured light image (i.e., the second offset corresponding to the line structured light image).

[0095] Step 211: For each line structured light image, record the mapping relationship between the image interval of the line structured light image and the second offset (i.e., the second offset of the line structured light image relative to the first line structured light image) in the compensation table. The image interval represents the number of images between the line structured light image and the first line structured light image.

[0096] Referring to Table 2, which is an example of a compensation table, b2-x represents the second offset of line structured light image a2 relative to line structured light image a1, b3-x represents the second offset of line structured light image a3 relative to line structured light image a1, and so on. Table 2

[0097] This completes the calibration process of the compensation table. The compensation table can be stored in the 3D imaging device, and vibration compensation can be performed based on the compensation table when the 3D imaging device is used at the factory.

[0098] In one possible implementation, referring to steps 204 and 209, it is necessary to divide the reference region (ROI) of the line structured light image. The reference region can be set based on user needs and there are no restrictions on it.

[0099] After dividing the reference region, the reference region can remain unchanged, meaning that all frames use the same reference region. For example, if the contour height and contour angle of the reference region remain stable, as shown in Figure 3A, which is a schematic diagram of stable contour height and contour angle, then all frames use the same reference region, meaning that the reference region of the current frame's line structured light image is the same as the reference region of the previous line structured light image.

[0100] The reference region may also change, meaning different frames may use different reference regions. For example, if the contour height and / or contour angle of the reference region changes, as shown in Figure 3B, which is a schematic diagram of the change in contour height and / or contour angle, then different frames may use different reference regions. That is, the reference region of the current frame's line structured light image may be different from the reference region of the previous line structured light image.

[0101] For example, considering that the laser or sample object may not move strictly along the reference region during actual scanning, it is necessary to support the assessment of misalignment vibration in the X direction. See Figure 3B, which illustrates the vibration difference between two adjacent frames and the misalignment scenario in the X direction. To support the assessment of misalignment vibration in the X direction, the reference region of the current frame's line structured light image can be determined based on the reference region of the previous line structured light image and the X-direction offset value. The X-direction offset value represents the X-direction offset between the reference region of the current frame's line structured light image and the reference region of the previous line structured light image.

[0102] For example, if there is an X-direction offset between the reference regions of two consecutive frames, the X-direction offset value can be determined (e.g., based on the moving speed of the laser or sample object, or the interval between two adjacent frames). The reference region of the current frame's line structured light image can be obtained by offsetting the reference region of the previous line structured light image by this X-direction offset value.

[0103] In one possible implementation, considering that the reference region of interest (ROI) is drawn within a frame contour (such as the image boundary of the first frame of the line structured light image), then when evaluating vibration in the line structured light images of the remaining frames, the reference region may have two modes: a following mode and a fixed mode. Depending on actual needs, the reference region can be determined using either the following mode or the fixed mode.

[0104] Referring to Figure 3C, which illustrates the following mode for the ROI coverage area (i.e., the reference area), the following mode can be used in application scenarios with small X-direction offsets. For example, in application scenarios with small X-direction offsets, the following mode can be used to determine the reference area. In the following mode, the reference area will not exceed the image boundary; that is, the portion exceeding the image boundary does not belong to the reference area.

[0105] For example, in Figure 3C, each row of the reference area includes 5 pixels. For the first row, the second row, the penultimate row, and the second-to-last row, the number of pixels in the reference area is less than 5 pixels, indicating that the part that exceeds the image boundary does not belong to the reference area. That is, the delineated part of the area does not belong to the reference area.

[0106] Based on this, in follow mode, when determining the reference region of the current frame's line structured light image based on the reference region of the previous line structured light image and the X-direction offset value, if a pixel after offsetting the reference region by the X-direction offset value exceeds the image boundary, then the pixel exceeding the image boundary does not belong to the reference region of the current frame's line structured light image. If a pixel after offsetting the reference region by the X-direction offset value does not exceed the image boundary, then the pixel not exceeding the image boundary belongs to the reference region of the current frame's line structured light image.

[0107] Referring to Figure 3D, this is a schematic diagram of a fixed mode for the ROI coverage area (i.e., the reference area). The fixed mode can be used in applications where the scanning direction is completely parallel to the X-axis (such as wire trace measurement on silicon wafers). For example, in applications where the scanning direction is completely parallel to the X-axis, the fixed mode can be used to determine the reference area. In the fixed mode, each row of the reference area includes the same number of pixels, and the pixels in each row of the reference area need to be aligned; that is, the pixels in all rows correspond to the same column.

[0108] For example, in Figure 3D, each row of the reference area consists of 5 pixels, the first pixel of all rows corresponds to the same column, the second pixel of all rows corresponds to the same column, and so on.

[0109] Based on this, in the fixed mode, when determining the reference region of the current frame's line structured light image based on the reference region and X-direction offset value of the previous line structured light image, if the X-direction offset value is applied to a certain row of pixels in the reference region, the reference region of the current frame's line structured light image can be obtained. That is, the reference region can be obtained by satisfying the requirement that all rows of pixels correspond to the same column.

[0110] Second, the vibration compensation process.

[0111] When a 3D imaging device is used at the factory, if the data of multiple frames of contour obtained from scanning the actual scene changes within the reference area, it is considered that the change is caused by vibration. The entire frame contour is corrected by calculating the rotation component and height translation component of this change. This process is called vibration compensation.

[0112] Referring to Figure 4, which is a flowchart of the vibration compensation process, the process may include:

[0113] Step 401: When the laser projects line structured light onto the object under test, acquire the current line structured light image of the object under test captured by the camera, and determine the initial three-dimensional coordinates of the object under test based on the current line structured light image.

[0114] For example, a laser is used to project line structured light onto the object under test, and a camera is used to capture the object under test to obtain a line structured light image w1 of the object under test. The initial three-dimensional coordinates of the object under test are determined based on the line structured light image w1. No vibration compensation is required in the first frame. A three-dimensional reconstructed image of the object under test can be generated based on the initial three-dimensional coordinates of the object under test. There are no restrictions on this process.

[0115] Then, move the object under test and / or the laser, use the laser to project line structured light onto the object under test, and use a camera to take a picture of the object under test to obtain a line structured light image w2 of the object under test. Based on the line structured light image w2, determine the initial three-dimensional coordinates of the object under test. These initial three-dimensional coordinates need to be vibrated. Vibration compensation is described in subsequent steps. In this way, multiple line structured light images can be obtained.

[0116] For example, the initial three-dimensional coordinates may include length coordinates, width coordinates, and height coordinates, with the height coordinates representing the distance between the surface of the object being measured and the laser.

[0117] Step 402: After obtaining the current line structured light image, determine the target image interval between the current line structured light image and the first line structured light image, that is, the number of images between these two frames.

[0118] For example, after obtaining the line structured light image w2, the target image interval between line structured light image w2 and line structured light image w1 can be 1. After obtaining the line structured light image w3, the target image interval between line structured light image w3 and line structured light image w1 can be 2, and so on.

[0119] Step 403: Query the compensation table based on the target image interval to obtain the target slope and target offset.

[0120] For example, as shown in Table 2, which is an example of a compensation table, this compensation table can include the mapping relationship between image intervals and slopes, and the mapping relationship between image intervals and offsets. Based on this, when querying this compensation table based on the target image interval, the target slope corresponding to the target image interval can be obtained, and the target offset corresponding to the target image interval can also be obtained.

[0121] Step 404: Determine the rotational compensation amount based on the target slope, that is, the compensation amount for the difference in rotational vibration.

[0122] For example, for each pixel in the current line structured light image, the x-coordinate (x), y-coordinate (y), and z-coordinate (z) of that pixel can be determined. The z-coordinate is the distance between the physical position of the pixel on the object being measured and the laser. The rotation compensation amount corresponding to the pixel can be determined based on the difference between the x-coordinate of the pixel and the x-coordinate of a specified pixel (such as the center pixel of the current line structured light image, the center pixel of the reference area of ​​the current line structured light image, or other pixels of the current line structured light image, without limitation), and the target slope. For example, the rotation compensation amount corresponding to the pixel can be determined using formula (1). In formula (1), z 补 This represents the rotation compensation amount corresponding to the pixel, where x represents the x-coordinate of the pixel. z中心 k represents the x-coordinate of a specified pixel. 当前帧 This represents the slope of the target. Clearly, the rotation compensation amount for each pixel in the current line structured light image can be obtained.

[0123] Step 405: Determine the height compensation amount based on the target offset, that is, the compensation amount for height vibration differences.

[0124] For each pixel in the current line structured light image, the target offset is determined as the height compensation amount corresponding to each pixel. That is, all pixels in the current line structured light image use the same height compensation amount.

[0125] Step 406: Based on the rotation compensation amount and the height compensation amount, perform vibration compensation on the height coordinate value of the initial three-dimensional coordinates to obtain the target three-dimensional coordinates of the object under test.

[0126] For example, for each pixel in the current line structured light image, vibration compensation can be performed on the height coordinate value of the pixel in the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount corresponding to the pixel. For example, the height coordinate value after vibration compensation can be the rotation compensation amount + the height compensation amount + the height coordinate value corresponding to the initial three-dimensional coordinates.

[0127] For example, after applying vibration compensation to the height coordinate value corresponding to each pixel in the initial 3D coordinate system, the vibration-compensated target 3D coordinate system can be obtained. The height coordinate value in the target 3D coordinate system is the vibration-compensated height coordinate value, while the length and width coordinate values ​​in the target 3D coordinate system remain unchanged from the length and width coordinate values ​​in the initial 3D coordinate system.

[0128] Step 407: Generate a 3D reconstructed image of the object under test based on the target's 3D coordinates.

[0129] As can be seen from the above technical solutions, in this embodiment, a compensation table can be pre-calibrated, and the error caused by vibration can be compensated by the target slope and target offset, reducing the impact of vibration on measurement accuracy. When the three-dimensional coordinates of the measured object are obtained, accurate and reliable three-dimensional coordinates can be obtained, and then an accurate three-dimensional reconstructed image can be obtained based on these three-dimensional coordinates. By evaluating the average vibration on the time axis, the relative vibration amplitude at each time point is calculated, and the vibration is evaluated by subtracting the point clouds of adjacent frames in the reference region. Using the mean vibration of all frames as the reference, all contours are aligned with the reference, effectively suppressing the impact of vibration on the contours of a single frame.

[0130] Based on the same application concept as the above method, this application proposes an image reconstruction device, which is applied to a three-dimensional imaging device, including a camera and a laser. Referring to Figure 5, which is a schematic diagram of the structure of the image reconstruction device 50, the device may include: an acquisition module 51, a determination module 52, and a reconstruction module 53.

[0131] The acquisition module 51 is used to acquire the current line structured light image of the object under test captured by the camera when the laser projects line structured light onto the object under test; determine the initial three-dimensional coordinates of the object under test based on the current line structured light image; and query the acquired compensation table based on the target image interval between the current line structured light image and the first line structured light image to obtain the target slope and target offset; wherein, the compensation table includes the mapping relationship between the image interval and the slope, and the compensation table includes the mapping relationship between the image interval and the offset.

[0132] The determining module 52 is used to determine the rotation compensation amount based on the target slope and the height compensation amount based on the target offset; and to perform vibration compensation on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the object under test.

[0133] The reconstruction module 53 is used to generate a three-dimensional reconstructed image of the object under test based on the target three-dimensional coordinates.

[0134] For example, when the acquisition module 51 acquires the compensation table, it is specifically used for: during the movement of the laser or the sample object, projecting multiple line structured lights onto the sample object through the laser, and acquiring multiple line structured light images of the sample object captured by the camera; for each line structured light image, determining the height coordinates corresponding to the line structured light image; determining the height difference based on the height coordinates corresponding to the line structured light image and the height coordinates corresponding to the previous line structured light image; determining the first slope corresponding to the line structured light image based on the height difference; determining the second slope of the line structured light image relative to the first line structured light image based on the first slope corresponding to the line structured light image and the first slope corresponding to each line structured light image preceding the line structured light image; and recording the mapping relationship between the image interval of the line structured light image and the second slope in the compensation table, where the image interval represents the number of images between the line structured light image and the first line structured light image.

[0135] For example, when the acquisition module 51 determines the first slope corresponding to the line structured light image based on the height difference, it is specifically used to: select multiple pixels from the reference area of ​​the line structured light image; construct multiple coordinate points corresponding to the multiple pixels in the coordinate system; wherein, for each coordinate point corresponding to a pixel, the horizontal coordinate of the coordinate point is the column identifier corresponding to the pixel in the line structured light image, and the vertical coordinate of the coordinate point is the height difference between the height coordinate of the pixel in the line structured light image and the height coordinate of the pixel in the previous line structured light image; and perform line fitting based on the multiple coordinate points to obtain the first slope corresponding to the line structured light image.

[0136] For example, when the acquisition module 51 acquires the compensation table, it specifically performs the following: for each line structured light image, it performs rotation compensation on the height coordinates corresponding to the line structured light image based on the first slope corresponding to the line structured light image to obtain the rotation-compensated height coordinates; it determines the rotation-compensated height difference based on the rotation-compensated height coordinates and the rotation-compensated height coordinates corresponding to the previous line structured light image; it determines the first offset corresponding to the line structured light image based on the rotation-compensated height difference; it determines the second offset of the line structured light image relative to the first line structured light image based on the first offset corresponding to the line structured light image and the first offset corresponding to each line structured light image preceding the line structured light image; and it records the mapping relationship between the image interval of the line structured light image and the second offset in the compensation table.

[0137] For example, when the acquisition module 51 determines the first offset corresponding to the line structured light image based on the height difference after rotation compensation, it is specifically used to: select multiple pixels from the reference area of ​​the line structured light image; acquire the height difference after rotation compensation corresponding to the multiple pixels; and determine the first offset corresponding to the line structured light image based on the average value of the height differences after rotation compensation corresponding to the multiple pixels; wherein, for each pixel, the height difference after rotation compensation corresponding to the pixel is the height difference between the height coordinate of the pixel after rotation compensation in the line structured light image and the height coordinate of the pixel after rotation compensation in the previous line structured light image.

[0138] For example, the acquisition module 51 is further configured to determine the reference region of the line structured light image based on the reference region of the previous line structured light image and the X-direction offset value before selecting multiple pixels from the reference region of the line structured light image; wherein the X-direction offset value represents the X-direction offset between the reference region of the line structured light image and the reference region of the previous line structured light image.

[0139] For example, when determining the rotation compensation amount based on the target slope and the height compensation amount based on the target offset, the determining module 52 specifically performs the following: for each pixel in the current line structured light image, based on the difference between the horizontal coordinate of the pixel and the horizontal coordinate of a specified pixel and the target slope, determines the rotation compensation amount corresponding to the pixel; and determines the target offset as the height compensation amount corresponding to each pixel. When the determining module 52 performs vibration compensation on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the object under test, it specifically performs the following: for each pixel in the current line structured light image, based on the rotation compensation amount and the height compensation amount corresponding to the pixel, performs vibration compensation on the height coordinate value corresponding to the pixel in the initial three-dimensional coordinates to obtain the target three-dimensional coordinates of the object under test.

[0140] Based on the same application concept as the above method, this application proposes an electronic device, as shown in FIG6. The electronic device includes: a processor 61 and a machine-readable storage medium 62, wherein the machine-readable storage medium 62 stores machine-executable instructions that can be executed by the processor 61; the processor 61 is used to execute the machine-executable instructions to implement the image reconstruction method disclosed in the above example of this application.

[0141] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the image reconstruction method disclosed in the above examples of this application.

[0142] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM, volatile memory, non-volatile memory, flash memory, storage drive, solid-state drive, any type of storage disk, or similar storage media, or combinations thereof.

[0143] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. An image reconstruction method, characterized in that, The method is applied to a three-dimensional imaging device, the three-dimensional imaging device including a camera and a laser, and the method includes: When the laser projects line structured light onto the object under test, the camera acquires the current line structured light image of the object under test; the initial three-dimensional coordinates of the object under test are determined based on the current line structured light image. Based on the target image interval between the current line structured light image and the first line structured light image, the obtained compensation table is queried to obtain the target slope and target offset; wherein, the compensation table includes the mapping relationship between image interval and slope, and the mapping relationship between image interval and offset; The rotation compensation amount is determined based on the target slope, and the height compensation amount is determined based on the target offset; Vibration compensation is performed on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the measured object. A three-dimensional reconstructed image of the object under test is generated based on the target's three-dimensional coordinates.

2. The method according to claim 1, characterized in that, The process of obtaining the compensation table includes: During the movement of the laser or the sample object, multiple line structured lights are projected onto the sample object through the laser, and multiple line structured light images of the sample object are acquired by the camera. For each line structured light image, determine the height coordinates corresponding to the line structured light image; determine the height difference based on the height coordinates corresponding to the line structured light image and the height coordinates corresponding to the previous line structured light image; determine the first slope corresponding to the line structured light image based on the height difference, where the first slope indicates the difference in rotational vibration between two adjacent images. Based on the first slope corresponding to the line structured light image and the first slope corresponding to each line structured light image preceding the line structured light image, a second slope is determined, wherein the second slope indicates the difference in rotational vibration of the line structured light image relative to the first line structured light image. The compensation table records the mapping relationship between the image interval of the line structured light image and the second slope. The image interval represents the number of images between the line structured light image and the first line structured light image.

3. The method according to claim 2, characterized in that, Determining the first slope corresponding to the line structured light image based on the height difference includes: Multiple pixels are selected from the reference region of the line structured light image; Construct multiple coordinate points corresponding to the multiple pixels in a coordinate system; wherein, for each coordinate point corresponding to a pixel, the horizontal coordinate of the coordinate point is the column identifier of the pixel in the line structured light image, and the vertical coordinate of the coordinate point is the height difference between the height coordinate of the pixel in the line structured light image and the height coordinate of the pixel in the previous line structured light image. Based on the multiple coordinate points, a straight line is fitted to obtain the first slope corresponding to the structured light image.

4. The method according to claim 2, characterized in that, The process of obtaining the compensation table also includes: For each line structured light image, the height coordinates of the line structured light image are rotated and compensated based on the first slope corresponding to the line structured light image to obtain the rotated and compensated height coordinates. The height difference after rotation compensation is determined based on the height coordinates after rotation compensation and the height coordinates after rotation compensation of the previous line structured light image, and the first offset corresponding to the line structured light image is determined based on the height difference after rotation compensation. Based on the first offset corresponding to the line structured light image and the first offset corresponding to each line structured light image preceding the line structured light image, the second offset of the line structured light image relative to the first line structured light image is determined. The compensation table records the mapping relationship between the image interval of the line structured light image and the second offset.

5. The method according to claim 4, characterized in that, The determination of the first offset corresponding to the line structured light image based on the height difference after rotation compensation includes: Multiple pixels are selected from the reference region of the line structured light image; Obtain the height difference after rotation compensation corresponding to the plurality of pixels, and determine the first offset corresponding to the line structured light image based on the average value of the height difference after rotation compensation corresponding to the plurality of pixels; wherein, for each pixel, the height difference after rotation compensation corresponding to the pixel is the height difference between the height coordinate of the pixel after rotation compensation in the line structured light image and the height coordinate of the pixel after rotation compensation in the previous line structured light image.

6. The method according to claim 3 or 5, characterized in that, Before selecting multiple pixels from the reference region of the line structured light image, the method further includes: The reference region of the current line structured light image is determined based on the reference region of the previous line structured light image and the X-direction offset value; wherein, the X-direction offset value represents the X-direction offset between the reference region of the current line structured light image and the reference region of the previous line structured light image.

7. The method according to any one of claims 1-5, characterized in that, The step of determining the rotation compensation amount based on the target slope and the height compensation amount based on the target offset includes: For each pixel in the current line structured light image, the rotation compensation amount corresponding to the pixel is determined based on the difference between the horizontal coordinate of the pixel and the horizontal coordinate of a specified pixel, and the target slope. The target offset is determined as the height compensation amount for each pixel.

8. The method according to any one of claims 1-5, characterized in that, The step of performing vibration compensation on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the measured object includes: For each pixel in the current line structured light image, based on the rotation compensation amount and the height compensation amount corresponding to the pixel, vibration compensation is performed on the height coordinate value corresponding to the pixel in the initial three-dimensional coordinates to obtain the target three-dimensional coordinates of the object under test.

9. An image reconstruction apparatus, characterized in that, The device is used in a three-dimensional imaging device, which includes a camera and a laser. The device includes: The acquisition module is configured to acquire a current line structured light image of the object under test captured by the camera when the laser projects line structured light onto the object under test; determine the initial three-dimensional coordinates of the object under test based on the current line structured light image; and query an acquired compensation table based on the target image interval between the current line structured light image and the first line structured light image to obtain the target slope and target offset; wherein, the compensation table includes the mapping relationship between image interval and slope, and the mapping relationship between image interval and offset; The determination module is used to determine the rotation compensation amount based on the target slope and the height compensation amount based on the target offset; and to perform vibration compensation on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the object under test. The reconstruction module is used to generate a three-dimensional reconstructed image of the object under test based on the target's three-dimensional coordinates.

10. The apparatus according to claim 9, Its features are, in, When the acquisition module acquires the compensation table, it is specifically used for: during the movement of the laser or the sample object, projecting multiple line structured lights onto the sample object through the laser, and acquiring multiple line structured light images of the sample object captured by the camera; for each line structured light image, determining the height coordinates corresponding to the line structured light image; determining the height difference based on the height coordinates corresponding to the line structured light image and the height coordinates corresponding to the previous line structured light image; and determining the first slope corresponding to the line structured light image based on the height difference, wherein the first slope indicates the difference in rotational vibration between two adjacent images. Based on the first slope corresponding to the line structured light image and the first slope corresponding to each line structured light image preceding the line structured light image, a second slope is determined. The second slope indicates the rotational vibration difference of the line structured light image relative to the first line structured light image. The mapping relationship between the image interval of the line structured light image and the second slope is recorded in the compensation table. The image interval represents the number of images between the line structured light image and the first line structured light image.

11. The apparatus according to claim 10, characterized in that, in, When the acquisition module determines the first slope corresponding to the line structured light image based on the height difference, it specifically performs the following steps: selecting multiple pixels from the reference region of the line structured light image; constructing multiple coordinate points corresponding to the multiple pixels in a coordinate system; wherein, for each coordinate point corresponding to a pixel, the horizontal coordinate of the coordinate point is the column identifier corresponding to the pixel in the line structured light image, and the vertical coordinate of the coordinate point is the height difference between the height coordinate of the pixel in the line structured light image and the height coordinate of the pixel in the previous line structured light image; and performing line fitting based on the multiple coordinate points to obtain the first slope corresponding to the line structured light image.

12. The apparatus according to claim 10, Its features are, Specifically, when the acquisition module acquires the compensation table, it performs the following steps: for each line structured light image, it performs rotation compensation on the height coordinates corresponding to the line structured light image based on the first slope corresponding to the line structured light image to obtain the rotated and compensated height coordinates; it determines the height difference after rotation compensation based on the rotated and compensated height coordinates of the previous line structured light image, and determines the first offset corresponding to the line structured light image based on the height difference; it determines the second offset of the line structured light image relative to the first line structured light image based on the first offset corresponding to the line structured light image and the first offset corresponding to each line structured light image preceding the line structured light image; and it records the mapping relationship between the image interval of the line structured light image and the second offset in the compensation table.

13. The apparatus according to claim 12, characterized in that, in, When the acquisition module determines the first offset corresponding to the line structured light image based on the height difference after rotation compensation, it is specifically used to: select multiple pixels from the reference area of ​​the line structured light image; acquire the height difference after rotation compensation corresponding to the multiple pixels; and determine the first offset corresponding to the line structured light image based on the average value of the height differences after rotation compensation corresponding to the multiple pixels; wherein, for each pixel, the height difference after rotation compensation corresponding to the pixel is the height difference between the height coordinate of the pixel after rotation compensation in the line structured light image and the height coordinate of the pixel after rotation compensation in the previous line structured light image.

14. The apparatus according to claim 11 or 13, characterized in that, in, The acquisition module is further configured to determine the reference region of the line structured light image based on the reference region of the previous line structured light image and the X-direction offset value before selecting multiple pixels from the reference region of the line structured light image; wherein the X-direction offset value represents the X-direction offset between the reference region of the line structured light image and the reference region of the previous line structured light image.

15. The apparatus according to any one of claims 9-13, characterized in that, in, The determining module determines the rotation compensation amount based on the target slope, and determines the height compensation amount based on the target offset. Specifically, it is used to: for each pixel in the current line structured light image, determine the rotation compensation amount corresponding to the pixel based on the difference between the horizontal coordinate of the pixel and the horizontal coordinate of a specified pixel and the target slope; and determine the target offset as the height compensation amount corresponding to each pixel.

16. The apparatus according to any one of claims 9-13, characterized in that, The determining module performs vibration compensation on the height coordinate value of the initial three-dimensional coordinates based on the rotation compensation amount and the height compensation amount to obtain the target three-dimensional coordinates of the object under test. Specifically, it performs vibration compensation on the height coordinate value corresponding to the pixel in the initial three-dimensional coordinates for each pixel in the current line structured light image, based on the rotation compensation amount and the height compensation amount corresponding to the pixel, to obtain the target three-dimensional coordinates of the object under test.

17. An electronic device, characterized in that, include: A processor and a non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute the machine-executable instructions to implement the method of any one of claims 1-8.

Images