Three-dimensional reconstruction method, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of three-dimensional reconstruction, and disclose a three-dimensional reconstruction method, an electronic device and a storage medium. The method comprises the following steps: acquiring an encoded projection pattern; projecting the encoded projection pattern to an object to be measured, and collecting a first image and a second image after projection; performing epipolar correction on the first image and the second image to obtain a corrected first image and a corrected second image; performing center line extraction on the corrected first image and the corrected second image respectively to obtain a first center line set and a second center line set; matching the center lines in the first center line set with the center lines in the second center line set to obtain a matched center line pair; and performing three-dimensional reconstruction on the object to be measured according to the matched center line pair. The present application projects a double-symmetrical projection pattern with a globally unique coding feature, thereby breaking the dependence on the natural texture of the tooth surface and improving the accuracy of center line extraction.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, specifically to a three-dimensional reconstruction method, electronic device, and storage medium. Background Technology

[0002] The core of digital dental restoration in clinical practice is the accurate acquisition of a three-dimensional model of the prepared tooth. The accuracy of this model directly determines the precision of undercut analysis and removal, which is crucial for successful restoration. Binocular scanning is an intraoral scanning technology based on the principle of binocular stereo parallax. It uses dual cameras to acquire images and complete three-dimensional reconstruction of the tooth, providing fundamental data for restorative design.

[0003] Currently, binocular scanning is mostly used for model acquisition after tooth preparation, but its imaging principle has inherent limitations. Scanning accuracy depends on the natural texture of the tooth surface, but the smooth surface of a prepared tooth lacks matching feature textures, easily leading to feature matching failure, resulting in missing undercut data and missed detection of minute undercuts. Furthermore, the oral cavity environment is complex; patient micro-movements, ambient light, and specular reflections from saliva can all severely interfere with the acquired images. Traditional grayscale-based binocular matching methods are prone to model distortion in such scenarios. Summary of the Invention

[0004] One objective of this application is to provide a three-dimensional reconstruction method, electronic device, and computer-readable storage medium to solve the technical problems in the prior art where binocular scanning relies on the natural texture of the tooth surface and has insufficient reconstruction accuracy on smooth surfaces and in complex optical environments.

[0005] This application provides a three-dimensional reconstruction method in a first aspect, the method comprising: Obtain an coded projection pattern, which is composed of multiple pattern units. The pattern units have central symmetry and axial symmetry, and the length of the symmetry axis of the pattern unit is encoded as a globally unique coded feature. The coded projection pattern is projected onto the surface of the object to be measured, and a first image and a second image are acquired after projection. Epipolar correction is performed on the first image and the second image to obtain a corrected first image and a corrected second image. Centerline extraction is performed on the corrected first image and the corrected second image to obtain a first centerline set and a second centerline set. The centerline extraction includes performing grayscale centroid extraction using a local window as the processing unit, eliminating false centers based on the relative positional relationship between the extracted centroid and the window center, and performing neighborhood density statistics on the retained candidate center points to generate centerlines; the centerlines in the first centerline set are matched with the centerlines in the second centerline set to obtain matched centerline pairs. Based on the matched centerline pairs, the object under test is reconstructed in three dimensions.

[0006] In conjunction with the first aspect, in one possible implementation, obtaining the encoded projection pattern includes: A right angle is constructed, and a mirror transformation is performed using one side of the right angle as the axis of symmetry. The mirrored graphic is combined with the original graphic. A mirror transformation is performed along the other side of the right angle to obtain the pattern unit. The pattern unit is randomly stretched and scaled and then combined to obtain the coded projection pattern. The length of the axis of symmetry of the pattern unit is randomly stretched to form the globally unique coded feature.

[0007] In conjunction with the first aspect, in one possible implementation, the epipolar correction of the first image and the second image includes: Obtain the rotation matrix and translation vector between the first camera acquiring the first image and the second camera acquiring the second image; perform half-angle decomposition on the rotation matrix to obtain a first half-angle rotation matrix and a second half-angle rotation matrix. The first half-angle rotation matrix is ​​used to rotate the optical axis of the first camera to a parallel state perpendicular to the baseline, and the second half-angle rotation matrix is ​​used to rotate the optical axis of the second camera to the parallel state; determine the baseline direction based on the translation vector, normalize the baseline direction and use it as the X-axis direction of the row alignment coordinate system, and construct the row alignment rotation matrix by combining the orthogonal vector of the optical axis direction in the parallel state with the X-axis direction; determine the first correction homography matrix corresponding to the first image and the second correction homography matrix corresponding to the second image based on the row alignment rotation matrix, the first half-angle rotation matrix, the second half-angle rotation matrix, and the intrinsic parameter matrices of the first and second cameras; perform homography mapping on the pixels in the first image through the first correction homography matrix, and perform homography mapping on the pixels in the second image through the second correction homography matrix to obtain the corrected first image and the corrected second image, where the row coordinates of corresponding pixels in the two mapped images are consistent.

[0008] In conjunction with the first aspect, in one possible implementation, the extraction of centerlines from the corrected first image and the corrected second image respectively includes: Based on the defocus characteristics of the small depth-of-field optical system, the maximum and minimum widths of the projected fringes within the depth-of-field range are determined. Based on the maximum and minimum widths, the size of the local window is determined, where the size of the local window is greater than the width of the current projected fringes but less than twice the width of the current projected fringes. Using the local window as a processing unit, gray-scale centroid extraction is performed in the corrected first and second images respectively. The gray-scale distribution of pixels within each local window is statistically analyzed, and candidate center points are obtained based on gray-scale weighting. Neighborhood density statistics are performed on the candidate center points, counting the number, spatial distribution density, and directional continuity of candidate center points appearing near each local region. Regions where the density of candidate center points is higher than a preset density threshold and the spatial distribution is continuous are determined as effective center regions. New center points are generated within the effective center regions based on the spatial distribution of the candidate center points, and these new center points are connected to form the first centerline set and the second centerline set.

[0009] In conjunction with the first aspect, in one possible implementation, the step of removing false centers based on the relative positional relationship between the extracted centroid and the window center includes: For each local window, the position of the candidate center point obtained by the gray-scale centroid extraction method within the local window is determined; if the distance between the candidate center point and the center of the local window exceeds a preset offset threshold, the candidate center point is determined to be a false center and is removed; if the candidate center point is located within the central region of the local window, it is retained.

[0010] In conjunction with the first aspect, in one possible implementation, matching the centerlines of the first set of centerlines with the centerlines of the second set of centerlines includes: Each centerline in the first centerline set and the second centerline set is numbered. The numbering is determined based on the correspondence between each centerline and the pattern unit in the coded projection pattern. The numbering criteria include the position, direction, length, curvature, endpoint relationship of the centerline, and the encoding features of the corresponding projection pattern. The epipolar constraint and the number of point spread pixels are used as constraints. The centerlines in the first centerline set and the second centerline set are matched according to the centerline features. The centerline features include at least one of the following: centerline length, centerline direction, centerline curvature, centerline endpoint position, spacing between adjacent centerlines, the symmetry axis length encoding of the projection pattern, the arrangement relationship of neighboring patterns, and local topology. The epipolar constraint is used to limit the search direction of the corresponding features. The number of point spread pixels is determined according to the degree of defocus to limit the local offset range under defocus conditions.

[0011] In conjunction with the first aspect, in one possible implementation, the step of performing three-dimensional reconstruction of the object under test based on the matched centerline pairs includes: In regions where corresponding centerlines exist in both the first centerline set and the second centerline set, disparity is calculated and three-dimensional coordinates are recovered based on the matched centerline pairs. In regions where at least one of the first centerline set and the second centerline set lacks a corresponding centerline, the grayscale matching search range is limited by the matched centerline pairs, neighborhood coding patterns, epipolar constraints, and local grayscale information, and the region is compensated and reconstructed.

[0012] In conjunction with the first aspect, in one possible implementation, after connecting the new center points into the first set of center lines and the second set of center lines, the method further includes: Repairing missing center points or broken line segments caused by the destruction of projection light in the first center line set and the second center line set; the repair includes supplementing the missing center points or broken line segments according to at least one of the following: the arrangement relationship of the original projection pattern, the symmetry axis direction of the pattern unit, the symmetry axis length encoding, the spatial relationship of adjacent center lines, and the extension direction of the broken endpoints.

[0013] This application provides an electronic device in a second aspect, comprising: At least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor to implement the three-dimensional reconstruction method described in the first aspect.

[0014] In a third aspect, this application provides a computer-readable storage medium storing computer instructions that are executed by a computer to implement the three-dimensional reconstruction method described in the first aspect.

[0015] The embodiments of this application can achieve the following technical effects: On the one hand, the embodiments of this application project an coded projection pattern composed of pattern units with central and axial symmetry, and encode the symmetry axis length of the pattern unit as a globally unique coded feature. This makes the structured light projection pattern itself a source of matchable features, completely eliminating the dependence of traditional binocular scanning on the natural texture of the tooth surface. The centerline and symmetry axis codes of the projection pattern can be stably extracted and identified in both left and right images, providing a reliable feature basis for subsequent binocular matching.

[0016] On the other hand, the embodiments of this application effectively eliminate false center points caused by saliva bright spots, specular reflection and uneven defocus diffusion by extracting the gray centroid based on the defocus window and determining the consistency of the window center. At the same time, the center points are regenerated in the effective center area by neighborhood density statistics and connected to form the center line, which makes up for the discrete error and breakage problem caused by single window extraction and significantly improves the continuity and accuracy of the center line in complex optical environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram illustrating a pattern that simultaneously possesses central symmetry and axial symmetry, provided as an embodiment of this application; Figure 2 This is a schematic diagram illustrating an example of an encoded projection pattern provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the principle of an epipolar correction method provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating an example of centerline extraction results provided in an embodiment of this application. Figure 5 This is a schematic diagram illustrating an example of a three-dimensional reconstruction result provided in an embodiment of this application. Figure 6 This is a flowchart illustrating a three-dimensional reconstruction method provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the various features in the embodiments of this application can be combined with each other, all within the scope of protection of this application. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. Moreover, the terms "first," "second," etc., used in this application do not limit the data or execution order, but only distinguish between identical or similar items with substantially the same function and effect.

[0020] The three-dimensional reconstruction method provided in this application embodiment can be applied to the scenario of three-dimensional reconstruction of teeth and gums in dental design software, and is particularly suitable for high-precision model reconstruction of smooth tooth preparation surfaces in complex optical environments in oral binocular scanning systems.

[0021] See Figure 6 The method includes the following steps: Step S10: Obtain the coded projection pattern, which is composed of multiple pattern units. The pattern units have central symmetry and axial symmetry, and the length of the symmetry axis of the pattern unit is encoded as a globally unique coded feature. Step S20: Project the coded projection pattern onto the surface of the object to be tested, and acquire the first and second images after projection; Step S30: Perform epipolar correction on the first image and the second image to obtain the corrected first image and the corrected second image; Step S40: Centerline extraction is performed on the corrected first image and the corrected second image respectively to obtain a first centerline set and a second centerline set. The centerline extraction includes performing grayscale centroid extraction with a local window as the processing unit, eliminating false centers according to the relative position relationship between the extracted centroid and the window center, and performing neighborhood density statistics on the retained candidate center points to generate centerlines. Step S50: Match the center lines in the first center line set with the center lines in the second center line set to obtain a matched center line pair; Step S60: Perform three-dimensional reconstruction of the object under test based on the matched centerline pair.

[0022] Specifically, see Figure 1 Pattern units can be prepared by two mirror transformations: construct a right angle, perform a mirror transformation with one side of the right angle as the axis of symmetry, combine the transformed graphic with the original graphic, and then perform a mirror transformation along the other side of the right angle to finally obtain a pattern unit that has both central symmetry and axial symmetry characteristics.

[0023] After two mirror transformations, the resulting pattern unit is symmetrical in both orthogonal directions, which makes the center line of the pattern unit a stable geometric feature that can be identified in subsequent projection and imaging processes.

[0024] After the pattern units are prepared, they are randomly stretched and scaled along different directions to change their symmetry axis lengths, thus forming combinations of symmetry axis lengths that are different across the entire field. Multiple pattern units that have undergone random stretching and scaling are then combined and arranged to obtain the coded projection pattern. (See [link to documentation]). Figure 2 .

[0025] Since the length of the axis of symmetry of each pattern unit is different after random stretching, the length of the axis of symmetry of each pattern unit can be used as a globally unique coding feature, providing a basis for matching corresponding patterns in subsequent left and right images.

[0026] In oral cavity scanning scenarios, the object to be tested can be a tooth preparation, gingival tissue, or other structures within the oral cavity. Projection can be achieved using a digital projector or laser projector, with the projection optical path and the acquisition optical path forming a certain angle, causing the structured light to produce deformations on the tooth surface modulated by the three-dimensional shape. The first and second projected images are simultaneously acquired using binocular cameras. The first image is acquired by the first camera, and the second image is acquired by the second camera. The spatial relationship between the two cameras is obtained during the system calibration phase through a calibration process.

[0027] Because it is difficult to ensure that the optical axes of the two cameras are completely parallel and the imaging planes are coplanar when the binocular camera is physically installed, the first and second images acquired contain epipolar tilt and perspective distortion. Direct feature matching between the two original images requires searching for corresponding points in two-dimensional space, which is computationally intensive and has a high mismatch rate.

[0028] Therefore, this embodiment requires epipolar correction of the first and second images, transforming the two images to a standard correction coordinate system with the epipolar lines aligned horizontally, so that the matching search range is reduced from two dimensions to one dimension.

[0029] The basic process of polar line correction is described in [link to documentation]. Figure 3 The rotation matrix and translation vector between the first and second cameras are obtained. The rotation matrix is ​​decomposed into half-angles to make the optical axes of the two cameras rotate synchronously to a parallel state. The baseline direction is determined according to the translation vector, and a row alignment rotation matrix is ​​constructed by combining the optical axis direction in the parallel state. The correction homography matrix is ​​then calculated by combining the intrinsic parameter matrices of the two cameras. The first image and the second image are transformed into the corrected first image and the corrected second image respectively through homography mapping.

[0030] The row coordinates of corresponding pixels in the two mapped images are consistent, and only horizontal disparity exists, without introducing new epipolar distortion. Considering that pixel coordinates may be non-integer during homography, this embodiment uses bilinear interpolation for image resampling. The gray value of the target pixel is calculated by weighting the gray values ​​of the four pixels surrounding the target pixel according to distance, ensuring the gray-level continuity of the corrected image.

[0031] After epipolar correction, centerlines were extracted from the corrected first and second images. Because intraoral scanning uses a shallow depth-of-field optical system, the image sharpness varies significantly across different depth regions. This results in varying degrees of defocus diffusion in the projected fringes at different depth positions, causing the fringe width to change accordingly.

[0032] In this embodiment, the maximum and minimum widths of the projected fringes within the depth of field are first determined based on the defocus characteristics of the small depth-of-field optical system, and then the local window size for center extraction using the gray-scale centroid method is determined accordingly.

[0033] For example, the local window size is greater than the width of the current projected stripe but less than twice the width of the current projected stripe. This range of values ​​ensures that the window can completely cover the grayscale distribution of the current stripe, while avoiding the introduction of interference from adjacent projected patterns, saliva bright spots or background noise due to an excessively large window.

[0034] Using local windows as processing units, gray-scale centroid extraction is performed in parallel in the corrected first and second images. For each local window, the weighted average of the pixel coordinates within the window is calculated using gray values ​​as weights. This weighted average is the position of the candidate center point.

[0035] After obtaining the candidate center points of each local window, false centers are eliminated based on the relative positional relationship between the extracted centroid and the window center: if the distance between the candidate center point and the window center exceeds the preset offset threshold, it indicates that the grayscale distribution in the window may be severely affected by saliva bright spots, specular reflection, or uneven defocus diffusion. In this case, the candidate center point is determined to be a false center and is eliminated; if the candidate center point is located within the window center area, it is retained.

[0036] Because the specular reflection bright spots formed by saliva in the mouth will produce strong grayscale response in local areas, causing the grayscale centroid to shift significantly towards the bright spot, such false centers affected by bright spots can be effectively identified and eliminated by window center consistency judgment.

[0037] Based on this, neighborhood density statistics are performed on the retained candidate center points. Specifically, the number of candidate center points appearing in the vicinity of each local area, their spatial distribution density, and directional continuity can be counted. Areas where the density of candidate center points is higher than a preset density threshold and their spatial distribution is continuous are identified as effective center areas.

[0038] Within the effective center region, new center points are generated based on the spatial distribution of candidate center points. This generation can employ a local weighted average, calculating the coordinates of the new center point representing the direction of the region's center based on the spatial location and grayscale weight of each candidate center point within the effective center region. These new center points are then connected sequentially according to spatial continuity, directional consistency, and local curvature relationships to form the first and second centerline sets. This method of regenerating center points through neighborhood density statistics compensates for the discrete errors and breaks introduced by single-window extraction, significantly improving the continuity and accuracy of centerlines in complex optical environments. (See also...) Figure 4 The extracted center line has a clear correspondence with the pattern unit in the projected pattern.

[0039] After obtaining the first and second centerline sets, the centerlines in both sets are matched. First, each centerline in both sets is numbered based on its correspondence with a pattern unit in the coded projection pattern. Since the symmetry axis length of each pattern unit in the coded projection pattern is globally unique, and there is a one-to-one correspondence between the extracted centerlines and the symmetry axes of the pattern units, each centerline can be assigned a number associated with its corresponding pattern unit based on this unique coding feature, ensuring that centerlines corresponding to the same pattern unit in the left and right images have an associative correspondence.

[0040] After numbering, epipolar constraints and dot spread pixel counts are used as constraints to match the centerlines in the first and second centerline sets based on centerline characteristics. Since the row coordinates of corresponding pixels in the two images are consistent after epipolar correction, the matching search only needs to be performed on the same horizontal row of the corrected images; the epipolar constraints limit the matching search range to this row direction. The dot spread pixel count is determined based on the degree of defocus and is used to limit the local offset range in the case of defocus. After centerline matching is completed, one-to-one matched centerline pairs are obtained.

[0041] It is easy to understand that there are regions with corresponding center lines in both the first and second center line sets, indicating that the projection structure of the region is complete and effective center line features have been successfully extracted in both the left and right images. At this time, the disparity can be directly calculated and the three-dimensional coordinates can be recovered based on the matched center line pairs.

[0042] Parallax is the difference in horizontal coordinates between matched centerline pairs in the corrected image. Combining the camera focal length and baseline distance, the three-dimensional spatial coordinates of the point can be calculated using triangulation principles. In regions where at least one of the first and second centerline sets lacks a corresponding centerline, the projection pattern of that region is corrupted, and centerline extraction fails. In this case, the matched centerline pairs, neighborhood encoded patterns, epipolar constraints, and local grayscale information are used to limit the grayscale matching search range, and compensation and reconstruction are performed on this region. See also... Figure 5 , Figure 5 The results of three-dimensional reconstruction of the object under test using this method are shown.

[0043] As a feasible implementation method, in the foregoing embodiments, obtaining the encoded projection pattern includes: A right angle is constructed, and a mirror transformation is performed using one side of the right angle as the axis of symmetry. The mirrored graphic is combined with the original graphic. A mirror transformation is performed along the other side of the right angle to obtain the pattern unit. The pattern unit is randomly stretched and scaled and then combined to obtain the coded projection pattern. The length of the axis of symmetry of the pattern unit is randomly stretched to form the globally unique coded feature.

[0044] In practice, the initial line segments forming the right angle can be generated by a computer with a preset length. For example, the initial line segment length is 100 pixels, and the two line segments intersect at a 90-degree angle at their endpoints. Mirror transformation can be mathematically achieved using a reflection matrix. Multiplying the coordinates of each vertex of the graphic by the corresponding reflection matrix yields the new coordinates of each vertex after mirroring.

[0045] The random stretching and scaling process employs independent random scaling factors in two orthogonal directions. For example, the horizontal scaling factor is randomly sampled between 0.8 and 1.2, and the vertical scaling factor is randomly sampled between 0.6 and 1.5, ensuring that the symmetry axis length of each pattern unit is different. After stretching and scaling, the pattern units are arranged according to a preset number of rows and columns. For example, a 10-row × 15-column arrangement is used, generating a total of 150 pattern units. The symmetry axis length of each pattern unit is unique globally, thus forming a reliable globally unique coding feature.

[0046] As a feasible implementation method, in the foregoing embodiments, the epipolar correction of the first image and the second image includes: Obtain the rotation matrix and translation vector between the first camera acquiring the first image and the second camera acquiring the second image; perform half-angle decomposition on the rotation matrix to obtain a first half-angle rotation matrix and a second half-angle rotation matrix. The first half-angle rotation matrix is ​​used to rotate the optical axis of the first camera to a parallel state perpendicular to the baseline, and the second half-angle rotation matrix is ​​used to rotate the optical axis of the second camera to the parallel state; determine the baseline direction based on the translation vector, normalize the baseline direction and use it as the X-axis direction of the row alignment coordinate system, and construct the row alignment rotation matrix by combining the orthogonal vector of the optical axis direction in the parallel state with the X-axis direction; determine the first correction homography matrix corresponding to the first image and the second correction homography matrix corresponding to the second image based on the row alignment rotation matrix, the first half-angle rotation matrix, the second half-angle rotation matrix, and the intrinsic parameter matrices of the first and second cameras; perform homography mapping on the pixels in the first image through the first correction homography matrix, and perform homography mapping on the pixels in the second image through the second correction homography matrix to obtain the corrected first image and the corrected second image, where the row coordinates of corresponding pixels in the two mapped images are consistent.

[0047] In practical implementation, half-angle decomposition can be achieved by calculating the matrix logarithm of the rotation matrix, representing the rotation as a rotation axis and rotation angle, and then constructing a new rotation matrix by taking half of the rotation angle. For example, if the rotation angle corresponding to the rotation matrix R is θ, then R... 0.5 This corresponds to a rotation matrix that rotates around the same rotation axis by an angle of θ / 2.

[0048] The intrinsic parameter matrix is ​​obtained using a checkerboard calibration board during the binocular camera calibration stage. The intrinsic parameter matrix includes the focal length and principal point coordinates. After homography, bilinear interpolation is used for resampling. The weights of the bilinear interpolation are determined by the relative distance between the target pixel's sub-pixel position in the original image and its four surrounding pixels, ensuring that the grayscale distribution of the corrected image maintains a continuous and natural transition.

[0049] As a feasible implementation method, in the foregoing embodiments, the step of extracting the centerline from the corrected first image and the corrected second image respectively includes: Based on the defocus characteristics of the small depth-of-field optical system, the maximum and minimum widths of the projected fringes within the depth-of-field range are determined. Based on the maximum and minimum widths, the size of the local window is determined, where the size of the local window is greater than the width of the current projected fringes but less than twice the width of the current projected fringes. Using the local window as a processing unit, gray-scale centroid extraction is performed in the corrected first and second images respectively. The gray-scale distribution of pixels within each local window is statistically analyzed, and candidate center points are obtained based on gray-scale weighting. Neighborhood density statistics are performed on the candidate center points, counting the number, spatial distribution density, and directional continuity of candidate center points appearing near each local region. Regions where the density of candidate center points is higher than a preset density threshold and the spatial distribution is continuous are determined as effective center regions. New center points are generated within the effective center regions based on the spatial distribution of the candidate center points, and these new center points are connected to form the first centerline set and the second centerline set.

[0050] In practice, the maximum and minimum stripe widths can be determined by combining offline calibration with online estimation. For example, in the offline stage, projected stripe images are acquired at different depths using a standard planar target at a known depth location, the stripe imaging width at each depth location is measured, and a calibration curve between depth and stripe width is established.

[0051] For the online phase, the stripe width range of the region to be processed can be obtained by looking up a table based on the depth range of the region to be processed. In the neighborhood density statistics, the preset density threshold can be adaptively determined based on the overall density distribution of the candidate center points. For example, the median of the density values ​​of all statistical units can be taken as the density threshold to avoid the insufficient adaptability of a fixed threshold to different regions.

[0052] The continuity of direction can be determined by calculating the direction vector between adjacent candidate center points. If the angle between adjacent direction vectors exceeds 45 degrees, it is determined to be discontinuous in direction. When connecting new center points to form a center line, the shortest path method can be used to prioritize connecting the nearest adjacent center points with the closest directions, so that the center line generated by the connection is smooth in space and conforms to the extension trend of the stripes.

[0053] As a feasible implementation method, in the foregoing embodiments, the step of removing false centers based on the relative positional relationship between the extracted centroid and the window center includes: For each local window, the position of the candidate center point obtained by the gray-scale centroid extraction method within the local window is determined; if the distance between the candidate center point and the center of the local window exceeds a preset offset threshold, the candidate center point is determined to be a false center and is removed; if the candidate center point is located within the central region of the local window, it is retained.

[0054] Specifically, the window center region refers to a circular area with the geometric center of the window as its center and a preset offset threshold as its radius. The preset offset threshold can be dynamically adjusted according to the window size. For example, the preset offset threshold is set to 25% to 35% of the window size. When the grayscale distribution within the window is relatively uniform, the grayscale centroid is located near the geometric center of the window, with a small distance deviation. When there are bright spots of saliva within the window, the high grayscale value of the bright spot area will cause the grayscale centroid to shift significantly towards the bright spot, resulting in the centroid moving away from the window center. This mechanism can simultaneously clean up erroneous center points caused by the complex oral optical environment during the center extraction stage, preventing false centers from entering the subsequent matching and reconstruction processes.

[0055] As a feasible implementation, in the foregoing embodiments, matching the centerlines in the first centerline set with the centerlines in the second centerline set includes: numbering each centerline in the first centerline set and the second centerline set, wherein the numbering is determined based on the correspondence between each centerline and the pattern unit in the coded projection pattern, and the numbering is based on the centerline's position, direction, length, curvature, endpoint relationship, and the encoding features of the corresponding projection pattern; using epipolar constraints and dot diffusion pixel quantity as constraints, matching the centerlines in the first centerline set with the centerlines in the second centerline set according to the centerline features; wherein the centerline features include at least one of centerline length, centerline direction, centerline curvature, centerline endpoint position, adjacent centerline spacing, the symmetry axis length encoding of the projection pattern, neighborhood pattern arrangement relationship, and local topology, wherein the epipolar constraints are used to limit the search direction of the corresponding features, and the dot diffusion pixel quantity is determined according to the defocus degree to limit the local offset range under defocus conditions.

[0056] In practice, the numbering process can begin by selecting a reference center line from the first set of center lines. Based on its position and coding characteristics, the corresponding pattern unit is located in the coded projection pattern. Then, the center line is assigned a number based on the globally unique code of that pattern unit. The remaining center lines are numbered sequentially in the same way. During matching, a coarse matching can be performed first using the symmetry axis length coding feature to quickly narrow down the candidate matching range. Then, a fine matching can be performed using the geometric features of the center lines.

[0057] For example, for a centerline in the first centerline set, the pattern unit corresponding to its axis of symmetry length encoding has a unique match or at most a few candidates in the second centerline set, and then the unique match is determined from the candidates based on the geometric features such as the position, direction and curvature of the centerline.

[0058] The number of pixels in the dot diffusion can be estimated based on the difference between the actual width and the minimum width of the projected stripes in the area. The larger the difference, the more severe the defocusing, and the larger the allowable vertical offset range, so that reliable matching can still be achieved in the defocused and blurred area.

[0059] As a feasible implementation method, in the foregoing embodiments, the step of performing three-dimensional reconstruction of the object under test based on the matched centerline pairs includes: In regions where corresponding centerlines exist in both the first centerline set and the second centerline set, disparity is calculated and three-dimensional coordinates are recovered based on the matched centerline pairs. In regions where at least one of the first centerline set and the second centerline set lacks a corresponding centerline, the grayscale matching search range is limited by the matched centerline pairs, neighborhood coding patterns, epipolar constraints, and local grayscale information, and the region is compensated and reconstructed.

[0060] In practice, for regions where corresponding centerlines exist, disparity is calculated as the difference in horizontal coordinates between the matching centerline pairs in the corrected image. The recovery of three-dimensional coordinates is calculated using triangulation principles based on focal length, baseline length, and disparity. For regions lacking corresponding centerlines, grayscale matching search uses normalized cross-correlation as a similarity metric, and the search window size is determined based on neighborhood coding patterns and local grayscale information.

[0061] For example, the horizontal range of the search window is ±20 pixels near the centerline parallax, and the vertical range is limited to ±1 pixel by the epipolar constraint. By utilizing matched adjacent centerline pairs and coded pattern information as guidance, reliable 3D coordinate estimates can still be obtained in areas where the projection structure is disrupted.

[0062] As a feasible implementation, in the foregoing embodiments, after connecting the new center points into the first centerline set and the second centerline set, the method further includes: Repairing missing center points or broken line segments caused by the destruction of projection light in the first center line set and the second center line set; the repair includes supplementing the missing center points or broken line segments according to at least one of the following: the arrangement relationship of the original projection pattern, the symmetry axis direction of the pattern unit, the symmetry axis length encoding, the spatial relationship of adjacent center lines, and the extension direction of the broken endpoints.

[0063] In practice, when repairing missing center points, the effective center points at the front and back ends of the missing section can be determined first, and the expected trajectory of the center points within the missing section can be determined based on the position and direction of the effective points at both ends.

[0064] For example, for a broken section of about 30 pixels long caused by saliva spots, the effective center point coordinates of the front end are (x1, y1) and the direction angle is θ1, and the effective center point coordinates of the rear end are (x2, y2) and the direction angle is θ2. Supplementary center points are generated by interpolation along a smooth transition trajectory with a step size of 2 pixels within the broken section. Each supplementary center point is then finely adjusted laterally according to the spacing of the adjacent unbroken parallel center lines at that position to ensure that the spatial distribution of the center lines after repair is consistent with the arrangement of the original projection pattern.

[0065] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in interchange, etc.

[0066] This application also provides an electronic device that can serve as a hardware entity for implementing the method described in this application. Specifically, the electronic device can be a computer running dental design software, a built-in processing unit of an oral scanning device, or a cloud server.

[0067] The electronic device includes one or more processors and a memory, the memory being connected to the one or more processors, for example, via a bus. The processors execute program instructions stored in the memory to implement the three-dimensional reconstruction method described in this application. The processors are configured to support the electronic device in performing the corresponding functions in the above-described method embodiments. The processor may be a central processing unit, a network processor, a hardware chip, or any combination thereof.

[0068] The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned programmable logic device can be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPRS), or any combination thereof. The memory is used to store program code, etc. The memory can include volatile memory, such as random access memory (RAM); it can also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory can also include combinations of the above types of memory. The memory can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the three-dimensional reconstruction method in the embodiments of this application. The processor executes various functional applications and data processing of the aforementioned method by running the non-volatile software programs, instructions, and modules stored in the memory.

[0069] This application also provides a computer-readable storage medium storing computer instructions that are executed by a computer to implement the methods described in the above embodiments.

[0070] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0071] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A three-dimensional reconstruction method, characterized in that, The method includes: Obtain an coded projection pattern, which is composed of multiple pattern units. The pattern units have central symmetry and axial symmetry, and the length of the symmetry axis of the pattern unit is encoded as a globally unique coded feature. The coded projection pattern is projected onto the surface of the object to be tested, and the first and second images after projection are acquired. Epipolar correction is performed on the first image and the second image to obtain the corrected first image and the corrected second image; Centerline extraction is performed on the corrected first image and the corrected second image respectively to obtain a first centerline set and a second centerline set. The centerline extraction includes performing grayscale centroid extraction with a local window as the processing unit, eliminating false centers according to the relative position relationship between the extracted centroid and the window center, and performing neighborhood density statistics on the retained candidate center points to generate centerlines. The center lines in the first center line set are matched with the center lines in the second center line set to obtain a matched center line pair; Based on the matched centerline pairs, the object under test is reconstructed in three dimensions.

2. The method according to claim 1, characterized in that, The acquisition of the encoded projection pattern includes: Construct a right angle, and then perform a mirror transformation using one of the sides forming the right angle as the axis of symmetry; Combine the image after mirroring with the image before mirroring; The pattern unit is obtained by mirroring along the other side that forms the right angle; The pattern units are randomly stretched and scaled and then combined to obtain the coded projection pattern. The length of the symmetry axis of the pattern units is randomly stretched to form the globally unique coded feature.

3. The method according to claim 1, characterized in that, The epipolar correction of the first image and the second image includes: Obtain the rotation matrix and translation vector between the first camera that captures the first image and the second camera that captures the second image; The rotation matrix is ​​decomposed into a first half-angle rotation matrix and a second half-angle rotation matrix. The first half-angle rotation matrix is ​​used to rotate the optical axis of the first camera to a parallel state perpendicular to the baseline, and the second half-angle rotation matrix is ​​used to rotate the optical axis of the second camera to the parallel state. The baseline direction is determined based on the translation vector. The baseline direction is normalized and used as the X-axis direction of the row alignment coordinate system. The row alignment rotation matrix is ​​constructed by combining the orthogonal vector of the optical axis direction in the parallel state and the X-axis direction. Based on the row alignment rotation matrix, the first half-angle rotation matrix, the second half-angle rotation matrix, and the intrinsic parameter matrices of the first and second cameras, determine the first correction homography matrix corresponding to the first image and the second correction homography matrix corresponding to the second image; The pixels in the first image are homographed using the first correction homograph matrix, and the pixels in the second image are homographed using the second correction homograph matrix to obtain the corrected first image and the corrected second image. The row coordinates of corresponding pixels in the two images are consistent.

4. The method according to claim 1, characterized in that, The step of extracting the centerline from the corrected first image and the corrected second image respectively includes: Based on the defocusing characteristics of a small depth-of-field optical system, determine the maximum and minimum widths of the projected fringes within the depth of field. The size of the local window is determined based on the maximum width and the minimum width, wherein the size of the local window is greater than the width of the current projection stripe and less than twice the width of the current projection stripe; Using the local window as a processing unit, gray-scale centroid extraction is performed in the corrected first image and the corrected second image respectively. The gray-scale distribution of pixels in each local window is statistically analyzed, and candidate center points are obtained based on gray-scale weighting. Neighborhood density statistics are performed on the candidate center points to count the number of candidate center points appearing in the vicinity of each local region, their spatial distribution density, and directional continuity. Regions where the density of candidate center points is higher than a preset density threshold and their spatial distribution is continuous are determined as effective center regions. New center points are generated within the effective center regions based on the spatial distribution of the candidate center points, and the new center points are connected to form the first centerline set and the second centerline set.

5. The method according to claim 4, characterized in that, The step of removing false centers based on the relative positional relationship between the extracted centroid and the window center includes: For each local window, determine the position of the candidate center point obtained by the gray-scale centroid extraction method within that local window; If the distance between the candidate center point and the center of the local window exceeds a preset offset threshold, the candidate center point is determined to be a false center and is removed. If the candidate center point is located within the central area of ​​the local window, it will be retained.

6. The method according to claim 1, characterized in that, The step of matching the centerlines of the first centerline set with the centerlines of the second centerline set includes: Each centerline in the first centerline set and the second centerline set is numbered. The numbering is determined based on the correspondence between each centerline and the pattern unit in the coded projection pattern. The numbering criteria include the position, direction, length, curvature, endpoint relationship of the centerline, and the coding features of the corresponding projection pattern. Using epipolar constraints and point diffusion pixel quantity as constraints, the center lines in the first center line set and the center lines in the second center line set are matched based on the center line characteristics. The centerline features include at least one of the following: centerline length, centerline direction, centerline curvature, centerline endpoint position, spacing between adjacent centerlines, symmetry axis length encoding of the projected pattern, neighborhood pattern arrangement relationship, and local topology. The epipolar constraint is used to limit the search direction of the corresponding feature. The number of point diffusion pixels is determined according to the degree of defocus to limit the local offset range under defocus conditions.

7. The method according to claim 1, characterized in that, The step of performing three-dimensional reconstruction of the object under test based on the matched centerline pairs includes: In regions where corresponding centerlines exist in both the first centerline set and the second centerline set, disparity is calculated and three-dimensional coordinates are recovered based on the matched centerline pairs. In regions where at least one of the first centerline set and the second centerline set is missing a corresponding centerline, the gray-scale matching search range is limited by the matched centerline pairs, neighborhood coding patterns, epipolar constraints, and local gray-scale information, and the region is compensated and reconstructed.

8. The method according to claim 4, characterized in that, After connecting the new center points to form the first centerline set and the second centerline set, the method further includes: Repair the missing center points or broken line segments in the first center line set and the second center line set caused by the destruction of the projection light; The repair includes supplementing the missing center point or broken line segment according to at least one of the following: the arrangement relationship of the original projection pattern, the symmetry axis direction of the pattern unit, the symmetry axis length encoding, the spatial relationship of adjacent center lines, and the extension direction of the broken endpoint.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions executable by the processor to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by a computer to implement the method of any one of claims 1 to 8.

Images