A three-dimensional model reconstruction method, device and equipment

Abstract

The present application provides a three-dimensional model reconstruction method, device and equipment, the reconstruction method comprising: acquiring a first image and a second image of a target object, wherein the first image and the second image are two-dimensional images collected from two different perspectives of the target object; matching a first pixel point in the first image and a corresponding second pixel point in the second image to construct an initial three-dimensional point cloud model of the target object; adjusting the depth value of each point cloud point in the initial three-dimensional point cloud model, generating a first sub-pixel point and a second sub-pixel point corresponding to each adjusted point cloud point in the first image and the second image, respectively, wherein the matching accuracy of the first sub-pixel point and the second sub-pixel point is higher than the matching accuracy of the first pixel point and the second pixel point; calculating the matching cost of the first sub-pixel point and the second sub-pixel point, and determining that the depth value corresponding to the minimum matching cost is the target depth value; and updating the three-dimensional point cloud model of the target object according to the target depth value.

Description

Technical Field

[0001] This specification relates to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional model reconstruction method, device and equipment. Background Art

[0002] In the field of 3D reconstruction, a common method for reconstructing an object is to capture 2D images of the object from multiple perspectives and use these images to restore and stitch together the complete 3D structure. However, this process faces significant challenges when it comes to objects with complex structures, such as those with attached hair. The filamentous structure and irregular arrangement of hair, coupled with its tiny gaps and light absorption, make data tracking easily lost and difficult to capture during the scanning process. These issues lead to deficiencies in the completeness and accuracy of the acquired data, thus affecting the quality of the final 3D model.

[0003] To overcome these difficulties, existing solutions often rely on external auxiliary objects to assist in stitching together incomplete scan data. This approach not only reduces the efficiency of 3D reconstruction but also limits the accuracy and reliability of the reconstruction results, making it unsuitable for applications requiring high-quality 3D reconstruction. Summary of the Invention

[0004] To overcome the problems existing in the related art, this specification provides a three-dimensional model reconstruction method, device and equipment.

[0005] According to a first aspect of an embodiment of this specification, a three-dimensional model reconstruction method is provided, the method comprising:

[0006] Acquire a first image and a second image of a target object, where the first image and the second image are two-dimensional images captured from two different perspectives of the target object;

[0007] Constructing an initial three-dimensional point cloud model of the target object by matching a first pixel in the first image with a corresponding second pixel in the second image;

[0008] Adjusting the depth value of each point cloud point in the initial three-dimensional point cloud model, and generating a first sub-pixel point and a second sub-pixel point corresponding to each adjusted point cloud point in the first image and the second image, respectively, wherein a matching accuracy of the first sub-pixel point and the second sub-pixel point is higher than a matching accuracy of the first pixel point and the second pixel point;

[0009] Calculate the matching cost of the first sub-pixel point and the second sub-pixel point, and determine the depth value corresponding to the minimum matching cost as the target depth value;

[0010] The three-dimensional point cloud model of the target object is updated according to the target depth value.

[0011] According to a second aspect of an embodiment of this specification, another three-dimensional model reconstruction method is provided, which is applicable to reconstructing a three-dimensional model of an object with hair, and the method comprises:

[0012] Acquire a first image and a second image of a target object with hair, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object;

[0013] constructing a three-dimensional point cloud model of the target object based on the first image and the second image;

[0014] Clustering the point cloud points in the three-dimensional point cloud model and deleting the point cloud points that do not meet a preset clustering condition, where the preset clustering condition is determined based on the attributes of the hair;

[0015] The clustering conditions at least include: a clustering radius and a clustering minimum point threshold, the clustering radius is 5*scanning point distance to 10*scanning point distance, and the clustering minimum point threshold is 100 to 500.

[0016] According to a third aspect of the embodiments of this specification, a three-dimensional model reconstruction device is provided, the device comprising:

[0017] a target object image acquisition module, configured to acquire a first image and a second image of the target object, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object;

[0018] an initial model building module, configured to build an initial three-dimensional point cloud model of the target object by matching a first pixel point in the first image with a corresponding second pixel point in the second image;

[0019] a depth adjustment and sub-pixel point generation module, configured to adjust the depth value of each point cloud point in the initial three-dimensional point cloud model, and generate a first sub-pixel point and a second sub-pixel point corresponding to each adjusted point cloud point in the first image and the second image, respectively, wherein the matching accuracy of the first sub-pixel point and the second sub-pixel point is higher than the matching accuracy of the first pixel point and the second pixel point;

[0020] A target depth determination module is configured to calculate a matching cost between the first sub-pixel point and the second sub-pixel point, and determine a depth value corresponding to a minimum matching cost as a target depth value;

[0021] The model updating module is used to update the three-dimensional point cloud model of the target object according to the target depth value.

[0022] According to a fourth aspect of the embodiments of this specification, another three-dimensional model reconstruction device is provided, suitable for reconstructing a three-dimensional model of an object with hair, the device comprising:

[0023] a hair target object image acquisition module, configured to acquire a first image and a second image of a target object with hair, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object;

[0024] a model building module, configured to build a three-dimensional point cloud model of the target object based on the first image and the second image;

[0025] a clustering module, configured to cluster the point cloud points in the three-dimensional point cloud model and delete the point cloud points that do not meet a preset clustering condition, wherein the preset clustering condition is determined based on the attributes of the hair;

[0026] The clustering conditions at least include: a clustering radius and a clustering minimum point threshold, the clustering radius is 5*scanning point distance to 10*scanning point distance, and the clustering minimum point threshold is 100 to 500.

[0027] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, storing a computer program, which implements the steps of any of the above-mentioned reconstruction methods when executed by a processor.

[0028] According to the sixth aspect of the embodiments of this specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the above-described reconstruction methods are implemented.

[0029] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0030] In the embodiments of this specification, depth values ​​are used to adjust the original pixel precision, constructing new pixels within the original pixels. This adjusts integer-level pixel precision to higher-precision pixels, significantly improving the matching accuracy of feature points during reconstruction and reducing reconstruction errors. This allows the reconstructed 3D model to more accurately reflect the characteristics of the target object. This is particularly true when dealing with complex structures with tiny pores, such as hair, enabling better capture of details and improving the model's realism and integrity. Furthermore, this method does not rely on external auxiliary objects, simplifying the scanning process and avoiding the potential bias introduced by auxiliary objects, ensuring the naturalness and accuracy of the reconstruction results.

[0031] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0033] Figure 1 This is a flowchart of a three-dimensional model reconstruction method according to an exemplary embodiment of this specification.

[0034] Figure 2 This is a flowchart of another three-dimensional model reconstruction method according to an exemplary embodiment of this specification.

[0035] Figure 3 This is a hardware structure diagram of the computer device where the three-dimensional model reconstruction device of the embodiment of this specification is located.

[0036] Figure 4 This is a block diagram of a three-dimensional model reconstruction device according to an exemplary embodiment of the present specification.

[0037] Figure 5 This specification provides a block diagram of another 3D model reconstruction device according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0039] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0040] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information without departing from the scope of this specification. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0041] 3D reconstruction is a technique that converts 2D images into 3D models. It recovers the 3D structure of an object by analyzing images acquired from different viewpoints. This process typically includes image acquisition, feature extraction and matching, motion recovery, point cloud generation, and surface reconstruction.

[0042] 1. Image Acquisition: In the initial stages of 3D reconstruction, a series of 2D images of the target object are acquired using a camera or other image acquisition device. These images are taken from different angles and positions to capture all the object's features. For example, for a hairy object, such as a human head, two high-resolution cameras can be used to capture images from different angles.

[0043] 2. Feature extraction and matching: Process the collected images to extract key feature points, such as corners and edges. Then, match the feature points between different images to find the same feature points under multiple viewing angles.

[0044] 3. Motion recovery: By matching feature points, we can estimate the camera's motion trajectory during the recording process, that is, the position and posture of the object when viewed from different perspectives. This process is called motion recovery and can be achieved through a variety of algorithms.

[0045] 4. Point cloud generation: By combining the results of motion recovery and feature point matching information, the position of each matching point in 3D space can be calculated to form a point cloud. A point cloud is composed of a large number of discrete points (referred to as point cloud points in this application), each of which represents a position in space.

[0046] 5. Surface reconstruction: After the point cloud is generated, it needs to be converted into a continuous 3D surface model. This is usually achieved through surface fitting, mesh generation and other technologies.

[0047] However, existing 3D reconstruction techniques face significant challenges when dealing with objects with complex structures, such as those with porous attachments.

[0048] For example, hair is composed of numerous fine filaments with numerous tiny pores between them. These pores can absorb or scatter light during scanning, reducing the amount of light reflected back to the scanner. This absorption reduces the contrast of hair in a 2D image, making feature point extraction and matching more difficult.

[0049] In addition, hair is naturally fluffy and disordered, which makes the distribution of hair feature points in images acquired from different angles highly irregular and random. This irregularity increases the complexity of feature matching, as the same hair may appear completely different in images from different perspectives.

[0050] Furthermore, due to the filamentous structure of hair, even high-resolution scanning equipment has difficulty distinguishing adjacent individual hairs. During the feature point matching stage, this can cause the feature points of multiple hair strands to be incorrectly matched as a single point, or make it difficult to find corresponding matching points, resulting in large errors in feature matching and, in turn, poor quality in the reconstructed 3D point cloud data.

[0051] Furthermore, feature point matching typically uses pixel-level feature points within an image. Because pixels are the smallest independent units in an image, this matching method fails to capture detailed spatial information, making it difficult to achieve the required accuracy when processing highly detailed or minutely structured objects, such as hair. Pixel-level matching is also susceptible to image noise, lighting variations, and feature point discontinuities, further increasing matching errors and reducing the accuracy of the reconstructed 3D model.

[0052] Furthermore, during the 3D point cloud generation phase, hair point cloud data often exhibits discontinuities due to the aforementioned reasons. This discontinuity can cause existing point cloud processing algorithms to mistakenly identify some valid data as noise or outliers, deleting them during subsequent processing. This results in a loss of important structural information, making the final point cloud data overly sparse, resulting in an incomplete model that cannot accurately reflect the true morphology and structure of hair.

[0053] To address the challenges of 3D reconstruction, the current common practice is to use external auxiliary objects for auxiliary stitching scanning. This method introduces a regularly shaped auxiliary object during the scanning process, which is scanned together with the target object. The scan data of the auxiliary object is used as a reference to approximately restore the 3D model of the target object. However, this method relies too much on the external object, which not only increases the complexity of the scan, but also, due to the essential differences between the auxiliary object and the target object, the reconstructed 3D model may not accurately capture the true structure and morphology of the hair, resulting in limited accuracy and authenticity of the model, making it difficult to achieve high-quality 3D reconstruction results.

[0054] In view of the above problems, the present application provides a three-dimensional model reconstruction method.

[0055] Figure 1 This is a flow chart of a three-dimensional model reconstruction method according to an exemplary embodiment of the present specification. Figure 1 As shown, the three-dimensional model reconstruction method includes the following steps:

[0056] S101 : Acquire a first image and a second image of a target object, where the first image and the second image are two-dimensional images collected from two different perspectives of the target object.

[0057] The first and second images can be obtained in a variety of ways, such as using a stereo camera, a laser scanner, or by capturing the target object from different positions using multiple conventional cameras. These images provide the basic visual information for subsequent 3D reconstruction.

[0058] S102: Construct an initial three-dimensional point cloud model of the target object by matching a first pixel point in the first image with a corresponding second pixel point in the second image.

[0059] The specific implementation process of step S102 may include the following stages:

[0060] Feature point extraction: First, for the first and second images, key feature points need to be extracted. These feature points are typically pixels with unique textures and shapes in the image, such as corners, edges, or other significant image structures. Existing feature extraction algorithms, such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Robust Features), and ORB (Oriented Fast and Rotated Brief), can be used to identify and describe these feature points.

[0061] Feature Point Matching: The extracted feature points need to be matched between the two images. This step aims to find the corresponding pixel in the second image for the first pixel in the first image. The matching process typically involves comparing the descriptors of the feature points in the two images. These descriptors are numerical representations of the pixels surrounding the feature point that capture the essential properties of the feature point. The matching algorithm calculates the similarity or distance between each feature point descriptor and finds the best match based on these metrics.

[0062] Initial 3D point cloud model construction: Once feature points are successfully matched between the two images, these matched points can be used to construct an initial 3D point cloud model. To do this, the relative motion between the two images (i.e., the camera pose) needs to be estimated. This can be achieved by solving a problem called "multi-view geometry." Common methods include calculating fundamental matrices or homography matrices, which describe the geometric relationship between the two images.

[0063] Triangulation: Given the camera poses and matched feature points, the position of each matching point in 3D space can be calculated through triangulation. Triangulation is a method that uses the positions of points in two views and their corresponding camera poses to determine the position of a point in space. For each pair of matching points, an epipolar line is constructed between the two view planes, and the point's position in 3D space is determined by solving the intersection of these epipolar lines.

[0064] Point cloud integration: All 3D points obtained through triangulation are integrated into a point cloud model. This point cloud model is a preliminary 3D representation of the target object, consisting of a series of discrete points on the object's surface. The positions of these points provide a preliminary reflection of the object's surface shape and structure.

[0065] S103. Adjust the depth value of each point cloud point in the initial three-dimensional point cloud model, and generate a first sub-pixel point and a second sub-pixel point corresponding to each adjusted point cloud point in the first image and the second image, respectively, wherein the matching accuracy of the first sub-pixel point and the second sub-pixel point is higher than the matching accuracy of the first pixel point and the second pixel point.

[0066] During the 3D reconstruction model building process, the spatial position of a point cloud point is determined by matching pixels in two images from different perspectives. The position of each point cloud point can be simply represented by its X, Y, and Z coordinates in 3D space, where the Z coordinate can be understood as the point's depth value, representing the distance from the observation camera. In other words, the depth value of a point cloud point directly maps to the pixels in the first and second images. Therefore, adjusting the depth value of a point cloud point can change the corresponding pixel in the first and second images.

[0067] To address the error and precision issues associated with pixel-level matching in existing 3D reconstruction techniques, this embodiment constructs new pixels between existing pixels by adjusting the depth values ​​of the point cloud points. These newly generated pixels are referred to herein as sub-pixels generated on the first and second images. Because the precision of the original pixels is typically at the integer level, these newly constructed sub-pixels enable a higher level of matching accuracy between the first and second images. This provides location information with higher precision than the original pixels, enabling the matching process to identify and manipulate details smaller than a single pixel, thereby capturing finer spatial information and improving matching accuracy and the quality of the reconstructed model.

[0068] Here, sub-pixel can refer to an image representation unit with a decimal level. For example, if the original pixel can be represented using integer coordinates such as (1,1) and (2,2), the sub-pixel can be represented using decimal coordinates such as (1.1,1.1), or other forms of improving image resolution. In addition to using decimals to describe the precision of sub-pixel points, the precision level of sub-pixel points can also be described using other expressions to describe higher precision than the integer level based on actual needs, which is not specifically limited in this embodiment.

[0069] For the embodiment of adjusting the depth value of the point cloud point, it can be achieved by offsetting the point cloud point. The offset amount is sufficient to generate sub-pixel points on the image, and the offset direction can be any direction, which is not specifically limited in this embodiment.

[0070] Furthermore, in the 3D reconstruction process, considering that each point cloud point has an associated normal vector, this normal vector is perpendicular to the tangent plane of the local surface where the point cloud point is located, and can be used to describe the shape and direction of the local surface of the point.

[0071] Based on this feature, in order to make the reconstructed point cloud data more closely fit the surface of the actual object, more accurately reflect the surface details of the target object, and improve the accuracy and quality of the three-dimensional model, in some embodiments, adjusting the depth value of each point cloud point in the initial three-dimensional point cloud model may include: determining the normal of each point cloud point in the initial three-dimensional point cloud model, and offsetting the point cloud point along the normal by a specified offset.

[0072] The normal can be determined by calculating the distribution and positional relationship of the point cloud points within a local area, for example, using least squares plane fitting or nearest neighbor normal estimation. The specified offset can be adjusted based on the desired level of sub-pixel accuracy required, and this embodiment does not impose any specific limitations.

[0073] S104: Calculate the matching cost of the first sub-pixel point and the second sub-pixel point, and determine the depth value corresponding to the minimum matching cost as the target depth value.

[0074] Matching cost is a key metric for evaluating the quality of the match between corresponding points in two images, and it is associated with the accuracy of the 3D model. A lower matching cost indicates a more accurate match between the two points, and thus a higher quality reconstructed 3D model.

[0075] In this embodiment, the matching costs of the first sub-pixel point and the second sub-pixel point at different depths are iteratively calculated and compared to identify the minimum matching cost. The depth value corresponding to the matching cost is determined to be the target depth value when the matching accuracy is the highest. The three-dimensional model updated according to the depth value is the most accurate three-dimensional model, which can better reflect the real structure of the target object.

[0076] Among them, the matching cost can be calculated based on various differences between corresponding points in the image, such as brightness difference, color difference, texture feature difference, etc. The specific matching cost calculation method can be selected according to actual needs, and this embodiment does not make specific limitations.

[0077] The following are some commonly used matching cost calculation formulas:

[0078] 1. Square difference: Calculate the sum of the squares of the differences between the brightness values ​​of two pixels I1(x,y) and I2(x′,y′).

[0079] D=∑[I1(x,y)-I2(x′,y′)] 2

[0080] 2. Normalized mutual information: It is a matching cost calculation method based on information theory. It measures the degree of matching between points in two images while taking into account their statistical properties.

[0081]

[0082] Among them, H(I1) and H(I2) are the entropy values ​​of the two images respectively, and K is a small constant to avoid division by zero.

[0083] 3. Hamming distance: In some cases, if the pixel features are represented as binary vectors, the Hamming distance can be used to calculate the difference between two vectors. The Hamming distance is the number of different positions between two strings of equal length and can be used to calculate the matching cost.

[0084] D=∑[bitwise_xor(I1(x,y)-I2(x′,y′))]

[0085] Among them, bitwise_xor is a bitwise exclusive OR operation.

[0086] 4. Structural Similarity Index: This is a full-reference image quality assessment method that takes into account similarity in brightness, contrast, and structure. The closer the structural similarity value is to 1, the more similar the match between the two images is.

[0087]

[0088] Among them, μ1 and μ2 are the means of the two images, σ1 and σ2 are the standard deviations, and σ 12 is the covariance, c1 and c2 are small constants added to avoid the denominator being zero.

[0089] S105: Update the three-dimensional point cloud model of the target object according to the target depth value.

[0090] After determining the target depth value, the position of corresponding points in the point cloud model can be adjusted using this value, resulting in a more accurate integrated point cloud model. The updated 3D point cloud model can then be reconstructed using various algorithms to convert the discrete point cloud data into a continuous 3D surface model. Common surface reconstruction techniques include the Poisson reconstruction algorithm, Delaunay triangulation, and other methods such as spherical blending and surface fitting.

[0091] During the surface reconstruction process, some additional data processing can be performed, such as removing outliers, filling holes, smoothing, etc., to improve the overall quality of the model, help eliminate noise and errors that may be introduced during data acquisition or processing, and ensure that the final three-dimensional model is more accurate.

[0092] In some embodiments, the target object may be an object with porous structure attachments, such as hair, vegetation, or other fibrous substances.

[0093] During the 3D reconstruction process of objects with porous attachments, the complexity and irregularity of their internal structure can easily lead to the appearance of numerous noise points or isolated point clusters in the point cloud data. These noise points or isolated point clusters are often caused by factors such as obstruction and scattering of scanning light by the porous structure, reflections within small spaces, scanning errors caused by the porous structure, and environmental interference, resulting in abnormal data. These abnormal data are typically characterized by a sparse number of points or a significant distance from the main point cluster, indicating that they are not part of the target object's attachments. If not effectively processed, these point cloud data that do not belong to the target object's structure may adversely affect the 3D reconstruction results, such as introducing false surface structures or causing model discontinuities.

[0094] Therefore, in some embodiments, as a further improvement to the above-mentioned embodiments, the point cloud points in the three-dimensional point cloud model can also be clustered, and the point cloud points that do not meet the preset clustering conditions can be deleted. The preset clustering conditions are determined based on the properties of the attachments; the clustering conditions include at least: the clustering radius and the minimum point number threshold of the clustering cluster.

[0095] In this embodiment, point cloud points are clustered to identify and remove those that do not belong to the target object, significantly improving the quality of the reconstructed model and ensuring that the resulting 3D model faithfully reflects the morphology and structure of the target object. Common clustering algorithms include K-means, DBSCAN, and Mean Shift, and can be selected based on actual needs. This embodiment does not specifically limit this.

[0096] To improve clustering accuracy and efficiency, the preset clustering conditions are adjusted based on the specific properties of the attachments. For example, the size, material, shape, and structural complexity of the attachments can all influence the setting of clustering conditions. In this embodiment, clustering conditions, such as the cluster radius and the minimum number of points in a cluster, can be adjusted based on the specific properties of the attachments to ensure that abnormal data can be effectively identified and deleted.

[0097] Furthermore, in some embodiments, the attachment is hair, the clustering radius is 5*scanning point distance to 10*scanning point distance, and the minimum point count threshold of the cluster is 100-500.

[0098] When the attachment is hair, appropriate clustering conditions can be selected based on the hair's thickness, texture, volume, continuity, and other properties to achieve the optimal clustering effect. The preset cluster radius in the clustering condition can be set to 5*scanning point distance to 10*scanning point distance. The scanning point distance generally refers to the distance between adjacent scanning points during the scanning process. This distance can be the actual physical distance or the sampling interval in the image or dataset. This value can capture the detailed features of the hair without causing too many noise points to be incorrectly classified due to being too small. This ensures that the clustering process can maintain hair details while avoiding excessive point cloud data redundancy, thereby improving the efficiency and accuracy of 3D reconstruction. Similarly, the preset cluster minimum point count threshold in the clustering condition can be set to 100-500, so that the threshold is set to a point count that can identify the continuity of hair while excluding isolated point cloud data that may be caused by measurement errors. This ensures that only when a certain number of point clouds are clustered together can a valid cluster be formed, thereby reflecting the true distribution and direction of the hair.

[0099] This specification also provides an interactive component for the user to select different operating modes, so that the user can select a certain operating mode through the interactive component. The user can use the interactive component to trigger the three-dimensional reconstruction process for the object with porous attachments described above. In some embodiments, the three-dimensional reconstruction process for the object with porous attachments can include the following steps:

[0100] Providing an interactive component and utilizing the interactive component to display a plurality of 3D reconstruction mode options, wherein the plurality of 3D reconstruction modes at least includes a 3D reconstruction mode for an object with a porous structure attachment;

[0101] When receiving a user-selected instruction for a 3D reconstruction mode of an object with porous structure attachments, the cluster radius is determined as a preset cluster radius, and the cluster minimum point threshold is determined as a preset cluster minimum point threshold.

[0102] Taking hair as an example, the specific process can refer to the following steps:

[0103] Providing an interactive component, and using the interactive component to display options for several 3D reconstruction modes, wherein the several 3D reconstruction modes at least include a 3D reconstruction mode for objects with hair;

[0104] When receiving a user-selected instruction for a 3D reconstruction mode of a hairy object, the cluster radius is determined to be 5*scanning point distance to 10*scanning point distance, and the cluster minimum point number threshold is determined to be 100-500.

[0105] In this embodiment, the interactive component can be a graphical user interface (GUI), a command line interface or other user interaction system, which aims to provide an intuitive and easy-to-operate way for users to select the 3D reconstruction mode that best suits their needs.

[0106] Each 3D reconstruction module can correspond to different clustering conditions according to the different properties of the attachments in its target object. When the user selects a certain 3D reconstruction mode, the system can adjust the clustering conditions according to the mode selected by the user.

[0107] In addition to the user triggering the 3D reconstruction method of this specification, in another embodiment, the process can also be triggered by automated image recognition. For example, the triggering action can be implemented by the following steps:

[0108] A two-dimensional image of a target object is collected, and when the target object is identified as an object with a porous structure attachment, the cluster radius is determined to be a preset cluster radius, and the cluster minimum point threshold is determined to be a preset cluster minimum point threshold.

[0109] Taking hair as an example, the specific process can refer to the following steps:

[0110] A two-dimensional image of the target object is collected, and when the target object is identified as an object with hair, the cluster radius is determined to be 5*scanning point distance to 10*scanning point distance, and the cluster minimum point threshold is determined to be 100-500.

[0111] In this embodiment, a two-dimensional image of the target object can be captured by a camera, mobile phone or other image capture device, and image recognition is performed on the captured two-dimensional image using a machine learning model, AI recognition or other image recognition methods. When the target object is identified as an object with hair, the parameters of the clustering algorithm will be adjusted according to the properties of the hair.

[0112] During the 3D reconstruction process, objects with porous structures and attachments may experience occlusion and multiple scattering of the scanning light within the pores due to their porous structure. This can cause geometric deviations in the data for the same physical point at different viewing angles. If not addressed, these deviations will accumulate, leading to geometric distortion in the reconstructed model, such as inaccurate pore connections or distortion of the overall structure.

[0113] At the same time, similar areas in porous structures may be captured multiple times in different scans, resulting in multiple duplicate points in the point cloud. Without effective processing, these duplicate points will be mistakenly regarded as different physical features, increasing the complexity of data processing and potentially creating spurious structures in the reconstructed model, affecting the model's detail expression and surface quality.

[0114] Therefore, in some embodiments, the three-dimensional reconstruction process of the present specification may also include: performing parallax consistency constraint detection and uniqueness constraint detection on the point cloud points in the three-dimensional point cloud model, deleting the point cloud points that do not satisfy the parallax consistency constraint and uniqueness constraint, and the parallax consistency constraint and uniqueness constraint are determined based on the properties of the attachment.

[0115] In this embodiment, the parallax consistency constraint ensures that point cloud data scanned from different perspectives is consistently mapped onto the actual surface of the object, thereby avoiding geometric errors caused by perspective differences. The uniqueness constraint reduces redundancy in the point cloud set by identifying and merging duplicate point cloud data, ensuring that each point in the point cloud represents a unique location on the object's surface, thereby improving model accuracy and efficiency. By ensuring the consistency and uniqueness of the point cloud, the accuracy and quality of the reconstructed model can be significantly improved, avoiding reconstruction errors caused by data inconsistency and duplication, resulting in a more realistic and reliable 3D model that meets the needs of subsequent applications.

[0116] When determining parallax consistency and uniqueness constraints, it's important to consider object attributes, such as shape, size, material, and structural complexity. These attributes will influence detection sensitivity and threshold settings. For example, for objects with complex textures, a higher consistency threshold can be set to avoid accidentally deleting valid point cloud data. Alternatively, for small or densely packed objects, a stricter uniqueness check can be employed to avoid excessive duplication of points.

[0117] During the 3D reconstruction process, objects with porous structural attachments, such as hair, may also have large coordinate differences between adjacent point clouds due to their natural curvature and fluffy characteristics. Such differences will cause the point cloud data to be broken in certain areas and the distribution to be discontinuous, presenting unnatural faults or jumps in the reconstructed model, making it impossible to accurately restore the true surface morphology of the target object.

[0118] Furthermore, the complex spatial orientation and irregular pore distribution of porous structures like hair can cause spatially uneven distribution of point cloud data during the scanning process, leading to the formation of data-sparse regions or holes. During other point cloud data processing stages, these sparse regions may be filtered out due to insufficient data, severely compromising the integrity and detail of the reconstructed model.

[0119] Therefore, in some embodiments, the 3D reconstruction process of this specification may further include:

[0120] Using a nonlinear filtering algorithm, new point cloud points are filled in the three-dimensional point cloud model so that the point cloud points in the filled three-dimensional point cloud model meet a preset continuous distribution condition, and the preset continuous distribution condition is determined based on the shape of the attachment.

[0121] In this embodiment, a nonlinear filtering algorithm is used to fill the point cloud data, which helps to improve the continuity of the point cloud data, reduce unnatural appearance faults, optimize the spatial distribution of the point cloud data, fill sparse areas or holes, effectively retain important structural details, and improve the integrity and accuracy of the reconstructed model.

[0122] Among them, in order to ensure that the filled point cloud points can more accurately describe the shape of the attachments of the target object, the preset continuous distribution conditions can be determined according to the shape of the attachments. The continuous distribution conditions may include the minimum spacing, maximum spacing, uniformity of point cloud density distribution, etc. The shape of the attachments may include smooth surfaces, curved surface changes, edges and textures, etc.

[0123] The reason for choosing a nonlinear filtering algorithm for point cloud filling is that compared with other linear filtering algorithms, the nonlinear filtering algorithm can more effectively process and reconstruct data with complex geometric shapes and topological structures, and is more in line with the natural curvature, fluffy characteristics and irregular spatial orientation of attachments such as hair, thereby more comprehensively and naturally restoring the morphological characteristics of the object surface.

[0124] In addition, during the three-dimensional reconstruction process, the point cloud model is often mesh-encapsulated and converted into a three-dimensional mesh model for subsequent use.

[0125] However, for objects with porous structures, such as hair, due to their structural complexity, it is difficult to capture every detail during scanning and point cloud generation, especially the connections and boundaries of the pores, which leads to incomplete or inaccurate meshes during the meshing process. At the same time, the interior of porous structures is difficult to scan, which may cause data loss in these areas and uneven spatial distribution of point cloud data. During meshing, some mesh pieces may become isolated due to lack of sufficient point cloud connection support. These isolated or incorrectly connected mesh pieces introduce incorrect geometric information into the reconstructed model, affecting the accuracy of the model and causing the model to fail to accurately reflect the surface morphology of the actual object.

[0126] Therefore, in some embodiments, the 3D reconstruction process of this specification may further include:

[0127] Performing grid processing on the three-dimensional point cloud model to convert it into a three-dimensional grid model;

[0128] Delete a grid in the three-dimensional grid model whose number of common points with other grids does not meet a preset common point number threshold, where the preset common point number threshold is determined based on the attributes of the attachment.

[0129] In this embodiment, by identifying and deleting meshes whose number of common points does not meet a preset threshold, the model quality is optimized to ensure that the 3D mesh model accurately and completely represents the surface features of the target object. The meshing process can use a variety of algorithms, such as Delaunay triangulation, Poisson reconstruction, and spherical segmentation.

[0130] Similarly, the preset common point number threshold may also be determined based on the properties of the attachment. For example, for attachments with complex structures and rich details, a lower common point number threshold may need to be set to retain more details.

[0131] When the attachment is hair, since hair is slender, soft, and continuous, the preset common point count threshold can be set to 100. This threshold is sufficient to preserve the continuity and details of the hair while effectively removing redundant meshes caused by noise or isolated points.

[0132] When converting 3D point cloud models into mesh models, objects with porous structures, such as hair, can experience abnormal mesh deformation during meshing due to their irregular natural shapes, such as bends, curls, or tangles, or because their tiny pores are difficult to scan. These abnormally deformed meshes can cause the model's geometry to deviate significantly from the actual object. This distortion visually manifests as sharp protrusions or irregular depressions, affecting the model's accuracy and authenticity.

[0133] Therefore, in some embodiments, the three-dimensional reconstruction process of the present specification may further include: deleting meshes in the three-dimensional mesh model whose mesh deformation curvature exceeds a preset deformation curvature threshold, where the preset deformation curvature threshold is determined based on the morphology of the attachment.

[0134] In this embodiment, the mesh deformation curvature can be used to describe the local shape change of the mesh. Deleting meshes whose mesh deformation curvature exceeds a preset deformation curvature threshold helps ensure that the model better reflects the surface characteristics of the actual object and improves the model quality.

[0135] For the detection and calculation of mesh deformation curvature, the geometric positions of mesh vertices can be mathematically analyzed to determine the local shape and deformation of the mesh.

[0136] The preset deformation curvature threshold can be determined based on the shape of the attachment. This threshold is used to determine whether the mesh is too distorted or deformed and whether it matches the expected shape of the target object attachment, which may affect the accuracy and authenticity of the reconstructed model.

[0137] When the attachment is hair, since hair has natural curling and bending characteristics, the preset deformation curvature threshold should adapt to the natural morphological changes of hair. The preset deformation curvature threshold can be set to 0.125~0.09. This threshold can effectively distinguish normal hair curvature from abnormal mesh deformation, while avoiding the loss of details caused by over-smoothing.

[0138] After deleting some meshes, a mesh model may exhibit numerous jagged edges and sharp edges, causing the model to lose its original smoothness and continuity, impacting its visual quality and the feasibility of subsequent applications. Therefore, to reduce jagged edges and sharp edges while preserving the model's key features and details, in some embodiments, the 3D mesh model can be smoothed after deleting meshes exceeding a preset deformation threshold. An appropriate smoothing algorithm, such as averaging, Gaussian smoothing, or Laplace smoothing, can be selected based on actual needs.

[0139] Figure 2 This is a flow chart of another three-dimensional model reconstruction method according to an exemplary embodiment of this specification. Figure 2 As shown, the three-dimensional model reconstruction method includes the following steps:

[0140] S201, acquiring a first image and a second image of a target object with hair, where the first image and the second image are two-dimensional images acquired from two different perspectives of the target object;

[0141] S202: constructing a three-dimensional point cloud model of the target object based on the first image and the second image;

[0142] S203, clustering the point cloud points in the three-dimensional point cloud model, and deleting the point cloud points that do not meet a preset clustering condition, where the preset clustering condition is determined based on the attributes of the hair;

[0143] S204 , the clustering conditions at least include: cluster radius and a cluster minimum point threshold, the cluster radius is 5*scanning point distance to 10*scanning point distance, and the cluster minimum point threshold is 100-500.

[0144] Furthermore, in some embodiments, the reconstruction method further includes:

[0145] Performing parallax consistency constraint detection and uniqueness constraint detection on the point cloud points in the three-dimensional point cloud model, and deleting the point cloud points that do not satisfy the parallax consistency constraint and uniqueness constraint, wherein the parallax consistency constraint and uniqueness constraint are determined based on the attributes of the hair.

[0146] Furthermore, in some embodiments, the reconstruction method further includes:

[0147] Using a nonlinear filtering algorithm, new point cloud points are filled in the three-dimensional point cloud model so that the point cloud points in the filled three-dimensional point cloud model meet a preset continuous distribution condition, and the preset continuous distribution condition is determined based on the morphology of the hair.

[0148] Furthermore, in some embodiments, the reconstruction method further includes:

[0149] Performing grid processing on the three-dimensional point cloud model to convert it into a three-dimensional grid model;

[0150] Deleting meshes in the three-dimensional hair mesh model whose number of common points with other meshes does not meet a preset common point number threshold, where the preset common point number threshold is determined based on the properties of the hair;

[0151] The preset common point number threshold is 100.

[0152] Furthermore, in some embodiments, the reconstruction method further includes:

[0153] Deleting meshes in the three-dimensional mesh model whose mesh deformation curvature exceeds a preset deformation curvature threshold, wherein the preset deformation curvature threshold is determined based on the morphology of the hair;

[0154] The preset deformation curvature threshold is 0.125 to 0.09;

[0155] Smoothing is performed on the three-dimensional mesh model after deleting the meshes that exceed the preset deformation threshold.

[0156] The above-mentioned another three-dimensional model reconstruction method is suitable for reconstructing a three-dimensional model of an object with hair. The specific implementation process of the steps can refer to the implementation process of the corresponding steps in the first three-dimensional model reconstruction method, which will not be repeated here.

[0157] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a three-dimensional model reconstruction device and a terminal used therein.

[0158] The embodiments of the three-dimensional model reconstruction device in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which the three-dimensional model is reconstructed reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 3 The figure is a hardware structure diagram of the computer device where the three-dimensional model reconstruction device of the embodiment of this specification is located, except Figure 3In addition to the processor 310, memory 330, network interface 320, and non-volatile memory 340 shown, the server or electronic device where the device is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0159] Figure 4 This is a block diagram of a three-dimensional model reconstruction device according to an exemplary embodiment of the present specification. Figure 4 As shown, the three-dimensional model reconstruction device includes:

[0160] A target object image acquisition module 401 is configured to acquire a first image and a second image of a target object, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object;

[0161] An initial model building module 402 is configured to build an initial three-dimensional point cloud model of the target object by matching a first pixel point in the first image with a corresponding second pixel point in the second image;

[0162] a depth adjustment and sub-pixel point generation module 403, configured to adjust the depth value of each point cloud point in the initial three-dimensional point cloud model, and generate a first sub-pixel point and a second sub-pixel point corresponding to each adjusted point cloud point in the first image and the second image, respectively, wherein the matching accuracy of the first sub-pixel point and the second sub-pixel point is higher than the matching accuracy of the first pixel point and the second pixel point;

[0163] A target depth determination module 404 is configured to calculate a matching cost between the first sub-pixel point and the second sub-pixel point, and determine a depth value corresponding to a minimum matching cost as a target depth value;

[0164] The model updating module 405 is configured to update the three-dimensional point cloud model of the target object according to the target depth value.

[0165] Figure 5 This is a block diagram of another 3D model reconstruction device according to an exemplary embodiment of the present specification. The 3D model reconstruction device is suitable for reconstructing a 3D model of an object with hair, such as Figure 4 As shown, the three-dimensional model reconstruction device includes:

[0166] The hair target object image acquisition module 501 is used to acquire a first image and a second image of a target object with hair, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object;

[0167] A model building module 502 is configured to build a three-dimensional point cloud model of the target object based on the first image and the second image;

[0168] A clustering module 503 is configured to cluster the point cloud points in the three-dimensional point cloud model and delete the point cloud points that do not meet a preset clustering condition, wherein the preset clustering condition is determined based on the attributes of the hair;

[0169] The clustering conditions at least include: a clustering radius and a clustering minimum point threshold, the clustering radius is 5*scanning point distance to 10*scanning point distance, and the clustering minimum point threshold is 100 to 500.

[0170] Accordingly, this specification also provides a computer-readable storage medium storing a computer program, which implements the steps of any of the above-mentioned reconstruction methods when executed by a processor.

[0171] Accordingly, this specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned reconstruction methods when executing the program.

[0172] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0173] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0174] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] Other embodiments of the present invention will readily occur to those skilled in the art upon consideration of the present invention and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present invention being indicated by the following claims.

[0176] It should be understood that the present description is not limited to the exact structure that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0177] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A three-dimensional model reconstruction method, characterized in that: The method comprises: Acquire a first image and a second image of a target object, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object by two image acquisition devices; Constructing an initial three-dimensional point cloud model of the target object by matching a first pixel point in the first image with a corresponding second pixel point in the second image; Adjusting the depth value of each point cloud point in the initial three-dimensional point cloud model, generating first sub-pixel points corresponding to each adjusted point cloud point between original pixel points of the first image, generating second sub-pixel points corresponding to each adjusted point cloud point between original pixel points of the second image, wherein the matching accuracy of the first sub-pixel points and the second sub-pixel points is higher than the matching accuracy of the first pixel points and the second pixel points; Calculate the matching cost of the first sub-pixel point and the second sub-pixel point, and determine that the depth value corresponding to the minimum matching cost is the target depth value; According to the target depth value, the three-dimensional point cloud model of the target object is updated.

2. The reconstruction method according to claim 1, characterized in that: Adjusting the depth value of each point cloud point in the initial three-dimensional point cloud model includes: The normal direction of each point cloud point in the initial three-dimensional point cloud model is determined, and the point cloud point is offset along the normal direction by a specified offset amount.

3. The reconstruction method according to claim 1, characterized in that: The target object is an object with porous structure attachments.

4. The reconstruction method according to claim 3, characterized in that: Also includes: Clustering the point cloud points in the three-dimensional point cloud model, and deleting the point cloud points that do not meet a preset clustering condition, wherein the preset clustering condition is determined based on the attribute of the attachment; The clustering conditions at least include: a clustering radius and a minimum point count threshold of a cluster.

5. The reconstruction method according to claim 4, characterized in that: When the attachment is hair, the cluster radius is 5*scanning point distance to 10*scanning point distance, and the minimum point number threshold of the cluster is 100 to 500.

6. The reconstruction method according to claim 4, characterized in that: Also includes: Providing an interactive component, and using the interactive component to display a plurality of options for three-dimensional reconstruction modes, wherein the plurality of three-dimensional reconstruction modes at least include a three-dimensional reconstruction mode for an object with a porous structure attachment; When receiving a user-selected instruction for a three-dimensional reconstruction mode of an object with porous structure attachments, the cluster radius is determined as a preset cluster radius, and the cluster minimum point count threshold is determined as a preset cluster minimum point count threshold.

7. The reconstruction method according to claim 4, characterized in that: Also includes: A two-dimensional image of the target object is acquired, and when the target object is identified as an object with a porous structure attachment, the clustering radius is determined to be a preset clustering radius, and the clustering minimum point count threshold is determined to be a preset clustering minimum point count threshold.

8. The reconstruction method according to claim 3, characterized in that: Also includes: Performing parallax consistency constraint detection and uniqueness constraint detection on the point cloud points in the three-dimensional point cloud model, and deleting the point cloud points that do not satisfy the parallax consistency constraint and uniqueness constraint, wherein the parallax consistency constraint and uniqueness constraint are determined based on the attributes of the attachment.

9. The reconstruction method according to claim 3, characterized in that: Also includes: Using a nonlinear filtering algorithm, new point cloud points are filled in the three-dimensional point cloud model so that the point cloud points in the filled three-dimensional point cloud model meet a preset continuous distribution condition, and the preset continuous distribution condition is determined based on the shape of the attachment.

10. The reconstruction method according to claim 3, characterized in that: Also includes: Performing grid processing on the three-dimensional point cloud model to convert it into a three-dimensional grid model; Deleting a grid in the three-dimensional grid model whose number of common points with other grids does not meet a preset common point number threshold, wherein the preset common point number threshold is determined based on the attribute of the attachment.

11. The reconstruction method according to claim 10, characterized in that: When the attachment is hair, the preset common point number threshold is 100.

12. The reconstruction method according to claim 10, characterized in that: Also includes: Deleting meshes in the three-dimensional mesh model whose mesh deformation curvature exceeds a preset deformation curvature threshold, wherein the preset deformation curvature threshold is determined based on the shape of the attachment; The three-dimensional mesh model is smoothed after deleting the meshes exceeding the preset deformation threshold.

13. The reconstruction method according to claim 12, characterized in that: When the attachment is hair, the preset deformation curvature threshold is 0.125-0.

09.

14. A three-dimensional model reconstruction device, characterized in that: The device comprises: A target object image acquisition module, used to acquire a first image and a second image of the target object, wherein the first image and the second image are two-dimensional images acquired from two different perspectives of the target object by two image acquisition devices; An initial model building module, used to match a first pixel point in the first image with a corresponding second pixel point in the second image to build an initial three-dimensional point cloud model of the target object; A depth adjustment and sub-pixel point generation module, used to adjust the depth value of each point cloud point in the initial three-dimensional point cloud model, generate first sub-pixel points corresponding to each adjusted point cloud point between original pixel points of the first image, and generate second sub-pixel points corresponding to each adjusted point cloud point between original pixel points of the second image, wherein the matching accuracy of the first sub-pixel point and the second sub-pixel point is higher than the matching accuracy of the first pixel point and the second pixel point; A target depth determination module, configured to calculate a matching cost between the first sub-pixel point and the second sub-pixel point, and determine a depth value corresponding to a minimum matching cost as a target depth value; The model updating module is used to update the three-dimensional point cloud model of the target object according to the target depth value.

15. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the steps of the reconstruction method according to any one of claims 1 to 13 are implemented.

16. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the reconstruction method according to any one of claims 1 to 13 are implemented.

Images