Product duplicating software system for importing and converting three-dimensional model

Abstract

The invention relates to the technical field of product duplicating, in particular to a product duplicating software system for importing and converting a three-dimensional model, and provides the following scheme that a primary three-dimensional model is generated by obtaining a primary view image and a secondary view image and extracting a first characteristic parameter and a second characteristic parameter; adjusting the target parameters based on the bidirectional mapping relation between the target parameters and the inference parameters to generate a parameter combination; the response coefficient of the inference parameter is corrected through reverse derivation, the bidirectional incidence matrix is optimized, and the consistency of the target parameter and the inference parameter is ensured. Through shadow area analysis and curved surface fitting technology of the multi-view image, the problem of data missing of the bottom area is solved, and accurate three-dimensional reconstruction of the surface and the bottom area of the object is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of product replication, and in particular to a product replication software system for importing and converting three-dimensional models. Background Art

[0002] In the modern product design and manufacturing process, accurate reproduction of three-dimensional models is one of the key technologies to improve production efficiency and precision. Traditional three-dimensional modeling methods mainly rely on laser scanning, computer vision technology and manual modeling. Although these methods can obtain the geometric information of objects more accurately, in complex application scenarios, such as when the bottom area is difficult to photograph, they still face the problem of not being able to accurately obtain the full picture or missing data. In addition, although the shadow analysis method in the existing technology can provide local information on the surface of an object, since the shadow itself is based on the relative position relationship between the object and the light source, its analysis is essentially an ill-posed inverse problem, and it is difficult to directly obtain the absolute three-dimensional properties of the object, resulting in large errors in the shadow area during the reconstruction process.

[0003] For example, a Chinese patent with authorization announcement number CN109598785B discloses a method for converting a three-dimensional mesh model view, including the following steps: importing a three-dimensional mesh model; establishing a first spatial rectangular coordinate system with the geometric mass of the three-dimensional mesh model as the origin O, and turning the major axis of the three-dimensional mesh model to a position that coincides with the Z axis of the first spatial rectangular coordinate system; locating the projection point coordinates; setting the pixel value of the projection view; and generating a projection view based on the projection point coordinates and pixel values. This invention turns the major axis of the three-dimensional mesh model to a position that coincides with the Z axis of the first spatial rectangular coordinate system, thereby projecting a larger number of points in the three-dimensional mesh model into the view to obtain a projection view with the maximum amount of information. This invention is used to convert a three-dimensional mesh model into a two-dimensional projection view.

[0004] The above existing technologies all have the problem raised by this background technology: how to accurately extract three-dimensional models and eliminate uncertainties in complex environments, especially in difficult-to-shoot areas such as the bottom. To solve the above problems, this application designs a product replication software system for importing and converting three-dimensional models. Summary of the Invention

[0005] The technical problem to be solved by this invention is to address the shortcomings of existing technologies and provide a product replication software system for importing and converting 3D models. By acquiring primary and secondary view images, the system extracts first and second characteristic parameters to generate a preliminary 3D model. Based on the bidirectional mapping relationship between target parameters and inferred parameters, the system adjusts the target parameters and generates parameter combinations. The system then reversely deduces the response coefficients of the inferred parameters and optimizes the bidirectional correlation matrix to ensure consistency between the target and inferred parameters. By analyzing shadow areas in multi-view images and using surface fitting techniques, the system resolves the problem of missing bottom-area data and achieves accurate 3D reconstruction of the object's surface and bottom areas.

[0006] To achieve the above object, the present invention provides the following technical solutions: A product replica software system for importing and converting three-dimensional models is applied to product replica software. The product replica software system includes an image acquisition module, a feature parameter extraction module, a model building module, and a model conversion module, wherein: The image acquisition module is used to obtain a main view image and multiple auxiliary view images of the product to be reproduced; The feature parameter extraction module is used to extract the first feature parameter, and perform shadow area segmentation on the primary view image and the secondary view image to extract the second feature parameter; The model building module is configured to import the first characteristic parameter and the second characteristic parameter into product replica software to generate a first model of the product to be replicated, wherein the first model generates inferred parameters based on the first characteristic parameter and the second characteristic parameter; The model conversion module is used to convert the first model according to the input target parameters to generate a second model.

[0007] The feature parameter extraction module includes a first feature extraction unit and a second feature extraction unit, wherein: The first feature extraction unit is configured to perform image processing on the primary view image and the secondary view image, extract edge information, contour lines, and key points of the primary view image and the secondary view image, and use the edge information, contour lines, and key points as first feature parameters; The second feature extraction unit is used to infer feature parameters of the remaining area based on the shadow parts of the primary view image and the secondary view image.

[0008] The second feature extraction unit includes a shadow extraction subunit, a shadow correction subunit, a shadow parameter calculation subunit, a shadow reasoning subunit and a shadow inversion subunit, wherein: The shadow extraction subunit is used to extract the shadow edges of the primary view image and the secondary view image; The shadow correction subunit is used to correct the shadow edge of the main view image according to the shadow edge of the auxiliary view image to generate a bottom shadow area; The shadow parameter calculation subunit is used to calculate the curvature tensor of the bottom shadow area, and obtain the transverse deformation parameter and the longitudinal deformation parameter according to the curvature tensor, wherein the curvature tensor is used to characterize the curvature change characteristics of the shadow area; The shadow reasoning subunit is used to jointly reason the lateral deformation parameters and the longitudinal deformation parameters to generate shadow point pairs that satisfy the differential homeomorphism constraint and the viewing angle energy function; The shadow inversion subunit is used to perform reverse inversion through surface fitting based on the shadow point pairs to extract the second characteristic parameters.

[0009] The shadow extraction subunit includes: Performing camera calibration on the primary view image and the secondary view image to obtain an intrinsic parameter matrix and an extrinsic parameter matrix, and establishing an epipolar geometric relationship between the two perspectives based on the intrinsic parameter matrix and the extrinsic parameter matrix; detecting shadow edges in the primary view image using an epipolar equation; According to the epipolar geometric relationship, edge points with the same gradient direction as the shadow edge of the primary view image are extracted within the epipolar range corresponding to the secondary view image to generate the shadow edge of the secondary view image.

[0010] The shadow correction subunit includes: Calculate the gradient structure tensor of the shadow edge of the main view image and generate an edge saliency map; Processing the edge saliency map using a region growing algorithm to obtain a first shadow, wherein the region growing algorithm includes starting from the centroid of the edge saliency map and gradually merging adjacent pixels that meet the direction consistency of the centroid structure tensor; Calculating an epipolar projection matching rate of the first shadow in the shadow edge of the secondary view image; If the epipolar projection matching rate is less than a preset matching threshold, Gaussian blurring is performed on the edge saliency map and region growing is performed again until the epipolar projection matching rate is greater than or equal to the preset matching threshold; The first shadow is completed according to the matching result, and the completed first shadow is output as the bottom shadow area.

[0011] The shadow reasoning subunit includes: A differential homeomorphism mapping relationship between the transverse deformation parameter and the longitudinal deformation parameter is established through a differential homeomorphism constraint condition, wherein the differential homeomorphism constraint condition indicates that the shadow edge deformation of the same three-dimensional surface point in the transverse and longitudinal directions satisfies the differential homeomorphism transformation; constructing a cross-viewing angle deformation parameter correlation matrix according to the horizontal deformation parameter and the vertical deformation parameter, and calculating a viewing angle energy function according to matrix characteristics of the cross-viewing angle deformation parameter correlation matrix; According to the differential homeomorphism mapping relationship and the viewing angle energy function, the shadow point pairs that satisfy the dual-viewing deformation consistency are solved by the least squares method.

[0012] The model conversion module includes a parameter mapping unit, a parameter adjustment unit and a model conversion unit, wherein: The parameter mapping unit is used to obtain the input target parameters and construct a bidirectional mapping between the target parameters and the inferred parameters; The parameter adjustment unit is configured to adjust the target parameter according to the bidirectional mapping and preset constraints to generate a parameter combination; The model conversion unit is used to convert the first model according to the parameter combination to generate a second model.

[0013] The parameter mapping unit includes: parsing the parameter topological relationship tree of the first model to obtain the process constraint chain of the inferred parameters; Establishing an initial association mapping table between the target parameters and the inferred parameters according to the process constraint chain; Generate a bidirectional correlation matrix, where the row vectors of the bidirectional correlation matrix represent the modification direction of the target parameter, the column vectors of the bidirectional correlation matrix represent the response coefficients of the inferred parameters, and the matrix element values ​​are jointly calibrated by the geometric constraints between the parameters and the historical modification data; The initial association mapping table is optimized according to the bidirectional association matrix to generate a bidirectional mapping.

[0014] The parameter adjustment unit includes: According to the parameter topology relationship tree, identifying directly associated parameters and indirectly associated parameters of the target parameter by a breadth-first traversal algorithm; Calling a preset parameter adjustment rule library, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules, and assembly constraint rules; Identifying the directly associated parameters and the indirectly associated parameters according to the rule engine of the parameter adjustment rule base, matching corresponding rule subsets and parsing the adjustment direction and magnitude; Adjust the target parameter according to the adjustment direction and amplitude to generate a parameter combination; The parameter combination is reversely deduced to the inferred parameter, the response coefficient of the inferred parameter is corrected, and the bidirectional correlation matrix is ​​modified according to the response coefficient.

[0015] A product replication method for importing and converting a three-dimensional model, applied to product replication software, comprising: Obtaining a primary view image and a secondary view image of the product to be reproduced, and extracting a first characteristic parameter; performing shadow area segmentation on the primary view image and the secondary view image to extract a second characteristic parameter; Importing the first characteristic parameter and the second characteristic parameter into product replica software to generate a first model of the product to be replicated, wherein the first model generates inferred parameters based on the first characteristic parameter and the second characteristic parameter; Obtaining input target parameters and constructing a bidirectional mapping between the target parameters and the inferred parameters; Adjusting the target parameters according to the bidirectional mapping and preset constraints to generate a parameter combination; The first model is transformed according to the parameter combination to generate a second model.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention, by combining shadow analysis techniques with multi-view images, successfully addresses the data loss issues of existing 3D modeling methods, particularly at the bottom and in hard-to-reach areas. Through camera calibration, epipolar geometry, and surface fitting, it accurately extracts the bottom shadow area of ​​an object and converts this local shadow information into complete 3D geometric data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 This is a module diagram of a product replica software system for importing and converting three-dimensional models according to embodiment 1 of the present invention; Figure 2 This is a flow chart of a product replica method for importing and converting a three-dimensional model according to embodiment 2 of the present invention; Figure 3 This is a flowchart of extracting the second characteristic parameter according to embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the first shadow extraction in Example 2 of the present invention; Figure 5 This is a schematic diagram of the first shadow completion according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] It should be noted that the technical solutions in this application can be applied to a variety of technical fields, including but not limited to automobile models.

[0020] Example 1 See also Figure 1 An embodiment of the present invention provides a product replica software system for importing and converting three-dimensional models, which is applied to product replica software. The product replica software system includes an image acquisition module, a feature parameter extraction module, a model building module, and a model conversion module, wherein: The image acquisition module is used to obtain a main view image and multiple auxiliary view images of the product to be reproduced; The feature parameter extraction module is used to extract the first feature parameter, and perform shadow area segmentation on the primary view image and the secondary view image to extract the second feature parameter; The model building module is configured to import the first characteristic parameter and the second characteristic parameter into product replica software to generate a first model of the product to be replicated, wherein the first model generates inferred parameters based on the first characteristic parameter and the second characteristic parameter; The model conversion module is used to convert the first model according to the input target parameters to generate a second model.

[0021] The feature parameter extraction module includes a first feature extraction unit and a second feature extraction unit, wherein: The first feature extraction unit is configured to perform image processing on the primary view image and the secondary view image, extract edge information, contour lines, and key points of the primary view image and the secondary view image, and use the edge information, contour lines, and key points as first feature parameters; The second feature extraction unit is used to infer feature parameters of the remaining area based on the shadow parts of the primary view image and the secondary view image.

[0022] The second feature extraction unit includes a shadow extraction subunit, a shadow correction subunit, a shadow parameter calculation subunit, a shadow reasoning subunit and a shadow inversion subunit, wherein: The shadow extraction subunit is used to extract the shadow edges of the primary view image and the secondary view image; The shadow correction subunit is used to correct the shadow edge of the main view image according to the shadow edge of the auxiliary view image to generate a bottom shadow area; The shadow parameter calculation subunit is used to calculate the curvature tensor of the bottom shadow area, and obtain the transverse deformation parameter and the longitudinal deformation parameter according to the curvature tensor, wherein the curvature tensor is used to characterize the curvature change characteristics of the shadow area; The shadow reasoning subunit is used to jointly reason the lateral deformation parameters and the longitudinal deformation parameters to generate shadow point pairs that satisfy the differential homeomorphism constraint and the viewing angle energy function; The shadow inversion subunit is used to perform reverse inversion through surface fitting based on the shadow point pairs to extract the second characteristic parameters.

[0023] The shadow extraction subunit includes: Performing camera calibration on the primary view image and the secondary view image to obtain an intrinsic parameter matrix and an extrinsic parameter matrix, and establishing an epipolar geometric relationship between the two perspectives based on the intrinsic parameter matrix and the extrinsic parameter matrix; detecting shadow edges in the primary view image using an epipolar equation; According to the epipolar geometric relationship, edge points with the same gradient direction as the shadow edge of the primary view image are extracted within the epipolar range corresponding to the secondary view image to generate the shadow edge of the secondary view image.

[0024] The shadow correction subunit includes: Calculate the gradient structure tensor of the shadow edge of the main view image and generate an edge saliency map; Processing the edge saliency map using a region growing algorithm to obtain a first shadow, wherein the region growing algorithm includes starting from the centroid of the edge saliency map and gradually merging adjacent pixels that meet the direction consistency of the centroid structure tensor; Calculating an epipolar projection matching rate of the first shadow in the shadow edge of the secondary view image; If the epipolar projection matching rate is less than a preset matching threshold, Gaussian blurring is performed on the edge saliency map and region growing is performed again until the epipolar projection matching rate is greater than or equal to the preset matching threshold; The first shadow is completed according to the matching result, and the completed first shadow is output as the bottom shadow area.

[0025] The shadow reasoning subunit includes: A differential homeomorphism mapping relationship between the transverse deformation parameter and the longitudinal deformation parameter is established through a differential homeomorphism constraint condition, wherein the differential homeomorphism constraint condition indicates that the shadow edge deformation of the same three-dimensional surface point in the transverse and longitudinal directions satisfies the differential homeomorphism transformation; constructing a cross-viewing angle deformation parameter correlation matrix according to the horizontal deformation parameter and the vertical deformation parameter, and calculating a viewing angle energy function according to matrix characteristics of the cross-viewing angle deformation parameter correlation matrix; According to the differential homeomorphism mapping relationship and the viewing angle energy function, the shadow point pairs that satisfy the dual-viewing deformation consistency are solved by the least squares method.

[0026] The model conversion module includes a parameter mapping unit, a parameter adjustment unit and a model conversion unit, wherein: The parameter mapping unit is used to obtain the input target parameters and construct a bidirectional mapping between the target parameters and the inferred parameters; The parameter adjustment unit is configured to adjust the target parameter according to the bidirectional mapping and preset constraints to generate a parameter combination; The model conversion unit is used to convert the first model according to the parameter combination to generate a second model.

[0027] The parameter mapping unit includes: parsing the parameter topological relationship tree of the first model to obtain the process constraint chain of the inferred parameters; Establishing an initial association mapping table between the target parameters and the inferred parameters according to the process constraint chain; Generate a bidirectional correlation matrix, where the row vectors of the bidirectional correlation matrix represent the modification direction of the target parameter, the column vectors of the bidirectional correlation matrix represent the response coefficients of the inferred parameters, and the matrix element values ​​are jointly calibrated by the geometric constraints between the parameters and the historical modification data; The initial association mapping table is optimized according to the bidirectional association matrix to generate a bidirectional mapping.

[0028] The parameter adjustment unit includes: According to the parameter topology relationship tree, identifying directly associated parameters and indirectly associated parameters of the target parameter by a breadth-first traversal algorithm; Calling a preset parameter adjustment rule library, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules, and assembly constraint rules; Identifying the directly associated parameters and the indirectly associated parameters according to the rule engine of the parameter adjustment rule base, matching corresponding rule subsets and parsing the adjustment direction and magnitude; Adjust the target parameter according to the adjustment direction and amplitude to generate a parameter combination; The parameter combination is reversely deduced to the inferred parameter, the response coefficient of the inferred parameter is corrected, and the bidirectional correlation matrix is ​​modified according to the response coefficient.

[0029] Example 2 See also Figure 2 The present invention provides an embodiment: a product replication method for importing and converting a three-dimensional model, which is applied to product replication software. The specific steps of the method are as follows: S1: Obtain the main view image and the secondary view image of the product to be reproduced; This embodiment solves the problem of acquiring multi-angle product data during the 3D modeling process by acquiring both a primary and secondary view of the product to be replicated. The primary view provides a frontal view of the product, while the secondary view provides additional perspective information from the side, thus comprising multiple images. During the image acquisition process, multiple light sources are used to provide fill illumination to highlight shadowed areas. The combination of these two perspectives enables a more comprehensive capture of the product's shape and geometric features, particularly details difficult to capture directly from other angles.

[0030] S2: extract the first characteristic parameter; In this embodiment, image processing technology is used to perform edge detection, contour extraction, and key point recognition on the primary and secondary images to extract the first characteristic parameters of the product. These parameters include the object's geometric features (such as edges, angles, and contour shape) and size information.

[0031] S3: extract the second characteristic parameter; In this embodiment, extracting the second characteristic parameter includes extracting shadow edge information from the primary and secondary images. By analyzing changes in the shadow area, the geometric shape of the object's base is inferred. By analyzing the changes in the shadow's shape, the underlying geometric features of the base are obtained, solving the problem of missing base details due to the difficulty of bottom-level photography.

[0032] S4: Importing the first characteristic parameter and the second characteristic parameter into the product replica software to generate a first model; In this embodiment, the extracted first and second characteristic parameters are imported into the replica software to generate a preliminary 3D model. By combining the primary view image, the secondary view image, and the characteristic data extracted by shadow analysis, the replica software can create a preliminary 3D model that conforms to the actual physical form.

[0033] S5: Obtain input target parameters and construct a bidirectional mapping between the target parameters and the first model inference parameters; In this embodiment, the target parameters can be product design specifications or process requirements, while the inferred parameters are physical parameters derived through 3D modeling and analysis. By establishing a bidirectional mapping between the target parameters and the inferred parameters of the first model, the target parameters can be automatically adjusted based on the inferred parameters when the target parameters change, ensuring that the 3D model meets actual design requirements.

[0034] S6: According to the bidirectional mapping and the preset constraints, the target parameters are adjusted to generate a parameter combination; In this embodiment, the target parameters are automatically adjusted based on the bidirectional mapping relationship and pre-set constraints (such as geometry, process constraints, and physical properties). These constraints limit the range of variation of the target parameters, ensuring that the adjusted target parameters still meet design and process requirements. Using these constraints, parameter combinations that meet actual requirements are generated, thereby optimizing the 3D model.

[0035] S7: transforming the first model according to the parameter combination to generate a second model; The technical solution proposed in this application aims to solve the problem of missing data or errors caused by the difficulty in obtaining bottom images in the current three-dimensional model reconstruction process. Traditional three-dimensional reconstruction methods usually rely on multi-view shooting, but in some application scenarios, the cost of shooting the bottom is high, and in many cases it is difficult to obtain clear images, so it is impossible to obtain complete three-dimensional information, especially in the bottom area. The existing technology usually adopts the method of shooting multi-angle images or using depth sensors, but these methods still have error accumulation in practical applications, especially under conditions of occlusion, reflection or uneven light. Through the method of this application, combined with the shadow area of ​​the main view image and the auxiliary view image, the three-dimensional structure of the bottom is inferred by the relative position relationship between the multi-view information and the shadow, which solves the technical problem of missing bottom parameters or large errors in traditional technologies.

[0036] Specifically, the problem that shadow analysis usually faces is that the shadow only reflects the relative relationship between the object and the light source, but cannot directly deduce the absolute three-dimensional properties of the object, which easily leads to multiple solutions. Therefore, the traditional shadow analysis method cannot effectively reflect the true shape of the object, especially in the speculation of the bottom area, which is prone to errors. However, the present application reduces this uncertainty by combining the geometric constraints in the top view and side view images and utilizing the geometric relationship between the surfaces of objects, thereby gradually approaching the real three-dimensional structure. Through the precise speculation of shadows and combined with the constraints, this method can effectively deduce the accurate three-dimensional data of the bottom and other hidden parts, thereby improving the overall accuracy of three-dimensional reconstruction.

[0037] The specific steps of S2 are as follows: S2.1: performing image processing on the primary view image and the secondary view image to extract edge information, contour lines, and key points of the primary view image and the secondary view image; S2.2: Use the edge information, contour lines and key points as first feature parameters.

[0038] See also Figure 3, the second characteristic parameter extraction flow chart of an embodiment of the present invention, in this embodiment, the second characteristic parameter is extracted through shadow analysis. Shadow analysis is essentially an ill-posed inverse problem, which means that since the shadow only provides the relative position relationship between the object and the light source, it cannot directly express the absolute three-dimensional properties of the object, resulting in a high degree of uncertainty in shadow analysis. This problem is mainly manifested in that the shadow can only reflect the interaction between the object surface and the light source, but cannot clearly define the three-dimensional form of the object itself. Especially at the bottom of the object or in the part that is difficult to observe, the change of shadow may correspond to multiple different three-dimensional structures, so there are often multiple solutions or uncertain situations.

[0039] In such situations, relying on traditional shadow analysis methods often fails to accurately determine the object's three-dimensional structure. This is because traditional techniques cannot eliminate the uncertainties introduced by factors such as light source position, viewing angle, and object reflectivity, leading to frequent errors in the reconstruction of 3D models. This is especially true in complex environments, where the estimation of hidden areas such as the bottom is often inaccurate.

[0040] In this embodiment, by combining the primary and secondary images, each perspective provides distinct spatial information. Light source angles and perspective differences also play a key role in shadow variations. This multi-perspective shadow analysis not only relies on edge information from a single perspective, but instead infers the object's three-dimensional form by analyzing the relative positional relationships of shadows from different perspectives. This comprehensive consideration of light sources, shadows, and the object's geometry significantly reduces uncertainty in shadow analysis, enabling a more accurate inference of the object's true shape.

[0041] Furthermore, epipolar geometry is incorporated into shadow edge extraction, ensuring accurate matching of shadow points across different viewpoints. The intrinsic and extrinsic parameter matrices derived from camera calibration not only accurately match shadow points in the two views but also effectively eliminate errors caused by differing viewpoints. This eliminates the uncertainty introduced by traditional shadow analysis due to differences in light source or viewpoint, ensuring precise matching of shadow point pairs and improving the stability and accuracy of inferring the object's 3D structure.

[0042] Furthermore, through the generation and inference of the aforementioned shadow point pairs, surface fitting technology can convert these 2D shadow points into 3D geometric information. Because surface fitting considers the geometric relationship between the shadow point pairs in 3D space, by minimizing the error between the shadow point pairs and the fitted surface, a 3D surface that approximates the surface of the real object can be generated. This process ensures that the geometry of the object is gradually deduced from the local to the global structure based on multi-view shadow information, eliminating the problem in traditional methods where local information cannot accurately deduce the global structure.

[0043] The specific steps for S3 are as follows: S3.1: Extracting shadow edges of the primary view image and the secondary view image; Specifically, the purpose of shadow edge extraction is to extract the shadow portion that reflects the relative positional relationship between the object surface and the light source from the primary and secondary view images. This step is based on image processing technology and typically uses edge detection algorithms (such as Canny, Sobel, or Laplacian operators) to extract significant edges in the image. The primary and secondary view images provide different perspectives of the object, and the changes in the shadow edge between the two perspectives reflect the geometric relationship between the object and the light source. Therefore, by extracting shadow edge information from these two images, the necessary preliminary data can be provided for subsequent shadow correction, reasoning, and 3D reconstruction.

[0044] Furthermore, while edge detection algorithms can extract basic image edges by calculating the edges of grayscale changes in an image, these methods have limitations when dealing with shadows and geometric features on the surface of an object. Especially in complex shadow areas of an object, edge detection algorithms are often susceptible to interference from noise and lighting changes, resulting in reduced edge extraction accuracy. This is especially true when the object is unevenly illuminated or the shadow is partially obscured or reflected. Edge detection algorithms are prone to false detections or missed detections, resulting in inaccurate extraction of shadow edges. For example, in complex light source environments, the morphology of shadows varies greatly, making it difficult for traditional algorithms to handle these changes. Furthermore, they cannot consider the relative geometric relationships between objects and therefore cannot provide reliable three-dimensional geometric information.

[0045] In this embodiment, the primary and secondary view images are linked through epipolar geometry, enabling detection of shadow edges in the primary view and finding corresponding edge points within the epipolar range of the secondary view. This epipolar geometry, derived from camera calibration, more accurately reflects the spatial relationship of objects in the two views. This approach not only relies on image grayscale variations but also incorporates the geometric relationship between the object and the light source, making shadow edge extraction more precise and reliable, ensuring spatial consistency and accuracy of shadow edges in images captured from different perspectives.

[0046] S3.2: Correcting the shadow edge of the primary view image according to the shadow edge of the secondary view image to generate a bottom shadow area; Specifically, traditional shadow analysis methods are often affected by the angle of the light source, viewing angle, and occlusion. Shadow analysis is inherently an ill-posed inverse problem (no single solution exists). Shadows can only reflect the relative position of the object and the light source, but cannot directly express the object's absolute three-dimensional properties. Relying solely on shadow edges in the primary view image can lead to inaccuracies in the bottom shadow area. By incorporating shadow edge information from the secondary view image, the uncertainty in shadow analysis can be reduced, effectively minimizing this error and improving the accuracy of the bottom shadow area estimation.

[0047] In this embodiment, since it's difficult to directly image the object's bottom, the secondary image provides information from a different angle, helping to correct the edges of the shadows in the primary image. In this step, the shadow edges in the secondary image serve as a reference for correcting the shadow edges in the primary image. The secondary image's shadow edges provide longitudinal information about the object, while the primary image primarily reflects its lateral information. By combining these two shadow edges, the geometry of the object's bottom, particularly the outline of the bottom shadow area, can be more accurately inferred.

[0048] Furthermore, by comparing and correcting the shadow edge of the secondary view image with the shadow edge of the primary view image, the shape and position of the bottom of the object can be accurately inferred, especially when the bottom cannot be directly photographed, the bottom data can be supplemented and improved.

[0049] S3.3: Calculate a curvature tensor of the bottom shadow area, and obtain a transverse deformation parameter and a longitudinal deformation parameter based on the curvature tensor, wherein the curvature tensor is used to characterize curvature change characteristics of the shadow area; Specifically, during 3D reconstruction, the geometric information of the bottom shadow area is often more complex than that of other parts of the object, as it involves lighting, surface features, and possible occlusion. Therefore, it is unrealistic to describe these complex shadow areas using simple linear relationships. The curvature tensor, as a mathematical tool, can describe the curvature of an object's surface in different directions in detail, providing precise data support for further 3D morphological inference.

[0050] In this embodiment, the curvature tensor of the bottom shadow region is used to quantitatively describe the curvature variation of the object's surface, thereby reflecting the object's geometric characteristics. By calculating the curvature tensor, the degree of deformation of the shadow region in different directions can be analyzed, including lateral (horizontal) and longitudinal (vertical) deformation. The curvature tensor provides a precise mathematical description of the object's surface morphology and is particularly suitable for processing complex geometric features such as the bottom shadow region.

[0051] Furthermore, the curvature tensor is obtained by mathematically modeling the image data of the shadow area. First, the gradient information of the shadow area (i.e., the rate of change of the edge) is extracted from the shadow area. This can be achieved by calculating the change in the grayscale value of the shadow area, usually using the Laplacian operator for gradient calculation. Next, the curvature tensor is obtained by calculating the second-order derivative of the gradient field. Specifically, the curvature tensor is obtained by calculating the Hessian matrix of the shadow area image (i.e., the second-order partial derivative matrix of the image). The Hessian matrix contains the curvature information of each point in the image, reflecting the curvature of the surface around the point. By solving this matrix, the principal curvature value of each point can be obtained, thereby obtaining the curvature change of the point in different directions.

[0052] Furthermore, the transverse deformation parameter and the longitudinal deformation parameter correspond to the principal curvature values ​​in different directions of the curvature tensor, respectively. The transverse deformation parameter is related to the principal curvature value in the horizontal direction, and the longitudinal deformation parameter is related to the principal curvature value in the vertical direction.

[0053] S3.4: performing joint reasoning on the lateral deformation parameters and the longitudinal deformation parameters to generate shadow point pairs that satisfy the differential homeomorphism constraint and the viewing angle energy function; Specifically, shadow analysis is inherently an ill-posed inverse problem, where a single solution often fails to satisfy geometric consistency under multi-view conditions. By introducing diffeomorphism constraints and a viewpoint energy function, this uncertainty can be effectively reduced, resulting in more accurate generation of shadow point pairs and avoiding the multiple solution problem found in traditional methods.

[0054] In this embodiment, the horizontal and vertical deformation parameters describe the horizontal and vertical variations of shadows, respectively. At this stage, a joint inference approach is used, combining diffeomorphism constraints and a viewpoint energy function to generate shadow point pairs. The diffeomorphism constraint ensures smooth and continuous deformation of the same 3D surface point under different viewpoints, while the viewpoint energy function helps optimize the matching of shadow point pairs, ensuring geometric consistency of the object surface under different viewpoints.

[0055] S3.5: Based on the shadow point pairs, perform inverse inversion by surface fitting to extract the second characteristic parameter; Specifically, during the 3D reconstruction of an object, the information provided by shadow regions is often local, reflecting only a portion of the object's surface morphology and failing to directly reveal the object's complete 3D geometry. To infer the object's complete shape from this localized shadow information, surface fitting and inverse inversion techniques are employed. Surface fitting converts shadow point pairs into 3D surface data, enabling accurate geometry to be obtained, particularly for bases and other hidden areas that are difficult to directly capture.

[0056] In this embodiment, the shadow point pairs generated in the previous steps are used as the basis for surface fitting. Each shadow point represents the deformation and geometric features of the object's surface at a specific viewing angle. By fitting these shadow points, the true shape of the object's surface can be inferred. The core of this process is to infer the object's three-dimensional surface by fitting the surface to which the shadow point pairs are attached. Surface fitting technology, particularly least squares-based fitting algorithms, ensures accuracy during the fitting process and preserves as much surface detail as possible, thereby generating an accurate three-dimensional geometric model.

[0057] Furthermore, through inverse inversion technology, the corresponding geometric features on the object's surface are inferred by the position and shape of the fitted surface in space. This process converts the two-dimensional shadow information into three-dimensional data, and by inferring the specific location of the shadow point, the secondary characteristic parameters, curvature and inclination, are obtained.

[0058] The specific steps of S3.1 are as follows: S3.1.1: Perform camera calibration on the primary view image and the secondary view image to obtain an intrinsic parameter matrix and an extrinsic parameter matrix, and establish an epipolar geometric relationship between the two view angles based on the intrinsic parameter matrix and the extrinsic parameter matrix; Specifically, camera calibration and epipolar geometry are key to ensuring spatial consistency of shadow edges extracted from different viewpoints. Because shadow edges are based on the relative position of the object and the light source, the morphological changes of shadows in an image depend on the viewpoint. Accurate camera calibration and epipolar geometry enable precise alignment of shadow edges across multiple viewpoints, ensuring that shadow edges extracted in the secondary view match those in the primary view.

[0059] In this embodiment, camera calibration first obtains the intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix describes the camera's imaging characteristics, such as focal length and principal point position, while the extrinsic matrix describes the camera's position and orientation in three-dimensional space. In stereoscopic vision, epipolar geometry refers to the geometric constraints between the image points of an object from one perspective and the image points of the object from another perspective when two cameras capture the same object from different perspectives. In other words, epipolar geometry describes how corresponding points of an object are connected in the two images.

[0060] Furthermore, this geometric relationship can be constructed using intrinsic and extrinsic matrices. Specifically, image points captured from one perspective can be mapped to an image from a second perspective after transformation using the intrinsic and extrinsic matrices. Epipolar geometry, through the definition of poles and epipolar lines, ensures that image points from one perspective and their corresponding points from another perspective must lie on a specific straight line, called an epipolar line. Points in the secondary image can be identified that correspond to shadow edges in the primary image.

[0061] S3.1.2: Detect shadow edges in the primary view image using an epipolar equation; Specifically, the epipolar equation is one of the core geometric relationships in stereo vision. It describes the trajectory of points in an image seen from one perspective that should fall on the image plane from another perspective. To extract shadow edges in the primary view, camera calibration is required to obtain the camera's intrinsic and extrinsic parameter matrices. These matrices describe the positional relationships of objects in three-dimensional space. Combined with the camera's parameter information, epipolar geometric relationships can be constructed, leading to the epipolar equation. In practice, the intrinsic and extrinsic matrixes obtained through camera calibration establish the geometric relationship between the two perspectives. The intrinsic matrix provides the camera's imaging parameters (such as focal length and principal point coordinates), while the extrinsic matrix describes the camera's position and orientation in space. Based on this information, the epipolar equation can be calculated using standard epipolar geometry principles. This equation describes the position of a point on an object when projected onto the image plane from one perspective and the line (i.e., the epipolar line) on which the corresponding point in the image should lie when projected from another perspective.

[0062] In this embodiment, by calculating the epipolar equation and determining the epipolar range in the primary view image, the shadow area in the image can be accurately located. Then, within these ranges, a traditional edge detection algorithm (such as the Sobel or Canny algorithm) is applied to further extract the shadow edge. The key to this step is that the epipolar equation provides a geometric constraint for the traditional edge detection algorithm, making the extraction of the shadow edge more accurate and avoiding errors caused by factors such as lighting changes and viewing angle differences. The shadow edge extraction process first limits the possible position range of the edge through the epipolar equation. In the primary view image, the corresponding shadow edge should be within the epipolar range. This means that through the epipolar equation, not only can the position of the shadow edge in the primary view image be limited, but it can also ensure that the shadow edge meets the spatial geometric constraints, thereby improving the accuracy of shadow edge extraction.

[0063] S3.1.3: Based on the epipolar geometric relationship, extract edge points with the same gradient direction as the shadow edge of the primary view image within the epipolar range corresponding to the secondary view image to generate the shadow edge of the secondary view image; Specifically, precise matching of shadow edges is crucial in 3D reconstruction. Traditional edge detection methods often fail to account for geometric relationships between different viewpoints. However, this approach, by introducing epipolar geometry, allows precise alignment of shadow edges extracted in the secondary view image with those in the primary view image. Consistent matching of gradient directions further ensures accurate edge extraction, avoiding edge extraction errors caused by illumination variations or object occlusions in conventional methods.

[0064] In this embodiment, after extracting the shadow edge in the primary view image, the epipolar geometric relationship obtained in the previous step can be used to determine the possible location of the shadow edge in the secondary view image using the epipolar equation. Shadow edge extraction in the secondary view image does not directly rely on traditional edge detection. Instead, it uses spatial information provided by camera calibration to extract edge points within the epipolar range of the secondary view image. Specifically, the epipolar geometric relationship defines the matching method for shadow points between the primary and secondary views, allowing accurate extraction of edge points in the secondary view image that share the same gradient direction as the shadow edge in the primary view image.

[0065] Furthermore, edge point extraction not only considers grayscale variations within the image but also incorporates viewing angle constraints, resulting in more accurate shadow edge extraction. In particular, gradient direction matching ensures geometric consistency between shadow edges extracted in the secondary view and those in the primary view. The ability to constrain the range of edge points in the secondary view based on epipolar geometry reduces unnecessary mismatches and incorrect extractions.

[0066] See also Figure 4 , schematic diagram of the first shadow extraction according to an embodiment of the present invention, Figure 4 This diagram shows how the primary view image, after processing the edge saliency map, is used to generate the first shadow using a region growing algorithm. Image processing techniques generate an edge saliency map for the primary view image, identifying potential shadow edge regions. Based on this saliency map, a region growing algorithm is used to expand outward from the edge centroid to generate a preliminary first shadow region.

[0067] It is understandable that in Figure 4 The three-dimensional part of the main image is the product to be reproduced, and the gray part is the shadow area.

[0068] The specific steps of S3.2 are as follows: S3.2.1: Calculate the gradient structure tensor of the shadow edge of the main view image and generate an edge saliency map; Specifically, gradient is an important feature that measures grayscale changes in an image and reflects the edge information between different image regions. To improve the accuracy of shadow edge detection, a gradient structure tensor is used, which can describe the change trend of local image regions. By calculating the gradient information of each pixel in the image, a gradient structure tensor can be obtained, which can represent the change in gradient direction in the image and the texture information of the local region. By measuring the directionality of grayscale changes in the image, the gradient structure tensor can enhance the response of edge and texture regions, especially in shadow regions, which is very effective for detecting object edges. Based on the calculated gradient structure tensor, an edge saliency map is generated. The edge saliency map highlights the areas in the image with significant gradient changes, that is, the areas where shadow edges are located, by calculating the response across the entire image. In this way, the generated saliency map can focus on the most important shadow information in the image.

[0069] In this embodiment, by applying the gradient structure tensor calculation to the shadow area, an edge saliency map is generated, which represents the saliency and directionality of the shadow edge. This process makes the shadow edge not limited to simple grayscale changes, but also takes into account the spatial geometric features in the image.

[0070] S3.2.2: Processing the edge saliency map using a region growing algorithm to obtain a first shadow, wherein the region growing algorithm includes starting from the centroid of the edge saliency map and gradually merging adjacent pixels that have a consistent direction of the centroid structure tensor; Specifically, shadow edges are typically continuous, but in images, especially in complex scenes, edge extraction may be discontinuous or noisy. Region growing algorithms effectively connect discrete edge information into a coherent whole, resulting in a more coherent and complete shadow region. This is particularly true for areas such as the bottom of an object that are difficult to photograph or directly capture. Region growing algorithms can infer the boundaries of these areas based on the geometric information in the image, generating a complete shadow region.

[0071] In this embodiment, after generating an edge saliency map, a region growing algorithm is used to further process the image. Region growing is a seed-based image segmentation algorithm. Its basic concept is to start from a starting point (seed point) and gradually expand to its neighboring pixels until a certain stopping criterion is met. The seed point is selected as the centroid region in the edge saliency map, i.e., the region in the image with the most significant grayscale changes and a high correlation with shadow edges. This selection method is based on gradient information and starts from the region with the highest saliency, ensuring that the growth starts from the shadow edge portion of the object surface without interference from noise or errors.

[0072] Furthermore, shadow edges have a certain directionality and are typically continuous. To effectively merge neighboring pixels, the region growing algorithm must ensure that the merged pixels adhere to certain directional constraints during the expansion process. Especially when processing shadow regions, the consistency of the gradient direction between pixels is an important criterion for determining whether they belong to the same region. This constraint ensures the continuity and stability of the shadow region, preventing mis-merging caused by viewing angle differences or changes in illumination. Starting from a seed point, the region growing algorithm gradually checks whether the gradient directions of the surrounding pixels are consistent with the gradient direction of the seed point. The algorithm calculates the gradient information of the neighboring pixels and compares it with the gradient direction of the seed point. If the gradient direction of a neighboring pixel is consistent with the direction of the seed point, the pixel is considered to be part of the shadow edge and can be merged with the seed point, thus expanding the shadow region. If the gradient direction of a neighboring pixel is inconsistent with the direction of the seed point, the pixel does not meet the merging criteria, and the algorithm stops expanding in that direction.

[0073] Furthermore, the core of the region growing algorithm is to gradually expand the region until the entire shadow area is completely extracted. The expansion process ensures the coherence and accuracy of the shadow area by merging neighboring pixels. To avoid excessive growth or unnecessary mismerging, the algorithm needs to set a stopping criterion. Generally, the stopping criterion can be based on the following aspects: Maximum area size. When the size of the shadow area reaches a certain threshold, the expansion stops.

[0074] Gradient direction consistency: when the gradient direction of the neighboring pixel differs from that of the current area by more than a certain threshold, the direction is stopped from being expanded.

[0075] Edge strength: When the edge strength of the merged neighborhood pixels is lower than a certain value, the growth stops.

[0076] In this embodiment, region growing continues until the entire shadow region is expanded and the gradient direction consistency of the neighboring pixels does not exceed the set threshold. This effectively controls the region growing process, avoids the accidental merging of unrelated pixels, and ensures the integrity of the shadow region.

[0077] S3.2.3: Calculate the epipolar projection matching rate of the first shadow in the shadow edge of the secondary view image; Specifically, epipolar geometry can be used to reduce errors in shadow edge matching between different viewpoints, ensuring accurate shadow edge matching across both viewpoints. Accurately matching shadow edges in the primary and secondary images indicates that the object's surface morphology is consistent in both images, and the inferred shadow region is reliable. Therefore, calculating the epipolar projection matching rate helps verify the accuracy of shadow edge extraction results, ensuring that the extracted shadow region matches the actual object's geometry.

[0078] In this embodiment, after generating the first shadow, epipolar geometry is used to further verify the accuracy of the shadow in the secondary view image. By calculating the epipolar projection matching rate of the first shadow in the secondary view image, the consistency of the shadow extracted from the primary view image in the secondary view image can be evaluated. Epipolar projection is a geometric constraint based on perspective transformation. According to the previous steps, theoretically, every point in the secondary view image should be within the corresponding epipolar range in the primary view image, reflecting the spatial consistency of the object's shadow. However, in practical applications, relying on simple geometric derivation often results in errors. These errors may come from the following aspects: 1. Although epipolar geometry theoretically provides a correspondence between points in the two viewpoints, in practice, slight deviations may occur in the actual image points due to factors such as camera calibration accuracy, image noise, and non-ideal object geometry (such as surface roughness or reflections). These deviations can cause the shadow points extracted from the primary view to project into the secondary view image in positions that do not completely match the theoretical positions, resulting in slight misalignment or offset.

[0079] 2. Shadow edges are often not absolutely clear, especially in complex lighting environments. Shadows can be affected by reflections, occlusions, or uneven light sources, resulting in blurred or inaccurate shadow edges. Even after edge saliency map extraction, shadow areas may still contain slight errors, making it impossible to accurately match the shadow edges in the secondary view image.

[0080] 3. In actual image processing, resolution and sampling accuracy are often limited. This is especially true due to factors such as camera or imaging device accuracy and image compression. The extraction and location of shadow edges can be subject to errors. These errors can make it impossible to directly find perfectly matching shadow points in the secondary image. Therefore, the epipolar projection matching rate is used to evaluate matching accuracy.

[0081] Therefore, although the theoretical shadow point positions are derived from the epipolar geometry, to ensure the accuracy of the shadow area, it is still necessary to evaluate the consistency of the shadow edges extracted from the primary view image in the secondary view image by calculating the epipolar projection matching rate. This calculation verifies that the actual projected shadow edge is accurately within the epipolar range of the secondary view image, further reducing potential errors and ensuring that the extracted shadow area is consistent with the actual object geometry.

[0082] Furthermore, the epipolar projection matching rate is calculated by projecting the first shadow onto the secondary image and calculating the degree of overlap between the projected position and the actual shadow edge in the secondary image. Specifically, the epipolar position of each shadow point in the secondary image is calculated based on the epipolar geometry. Then, a determination is made as to whether the actual shadow points within the epipolar range are consistent with the theoretical positions. By calculating the projection matching degree of these shadow points (e.g., by calculating the projection error or overlap), the positional accuracy of the first shadow in the secondary image can be assessed.

[0083] S3.2.4: If the epipolar projection matching rate is less than a preset matching threshold, Gaussian blur the edge saliency map and re-perform region growing until the epipolar projection matching rate is greater than or equal to the preset matching threshold; See also Figure 5 , a schematic diagram of the first shadow completion according to an embodiment of the present invention, Figure 5 The figure shows the process of completing the first shadow region after projective matching of the secondary view image. As can be seen, the initial first shadow region is projectively matched with the shadow edge of the secondary view image. If the matching rate does not meet the requirements, the shadow region is regenerated through Gaussian blurring and region growing. Finally, the corrected and completed shadow region is output as the bottom shadow region for further 3D modeling and analysis.

[0084] S3.2.5: Complete the first shadow according to the matching result, and output the completed first shadow as the bottom shadow area.

[0085] The specific steps of S3.4 are as follows: S3.4.1: Establish a diffeomorphic mapping relationship between the lateral deformation parameters and the longitudinal deformation parameters through a diffeomorphic constraint, wherein the diffeomorphic constraint indicates that the shadow edge deformation of the same three-dimensional surface point in the lateral and longitudinal directions satisfies the diffeomorphic transformation; Specifically, by establishing a differential homeomorphism mapping relationship between the lateral and longitudinal deformation parameters, we ensure that the geometric deformation of the shadow region follows the same rules under different viewpoints. The deformation of shadow points is not only due to differences in viewpoint, but also affected by the geometric characteristics of the object surface. Through differential homeomorphism mapping, inconsistencies in the shadow region can be effectively eliminated, ensuring that shadow points extracted from different viewpoints are consistent.

[0086] In this embodiment, during the three-dimensional modeling process, the deformation of the shadow can be regarded as a reflection of the morphological changes between different areas on the surface. In order to accurately describe the edge deformation of the shadow of the object surface at different viewing angles, it is first necessary to establish a differential homeomorphism mapping relationship between the lateral deformation parameters and the longitudinal deformation parameters. Differential homeomorphism mapping means that the shadow deformation of the same three-dimensional surface point at different viewing angles maintains continuity and smoothness, that is, there is no breakage or irregular deformation. This means that through differential homeomorphism constraints, the changes in the shadow deformation in the horizontal direction (lateral direction) and the vertical direction (longitudinal direction) should be able to be mapped to each other, maintaining the consistency of the geometric morphology of the object surface.

[0087] S3.4.2: Constructing a cross-view deformation parameter correlation matrix based on the transverse deformation parameters and the longitudinal deformation parameters, and calculating a view energy function based on matrix properties of the cross-view deformation parameter correlation matrix; Specifically, the cross-view deformation parameter correlation matrix is ​​constructed to account for the mutual influence of shadow deformation parameters at different viewpoints. Shadow deformation within each viewpoint is not limited to a single direction, but involves both horizontal and vertical deformations. Therefore, the relationship between these two types of deformations must be established. This matrix allows for a systematic representation of these relationships, providing the necessary data foundation for the subsequent calculation of the viewpoint energy function.

[0088] In this embodiment, after establishing the differential homeomorphism mapping relationship between the horizontal and vertical deformation parameters, the next step is to construct a cross-view deformation parameter correlation matrix. This matrix describes how the horizontal and vertical deformation parameters relate to each other at different viewpoints and how they affect the deformation of the shadow area on the object's surface. By analyzing the deformation of the shadow area at two viewpoints, a matrix can be constructed to record the relationship between the horizontal and vertical deformations at different viewpoints. Each element in the matrix represents the degree of change in the shadow deformation between viewpoints. Using the cross-view deformation parameter correlation matrix, the relationship between the shadow deformations at different viewpoints can be accurately calculated, thereby inferring the actual shape of the object at different viewpoints.

[0089] Furthermore, the matrix's properties are used to calculate the view energy function. The goal of this function is to optimize the matching accuracy of shadow points by quantifying the inconsistencies in shadow deformation between different viewpoints. Since shadows may deform under different viewpoints, the view energy function aims to measure and minimize these differences, thereby ensuring consistency in the spatial position and geometric form of shadow points extracted from different viewpoints.

[0090] Furthermore, the calculation of the view energy function is based on the matrix properties of the cross-view deformation parameter correlation matrix, combined with the differences in shadow deformation under different viewpoints to quantify the energy. The essence of the view energy function is to measure the position error of shadow points between different viewpoints and minimize this error through an optimization algorithm. Generally, the view energy function can be expressed as deformation difference and geometric consistency, where the deformation difference calculates the difference in horizontal and vertical deformation parameters under different viewpoints to measure the consistency of the deformation between the two under different viewpoints. Geometric consistency measures the geometric consistency of the object surface under different viewpoints by comparing the positions of shadow points under different viewpoints, ensuring that the shadow information extracted from one viewpoint can be accurately matched in another viewpoint.

[0091] Specifically, based on the cross-viewpoint correlation matrix of the horizontal and vertical deformation parameters, the difference in each deformation parameter between different viewpoints is calculated. The error of each shadow point is calculated based on its projected position in the primary and secondary viewpoints. The weighted sum of the projection errors of all shadow points is then used to generate the overall viewpoint energy function. This function represents the consistency of the shadow deformation between different viewpoints, and the goal is to minimize this value, thereby optimizing the matching accuracy of the shadow points.

[0092] S3.4.3: Based on the diffeomorphic mapping relationship and the view energy function, solve the shadow point pairs that satisfy the two-view deformation consistency by the least squares method; Specifically, the least squares method is used to optimize the positions of shadow points from different viewpoints. In real-world scenarios, due to factors such as perspective differences and surface complexity, shadow point deformations can exhibit errors. The least squares method effectively minimizes these errors, ensuring geometric consistency of shadow point pairs from both viewpoints. By optimizing the positions and deformation parameters of these shadow point pairs, consistent 3D shapes can be achieved from different viewpoints, thereby improving the accuracy and stability of 3D reconstruction.

[0093] It should be noted that although the deduction here is based on the bottom shadow area, in practice, a preliminary shadow area at the bottom of the object is obtained through shadow edge extraction and correction of multi-view images. However, this preliminary shadow area may not completely conform to the object's true three-dimensional shape, especially the bottom area. Due to the object's projective deformation, the shadow edge may appear misaligned or asymmetric at different viewpoints. Even if correction is performed using epipolar geometric relationships and edge saliency maps, there will still be slight geometric deviations due to differences in viewpoint. At the same time, the shadow edge is affected by factors such as the object's surface texture and light source distribution. The shadow boundary may not be an ideal smooth curve, but may have an irregular or segmented shape. In this case, although the shadow area has been preliminarily extracted, further optimization is still needed to correct these irregularities to ensure that the obtained shadow area accurately reflects the actual shape of the object.

[0094] In this embodiment, shadow point pairs are matching points on the shadow edge extracted from two different viewpoints. Ideally, these points should correspond to the same position on the surface of the same object. However, in actual shooting, due to factors such as perspective differences and the complexity of the object surface, the deformation of the shadow points may contain errors. The goal of the least squares method is to optimize the spatial position of these points by minimizing the error between the shadow point pairs in the two viewpoints, so that the shadow points are as consistent as possible in both viewpoints.

[0095] Furthermore, in the least squares method, the error model is typically constructed based on the geometric differences between shadow point pairs. For example, the projection error of a shadow point in two viewpoints can be calculated, and the sum of the squared errors of all shadow points can be used as the optimization objective function. By minimizing this objective function, the least squares method adjusts the positions and deformation parameters of the shadow points to reduce the projection error and achieve more accurate shadow point matching.

[0096] Furthermore, the deformation of the shadow region may occur simultaneously in the horizontal and vertical directions under both viewing angles. Therefore, the horizontal and vertical deformation parameters need to be jointly optimized. Specifically, during the optimization of shadow point pairs, the least squares method not only adjusts the positions of the shadow points but also optimizes the horizontal and vertical deformation parameters. By adjusting these deformation parameters, the least squares method ensures the consistency of the shadow point pairs under both viewing angles, thereby improving reconstruction accuracy.

[0097] Furthermore, the least squares method uses iterative solutions to gradually adjust the positions and deformation parameters of shadow point pairs until the error converges to a minimum. Each iteration reduces the error between shadow point pairs, and the resulting optimized shadow point positions and deformation parameters accurately reflect the three-dimensional shape of the object's surface.

[0098] The specific steps of S5 are as follows: S5.1: Analyze the parameter topology relationship tree of the first model to obtain the process constraint chain of the inferred parameters; Specifically, by analyzing the parameter topology tree of the first model and the process constraint chain of the inferred parameters, we can systematically understand the relationship between the target and inferred parameters and determine the role of each parameter in the model. The purpose of this process is to provide the data foundation and constraints for subsequent association mapping, ensuring that the mapping relationship meets design and process requirements.

[0099] In this embodiment, during the 3D modeling process, a parameter topology tree is used to describe the hierarchical structure and interdependencies between the various parameters in the first model. Each parameter plays a specific role in the model, and these parameters are interconnected through specific constraints, forming a tree structure. By analyzing the parameter topology tree of the first model, the interactions and dependencies between the various model parameters can be clearly understood.

[0100] Furthermore, to further understand the relationship between target parameters and inferred parameters, it is necessary to analyze the relationship between each inferred parameter and the process constraint chain. The process constraint chain is a set of constraints related to the manufacturing process, physical limitations, and design requirements. These constraints limit the range of variation of the inferred parameters and the mutual influence between the inferred parameters. The process constraint chain ensures that the target parameters do not exceed the reasonable physical or design range during the mapping process.

[0101] S5.2: Establishing an initial association mapping table between the target parameters and the inferred parameters based on the process constraint chain; Specifically, the constraints of the process constraint chain ensure that the mapping between target and inferred parameters meets actual manufacturing and design requirements, avoiding unreasonable values ​​that may appear during the mapping process. By constructing an initial mapping table, the foundation is laid for the subsequent optimization process, ensuring the rationality and feasibility of the association relationship.

[0102] In this embodiment, the association mapping table between target parameters and inferred parameters serves as the fundamental data structure for establishing the relationship between them. When constructing this association mapping table, the constraints within the process constraint chain must first be considered. These constraints may involve physical properties (e.g., material strength, dimensional limitations), manufacturing processes (e.g., assembly sequence, machining techniques), and design specifications. These process constraints allow for a clear and rational mapping relationship between target parameters and inferred parameters, providing preliminary reference data for subsequent optimization.

[0103] To generate an initial correlation mapping table, we first analyze the constraints in the process constraint chain to determine the degree of mutual influence between each target parameter and the inferred parameter. Then, based on these relationships, we construct an initial mapping table, where each row represents the modification direction of a target parameter and each column represents the response coefficient of the inferred parameter. Through these mapping relationships, we preliminarily determine the interdependence between the target and inferred parameters.

[0104] S5.3: Generate a bidirectional correlation matrix, where the row vectors of the bidirectional correlation matrix represent the modification direction of the target parameter, the column vectors of the bidirectional correlation matrix represent the response coefficients of the inferred parameters, and the matrix element values ​​are jointly calibrated by the geometric constraints between the parameters and the historical modification data; Specifically, the bidirectional correlation matrix is ​​constructed to systematically describe the mutual influence between target and inferred parameters. By matrixing the modification directions and response coefficients, the relationship between parameters can be clearly and intuitively represented, providing precise mathematical support for the subsequent optimization process. Furthermore, by combining calibrated geometric constraints with historical data, the specific impact of target parameter changes on inferred parameters can be more accurately determined, avoiding error accumulation.

[0105] In this embodiment, a bidirectional correlation matrix is ​​used to describe the mutual influence between target parameters and inferred parameters. In this matrix, the row vectors represent the modification direction of the target parameter, that is, the direction in which the target parameter is adjusted in the 3D model; the column vectors represent the response coefficients of the inferred parameters, that is, the impact of changes in the target parameter on the inferred parameter.

[0106] Furthermore, each element of the matrix is ​​calibrated using a combination of geometric constraints between parameters and historical modification data. Geometric constraints primarily consider factors such as the object's surface, geometry, and spatial position. These constraints ensure that changes between the target and inferred parameters conform to actual physical laws. Historical modification data records the relationship between the target and inferred parameters during previous modification processes. Analysis of this data allows for more accurate predictions of the inferred parameter's response to target parameter adjustments.

[0107] S5.4: Optimizing the initial association mapping table according to the bidirectional association matrix to generate a bidirectional mapping; Specifically, using the constructed bidirectional correlation matrix, the initial correlation mapping table needs to be optimized. The goal of optimization is to minimize the errors that may occur during the mapping process and ensure a more accurate relationship between the target parameters and the inferred parameters. In this process, the bidirectional correlation matrix provides the geometric and physical relationship between the parameters. The optimization process adjusts the elements in the matrix to make the mapping relationship between the target parameters and the inferred parameters more precise. The optimization process is mainly based on the characteristics of the matrix, and by adjusting the weights of the row vectors and column vectors, the matching degree between the target parameters and the inferred parameters is optimized. In this way, the deviation caused by historical data errors or incomplete geometric constraints can be reduced, further improving the accuracy of the mapping relationship.

[0108] The specific steps of S6 are as follows: S6.1: Identify directly associated parameters and indirectly associated parameters of the target parameter using a breadth-first traversal algorithm based on the parameter topology relationship tree; Specifically, the relationship between target parameters and inferred parameters is complex and diverse, often not a single linear relationship. Using a breadth-first traversal algorithm, we can systematically analyze the hierarchical structure of parameters, identifying directly and indirectly related parameters of the target parameter. Directly related parameters are those directly affected by the target parameter, while indirectly related parameters are those that indirectly affect the target parameter through other parameters.

[0109] In this embodiment, a breadth-first traversal algorithm is used to first identify the directly associated parameters of the target parameter, and then further identify the indirectly associated parameters. This process ensures a comprehensive understanding of all relationships between the target parameter and other parameters, providing a foundation for subsequent rule matching and parameter adjustment. Starting from the target parameter, all other directly and indirectly associated parameters are sequentially identified. The breadth-first traversal algorithm is a graph traversal algorithm that typically begins at the root node (the target parameter) and first visits the parameters directly associated with the target parameter, then visits the other parameters connected to these parameters layer by layer until all relevant parameters are identified.

[0110] S6.2: Calling a preset parameter adjustment rule library, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules, and assembly constraint rules; Specifically, due to the complex and mutually constrained relationship between target parameters and inferred parameters, certain rules and constraints must be followed when adjusting target parameters to ensure that the final result not only meets design requirements but also meets the feasibility of actual manufacturing. Using a pre-set rule library effectively ensures the rationality of target parameter adjustments, avoiding excessive or non-compliant adjustments that render the model unmanufacturable or unfit for assembly. The rule library provides a systematic framework that makes the parameter adjustment process more standardized and automated.

[0111] In this embodiment, the rule base includes three main types of rules: geometric constraint rules, engineering constraint rules, and assembly constraint rules. These rules constrain the adjustable range and adjustment method of parameters, as well as their interrelationships, based on factors such as the product's physical properties, design requirements, and manufacturing process. For example, geometric constraint rules may involve geometric restrictions such as surface smoothness and angles; engineering constraint rules cover material strength and load limits; and assembly constraint rules cover the assembly sequence and compatibility between components.

[0112] S6.3: Identify the directly associated parameters and the indirectly associated parameters according to the rule engine of the parameter adjustment rule base, match the corresponding rule subsets, and analyze the adjustment direction and magnitude; Specifically, since adjusting target parameters isn't simply a linear process, often involving multiple factors and constraints, a rules engine can efficiently and automatically match parameters to relevant rules, ensuring that each parameter adjustment meets design and process requirements. The rules engine can select the most appropriate subset of rules based on the characteristics of each parameter, allowing adjustments to be made while still meeting design requirements. In this way, the rules engine significantly simplifies the adjustment process, improving automation and precision.

[0113] In this embodiment, the core function of the rules engine is to select a subset of rules that match the parameter type, constraints, and actual requirements, and then calculate the direction and magnitude of target parameter adjustment based on these rules. The rules engine uses logical reasoning, condition matching, and other methods to associate parameters with applicable rules and determine the adjustment method.

[0114] S6.4: Adjust the target parameter according to the adjustment direction and magnitude to generate a parameter combination; S6.5: Reversely derive the parameter combination to the inferred parameter, correct the response coefficient of the inferred parameter, and modify the bidirectional correlation matrix according to the response coefficient.

[0115] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A product replica software system for importing and converting 3D models, applied to product replica software, characterized by: The product replication software system includes an image acquisition module, a feature parameter extraction module, a model building module and a model conversion module, wherein: The image acquisition module is used to obtain a main view image and multiple auxiliary view images of the product to be reproduced; The feature parameter extraction module is used to extract the first feature parameter, and perform shadow area segmentation on the primary view image and the secondary view image to extract the second feature parameter; The model building module is configured to import the first characteristic parameter and the second characteristic parameter into product replica software to generate a first model of the product to be replicated, wherein the first model generates inferred parameters based on the first characteristic parameter and the second characteristic parameter; The model conversion module is used to convert the first model according to the input target parameters to generate a second model.

2. A product replica software system for importing and converting three-dimensional models according to claim 1, characterized in that: The feature parameter extraction module includes a first feature extraction unit and a second feature extraction unit, wherein: The first feature extraction unit is configured to perform image processing on the primary view image and the secondary view image, extract edge information, contour lines, and key points of the primary view image and the secondary view image, and use the edge information, contour lines, and key points as first feature parameters; The second feature extraction unit is used to infer feature parameters of the remaining area based on the shadow parts of the primary view image and the secondary view image.

3. A product replica software system for importing and converting three-dimensional models according to claim 2, characterized in that: The second feature extraction unit includes a shadow extraction subunit, a shadow correction subunit, a shadow parameter calculation subunit, a shadow reasoning subunit and a shadow inversion subunit, wherein: The shadow extraction subunit is used to extract the shadow edges of the primary view image and the secondary view image; The shadow correction subunit is used to correct the shadow edge of the main view image according to the shadow edge of the auxiliary view image to generate a bottom shadow area; The shadow parameter calculation subunit is used to calculate the curvature tensor of the bottom shadow area, and obtain the transverse deformation parameter and the longitudinal deformation parameter according to the curvature tensor, wherein the curvature tensor is used to characterize the curvature change characteristics of the shadow area; The shadow reasoning subunit is used to jointly reason the lateral deformation parameters and the longitudinal deformation parameters to generate shadow point pairs that satisfy the differential homeomorphism constraint and the viewing angle energy function; The shadow inversion subunit is used to perform reverse inversion through surface fitting based on the shadow point pairs to extract the second characteristic parameters.

4. A product replica software system for importing and converting three-dimensional models according to claim 3, characterized in that: The shadow extraction subunit includes: Performing camera calibration on the primary view image and the secondary view image to obtain an intrinsic parameter matrix and an extrinsic parameter matrix, and establishing an epipolar geometric relationship between the two perspectives based on the intrinsic parameter matrix and the extrinsic parameter matrix; detecting shadow edges in the primary view image using an epipolar equation; According to the epipolar geometric relationship, edge points with the same gradient direction as the shadow edge of the primary view image are extracted within the epipolar range corresponding to the secondary view image to generate the shadow edge of the secondary view image.

5. The product replica software system for importing and converting three-dimensional models according to claim 3, characterized in that: The shadow correction subunit includes: Calculate the gradient structure tensor of the shadow edge of the main view image and generate an edge saliency map; Processing the edge saliency map using a region growing algorithm to obtain a first shadow, wherein the region growing algorithm includes starting from the centroid of the edge saliency map and gradually merging adjacent pixels that meet the direction consistency of the centroid structure tensor; Calculating an epipolar projection matching rate of the first shadow in the shadow edge of the secondary view image; If the epipolar projection matching rate is less than a preset matching threshold, Gaussian blurring is performed on the edge saliency map and region growing is performed again until the epipolar projection matching rate is greater than or equal to the preset matching threshold; The first shadow is completed according to the matching result, and the completed first shadow is output as the bottom shadow area.

6. A product replica software system for importing and converting three-dimensional models according to claim 3, characterized in that: The shadow reasoning subunit includes: A differential homeomorphism mapping relationship between the transverse deformation parameter and the longitudinal deformation parameter is established through a differential homeomorphism constraint condition, wherein the differential homeomorphism constraint condition indicates that the shadow edge deformation of the same three-dimensional surface point in the transverse and longitudinal directions satisfies the differential homeomorphism transformation; constructing a cross-viewing angle deformation parameter correlation matrix according to the horizontal deformation parameter and the vertical deformation parameter, and calculating a viewing angle energy function according to matrix characteristics of the cross-viewing angle deformation parameter correlation matrix; According to the differential homeomorphism mapping relationship and the viewing angle energy function, the shadow point pairs that satisfy the dual-viewing deformation consistency are solved by the least squares method.

7. The product replica software system for importing and converting three-dimensional models according to claim 1, characterized in that: The model conversion module includes a parameter mapping unit, a parameter adjustment unit and a model conversion unit, wherein: The parameter mapping unit is used to obtain the input target parameters and construct a bidirectional mapping between the target parameters and the inferred parameters; The parameter adjustment unit is configured to adjust the target parameter according to the bidirectional mapping and preset constraints to generate a parameter combination; The model conversion unit is used to convert the first model according to the parameter combination to generate a second model.

8. A product replica software system for importing and converting three-dimensional models according to claim 7, characterized in that: The parameter mapping unit includes: parsing the parameter topological relationship tree of the first model to obtain the process constraint chain of the inferred parameters; Establishing an initial association mapping table between the target parameters and the inferred parameters according to the process constraint chain; Generate a bidirectional correlation matrix, where the row vectors of the bidirectional correlation matrix represent the modification direction of the target parameter, the column vectors of the bidirectional correlation matrix represent the response coefficients of the inferred parameters, and the matrix element values ​​are jointly calibrated by the geometric constraints between the parameters and the historical modification data; The initial association mapping table is optimized according to the bidirectional association matrix to generate a bidirectional mapping.

9. A product replica software system for importing and converting three-dimensional models according to claim 8, characterized in that: The parameter adjustment unit includes: According to the parameter topology relationship tree, identifying directly associated parameters and indirectly associated parameters of the target parameter by a breadth-first traversal algorithm; Calling a preset parameter adjustment rule library, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules, and assembly constraint rules; Identifying the directly associated parameters and the indirectly associated parameters according to the rule engine of the parameter adjustment rule base, matching corresponding rule subsets and parsing the adjustment direction and magnitude; Adjust the target parameter according to the adjustment direction and amplitude to generate a parameter combination; The parameter combination is reversely deduced to the inferred parameter, the response coefficient of the inferred parameter is corrected, and the bidirectional correlation matrix is ​​modified according to the response coefficient.

10. A product replica method for importing and converting a three-dimensional model, implemented based on a product replica software system for importing and converting a three-dimensional model according to any one of claims 1 to 9, and applied to product replica software, characterized in that: The method comprises: Obtaining a primary view image and a secondary view image of the product to be reproduced, and extracting a first characteristic parameter; performing shadow area segmentation on the primary view image and the secondary view image to extract a second characteristic parameter; Importing the first characteristic parameter and the second characteristic parameter into product replica software to generate a first model of the product to be replicated, wherein the first model generates inferred parameters based on the first characteristic parameter and the second characteristic parameter; Obtaining input target parameters and constructing a bidirectional mapping between the target parameters and the inferred parameters; Adjusting the target parameters according to the bidirectional mapping and preset constraints to generate a parameter combination; The first model is transformed according to the parameter combination to generate a second model.

Images