A three-dimensional model import and conversion product copying software system

Abstract

The application relates to the technical field of product copying, in particular to a product copying software system for three-dimensional model import and conversion, and discloses the following scheme: a primary three-dimensional model is generated by acquiring a main view image and a sub-view image, extracting a first characteristic parameter and a second characteristic parameter; target parameter adjustment is carried out based on the bidirectional mapping relationship between the target parameter and an inferred parameter, and a parameter combination is generated; the response coefficient of the inferred parameter is corrected through reverse deduction, a bidirectional correlation matrix is optimized, and the consistency of the target parameter and the inferred parameter is ensured. Through shadow area analysis and surface fitting technology of multi-view images, the problem of bottom area data loss is solved, and accurate three-dimensional reconstruction of the object surface and the bottom area is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of product replication, and in particular to a product replication software system for three-dimensional model import and conversion. BACKGROUND

[0002] In modern product design and manufacturing processes, accurate replication of three-dimensional models is one of the key technologies for improving production efficiency and precision. Traditional three-dimensional modeling methods mainly rely on laser scanning, computer vision technology and manual modeling, etc. Although these methods can accurately obtain the geometric information of objects, they still face problems such as being unable to accurately obtain the overall appearance or producing data missing in complex application scenarios, such as difficult-to-shoot bottom areas. In addition, although the shadow analysis method in the prior art can provide local information of the object surface, since the shadow itself is based on the relative position relationship between the object and the light source, its analysis essentially belongs to an ill-posed inverse problem, which is difficult to directly obtain the absolute three-dimensional attributes of the object, resulting in a large error in the reconstruction process of the shadow area.

[0003] For example, the Chinese patent with the granted announcement number CN109598785B discloses a three-dimensional grid model view conversion method, including the following steps: importing a three-dimensional grid model; establishing a first space rectangular coordinate system with the geometric particles of the three-dimensional grid model as the origin O, and turning the long principal axis of the three-dimensional grid model to a position coinciding with the Z-axis of the first space rectangular coordinate system; positioning the projection point coordinates; setting the pixel value of the projection view; and generating the projection view according to the projection point coordinates and the pixel value. The invention turns the long principal axis of the three-dimensional grid model to a position coinciding with the Z-axis of the first space rectangular coordinate system, which can project more points in the three-dimensional grid model into the view to obtain a projection view with the maximum amount of information. The invention is used for converting a three-dimensional grid model into a two-dimensional projection view.

[0004] The above prior art all have the problem proposed in the background: how to accurately extract three-dimensional models and eliminate uncertainty in complex environments, especially in difficult-to-shoot areas such as the bottom. To solve the above problems, the present application designs a product replication software system for three-dimensional model import and conversion. SUMMARY

[0005] The technical problem solved by the present application is to provide a product copying software system for three-dimensional model import and conversion, which generates a preliminary three-dimensional model by acquiring a main view image and a sub-view image, extracting first feature parameters and second feature parameters; adjusts target parameters based on a bidirectional mapping relationship between the target parameters and inferred parameters, generates parameter combinations; optimizes a bidirectional correlation matrix by correcting the response coefficient of the inferred parameters through reverse deduction, and ensures the consistency of the target parameters and the inferred parameters. Through shadow area analysis and surface fitting technology of multi-view images, the problem of missing data in the bottom area is solved, and accurate three-dimensional reconstruction of the object surface and the bottom area is realized.

[0006] To achieve the above object, the present application provides the following technical scheme.

[0007] A product copying software system for three-dimensional model import and conversion is applied to product copying software, and comprises an image acquisition module, a feature parameter extraction module, a model modeling module and a model conversion module, wherein:

[0008] The image acquisition module is used to acquire a main view image and multiple sub-view images of a product to be copied.

[0009] The feature parameter extraction module is used to extract first feature parameters, and to perform shadow area segmentation on the main view image and the sub-view images to extract second feature parameters.

[0010] The model modeling module is used to import the first feature parameters and the second feature parameters into the product copying software to generate a first model of the product to be copied, wherein the first model generates inferred parameters according to the first feature parameters and the second feature parameters.

[0011] The model conversion module is used to convert the first model according to input target parameters to generate a second model.

[0012] The feature parameter extraction module comprises a first feature extraction unit and a second feature extraction unit, wherein:

[0013] The first feature extraction unit is used to perform image processing on the main view image and the sub-view images, extract edge information, contour lines and key points of the main view image and the sub-view images, and take the edge information, contour lines and key points as first feature parameters.

[0014] The second feature extraction unit is used to infer feature parameters of the remaining area according to the shadow part of the main view image and the sub-view images.

[0015] The second feature extraction unit comprises a shadow extraction subunit, a shadow correction subunit, a shadow parameter calculation subunit, a shadow inference subunit and a shadow inversion subunit, wherein:

[0016] The shadow extraction subunit is configured to extract shadow edges of the main view image and the auxiliary view image.

[0017] The shadow correction subunit is configured to correct the shadow edges of the main view image according to the shadow edges of the auxiliary view image to generate a bottom shadow region.

[0018] The shadow parameter calculation subunit is configured to calculate a curvature tensor of the bottom shadow region, and obtain a transverse deformation parameter and a longitudinal deformation parameter according to the curvature tensor, wherein the curvature tensor is used to represent a curvature variation feature of the shadow region.

[0019] The shadow inference subunit is configured to jointly infer the transverse deformation parameter and the longitudinal deformation parameter to generate a shadow point pair satisfying a differential homeomorphism constraint and a perspective energy function.

[0020] The shadow inversion subunit is configured to inversely invert by surface fitting according to the shadow point pair to extract a second feature parameter.

[0021] The shadow extraction subunit comprises:

[0022] Camera calibration is performed on the main view image and the auxiliary view image to obtain an intrinsic matrix and an extrinsic matrix, and a epipolar geometry relationship between two view angles is established according to the intrinsic matrix and the extrinsic matrix.

[0023] The shadow edges are detected in the main view image by an epipolar equation.

[0024] According to the epipolar geometry relationship, edge points in the same gradient direction as the shadow edges of the main view image are extracted in the epipolar line range of the auxiliary view image to generate the shadow edges of the auxiliary view image.

[0025] The shadow correction subunit comprises:

[0026] A gradient structure tensor of the shadow edges of the main view image is calculated to generate an edge saliency map.

[0027] The edge saliency map is processed by a region growing algorithm to obtain a first shadow, wherein the region growing algorithm comprises gradually merging adjacent pixels consistent with the direction of the structure tensor of the center of the edge saliency map from the center of the edge saliency map.

[0028] The epipolar projection matching rate of the first shadow in the shadow edges of the auxiliary view image is calculated.

[0029] If the epipolar line projection matching rate is less than a preset matching threshold, the edge saliency map is Gaussian blurred and region growing is performed again until the epipolar line projection matching rate is greater than or equal to the preset matching threshold.

[0030] According to the matching result, the first shadow is completed, and the completed first shadow is output as a bottom shadow region.

[0031] The shadow inference subunit comprises:

[0032] A differential homeomorphism constraint condition is used to establish a differential homeomorphism mapping relationship between the lateral deformation parameter and the longitudinal deformation parameter, wherein the differential homeomorphism constraint condition indicates that the shadow edge deformation of the same three-dimensional surface point in the lateral direction and the longitudinal direction satisfies a differential homeomorphism transformation;

[0033] A cross-view-angle deformation parameter correlation matrix is constructed according to the lateral deformation parameter and the longitudinal deformation parameter, and a view-angle energy function is calculated according to the matrix characteristics of the cross-view-angle deformation parameter correlation matrix;

[0034] According to the differential homeomorphism mapping relationship and the view-angle energy function, a shadow point pair satisfying the consistency of the double-view-angle deformation is solved by a least square method.

[0035] The model conversion module comprises a parameter mapping unit, a parameter adjustment unit and a model conversion unit, wherein:

[0036] The parameter mapping unit is configured to obtain an input target parameter and construct a bidirectional mapping between the target parameter and the inference parameter;

[0037] The parameter adjustment unit is configured to adjust the target parameter according to the bidirectional mapping and a preset constraint condition to generate a parameter combination;

[0038] The model conversion unit is configured to convert the first model according to the parameter combination to generate a second model.

[0039] The parameter mapping unit comprises:

[0040] The parameter topology relationship tree of the first model is parsed, and a process constraint chain of the inference parameter is obtained;

[0041] According to the process constraint chain, an initial correlation mapping table between the target parameter and the inference parameter is established;

[0042] A bidirectional correlation matrix is generated, wherein a row vector of the bidirectional correlation matrix represents a modification direction of the target parameter, an array vector of the bidirectional correlation matrix represents a response coefficient of the inference parameter, and a matrix element value is jointly calibrated by a geometric constraint relationship between parameters and historical modification data;

[0043] According to the bidirectional association matrix, the initial association mapping table is optimized to generate a bidirectional mapping.

[0044] The parameter adjustment unit comprises:

[0045] According to the parameter topology relationship tree, direct associated parameters and indirect associated parameters of the target parameter are identified through a breadth-first search algorithm.

[0046] A preset parameter adjustment rule library is called, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules and assembly constraint rules.

[0047] According to the rule engine of the parameter adjustment rule library, the direct associated parameters and the indirect associated parameters are identified, a corresponding rule subset is matched, and an adjustment direction and an amplitude are parsed out.

[0048] According to the adjustment direction and the amplitude, the target parameter is adjusted to generate a parameter combination.

[0049] The parameter combination is reversely deduced to an inferred parameter, a response coefficient of the inferred parameter is corrected, and the bidirectional association matrix is modified according to the response coefficient.

[0050] A product copying method for three-dimensional model import and conversion, applied to product copying software, the method comprising:

[0051] Obtaining a front view image and a side view image of a product to be copied, and extracting a first feature parameter;

[0052] The front view image and the side view image are segmented into shadow regions, and a second feature parameter is extracted;

[0053] The first feature parameter and the second feature parameter are imported into the product copying software to generate a first model of the product to be copied, wherein the first model generates an inferred parameter according to the first feature parameter and the second feature parameter;

[0054] Obtaining an input target parameter, and constructing a bidirectional mapping between the target parameter and the inferred parameter;

[0055] According to the bidirectional mapping and a preset constraint condition, the target parameter is adjusted to generate a parameter combination;

[0056] According to the parameter combination, the first model is converted to generate a second model.

[0057] Compared with the prior art, the beneficial effects of the present application are:

[0058] The application successfully solves the data missing problem of the bottom and difficult-to-obtain area of the existing three-dimensional modeling method by combining the shadow analysis technology of multi-view images. Through camera calibration, polar geometric relationship and surface fitting technology, the bottom shadow area of the object can be accurately extracted, and the local shadow information can be converted into complete three-dimensional geometric data. BRIEF DESCRIPTION OF DRAWINGS

[0059] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings:

[0060] Figure 1 A product copying software system module diagram for three-dimensional model import and conversion of embodiment 1 of the application;

[0061] Figure 2 A flowchart of a product copying method for three-dimensional model import and conversion of embodiment 2 of the application;

[0062] Figure 3 A second feature parameter extraction flowchart of embodiment 2 of the application;

[0063] Figure 4 A first shadow extraction schematic diagram of embodiment 2 of the application;

[0064] Figure 5 A first shadow completion schematic diagram of embodiment 2 of the application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments.

[0066] It should be noted that the technical solutions in the application can be applied to various technical fields, including but not limited to automobile models.

[0067] Embodiment 1

[0068] Please refer to Figure 1 The application provides an embodiment: a product copying software system for three-dimensional model import and conversion, applied to product copying software. The product copying software system includes an image acquisition module, a feature parameter extraction module, a model modeling module and a model conversion module, wherein:

[0069] The image acquisition module is used to acquire a main view image and multiple auxiliary view images of a product to be copied.

[0070] The feature parameter extraction module is used to extract first feature parameters and perform shadow area segmentation on the main view image and the auxiliary view images to extract second feature parameters.

[0071] The model modeling module is configured to import the first characteristic parameter and the second characteristic parameter into product copying software to generate a first model of a product to be copied, wherein the first model generates inferred parameters according to the first characteristic parameter and the second characteristic parameter.

[0072] The model conversion module is configured to convert the first model according to input target parameters to generate a second model.

[0073] The characteristic parameter extraction module includes a first characteristic extraction unit and a second characteristic extraction unit, wherein:

[0074] The first characteristic extraction unit is configured to perform image processing on the main view image and the auxiliary view image, extract edge information, contour lines and key points of the main view image and the auxiliary view image, and take the edge information, contour lines and key points as first characteristic parameters.

[0075] The second characteristic extraction unit is configured to infer characteristic parameters of a remaining area according to a shadow part of the main view image and the auxiliary view image.

[0076] The second characteristic extraction unit includes a shadow extraction subunit, a shadow correction subunit, a shadow parameter calculation subunit, a shadow inference subunit and a shadow inversion subunit, wherein:

[0077] The shadow extraction subunit is configured to extract a shadow edge of the main view image and the auxiliary view image.

[0078] The shadow correction subunit is configured to correct the shadow edge of the main view image according to the shadow edge of the auxiliary view image to generate a bottom shadow area.

[0079] The shadow parameter calculation subunit is configured to calculate a curvature tensor of the bottom shadow area, and obtain a transverse deformation parameter and a longitudinal deformation parameter according to the curvature tensor, wherein the curvature tensor is used to represent a curvature change feature of the shadow area.

[0080] The shadow inference subunit is configured to jointly infer the transverse deformation parameter and the longitudinal deformation parameter to generate a shadow point pair satisfying a differential homeomorphism constraint and a view angle energy function.

[0081] The shadow inversion subunit is configured to extract second characteristic parameters by reverse inversion through surface fitting according to the shadow point pair.

[0082] The shadow extraction subunit includes:

[0083] Camera calibration is performed on the primary image and the secondary image to obtain an intrinsic matrix and an extrinsic matrix, and an epipolar geometric relationship between the two views is established according to the intrinsic matrix and the extrinsic matrix;

[0084] The shadow edge is detected in the primary image through an epipolar equation;

[0085] According to the epipolar geometric relationship, edge points with the same gradient direction as the shadow edge of the primary image are extracted within the epipolar range corresponding to the secondary image to generate the shadow edge of the secondary image.

[0086] The shadow correction subunit comprises:

[0087] The gradient structure tensor of the shadow edge of the primary image is calculated to generate an edge saliency map;

[0088] The edge saliency map is processed through a region growing algorithm to obtain a first shadow, wherein the region growing algorithm comprises gradually merging adjacent pixels that are consistent with the direction of the centroid structure tensor from the centroid of the edge saliency map;

[0089] The epipolar projection matching rate of the first shadow in the shadow edge of the secondary image is calculated;

[0090] If the epipolar projection matching rate is less than a preset matching threshold, the edge saliency map is Gaussian blurred and region growing is performed again until the epipolar projection matching rate is greater than or equal to the preset matching threshold;

[0091] The first shadow is completed according to the matching result, and the completed first shadow is output as a bottom shadow region.

[0092] The shadow inference subunit comprises:

[0093] A differential homeomorphism mapping relationship between the lateral deformation parameter and the longitudinal deformation parameter is established through a differential homeomorphism constraint condition, wherein the differential homeomorphism constraint condition indicates that the shadow edge deformation of the same three-dimensional surface point in the lateral direction and the longitudinal direction satisfies a differential homeomorphism transformation;

[0094] A cross-view deformation parameter correlation matrix is constructed according to the lateral deformation parameter and the longitudinal deformation parameter, and a view energy function is calculated according to the matrix characteristics of the cross-view deformation parameter correlation matrix;

[0095] According to the differential homeomorphism mapping relationship and the view energy function, a shadow point pair that satisfies the consistency of the two-view deformation is solved through a least square method.

[0096] The model conversion module comprises a parameter mapping unit, a parameter adjustment unit and a model conversion unit, wherein:

[0097] The parameter mapping unit is configured to obtain an input target parameter and construct a bidirectional mapping between the target parameter and the inferred parameter.

[0098] The parameter adjustment unit is configured to adjust the target parameter according to the bidirectional mapping and a preset constraint condition to generate a parameter combination.

[0099] The model conversion unit is configured to convert the first model according to the parameter combination to generate a second model.

[0100] The parameter mapping unit comprises:

[0101] The parameter topology relationship tree of the first model is parsed to obtain a process constraint chain of the inferred parameter.

[0102] An initial associated mapping table between the target parameter and the inferred parameter is established according to the process constraint chain.

[0103] A bidirectional associated matrix is generated, wherein a row vector of the bidirectional associated matrix represents a modification direction of the target parameter, a column vector of the bidirectional associated matrix represents a response coefficient of the inferred parameter, and a matrix element value is jointly calibrated by a geometric constraint relationship between parameters and historical modification data.

[0104] The initial associated mapping table is optimized according to the bidirectional associated matrix to generate a bidirectional mapping.

[0105] The parameter adjustment unit comprises:

[0106] Directly associated parameters and indirectly associated parameters of the target parameter are identified by a breadth-first search algorithm according to the parameter topology relationship tree.

[0107] A preset parameter adjustment rule library is called, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules and assembly constraint rules.

[0108] The directly associated parameters and the indirectly associated parameters are identified according to a rule engine of the parameter adjustment rule library, a corresponding rule subset is matched, and an adjustment direction and an amplitude are parsed.

[0109] The target parameter is adjusted according to the adjustment direction and the amplitude to generate a parameter combination.

[0110] The parameter combination is reversely deduced to the inferred parameter, a response coefficient of the inferred parameter is corrected, and the bidirectional associated matrix is modified according to the response coefficient.

[0111] Embodiment 2

[0112] See Figure 2The application provides a product replication method based on three-dimensional model import and conversion, which is applied to product replication software, and specific steps of the method are as follows:

[0113] S1: Obtain a front view image and a side view image of a product to be replicated;

[0114] In this embodiment, by obtaining the front view image and the side view image of the product to be replicated, the problem of obtaining multi-angle data of the product in the three-dimensional modeling process is solved. The front view image provides a front view of the product, and the side view image provides additional perspective information from the side, so the side view image contains multiple images. In the image acquisition process, light is supplemented by multiple light sources to highlight the shadow area. Through the combination of the two perspectives, the shape and geometric features of the product can be captured more comprehensively, especially the detailed information that is difficult to directly capture from other angles.

[0115] S2: Extract first feature parameters;

[0116] In this embodiment, the first feature parameters of the product are extracted by performing edge detection, contour extraction and key point recognition on the front view image and the side view image through image processing technology. These parameters include geometric features (such as edges, angles, contour shapes) and size information of the object.

[0117] S3: Extract second feature parameters;

[0118] In this embodiment, extracting the second feature parameters includes extracting shadow edge information from the front view image and the side view image. By analyzing the changes of the shadow area, the geometric shape of the bottom of the object is inferred. By analyzing the shape change of the shadow, the implicit geometric features of the bottom are obtained, solving the problem of missing details of the bottom caused by difficult shooting of the bottom.

[0119] S4: Import the first feature parameters and the second feature parameters into the product replication software to generate a first model;

[0120] In this embodiment, the extracted first feature parameters and second feature parameters are imported into the replication software to generate a preliminary three-dimensional model. By combining the feature data extracted from the front view image, the side view image and the shadow analysis, the replication software can create a preliminary three-dimensional model that conforms to the actual physical shape.

[0121] S5: Obtain input target parameters and construct a bidirectional mapping between the target parameters and the first model inference parameters;

[0122] In this embodiment, the target parameters can be the design specifications or process requirements of the product, while the inferred parameters are physical parameters obtained through three-dimensional modeling and analysis. By constructing a bidirectional mapping between the target parameters and the first model inferred parameters, the corresponding target parameters can be automatically adjusted according to the inferred parameters when the target parameters change, ensuring that the three-dimensional model meets the actual design requirements.

[0123] S6: Adjust the target parameters according to the bidirectional mapping and the preset constraints to generate a parameter combination;

[0124] In this embodiment, the target parameters are automatically adjusted according to the bidirectional mapping relationship and the preset constraints (such as geometric shape, process constraints, physical properties, etc.). These constraints can limit the range of changes in the target parameters, ensuring that the adjusted target parameters still meet the design and process requirements. Through these conditions, a parameter combination that meets the actual requirements is generated, thereby optimizing the three-dimensional model.

[0125] S7: Convert the first model according to the parameter combination to generate a second model;

[0126] The technical solution proposed in this application aims to solve the problem of data loss or error caused by the difficulty in obtaining bottom images in the current three-dimensional model reconstruction process. Traditional three-dimensional reconstruction methods usually rely on multi-angle shooting, but in some application scenarios, the cost of shooting from the bottom is high, and in many cases it is difficult to obtain clear images, so it is impossible to obtain complete three-dimensional information, especially in the bottom area. Existing technologies usually use multi-angle shooting or use depth sensors, but these methods still have error accumulation in practical application, especially in conditions of occlusion, reflection or uneven light. Through the method of this application, the relative position relationship between the multi-angle information and the shadow is used to infer the three-dimensional structure of the bottom by combining the shadow area of the main view and the secondary view, solving the technical problems of missing or large error of bottom parameters in traditional technology.

[0127] Specifically, the problem that shadow analysis usually faces is that shadow only reflects the relative relationship between the object and the light source, and cannot directly deduce the absolute three-dimensional attributes of the object, which easily leads to the problem of multiple solutions. Therefore, the traditional shadow analysis method cannot effectively reflect the true shape of the object, especially in the inference of the bottom area. However, this application reduces this uncertainty by combining the geometric constraints in the top view and side view images and using the geometric relationship between the object surfaces, thereby gradually approaching the true three-dimensional structure. Through accurate inference of the shadow and combining the constraint conditions, this method can effectively calculate the accurate three-dimensional data of the bottom and other hidden parts, improving the overall accuracy of three-dimensional reconstruction.

[0128] The specific steps of S2 are as follows:

[0129] S2.1: image processing is performed on the main view image and the sub-view image to extract edge information, contour lines and key points of the main view image and the sub-view image;

[0130] S2.2: the edge information, contour lines and key points are taken as first feature parameters.

[0131] Referring to Figure 3 , a second feature parameter extraction flowchart of the embodiment of the present application, in the embodiment, the extraction of the second feature parameter is performed through shadow analysis, shadow analysis is essentially an ill-posed problem, which means that shadow only provides the relative position relationship between the object and the light source, and cannot directly express the absolute three-dimensional attribute of the object, resulting in high uncertainty in shadow analysis. This problem mainly manifests that shadow can only reflect the interaction between the object surface and the light source, and cannot clearly determine the three-dimensional form of the object itself, especially at the bottom of the object or the part that is difficult to observe, the change of shadow may correspond to multiple different three-dimensional structures, so there are often multiple solutions or indeterminate cases.

[0132] In this case, the traditional shadow analysis method often cannot obtain accurate object three-dimensional structure. This is because the traditional technology cannot eliminate the uncertainty caused by the light source position, viewing angle, object reflectivity and other factors, so that the reconstruction of the three-dimensional model often has errors, especially in complex environment, the speculation of the hidden part at the bottom is often not accurate enough.

[0133] In the embodiment, through the combination of the main view image and the sub-view image, each viewing angle provides different spatial information, and the light source angle and the viewing angle difference also play a key role in the change of shadow. Through this multi-view shadow analysis, not only the edge information of a single viewing angle is relied on, but also the relative position relationship of the shadow in different viewing angles is used to deduce the three-dimensional form of the object. In this way, the light source, the shadow and the geometry of the object are all considered comprehensively, which greatly reduces the uncertainty in shadow analysis, and thus the real shape of the object can be more accurately speculated.

[0134] Further, the epipolar geometry relationship is introduced into the shadow edge extraction, which ensures the matching accuracy of the shadow points in different viewing angles. Through the internal and external parameter matrices obtained by camera calibration, the shadow points in the two views can be accurately corresponded, and the errors caused by different viewing angles can also be effectively eliminated. The uncertainty caused by the difference in light source or viewing angle in the traditional shadow analysis is eliminated, the accurate matching of the shadow point pairs is ensured, and the stability and precision of the object three-dimensional structure speculation are improved.

[0135] Furthermore, through the generation and inference of shadow point pairs mentioned earlier, surface fitting technology can convert these two-dimensional shadow points into three-dimensional geometric information. Since surface fitting considers the geometric relationship of shadow point pairs in three-dimensional space, by minimizing the error between the shadow point pairs and the fitted surface, a three-dimensional surface that approximates the real object's surface can be generated. This process ensures that, based on multi-view shadow information, the object's geometry is gradually deduced from local to global, thus eliminating the problem in traditional methods where local information cannot accurately deduce the overall structure.

[0136] The specific steps for S3 are as follows:

[0137] S3.1: Extract the shadow edges of the front view image and the secondary view image;

[0138] Specifically, the purpose of shadow edge extraction is to extract the shadow portion reflecting the relative positional relationship between the object's surface and the light source from the front and back view images. This step is based on image processing techniques, typically using edge detection algorithms (such as Canny, Sobel, or Laplacian operators) to extract salient edges in the image. The front and back view images provide different perspectives of the object, and the changes in shadow edges in these two perspectives reflect the geometric relationship between the object and the light source. Therefore, by extracting shadow edge information from these two images, necessary preliminary data can be provided for subsequent shadow correction, inference, and 3D reconstruction.

[0139] Furthermore, while edge detection algorithms can extract basic image edges by calculating edges with varying grayscale values ​​in an image, these methods have limitations when dealing with surface shadows and geometric features. Especially in complex shadow areas, edge detection algorithms are often susceptible to noise and lighting variations, leading to reduced accuracy in edge extraction. Particularly in cases of uneven lighting, occlusion in shadows, or reflections, edge detection algorithms are prone to false positives or false negatives, resulting in inaccurate shadow edge extraction. For example, in complex lighting environments, shadow shapes vary significantly, making it difficult for traditional algorithms to handle these variations and failing to consider the relative geometric relationships between objects, thus failing to provide reliable three-dimensional geometric information.

[0140] In this embodiment, the front view image and the back view image are associated through epipolar geometry, enabling the detection of shadow edges in the front view image and the location of corresponding edge points within the epipolar range of the back view image. Epipolar geometry, derived from camera calibration, more accurately reflects the spatial relationship of objects in the two views. In this way, shadow extraction not only relies on image grayscale variations but also incorporates the geometric relationship between the object and the light source, making shadow edge extraction more precise and reliable, ensuring spatial consistency and accuracy of shadow edges in images taken from different perspectives.

[0141] S3.2: correcting the shadow edge of the main view image according to the shadow edge of the sub-view image, to generate a bottom shadow area;

[0142] Specifically, the shadow analysis in the traditional method is often affected by the light source angle, viewing angle and occlusion. The shadow analysis essentially belongs to an ill-posed problem (single solution does not exist), and the shadow can only reflect the relative position relationship between the object and the light source, but cannot directly express the absolute three-dimensional attributes of the object itself. Relying solely on the shadow edge in the main view image will lead to inaccuracy of the bottom shadow area. By combining the shadow edge information in the sub-view image, the uncertainty in shadow analysis can be reduced, which can effectively reduce this error and improve the accuracy of the bottom shadow area.

[0143] In this embodiment, since the image of the bottom of the object is difficult to obtain directly, the sub-view image provides information from different angles, which helps to correct the edge of the shadow in the main view image. In this step, the shadow edge in the sub-view image will be used as a reference to correct the shadow edge in the main view image. The shadow edge of the sub-view image provides longitudinal information of the object, while the main view image mainly reflects the lateral information of the object. By combining the shadow edges of the two, the geometry of the bottom of the object, especially the outline of the bottom shadow area, can be more accurately inferred.

[0144] Further, by comparing and correcting the shadow edge of the sub-view image with the shadow edge of the main view image, the shape and position of the bottom of the object can be accurately inferred, especially in the case where the bottom cannot be directly photographed, the bottom data can be supplemented and improved.

[0145] S3.3: calculating a curvature tensor of the bottom shadow area, and obtaining a lateral deformation parameter and a longitudinal deformation parameter according to the curvature tensor, wherein the curvature tensor is used to represent the curvature variation characteristics of the shadow area;

[0146] Specifically, in the process of three-dimensional reconstruction, the geometric information of the bottom shadow area is usually more complex than that of other parts of the object, because it involves lighting, object surface features and possible occlusion. Therefore, it is unrealistic to use a simple linear relationship to describe these complex shadow areas. As a mathematical tool, the curvature tensor can describe the bending of the object surface in different directions in detail, thereby providing accurate data support for further three-dimensional shape inference.

[0147] In this embodiment, the curvature tensor of the bottom shadow region is used to quantitatively describe the curvature variation of the object surface, thereby reflecting the geometric characteristics of the object. By calculating the curvature tensor, the deformation degree of the shadow region in different directions, including horizontal and vertical deformation, can be analyzed. The curvature tensor provides an accurate mathematical description of the surface morphology of the object, which is particularly suitable for handling such complex geometric features as the bottom shadow region.

[0148] Further, the curvature tensor is obtained by mathematical modeling of the image data of the shadow region. First, the gradient information of the region (i.e., the rate of change of the edge) is extracted from the shadow region. This can be achieved by calculating the gray value variation of the shadow region, usually using the Laplacian operator for gradient calculation. Then, the curvature tensor is obtained by calculating the second derivative of the gradient field, specifically, the curvature tensor is obtained by calculating the Hessian matrix (i.e., the second-order partial derivative matrix of the image) of the shadow region image. The Hessian matrix contains the curvature information of each point in the image, reflecting the surface curvature around the point. By solving the matrix, the principal curvature value of each point can be obtained, thereby obtaining the curvature variation of the point in different directions.

[0149] Further, the horizontal deformation parameter and the vertical deformation parameter correspond to the principal curvature values in different directions of the curvature tensor, respectively. The horizontal deformation parameter is related to the principal curvature value in the horizontal direction, and the vertical deformation parameter is related to the principal curvature value in the vertical direction.

[0150] S3.4: jointly infer the horizontal deformation parameter and the vertical deformation parameter to generate a shadow point pair satisfying the differential homeomorphism constraint and the perspective energy function;

[0151] Specifically, shadow analysis is essentially an ill-posed inverse problem, and a single solution often cannot satisfy the geometric consistency under multiple perspective conditions. By introducing the differential homeomorphism constraint and the perspective energy function, this uncertainty can be effectively reduced, making the generation of shadow point pairs more accurate and avoiding the multi-solution problem in traditional methods.

[0152] In this embodiment, the horizontal and vertical deformation parameters describe the changes of the shadow in the horizontal and vertical directions, respectively. In this stage, a joint inference method is used to generate a shadow point pair in combination with the differential homeomorphism constraint and the perspective energy function. The differential homeomorphism constraint ensures that the deformation of the same three-dimensional surface point under different perspectives is smooth and continuous, while the perspective energy function helps to optimize the matching of the shadow point pair, ensuring the geometric consistency of the object surface under different perspectives.

[0153] S3.5: according to the shadow point pair, inverse inversion is performed through surface fitting to extract a second feature parameter;

[0154] Specifically, in the process of three-dimensional reconstruction of the object, the information provided by the shadow area is usually local, i.e. only reflects part of the morphology of the object surface, and cannot directly give the complete three-dimensional geometric structure of the object. In order to infer the complete shape of the object from these local shadow information, the surface fitting and reverse inversion techniques are used. The surface fitting technique can convert the shadow point pairs into object surface data in three-dimensional space, so as to obtain accurate geometric morphology, especially the bottom and other hidden parts which are difficult to obtain directly.

[0155] In the present embodiment, the shadow point pairs generated by the previous steps are used as the basis for surface fitting. Each shadow point represents the deformation and geometric features of the object surface at a specific viewing angle. By fitting these shadow points, the true morphology on the object surface can be inferred. The core of this process is to infer the three-dimensional surface of the object by fitting the surface on which the shadow point pairs are located. The surface fitting technique, especially the fitting algorithm based on least squares method, can ensure the accuracy in the fitting process and preserve the details of the object surface as much as possible, so as to generate an accurate three-dimensional geometric model.

[0156] Further, by using the position and shape of the fitted surface in space, the corresponding geometric features on the object surface are calculated by reverse inversion technique. This process converts two-dimensional shadow information into three-dimensional data, and obtains the second feature parameter, curvature and inclination by inferring the specific position of the shadow point.

[0157] The specific steps of S3.1 are as follows:

[0158] S3.1.1: Camera calibration is performed on the main view image and the secondary view image to obtain the intrinsic matrix and the extrinsic matrix, and the epipolar geometric relationship between the two views is established according to the intrinsic matrix and the extrinsic matrix;

[0159] Specifically, camera calibration and establishment of epipolar geometric relationship are the key to ensure the spatial consistency of the shadow edges extracted from different viewing angles. Since the shadow edge is based on the relative position relationship between the object and the light source, the morphology of the shadow in the image depends on the viewing angle difference. Through accurate camera calibration and epipolar geometric relationship, accurate correspondence of multi-view shadow edges can be realized, and it is ensured that the shadow edges extracted in the secondary view image can match the shadow edges in the main view image.

[0160] In this embodiment, through camera calibration, the intrinsic matrix and extrinsic matrix are first obtained, where the intrinsic matrix describes the imaging characteristics of the camera such as focal length, principal point position, etc., and the extrinsic matrix describes the position and orientation of the camera in three-dimensional space. The epipolar geometry refers to the geometric constraints between the image points of an object in one view and the image points of the object in another view when two cameras capture the same object from different perspectives. In other words, the epipolar geometry describes how the corresponding points of an object in two images are related.

[0161] Further, through the intrinsic matrix and extrinsic matrix, this geometric relationship can be constructed. Specifically, the image points captured in the first view, after conversion through the intrinsic matrix and extrinsic matrix, can be mapped to the image in the second view. The epipolar geometry, through the definition of epipoles and epipolar lines, ensures that the corresponding points of an image point in one view must fall on a specific straight line in another view, which is called the epipolar line. The points corresponding to the shadow edges in the main view image can be determined in the sub-view image.

[0162] S3.1.2: Detecting shadow edges in the main view image through the epipolar equation;

[0163] Specifically, the epipolar equation is one of the core geometric relationships in stereo vision. It is used to describe the trajectory of a point in one view that should fall in another view. In order to extract the shadow edges in the main view image, the intrinsic and extrinsic matrices of the camera must be obtained through camera calibration. These matrices describe the positional relationship of the object in three-dimensional space, and combined with the parameter information of the camera, the epipolar geometry can be constructed, and then the epipolar equation is obtained. In actual operation, the intrinsic matrix and extrinsic matrix obtained through camera calibration can establish the geometric relationship between the two views. The intrinsic matrix provides the imaging parameters of the camera (such as focal length, principal point coordinates, etc.), and the extrinsic matrix describes the position and orientation of the camera in space. Based on this information, the epipolar equation can be calculated through standard epipolar geometry principles. This equation describes the position of a point of an object projected onto the image plane from one view, and the straight line (i.e. epipolar line) that the corresponding point in the image should be located in when projected from another view.

[0164] In this embodiment, by calculating the epipolar equation and determining the epipolar range in the main view image, the shadow area in the image can be accurately located. Then, the traditional edge detection algorithm (such as Sobel or Canny algorithm) is applied within these ranges to further extract the shadow edge. The key of this step is that the epipolar equation provides a geometric constraint for the traditional edge detection algorithm, making the extraction of the shadow edge more accurate and avoiding errors caused by factors such as lighting changes and viewing angle differences. The extraction process of the shadow edge first limits the possible position range of the edge through the epipolar equation. In the main view image, the corresponding shadow edge should be within the epipolar range. This means that through the epipolar equation, not only can the position of the shadow edge in the main view image be limited, but also the shadow edge can be ensured to conform to the spatial geometric constraint, thereby improving the accuracy of the shadow edge extraction.

[0165] S3.1.3: According to the epipolar geometric relationship, extract the edge points with the same gradient direction as the shadow edge in the main view image within the epipolar range corresponding to the secondary view image to generate the shadow edge of the secondary view image;

[0166] Specifically, the accurate matching of the shadow edge is the key in three-dimensional reconstruction. Traditional edge detection methods usually cannot handle the geometric relationship between different viewing angles, while the present scheme introduces the epipolar geometric relationship, so that the shadow edge extracted in the secondary view image can be accurately aligned with the shadow edge in the main view image. The consistency of the gradient direction further ensures the accuracy of the edge extraction, avoiding the edge extraction errors caused by lighting changes or object occlusion in the conventional method.

[0167] In this embodiment, after extracting the shadow edge in the main view image, according to the epipolar geometric relationship obtained in the foregoing steps, the possible position of the shadow edge in the secondary view image can be determined through the epipolar equation. The extraction of the shadow edge in the secondary view image is not directly dependent on the traditional edge detection, but through the spatial information provided by the camera calibration, the edge point extraction is carried out within the epipolar range of the secondary view image. Specifically, the epipolar geometric relationship defines the matching mode of the shadow points between the main view image and the secondary view image, so that the edge points with the same gradient direction as the shadow edge in the main view image can be accurately extracted in the secondary view image.

[0168] Further, the edge point extraction not only considers the gray scale change in the image, but also combines the viewing angle constraint, making the extraction of the shadow edge more accurate. In particular, the matching of the gradient direction can ensure that the shadow edge extracted in the secondary view image has consistency with the shadow edge in the main view image in geometry. The range of the edge points in the secondary view image can be limited according to the epipolar geometric relationship, reducing unnecessary false matching and false extraction.

[0169] Please refer to Figure 4 , the first shadow extraction schematic diagram of the embodiment of the present application, Figure 4The process of forming the first shadow from the main view image through the edge saliency map processing and the region growing algorithm is shown. The main view image generates an edge saliency map through image processing techniques, which identifies the potential shadow edge region. On the basis of the saliency map, the region growing algorithm is used to expand outward from the edge center, generating a preliminary first shadow region.

[0170] It can be understood that in the main view image Figure 4 The stereoscopic part of the main view image is the product to be engraved, and the gray part is the shadow area.

[0171] The specific steps of S3.2 are as follows:

[0172] S3.2.1: Calculate the gradient structure tensor of the shadow edge of the main view image to generate an edge saliency map;

[0173] Specifically, the gradient is an important feature that measures the gray scale change in the image, reflecting the edge information between different regions in the image. In order to improve the detection accuracy of the shadow edge, the gradient structure tensor is used, which can describe the change trend of the local region of the image. By calculating the gradient information of each pixel in the image, a gradient structure tensor can be obtained, which can represent the change of the gradient direction and the texture information of the local region in the image. The gradient structure tensor can enhance the response of the edge and texture region by measuring the directionality of the gray scale change in the image, especially in the shadow area, which is very effective for detecting object edges. Based on the calculated gradient structure tensor, an edge saliency map is generated. The edge saliency map highlights the regions with significant gradient changes in the image, i.e. the regions where the shadow edge is located, by calculating the response in the entire image. In this way, the generated saliency map can focus on the most important shadow information in the image.

[0174] In this embodiment, an edge saliency map is generated by applying gradient structure tensor calculation to the shadow area, which represents the saliency and directionality information of the shadow edge. This process makes the shadow edge not only limited to simple gray scale change, but also considers the spatial geometric features in the image.

[0175] S3.2.2: Process the edge saliency map through a region growing algorithm to obtain the first shadow, wherein the region growing algorithm includes gradually merging adjacent pixels consistent with the direction of the center structure tensor from the center of the edge saliency map;

[0176] Specifically, the shadow edge is usually continuous, but in the image, especially in complex scenes, the extraction of the edge may have discontinuities or noise. The region growing algorithm can effectively connect the discrete edge information into a whole, so as to obtain a more coherent and complete shadow region. In particular, for parts such as the bottom of the object that are difficult to shoot or directly obtain, the region growing algorithm can infer the boundaries of these regions according to the geometric information in the image, and generate a complete shadow region.

[0177] In this embodiment, after the edge saliency map is generated, the region growing algorithm is used to further process the image. Region growing is a seed point-based image segmentation algorithm, and its basic idea is to start from a starting point (seed point) and gradually expand to its neighborhood pixels until a certain stopping criterion is met. The seed point is selected as the center of mass region in the edge saliency map, that is, the region with the most significant gray scale change in the image and a high correlation with the shadow edge. This selection method is based on gradient information, starting from the region with the highest saliency, ensuring that the growth starts from the shadow edge part of the object surface, and is not disturbed by noise or errors.

[0178] Further, the shadow edge has a certain directionality, and the edge is usually continuous. In order to effectively merge the neighborhood pixels, the region growing algorithm must ensure that the merged pixels meet certain directionality constraints during the expansion process. In particular, when processing the shadow region, the consistency of the gradient direction between pixels is an important basis for determining whether they belong to the same region. Through this constraint, the continuity and stability of the shadow region can be ensured, and false merging caused by differences in viewing angle or changes in illumination can be avoided. Starting from the seed point, the region growing algorithm gradually checks whether the gradient direction of the surrounding neighbor pixels is consistent with the gradient direction of the seed point. The algorithm calculates the gradient information of the neighborhood pixels and compares it with the gradient direction of the seed point. If the gradient direction of the neighborhood pixel is consistent with the direction of the seed point, it is considered that the pixel belongs to part of the shadow edge and can be merged with the seed point to expand the shadow region. If the gradient direction of the neighborhood pixel is not consistent with the direction of the seed point, the pixel does not meet the merging condition, and the algorithm will stop expanding in that direction.

[0179] Further, the core of the region growing algorithm is to gradually expand the region until the entire shadow region is completely extracted. The expansion process ensures the continuity and accuracy of the shadow region through the merging of neighborhood pixels. In order to avoid overgrowth or unnecessary false merging, the algorithm needs to set a stopping criterion. Generally, the stopping criterion can be based on the following aspects:

[0180] Maximum region size, when the size of the shadow region reaches a certain threshold, stop expanding.

[0181] Gradient direction consistency, when the gradient direction of the neighborhood pixel differs from the direction of the current region by more than a certain threshold, stop expanding in that direction.

[0182] Edge strength, stop growing when the edge strength of the merged neighborhood pixels is below a certain value.

[0183] In this embodiment, the region growing will continue until the entire shadow region is expanded and the consistency of the gradient direction of the neighborhood pixels does not exceed the set threshold. The region growing process can be effectively controlled to avoid merging unrelated pixels while ensuring the integrity of the shadow region.

[0184] S3.2.3: Calculate the epipolar line projection matching rate of the first shadow in the shadow edge of the secondary view image;

[0185] Specifically, through the epipolar geometry, the error of the shadow edge under different viewing angles can be reduced, ensuring the matching accuracy of the shadow edge in the two viewing angles. If the shadow edges in the primary view image and the secondary view image can be accurately matched, it means that the object surface morphology in the two images is consistent, and the inferred shadow region is reliable. Therefore, calculating the epipolar line projection matching rate helps to verify the accuracy of the shadow edge extraction result and ensures that the extracted shadow region conforms to the geometric morphology of the actual object.

[0186] In this embodiment, after generating the first shadow, the epipolar geometry is used to match in the secondary view image in order to further verify the accuracy of the shadow. By calculating the epipolar line projection matching rate of the first shadow in the secondary view image, the consistency of the shadow extracted from the primary view image in the secondary view image can be evaluated. The epipolar line projection is based on the geometric constraints of viewing angle transformation. According to the previous steps, theoretically, every point in the secondary view image should be within the corresponding epipolar line range in the primary view image, reflecting the spatial consistency of the object shadow, but in actual application, relying on simple geometric derivation often has errors. These errors may come from the following aspects:

[0187] 1. Although the epipolar geometry theoretically provides a corresponding relationship for points under two viewing angles, in actual application, due to factors such as camera calibration accuracy, image noise, non-ideal geometric morphology of the object (such as surface unevenness or reflection effect), the actual captured image points may have slight deviations. These deviations will cause the shadow points extracted from the primary view image to have a slightly misplaced or offset projection position in the secondary view image.

[0188] 2. The shadow edge is usually not absolutely clear, especially in complex lighting environments, the shadow part may be affected by reflection, occlusion or uneven light sources, resulting in blurred or inaccurate shadow edges. Even after edge saliency map extraction, there may be small errors in the shadow region, which cannot completely accurately match the shadow edge in the secondary view image.

[0189] 3. In actual image processing, the resolution and sampling accuracy are usually limited, especially under the influence of factors such as the accuracy of the camera or imaging device and image compression, the extraction and positioning of the shadow edge may have certain errors. These errors may result in the inability to directly find a perfect match of the shadow point in the sub-view image, so it is necessary to evaluate the matching accuracy by the epipolar projection matching rate.

[0190] Therefore, although the theoretical shadow point position is derived from the epipolar geometric relationship, in order to ensure the accuracy of the shadow area, it is still necessary to evaluate the consistency of the shadow edge extracted from the main view image in the sub-view image by calculating the epipolar projection matching rate. The epipolar projection matching rate calculation can verify whether the actually projected shadow edge is accurately located within the epipolar range of the sub-view image, further reducing possible errors and ensuring that the extracted shadow area is consistent with the geometric shape of the actual object.

[0191] Further, the process of calculating the epipolar projection matching rate is to project the first shadow into the sub-view image and calculate the degree of coincidence of the projection position with the actual shadow edge in the sub-view image. Specifically, first, the epipolar position of each shadow point in the sub-view image is calculated according to the epipolar geometric relationship, and then it is judged whether the actual existing shadow point within the epipolar range is consistent with the theoretical position. By calculating the projection matching degree of these shadow points (for example, by calculating the projection error or the degree of coincidence), the position accuracy of the first shadow in the sub-view image can be evaluated.

[0192] S3.2.4: If the epipolar projection matching rate is less than the preset matching threshold, Gaussian blur is performed on the edge saliency map and region growing is performed again until the epipolar projection matching rate is greater than or equal to the preset matching threshold;

[0193] Please refer to Figure 5 , the first shadow completion schematic diagram of the embodiment of the present application, Figure 5 shows the process of completing the first shadow area after the projection matching of the sub-view image. As can be seen from the figure, the initial first shadow area is projected and matched with the shadow edge of the sub-view image, and when the matching rate does not meet the requirement, the shadow area is regenerated by Gaussian blur and region growing. Finally, the corrected and completed shadow area is output as the bottom shadow area, which is used for further three-dimensional modeling analysis.

[0194] S3.2.5: According to the matching result, the first shadow is completed, and the completed first shadow is output as the bottom shadow area.

[0195] The specific steps of S3.4 are as follows:

[0196] S3.4.1: Establish a differential homeomorphism mapping relationship between the lateral deformation parameter and the longitudinal deformation parameter by a differential homeomorphism constraint condition, wherein the differential homeomorphism constraint condition represents that the shadow edge deformation of the same three-dimensional surface point in the lateral direction and the longitudinal direction satisfies a differential homeomorphism transformation;

[0197] Specifically, by establishing a differential homeomorphism mapping relationship between the lateral deformation parameter and the longitudinal deformation parameter, it can be ensured that the geometric deformation of the shadow area under different viewing angles follows the same rule. The deformation of the shadow point is not only due to the difference in viewing angle, but also affected by the geometric features of the object surface. Through differential homeomorphism mapping, inconsistencies in the shadow area can be effectively eliminated, so that the shadow points extracted from different viewing angles have consistency.

[0198] In this embodiment, in the three-dimensional modeling process, the deformation of the shadow can be regarded as a reflection of the morphological change between different areas on the surface. In order to accurately describe the shadow edge deformation of the object surface under different viewing angles, it is necessary to first establish a differential homeomorphism mapping relationship between the lateral deformation parameter and the longitudinal deformation parameter. Differential homeomorphism mapping means that the shadow deformation of the same three-dimensional surface point under different viewing angles maintains continuity and smoothness, that is, no fracture or irregular deformation occurs. This means that, through the differential homeomorphism constraint, the changes in the shadow deformation in the horizontal direction (lateral direction) and the vertical direction (longitudinal direction) should be able to map each other, maintaining the consistency of the geometric shape of the object surface.

[0199] S3.4.2: Construct a cross-view deformation parameter correlation matrix according to the lateral deformation parameter and the longitudinal deformation parameter, and calculate a view energy function according to the matrix characteristics of the cross-view deformation parameter correlation matrix;

[0200] Specifically, the establishment of the cross-view deformation parameter correlation matrix is to deal with the mutual influence between the shadow deformation parameters under different viewing angles. The shadow deformation in each viewing angle is not only in a single direction, but also involves deformation in the lateral and longitudinal directions, so the relationship between the two deformations must be established. Through the form of a matrix, these relationships can be systematically expressed and provide the necessary data basis for subsequent view energy function calculation.

[0201] In this embodiment, after the differential homeomorphism mapping relationship between the lateral and longitudinal deformation parameters is established, the next step is to construct a cross-view deformation parameter correlation matrix. This matrix describes how the lateral and longitudinal deformation parameters are related to each other under different viewing angles and how they affect the deformation of the shadow area of the object surface. By analyzing the deformation of the shadow area under two viewing angles, a matrix can be constructed to record the relationship between the lateral and longitudinal deformations in different viewing angles. Each element in the matrix represents the degree of change in the shadow deformation between viewing angles. Through the cross-view deformation parameter correlation matrix, the relationship between the shadow deformations under different viewing angles can be accurately calculated, so as to deduce the actual shape of the object under different viewing angles.

[0202] Further, the view angle energy function is calculated based on the properties of the matrix. The goal of the view angle energy function is to optimize the matching accuracy of the shadow points by quantifying the inconsistency of shadow deformation across different view angles. Since shadows may deform under different view angles, the view angle energy function aims to measure these deformations and minimize the differences, ensuring the consistency of shadow points in spatial position and geometric shape extracted from different view angles.

[0203] Further, the calculation of the view angle energy function is based on the matrix properties of the cross-view deformation parameter correlation matrix, combining the differences in shadow deformation under different view angles to quantify the energy. The essence of the view angle energy function is to measure the position error of shadow points between different view angles, and to minimize this error through optimization algorithms. Generally, the view angle energy function can be represented as deformation difference and geometric consistency, where the deformation difference calculates the difference between the horizontal and vertical deformation parameters under different view angles, measures the consistency of the deformations under different view angles, and the geometric consistency measures the geometric consistency of the object surface under different view angles by comparing the positions of shadow points in different view angles, ensuring that the shadow information extracted from one view angle can be accurately matched in another view angle.

[0204] Specifically, according to the cross-view deformation parameter correlation matrix of the horizontal and vertical deformation parameters, the difference in the change of each deformation parameter between different view angles is calculated, and the error of each shadow point is calculated according to its projection position in the main view image and the secondary view image. The projection errors of all shadow points are weighted and summed to obtain the overall view angle energy function value. This function value represents the consistency of shadow deformation between different view angles, and the goal is to minimize this value to optimize the matching accuracy of the shadow points.

[0205] S3.4.3: According to the differential homeomorphism mapping relationship and the view angle energy function, the shadow point pairs satisfying the consistency of the dual-view deformation are solved by the least squares method;

[0206] Specifically, the least squares method is used to solve the optimal matching position of the shadow points under different view angles through optimization algorithms. Because in the actual scene, due to the difference in view angle, the complexity of the object surface and other factors, there are errors in the deformation of shadow points, and the least squares method can effectively reduce these errors, ensuring the geometric consistency of the shadow point pairs under two view angles. By optimizing the positions and deformation parameters of these shadow point pairs, consistent three-dimensional shapes can be obtained under different view angles, thereby improving the accuracy and stability of three-dimensional reconstruction.

[0207] It should be noted that although the derivation here is based on the bottom shadow area, in the actual process, the preliminary shadow area of the object bottom is obtained through shadow edge extraction and correction of multi-view images. However, this preliminary shadow area may not fully conform to the true three-dimensional shape of the object, especially the bottom area. Due to the projection deformation of the object, the shadow edges under different viewing angles may be misaligned or asymmetric. Even with the correction of epipolar geometry and edge saliency map, there may still be slight geometric deviations due to viewing angle differences. At the same time, the shadow edge is affected by factors such as object surface texture and light source distribution, and the shadow boundary may not be an ideal smooth curve, but may have irregular or segmented morphology. In this case, although the shadow area has been preliminarily extracted, further optimization is still needed to correct these irregularities and ensure that the obtained shadow area accurately reflects the actual shape of the object.

[0208] In this embodiment, the shadow point pair is a matching point on the shadow edge extracted under two viewing angles. These points should correspond to the same position on the surface of the object in an ideal case. However, due to factors such as viewing angle difference and complexity of the object surface, the deformation of the shadow point may have errors. The goal of the least squares method is to minimize the error of the shadow point pair in the two viewing angles, optimize the spatial position of these points, and make the shadow points consistent in the two viewing angles as much as possible.

[0209] Further, in the least squares method, the error model is usually constructed based on the geometric difference of the shadow point pair. For example, the projection error of the shadow point in the two viewing angles can be calculated, and the sum of the squares of the errors of all shadow points is taken as the optimization objective function. By minimizing this objective function, the least squares method adjusts the position and deformation parameters of the shadow points to reduce the projection error and achieve more accurate matching of the shadow point pair.

[0210] Further, in the two viewing angles, the deformation of the shadow area may occur in both horizontal and vertical directions, so the horizontal and vertical deformation parameters need to be jointly optimized. That is, in the optimization process of the shadow point pair, the least squares method not only adjusts the position of the shadow point, but also optimizes the horizontal and vertical deformation parameters. The least squares method adjusts these deformation parameters to ensure the consistency of the shadow point pair in the two viewing angles, thereby improving the accuracy of the reconstruction.

[0211] Further, the least squares method is solved iteratively, gradually adjusting the position and deformation parameters of the shadow point pair until the error converges to a minimum value. Each iteration reduces the error between the shadow point pairs, and the final optimized shadow point position and deformation parameter accurately reflect the three-dimensional shape of the object surface.

[0212] The specific steps of S5 are as follows:

[0213] S5.1: Analyze the parameter topological relationship tree of the first model to obtain the process constraint chain of the inferred parameters;

[0214] Specifically, by analyzing the parameter topological relationship tree of the first model and the process constraint chain of the inferred parameters, the relationship between the target parameters and the inferred parameters can be systematically understood, and the role of each parameter in the model can be determined. The purpose of this process is to provide a data basis and constraint condition for subsequent correlation mapping, ensuring that the mapping relationship can meet the design and process requirements.

[0215] In this embodiment, during the three-dimensional modeling process, the parameter topological relationship tree is used to describe the hierarchical structure and mutual dependency between parameters in the first model. Each parameter has its specific role in the model, and these parameters are connected through specific constraint relationships to form a tree structure. By analyzing the parameter topological relationship tree of the first model, the interaction and dependency between parameters in the model can be clearly understood.

[0216] Further, in order to further understand the relationship between the target parameters and the inferred parameters, the relationship between each inferred parameter and the process constraint chain must be analyzed. The process constraint chain is a series of constraint conditions related to manufacturing processes, physical limitations, and design requirements, which limit the variation range of the inferred parameters and the mutual influence between the inferred parameters. Through the process constraint chain, it can be ensured that the target parameters will not exceed the reasonable physical or design range during the mapping process.

[0217] S5.2: According to the process constraint chain, establish an initial correlation mapping table between the target parameters and the inferred parameters;

[0218] Specifically, through the constraints of the process constraint chain, it can be ensured that the correlation mapping of the target parameters and the inferred parameters conforms to the actual manufacturing and design requirements, avoiding unreasonable values that may occur during the mapping process. By constructing the initial correlation mapping table, a foundation is laid for the subsequent optimization process, ensuring the reasonableness and feasibility of the correlation relationship.

[0219] In this embodiment, the correlation mapping table between the target parameters and the inferred parameters is the basic data structure for establishing the relationship between them. When constructing the correlation mapping table, the limiting conditions in the process constraint chain must be considered first. These constraint conditions may involve physical properties (such as material strength, size limitations), manufacturing processes (such as assembly sequence, processing technology), and design specifications, etc. Through these process constraints, the reasonable mapping relationship between the target parameters and the inferred parameters can be clearly defined, and preliminary reference data can be provided for the subsequent optimization process.

[0220] Further, in order to generate the initial correlation mapping table, first analyze the constraints in the process constraint chain to determine the degree of influence of each target parameter on the inferred parameter. Then, based on these relationships, an initial mapping table is constructed, where each row represents the modification direction of a target parameter, and each column represents the response coefficient of the inferred parameter. Through these mapping relationships, the interdependence between the target parameter and the inferred parameter is preliminarily determined.

[0221] S5.3: Generate a bidirectional correlation matrix, wherein the row vector of the bidirectional correlation matrix represents the modification direction of the target parameter, the column vector of the bidirectional correlation matrix represents the response coefficient of the inferred parameter, and the matrix element value is jointly calibrated by the geometric constraint relationship between the parameters and the historical modification data;

[0222] Specifically, the bidirectional correlation matrix is established to systematically describe the mutual influence between the target parameter and the inferred parameter. By matrixing the modification direction and the response coefficient, the relationship between the parameters can be clearly and intuitively represented, and accurate mathematical support can be provided for the subsequent optimization process. At the same time, by jointly calibrating the geometric constraint relationship and the historical data, the specific influence of the target parameter change on the inferred parameter can be more accurately determined, avoiding error accumulation.

[0223] In the present embodiment, the bidirectional correlation matrix is used to describe the mutual influence between the target parameter and the inferred parameter. In this matrix, the row vector represents the modification direction of the target parameter, i.e. the adjustment direction of the target parameter in the three-dimensional model; the column vector represents the response coefficient of the inferred parameter, i.e. the influence of the change of the target parameter on the inferred parameter.

[0224] Further, each element value of the matrix is jointly calibrated by the geometric constraint relationship between the parameters and the historical modification data. The geometric constraint relationship mainly considers factors such as object surface, geometric shape and spatial position, etc. These constraints ensure that the changes between the target parameter and the inferred parameter conform to the actual physical laws; the historical modification data records the relationship between the target parameter and the inferred parameter in the past modification process, and through the analysis of these data, the response of the inferred parameter when the target parameter is adjusted can be more accurately predicted.

[0225] S5.4: Optimize the initial correlation mapping table according to the bidirectional correlation matrix to generate a bidirectional mapping;

[0226] Specifically, by constructing the bidirectional association matrix, the initial association mapping table needs to be optimized next. The goal of optimization is to minimize the errors that may occur in the mapping process, ensuring that the relationship between the target parameters and the inferred parameters is more accurate. In this process, the bidirectional association matrix provides the geometric and physical relationship between the parameters, and the optimization process adjusts the elements in the matrix to make the mapping relationship between the target parameters and the inferred parameters more accurate. The optimization process is mainly based on the characteristics of the matrix, by adjusting the weights of the row vectors and column vectors, to optimize the matching degree between the target parameters and the inferred parameters. In this way, it can reduce the deviation caused by historical data errors or incomplete geometric constraints, and further improve the accuracy of the mapping relationship.

[0227] The specific steps of S6 are as follows:

[0228] S6.1: According to the parameter topological relationship tree, the direct associated parameters and indirect associated parameters of the target parameter are identified by the breadth-first search algorithm.

[0229] Specifically, the relationship between the target parameters and the inferred parameters is complex and diversified, and it is not usually a single linear relationship. By the breadth-first search algorithm, the hierarchical structure between parameters can be systematically analyzed to identify the direct associated parameters and indirect associated parameters of the target parameter. Direct associated parameters refer to parameters directly affected by the target parameter, while indirect associated parameters refer to parameters indirectly affecting the target parameter through other parameters.

[0230] In this embodiment, by the breadth-first search algorithm, the direct associated parameters of the target parameter are first identified, and further the indirect associated parameters associated therewith are identified. This process can ensure a comprehensive understanding of all relationships between the target parameter and other parameters, providing a basis for subsequent rule matching and parameter adjustment, i.e., starting from the target parameter, all other parameters directly and indirectly associated therewith are identified in turn. The breadth-first search algorithm is a graph traversal algorithm, which usually starts from the root node (target parameter), first visits the parameters directly associated with the target parameter, and then visits other parameters connected to these parameters layer by layer until all relevant parameters are identified.

[0231] S6.2: A preset parameter adjustment rule library is called, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules and assembly constraint rules.

[0232] Specifically, due to the complex and interdependent relationship between the target parameters and the inferred parameters, certain rules and constraints must be followed when adjusting the target parameters to ensure that the final result not only meets the design requirements but also meets the feasibility of actual manufacturing. Using a pre-set rule library can effectively ensure the rationality of target parameter adjustment and avoid excessive or irregular adjustments that may result in a model that cannot be manufactured or assembled. The rule library provides a systematic framework that makes the parameter adjustment process more standardized and automated.

[0233] In this embodiment, the rule library includes three main types of rules: geometric constraint rules, engineering constraint rules, and assembly constraint rules. These rules constrain the adjustable range, adjustment method, and mutual relationship of parameters based on factors such as the physical properties of the product, design requirements, and manufacturing processes. For example, geometric constraint rules may involve geometric limitations such as the smoothness of object surfaces and angles; engineering constraint rules involve material strength and load limitations; and assembly constraint rules cover the assembly sequence and compatibility between components.

[0234] S6.3: According to the rule engine of the parameter adjustment rule library, identify the direct and indirect associated parameters, match the corresponding rule subsets, and analyze the adjustment direction and amplitude;

[0235] Specifically, the adjustment of target parameters is not just a simple linear process, but usually involves multiple factors and constraints. By using a rule engine, parameters can be efficiently and automatically matched with relevant rules to ensure that each parameter adjustment meets design and process requirements. The rule engine can select the most suitable rule subset based on the characteristics of each parameter, thereby adjusting under the premise of meeting design requirements. In this way, the rule engine greatly simplifies the adjustment process and improves automation and precision.

[0236] In this embodiment, the core role of the rule engine is to select a rule subset that matches the type, constraint relationship, and actual requirements of each parameter, and calculate the direction and amplitude of target parameter adjustment based on these rules. The rule engine associates parameters with applicable rules through logical judgment, condition matching, and other methods to determine the adjustment method.

[0237] S6.4: Adjust the target parameters according to the adjustment direction and amplitude to generate a parameter combination;

[0238] S6.5: Reverse deduce the parameter combination to the inferred parameters, correct the response coefficients of the inferred parameters, and modify the bidirectional association matrix according to the response coefficients.

[0239] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that variations, modifications, substitutions and changes can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A product replication software system for importing and converting 3D models, applied to product replication software, characterized in that, The product replication software system includes an image acquisition module, a feature parameter extraction module, a model modeling module, and a model conversion module, wherein: The image acquisition module is used to acquire the main view image and multiple secondary view images of the product to be replicated; The feature parameter extraction module is used to extract the first feature parameter, and to perform shadow region segmentation on the front view image and the sub-view image to extract the second feature parameter; The modeling module is used to import the first feature parameters and the second feature parameters into the product replication software to generate a first model of the product to be replicated, wherein the first model generates inference parameters based on the first feature parameters and the second feature parameters; The model conversion module is used to convert the first model according to the input target parameters to generate a second model; The feature parameter extraction module includes a first feature extraction unit and a second feature extraction unit, wherein: The first feature extraction unit is used to perform image processing on the front view image and the sub-view image, extract edge information, contour lines and key points of the front view image and the sub-view image, and use the edge information, contour lines and key points as the first feature parameters; The second feature extraction unit is used to infer the feature parameters of the remaining region based on the shadowed parts of the front view image and the secondary view image; The second feature extraction unit includes a shadow extraction subunit, a shadow correction subunit, a shadow parameter calculation subunit, a shadow inference subunit, and a shadow inversion subunit, wherein: The shadow extraction subunit is used to extract the shadow edges of the main view image and the secondary view image; The shadow correction subunit is used to correct the shadow edge of the main view image based on the shadow edge of the secondary view image, thereby generating a bottom shadow area. The shadow parameter calculation subunit is used to calculate the curvature tensor of the bottom shadow region, and obtain the lateral deformation parameters and longitudinal deformation parameters based on the curvature tensor, wherein the curvature tensor is used to characterize the curvature change characteristics of the shadow region; The shadow inference subunit is used to perform joint inference on the lateral deformation parameters and the longitudinal deformation parameters to generate shadow point pairs that satisfy the differential homeomorphism constraint and the viewpoint energy function. The shadow inversion subunit is used to perform inverse inversion based on the shadow point pairs through surface fitting to extract the second feature parameters; The model conversion module includes a parameter mapping unit, a parameter adjustment unit, and a model conversion unit, wherein: The parameter mapping unit is used to obtain the input target parameters and construct a bidirectional mapping between the target parameters and the inferred parameters; The parameter adjustment unit is used to adjust the target parameters according to the bidirectional mapping and preset constraints to generate parameter combinations. The model conversion unit is used to convert the first model according to the parameter combination to generate a second model.

2. The product replication software system for importing and converting three-dimensional models according to claim 1, characterized in that, The shadow extraction subunit includes: Camera calibration is performed on the front view image and the back view image to obtain the intrinsic parameter matrix and the extrinsic parameter matrix. The epipolar geometric relationship between the two viewpoints is established based on the intrinsic parameter matrix and the extrinsic parameter matrix. Detecting shadow edges in the main view image using the epipolar equation; Based on the epipolar geometry, edge points with the same gradient direction as the shadow edge of the main image are extracted within the epipolar range corresponding to the sub-view image to generate the shadow edge of the sub-view image.

3. The product replication software system for importing and converting three-dimensional models according to claim 1, characterized in that, The shadow correction subunit includes: Calculate the gradient structure tensor of the shadow edge of the front view image to generate an edge saliency map; The edge saliency map is processed by a region growing algorithm to obtain a first shadow, wherein the region growing algorithm includes starting from the centroid of the edge saliency map and gradually merging adjacent pixels that conform to the centroid structure tensor direction consistency; Calculate the epipolar projection matching rate of the first shadow in the shadow edge of the sub-view image; If the epipolar projection matching rate is less than the preset matching threshold, Gaussian blur is applied to the edge saliency map and region growing is performed again until the epipolar projection matching rate is greater than or equal to the preset matching threshold. The first shadow is completed based on the matching result, and the completed first shadow is output as the bottom shadow area.

4. The product replication software system for importing and converting three-dimensional models according to claim 1, characterized in that, The shadow inference subunit includes: By using the differential homeomorphic constraint condition, a differential homeomorphic mapping relationship between the transverse deformation parameter and the longitudinal deformation parameter is established, wherein the differential homeomorphic constraint condition indicates that the shadow edge deformation of the same three-dimensional surface point in the transverse and longitudinal directions satisfies the differential homeomorphic transformation. A cross-view deformation parameter correlation matrix is ​​constructed based on the lateral deformation parameters and the longitudinal deformation parameters, and the view energy function is calculated based on the matrix characteristics of the cross-view deformation parameter correlation matrix. Based on the differential homeomorphism mapping relationship and the view energy function, the shadow point pairs that satisfy the dual-view deformation consistency are solved by the least squares method.

5. The product replication software system for importing and converting three-dimensional models according to claim 1, characterized in that, The parameter mapping unit includes: Parse the parameter topology tree of the first model to obtain the process constraint chain of the inferred parameters; Based on the process constraint chain, establish an initial association mapping table between the target parameters and the inferred parameters; Generate a bidirectional correlation matrix, where the row vectors of the bidirectional correlation matrix represent the modification direction of the target parameter, the column vectors of the bidirectional correlation matrix represent the response coefficients of the inferred parameter, and the matrix element values ​​are jointly calibrated by the geometric constraint relationship between parameters and historical modification data; The initial association mapping table is optimized based on the bidirectional association matrix to generate a bidirectional mapping.

6. The product replication software system for importing and converting three-dimensional models according to claim 5, characterized in that, The parameter adjustment unit includes: Based on the parameter topology tree, the direct and indirect related parameters of the target parameter are identified by a breadth-first traversal algorithm. Call the preset parameter adjustment rule library, wherein the parameter adjustment rules include geometric constraint rules, engineering constraint rules and assembly constraint rules; The rule engine of the rule base is adjusted according to the parameters to identify the directly related parameters and indirectly related parameters, match the corresponding rule subsets, and parse out the adjustment direction and magnitude. The target parameters are adjusted according to the adjustment direction and magnitude to generate parameter combinations; The parameter combination is reverse-engineered to the inferred parameters, the response coefficients of the inferred parameters are corrected, and the bidirectional correlation matrix is ​​modified based on the response coefficients.

7. A method for importing and converting 3D models for product replication, implemented based on a 3D model importing and converting product replication software system according to any one of claims 1-6, and applied to product replication software, characterized in that... The method includes: Obtain the front view image and the secondary view image of the product to be replicated, and extract the first feature parameter; The shadow regions of the front view image and the secondary view image are segmented, and the second feature parameters are extracted. The first feature parameter and the second feature parameter are imported into the product replication software to generate a first model of the product to be replicated, wherein the first model generates inference parameters based on the first feature parameter and the second feature parameter; Obtain the input target parameters and construct a bidirectional mapping between the target parameters and the inferred parameters; Based on the bidirectional mapping and preset constraints, the target parameters are adjusted to generate parameter combinations; The first model is transformed based on the parameter combination to generate the second model.

Images