A 3D model reconstruction method for remanufacturing blanks based on deep learning

Abstract

The present application provides a method for reconstructing a three-dimensional model of a remanufactured blank based on deep learning, the method comprising: collecting images of the remanufactured blank according to pre-set acquisition parameters; generating sparse point cloud data of the remanufactured blank based on the images; extracting features of the remanufactured blank for each image to obtain feature images of the remanufactured blank of different sizes; generating feature-enhanced point cloud data based on the sparse point cloud data and feature images, and optimizing the feature-enhanced point cloud data using a deep learning network to obtain the optimal feature-enhanced point cloud data; generating a three-dimensional model of the remanufactured blank based on the optimal feature-enhanced point cloud data. The method achieves the improvement of the three-dimensional reconstruction accuracy and reconstruction efficiency of the remanufactured blank by pre-setting acquisition parameters and optimizing the feature-enhanced point cloud data using a deep learning network.

Description

Technical Field

[0001] The present application belongs to the field of additive remanufacturing technology, and in particular, relates to a three-dimensional model reconstruction method of a remanufactured blank based on deep learning. Background Art

[0002] Remanufacturing is a method of repairing remanufactured blanks through high-tech to restore their performance to or even exceed the performance of new parts. It is one of the effective ways to achieve carbon neutrality and carbon peak. Additive remanufacturing mainly includes multiple steps such as remanufacturing evaluation of damaged blanks, damaged area treatment, construction of three-dimensional digital models of remanufactured blanks, acquisition of damaged area models, additive remanufacturing, etc. Among them, the construction of three-dimensional models of remanufactured blanks is the key link in the entire additive remanufacturing process. Remanufactured blanks have the characteristics of complex shapes of damaged areas and are difficult to measure. Image-based three-dimensional reconstruction of remanufactured blanks has the advantages of low cost and high efficiency, but there is still room for further improvement in reconstruction accuracy and efficiency. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a three-dimensional model reconstruction method for remanufactured blanks based on deep learning, and to improve the three-dimensional reconstruction accuracy and reconstruction efficiency of the remanufactured blanks by improving the image acquisition method and three-dimensional reconstruction method of the remanufactured blanks.

[0004] The present application provides a method for reconstructing a three-dimensional model of a remanufactured blank based on deep learning, characterized in that the three-dimensional model reconstruction method comprises:

[0005] S1. According to pre-set acquisition parameters, an image of the remanufactured blank is acquired; wherein the acquisition parameters include: image quality, number of images, shooting distance, and number of viewing angles;

[0006] S2. generating sparse point cloud data of the remanufactured blank based on the image;

[0007] S3, for each of the images, extracting the features of the remanufactured blank to obtain feature images of the remanufactured blanks of different sizes;

[0008] S4, generating feature-enhanced point cloud data based on the sparse point cloud data and the feature image, and optimizing the feature-enhanced point cloud data using a deep learning network to obtain the optimal feature-enhanced point cloud data;

[0009] S5. Generate a three-dimensional model of the remanufactured blank based on the optimal feature-enhanced point cloud data.

[0010] Furthermore, the generating of feature-enhanced point cloud data based on the sparse point cloud data and the feature image, and optimizing the feature-enhanced point cloud data using a deep learning network to obtain the optimal feature-enhanced point cloud data includes:

[0011] S4a, based on the sparse point cloud data, using the feature image of a specific size, performing rough depth prediction to generate an initial depth map;

[0012] S4b, based on the initial depth map, extracting three-dimensional coordinate points, and generating hypothetical points corresponding to the three-dimensional coordinate points and probability scalars corresponding to the hypothetical points;

[0013] S4c, for each of the hypothetical points, extracting the two-dimensional features of the hypothetical point from the feature image to generate the feature enhanced point cloud data;

[0014] S4d, generating a depth residual map based on the probability scalar and the feature enhanced point cloud data;

[0015] S4e, using the depth residual map to optimize the initial depth map, and repeating steps S4b-S4e until the optimization condition is met, generating the optimal initial depth map, so as to obtain the optimal feature enhanced point cloud data.

[0016] Furthermore, the extracting of three-dimensional coordinate points based on the initial depth map, and generating hypothetical points corresponding to the three-dimensional coordinate points and probability scalars corresponding to the hypothetical points include:

[0017] S4b1, based on the initial depth map, generating spatial point cloud data by non-projection method, and extracting three-dimensional coordinate points in the spatial point cloud data;

[0018] S4b2, identifying the three-dimensional coordinate point and at least one neighboring point of the three-dimensional coordinate point from the feature image of a specific size;

[0019] S4b3, performing edge convolution operations on the three-dimensional coordinate point and the neighboring points respectively to obtain at least one local structure information of the three-dimensional coordinate point;

[0020] S4b4, combining the local structure information to obtain the local structure features of the three-dimensional coordinate point;

[0021] S4b5. Use a hypothesis point prediction network to transform the local structural features of the three-dimensional coordinate point to obtain the hypothesis point corresponding to the three-dimensional coordinate point and the probability scalar corresponding to the hypothesis point.

[0022] Furthermore, the hypothesis point prediction network is obtained in advance in the following manner:

[0023] Acquiring an image of a test blank to obtain an initial depth map and a feature image of the test blank;

[0024] Based on the initial depth map of the test blank, generating spatial point cloud data of the test blank in a non-projection manner, and extracting three-dimensional coordinate points of the test blank;

[0025] For each three-dimensional coordinate point of the test blank, based on the characteristic image of the test blank, generate a hypothetical point corresponding to the three-dimensional coordinate point of the test blank;

[0026] Generate local structural features of the three-dimensional coordinate points of the test blank according to the three-dimensional coordinate points of the test blank and the hypothetical points corresponding to the three-dimensional coordinate points;

[0027] The local structural features of the three-dimensional coordinate points of the test blank are used as the input layer, and the hypothetical points corresponding to the three-dimensional coordinate points of the test blank are used as the output layer. The multi-layer shared perceptron is trained to generate the hypothetical point prediction network in which the local structural features of the three-dimensional coordinate points and the hypothetical points corresponding to the three-dimensional coordinate points have a mapping relationship.

[0028] Furthermore, for each three-dimensional coordinate point of the test blank, based on the characteristic image of the test blank, generating a hypothetical point corresponding to the three-dimensional coordinate point of the test blank includes:

[0029] Projecting the feature image of the test blank of a specific size into the attribute space to obtain a feature projection image of the test blank;

[0030] For each three-dimensional coordinate point of the test blank, based on the feature projection image of the test blank, determine the displacement depth of the three-dimensional coordinate point of the test blank;

[0031] A plurality of hypothetical points corresponding to the three-dimensional coordinate points of the test blank having displacement depth characteristics are generated along the direction of the acquisition device; wherein the hypothetical points of the test blank have the same depth direction as the three-dimensional coordinate points of the test blank.

[0032] Furthermore, the generating of the local structural features of the three-dimensional coordinate points of the test blank according to the three-dimensional coordinate points of the test blank and the hypothetical points corresponding to the three-dimensional coordinate points includes:

[0033] Constructing a directed edge between each hypothetical point corresponding to the three-dimensional coordinate point of the test blank and the three-dimensional coordinate point of the test blank;

[0034] The three-dimensional coordinate points of the test blank are respectively subjected to edge convolution operations with the hypothetical points corresponding to the three-dimensional coordinate points of the test blank to generate local structural features of the three-dimensional coordinate points of the test blank.

[0035] Furthermore, for each of the hypothetical points, extracting the two-dimensional features of the hypothetical point from the feature image to generate the feature-enhanced point cloud data includes:

[0036] S4c1: for each of the hypothetical points, performing variance calculation on the two-dimensional features of the hypothetical point identified from the feature images of the same size, to obtain two-dimensional variance features of the hypothetical points of different sizes;

[0037] S4c2: for each of the hypothetical points, fusing the two-dimensional variance features of different sizes to obtain the two-dimensional features of the hypothetical point;

[0038] S4c3: Combining the two-dimensional features with the spatial position features of the hypothetical points to generate feature-enhanced point cloud data.

[0039] Furthermore, generating a depth residual map based on the probability scalar and the feature enhanced point cloud data includes:

[0040] For each of the hypothetical points, a displacement depth feature of the hypothetical point is identified to obtain a depth displacement of the hypothetical point, and a product of the depth displacement and the probability scalar is calculated to obtain an offset depth of the hypothetical point;

[0041] Performing weighted sum calculation on the offset depth of each of the hypothetical points to obtain the displacement of the three-dimensional coordinate point;

[0042] The displacement of the three-dimensional coordinate point is projected onto the initial depth map to obtain a depth residual map.

[0043] Furthermore, the preset acquisition parameters are determined in the following manner:

[0044] Based on the acquisition parameters, an orthogonal table with 4 factors and 3 levels was constructed using an orthogonal test method;

[0045] Based on the orthogonal table, 9 groups of factors and levels are selected to set acquisition parameters to acquire images of the test blank, and a three-dimensional model reconstruction test is performed to obtain test results;

[0046] Based on the test results, three evaluation indicators are selected to evaluate the nine groups of three-dimensional model reconstruction tests to obtain evaluation results;

[0047] The combination of factors and levels corresponding to the optimal evaluation result is determined as a preset acquisition parameter.

[0048] The deep learning-based three-dimensional model reconstruction method of the remanufactured blank provided in the present application improves the image acquisition method and the three-dimensional reconstruction method of the remanufactured blank, thereby improving the three-dimensional reconstruction accuracy and reconstruction efficiency of the remanufactured blank. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of a method for reconstructing a three-dimensional model of a remanufactured blank based on deep learning provided in an embodiment of the present application is shown;

[0050] Figure 2 A reference schematic diagram showing the position of a collection device when collecting an image of a test blank according to preset collection parameters provided in an embodiment of the present application is shown;

[0051] Figure 3 A schematic diagram showing an image of a test blank collected according to preset collection parameters provided in an embodiment of the present application;

[0052] Figure 4 A comparison diagram of the three-dimensional model generated by the actual shooting of the guide shoe, MVSNet reconstruction, COLMAP reconstruction and the reconstruction of the present invention provided in the embodiment of the present application is shown;

[0053] Figure 5 A comparison chart of the cam bearing cover real shot, MVSNet reconstruction, COLMAP reconstruction and the three-dimensional model generated by the reconstruction of the present invention provided in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the technical solution more clear, the technical solution is further described in detail below in conjunction with specific implementation methods. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the technical solution.

[0055] See also Figure 1 , Figure 1 A flowchart of a method for reconstructing a three-dimensional model of a remanufactured blank based on deep learning provided in an embodiment of the present application. Figure 1 As shown, a three-dimensional model reconstruction method of a remanufactured blank based on deep learning includes:

[0056] S101. Collect images of remanufactured blanks according to preset acquisition parameters; wherein the acquisition parameters include: image quality, number of images, shooting distance, and number of viewing angles.

[0057] In this step, the applicant has found through research that the acquisition parameters: image quality, number of images, shooting distance and number of viewing angles are factors that affect the accuracy of three-dimensional model reconstruction of the remanufactured blank. Therefore, the embodiment of the present application acquires images of the remanufactured blank through pre-set acquisition parameters to improve the accuracy of three-dimensional model reconstruction.

[0058] In specific implementation, the preset acquisition parameters can be determined in the following ways:

[0059] Step 1011: Based on the acquisition parameters, an orthogonal table of 4 factors and 3 levels is constructed using an orthogonal test method.

[0060] In this step, the orthogonal experiment method is a design method for studying multiple factors and multiple levels. Four acquisition parameters: image quality, number of images, shooting distance and number of viewing angles are used as factors of the orthogonal experiment. Different values ​​are set for each factor, that is, 3 levels are selected to construct an orthogonal table of 4 factors and 3 levels.

[0061] As an example, in order to make the set acquisition parameters have higher accuracy, the embodiment of the present application uses the guide shoe and the camshaft cover as test blanks for orthogonal experiments, that is, an orthogonal table with 4 factors and 3 levels is constructed for the guide shoe and the camshaft cover respectively.

[0062] Step 1012: Based on the orthogonal table, 9 groups of factor and level combinations are selected to set acquisition parameters to acquire images of the test blanks, and three-dimensional model reconstruction tests are performed to obtain test results.

[0063] In this step, different combinations of factors and levels can be obtained based on the orthogonal table, and acquisition parameters are set according to different combinations. Images of the test blanks are collected according to 9 groups of different acquisition parameters, and 9 groups of 3D model reconstruction tests are carried out to obtain test results. The test results here are the enhanced point cloud data obtained through the 3D model reconstruction test.

[0064] As an example, based on the orthogonal table of the guide shoe, the embodiment of the present application selects 9 groups of factors and levels to set the acquisition parameters to acquire the image of the guide shoe, and respectively performs 9 groups of 3D model reconstruction tests on the guide shoe to obtain the test results of the guide shoe. For the camshaft cover, the same operation as above is performed to obtain the test results of 9 groups of camshaft covers.

[0065] Step 1013: Based on the test results, three evaluation indicators are selected to evaluate the nine groups of three-dimensional model reconstruction tests to obtain evaluation results.

[0066] In this step, the test results are analyzed by evaluation indicators to evaluate the 3D model reconstruction test. The three evaluation indicators are: accuracy of test results, completeness of reconstruction and reconstruction efficiency. Furthermore, corresponding weights are assigned to the above evaluation indicators respectively, and the evaluation results of each group of 3D model reconstruction tests are obtained using the following formula (1):

[0067]

[0068] In the formula, n is the number of indicators, m is the number of experts, and A ij is the score of the i-th expert on the j-th indicator of the evaluation object, w j is the weight of the j indicator item, and Where 0<A i ≤1.

[0069] As an example, the present embodiment of the present invention selects the above three evaluation indicators based on the test results of the guide shoe, and assigns 35%, 35% and 30% weights to the above evaluation indicators: accuracy of the test results, completeness of reconstruction and reconstruction efficiency, respectively. The above formula (1) is used to evaluate the three-dimensional model reconstruction test of 9 groups of guide shoes, and the evaluation results of the three-dimensional model reconstruction test of 9 groups of guide shoes are obtained. For the camshaft cover, the same operation as above is performed to obtain the evaluation results of the three-dimensional model reconstruction test of 9 groups of camshaft covers.

[0070] Step 1014: Determine the combination of factors and levels corresponding to the optimal evaluation result as a pre-set acquisition parameter.

[0071] In this step, the best evaluation result is selected from the evaluation results of the nine groups of three-dimensional model reconstruction tests, and the combination of factors and levels corresponding to the best evaluation result is determined as the pre-set acquisition parameters.

[0072] As an example, the acquisition parameters preset in the embodiment of the present application are: low image quality (resolution 3072×3072, image size 500KB), 48 images, shooting distance 15cm, and 2 viewing angles. Figure 2 and Figure 3 , Figure 2 A reference schematic diagram of the position of the acquisition device when acquiring an image of a test blank according to pre-set acquisition parameters provided in an embodiment of the present application, wherein: Figure 2 The test blank on the left is a guide shoe, and the test blank on the right is a camshaft cover; Figure 3 A schematic diagram of an image of a test blank collected according to pre-set collection parameters provided in an embodiment of the present application, wherein the test blank is a guide shoe. Figure 2 At the shooting distance of 15 cm, the following images were collected from two viewing angles: Figure 348 images shown.

[0073] S102 . For each of the images, extract the features of the remanufactured blank to obtain feature images of the remanufactured blank of different sizes.

[0074] In this step, for each image, three levels of feature extraction are performed, each level contains multiple feature images of the same size, and the feature images between levels have different sizes.

[0075] S103: Generate sparse point cloud data of the remanufactured blank based on the image.

[0076] In this step, for each image, feature points of the image are extracted, feature matching is performed on the image based on the feature points, and sparse point cloud data of the remanufactured blank is generated.

[0077] S104: Generate feature-enhanced point cloud data based on the sparse point cloud data and the feature image, and optimize the feature-enhanced point cloud data using a deep learning network to obtain optimal feature-enhanced point cloud data.

[0078] In specific implementation, feature-enhanced point cloud data can be generated based on sparse point cloud data and feature images in the following ways:

[0079] Step 1041: Based on the sparse point cloud data, use the feature image of a specific size to perform rough depth prediction to generate an initial depth map.

[0080] In this step, a feature image with a size of one eighth of the remanufactured blank image is used to perform rough depth prediction and generate an initial depth map.

[0081] Step 1042: extract three-dimensional coordinate points based on the initial depth map, and generate hypothetical points corresponding to the three-dimensional coordinate points and probability scalars corresponding to the hypothetical points.

[0082] In specific implementation, the hypothesis point corresponding to the three-dimensional coordinate point and the probability scalar corresponding to the hypothesis point can be generated in the following way:

[0083] Step 201: Generate spatial point cloud data in a non-projection manner based on the initial depth map, and extract three-dimensional coordinate points in the spatial point cloud data.

[0084] In this step, the spatial point cloud data can be expressed as: X = [X1, X2, ..., X n ], where n is the number of three-dimensional coordinate points in the spatial point cloud, and each three-dimensional coordinate point X p The spatial position feature can be expressed as X p =[x, y, z].

[0085] Step 202: Identify the three-dimensional coordinate point and at least one neighboring point of the three-dimensional coordinate point from the feature image of a specific size.

[0086] In this step, the three-dimensional coordinate points are identified by using a feature image of a specific size in order to reduce the amount of calculation. Specifically, multiple three-dimensional coordinate points X are identified from the feature image of a specific size. p , for each 3D coordinate point, identify at least one neighboring point X of the 3D coordinate point q .

[0087] Step 203: Perform edge convolution operations on the three-dimensional coordinate point and the neighboring points respectively to obtain at least one local structure information of the three-dimensional coordinate point.

[0088] Step 204: Combine the local structure information to obtain the local structure features of the three-dimensional coordinate points.

[0089] Step 205: Use a hypothesis point prediction network to transform the local structural features of the three-dimensional coordinate point to obtain a hypothesis point corresponding to the three-dimensional coordinate point and a probability scalar corresponding to the hypothesis point.

[0090] In specific implementation, the hypothesis point prediction network can be obtained in advance by the following methods:

[0091] Step 2051: Capture an image of the test blank to obtain an initial depth map and a feature image of the test blank.

[0092] Step 2052: Based on the initial depth map of the test blank, generate spatial point cloud data of the test blank in a non-projection manner, and extract the three-dimensional coordinate points of the test blank.

[0093] Step 2053: for each three-dimensional coordinate point of the test blank, generate a hypothetical point corresponding to the three-dimensional coordinate point of the test blank based on the feature image of the test blank.

[0094] In specific implementation, the hypothetical points corresponding to the three-dimensional coordinate points of the test blank can be generated in the following manner:

[0095] Step 301: Project the feature image of the test blank of a specific size into the attribute space to obtain the feature projection image of the test blank.

[0096] Step 302: for each three-dimensional coordinate point of the test blank, determine the displacement depth of the three-dimensional coordinate point of the test blank based on the feature projection image of the test blank.

[0097] Step 303, generating a plurality of hypothetical points corresponding to the three-dimensional coordinate points of the test blank having displacement depth characteristics along the direction of the acquisition device; wherein the hypothetical points of the test blank have the same depth direction as the three-dimensional coordinate points of the test blank.

[0098] In this step, each 3D coordinate point corresponds to 2m hypothetical points. The relationship between the 3D coordinate point and the hypothetical point is shown in Figure (2):

[0099]

[0100] In the formula, is the assumed point set, X p is a three-dimensional coordinate point, t is the direction of the acquisition device, and s represents the displacement depth.

[0101] Step 2054: Generate local structural features of the three-dimensional coordinate points of the test blank according to the three-dimensional coordinate points of the test blank and the hypothetical points corresponding to the three-dimensional coordinate points.

[0102] In specific implementation, the local structural features of the test blank can be generated in the following ways:

[0103] Step 401: construct a directed edge between each hypothetical point corresponding to the three-dimensional coordinate point of the test blank and the three-dimensional coordinate point of the test blank.

[0104] Step 402: Perform edge convolution operations on the three-dimensional coordinate points of the test blank and the hypothetical points corresponding to the three-dimensional coordinate points of the test blank to generate local structural features of the three-dimensional coordinate points of the test blank.

[0105] Step 2055: Use the local structural features of the three-dimensional coordinate points of the test blank as the input layer, and use the hypothetical points corresponding to the three-dimensional coordinate points of the test blank as the output layer, and train the multi-layer shared perceptron to generate the hypothetical point prediction network in which the local structural features of the three-dimensional coordinate points and the hypothetical points corresponding to the three-dimensional coordinate points have a mapping relationship.

[0106] Step 1043: For each of the hypothetical points, extract the two-dimensional features of the hypothetical point from the feature image to generate the feature-enhanced point cloud data.

[0107] In specific implementation, the two-dimensional features of the hypothetical points can be extracted from the feature image in the following manner to generate feature enhanced point cloud data:

[0108] Step 501: For each of the hypothetical points, variance calculation is performed on the two-dimensional features of the hypothetical point identified from the feature images of the same size to obtain the two-dimensional variance features of the hypothetical points of different sizes.

[0109] Step 502: For each of the hypothetical points, the two-dimensional variance features of different sizes are fused to obtain the two-dimensional features of the hypothetical point.

[0110] Step 503: Combine the two-dimensional features with the spatial position features of the hypothetical points to generate feature-enhanced point cloud data.

[0111] As an example, the following formula (3) is used to express the assumed point X p The two-dimensional variance feature C p :

[0112]

[0113] In the formula, is the two-dimensional feature on feature image i, and N is the variance measure of the view.

[0114] Among them, the embodiment of the present application selects three levels of two-dimensional features to obtain the two-dimensional features of the assumed point, then, the assumed point X p The two-dimensional features at the jth layer are expressed by the following formula (4):

[0115]

[0116] Where C is the variance loss metric, j is the pyramid feature of different sizes, and j = 1, 2, 3.

[0117] Furthermore, the two-dimensional variance features C under different sizes (i.e. different levels j) are p Fusion is performed to obtain the two-dimensional features of the hypothetical point The two-dimensional features are transformed into And the spatial position feature X of the hypothesis point p Combine to get feature enhanced point cloud

[0118]

[0119] In specific implementation, the feature-enhanced point cloud data can be optimized using a deep learning network in the following ways to obtain the optimal feature-enhanced point cloud data:

[0120] Step 1044: Generate a depth residual map based on the probability scalar and the feature enhanced point cloud data.

[0121] In specific implementation, the depth residual map can be generated by:

[0122] Step 601: for each of the hypothetical points, identify the displacement depth feature of the hypothetical point, obtain the depth displacement of the hypothetical point, and calculate the product of the depth displacement and the probability scalar to obtain the offset depth of the hypothetical point.

[0123] In this step, the offset depth of the hypothetical point is calculated using the following formula (6):

[0124]

[0125] Where E(ks) is the migration depth, is the set of assumed points, is a probability scalar.

[0126] Step 602: Perform weighted sum calculation on the offset depth of each of the hypothetical points to obtain the displacement of the three-dimensional coordinate point.

[0127] Step 603: Project the displacement of the three-dimensional coordinate point onto the initial depth map to obtain a depth residual map.

[0128] Step 1045: Use the depth residual map to optimize the initial depth map, and repeat steps S1042-S1045 until the optimization condition is met, thereby generating the optimal initial depth map to obtain the optimal feature-enhanced point cloud data.

[0129] In this step, the depth residual map is added to the initial depth map to optimize the initial depth map. Further, the optimization condition is to determine whether the optimized initial depth map satisfies the loss function. If the loss function is satisfied, the optimal initial depth map is generated. If the loss function is not satisfied, steps S1042-S1045 are repeated until the optimization condition is satisfied. Here, the loss function is obtained by calculating the absolute error between the optimized initial depth map and the true depth map.

[0130] S105 . Generate a three-dimensional model of the remanufactured blank based on the optimal feature-enhanced point cloud data.

[0131] In this step, a 3D model of the remanufactured blank is generated based on the feature-enhanced point cloud data. Figure 4 and Figure 5 , Figure 4 A comparison diagram of the three-dimensional model generated by the actual shooting of the guide shoe, MVSNet reconstruction, COLMAP reconstruction and the reconstruction of the present invention provided in the embodiment of the present application is shown; Figure 5 A comparison chart of the cam bearing cover real shot, MVSNet reconstruction, COLMAP reconstruction and the 3D model generated by the reconstruction of the present invention is shown. Figure 4 and 5It can be seen that the three-dimensional model of the remanufactured blank generated by the present application has higher accuracy and is closer to the actual image.

[0132] The above contents are only preferred embodiments of the present invention. For ordinary technicians in this field, many changes can be made in the specific implementation methods and application scopes based on the ideas of the present technical content. As long as these changes do not deviate from the concept of the present invention, they all fall within the scope of protection of this patent.

Claims

1. A method for reconstructing a three-dimensional model of a remanufactured blank based on deep learning, characterized in that: The three-dimensional model reconstruction method comprises: S1. According to pre-set acquisition parameters, an image of the remanufactured blank is acquired; wherein the acquisition parameters include: image quality, number of images, shooting distance, and number of viewing angles; S2. generating sparse point cloud data of the remanufactured blank based on the image; S3, for each of the images, extracting the features of the remanufactured blank to obtain feature images of the remanufactured blanks of different sizes; S4, generating feature-enhanced point cloud data based on the sparse point cloud data and the feature image, and optimizing the feature-enhanced point cloud data using a deep learning network to obtain the optimal feature-enhanced point cloud data; S5. generating a three-dimensional model of the remanufactured blank based on the optimal feature-enhanced point cloud data; The generating feature-enhanced point cloud data based on the sparse point cloud data and the feature image, and optimizing the feature-enhanced point cloud data using a deep learning network to obtain the optimal feature-enhanced point cloud data, includes: S4a, based on the sparse point cloud data, using the feature image of a specific size, performing rough depth prediction to generate an initial depth map; S4b, based on the initial depth map, extracting three-dimensional coordinate points, and generating hypothetical points corresponding to the three-dimensional coordinate points and probability scalars corresponding to the hypothetical points; S4c, for each of the hypothetical points, extracting the two-dimensional features of the hypothetical point from the feature image to generate the feature enhanced point cloud data; S4d, generating a depth residual map based on the probability scalar and the feature enhanced point cloud data; S4e, optimizing the initial depth map using the depth residual map, and repeating steps S4b-S4e until the optimization condition is met, generating the optimal initial depth map, and obtaining the optimal feature-enhanced point cloud data.

2. The method according to claim 1, characterized in that The extracting of three-dimensional coordinate points based on the initial depth map, and generating hypothetical points corresponding to the three-dimensional coordinate points and probability scalars corresponding to the hypothetical points include: S4b1, based on the initial depth map, generating spatial point cloud data in a non-projection manner, and extracting three-dimensional coordinate points in the spatial point cloud data; S4b2, identifying the three-dimensional coordinate point and at least one neighboring point of the three-dimensional coordinate point from the feature image of a specific size; S4b3, performing edge convolution operations on the three-dimensional coordinate point and the neighboring points respectively to obtain at least one local structure information of the three-dimensional coordinate point; S4b4, combining the local structure information to obtain the local structure features of the three-dimensional coordinate point; S4b5. Use a hypothesis point prediction network to transform the local structural features of the three-dimensional coordinate point to obtain the hypothesis point corresponding to the three-dimensional coordinate point and the probability scalar corresponding to the hypothesis point.

3. The method according to claim 2, characterized in that The hypothesis point prediction network is obtained in advance in the following way: Acquiring an image of a test blank to obtain an initial depth map and a feature image of the test blank; Based on the initial depth map of the test blank, generating spatial point cloud data of the test blank in a non-projection manner, and extracting three-dimensional coordinate points of the test blank; For each three-dimensional coordinate point of the test blank, based on the characteristic image of the test blank, generate a hypothetical point corresponding to the three-dimensional coordinate point of the test blank; Generate local structural features of the three-dimensional coordinate points of the test blank according to the three-dimensional coordinate points of the test blank and the hypothetical points corresponding to the three-dimensional coordinate points; The local structural features of the three-dimensional coordinate points of the test blank are used as the input layer, and the hypothetical points corresponding to the three-dimensional coordinate points of the test blank are used as the output layer. The multi-layer shared perceptron is trained to generate the hypothetical point prediction network in which the local structural features of the three-dimensional coordinate points and the hypothetical points corresponding to the three-dimensional coordinate points have a mapping relationship.

4. The method according to claim 3, characterized in that For each three-dimensional coordinate point of the test blank, generating a hypothetical point corresponding to the three-dimensional coordinate point of the test blank based on the characteristic image of the test blank includes: Projecting the feature image of the test blank of a specific size into the attribute space to obtain a feature projection image of the test blank; For each three-dimensional coordinate point of the test blank, based on the feature projection image of the test blank, determine the displacement depth of the three-dimensional coordinate point of the test blank; A plurality of hypothetical points corresponding to the three-dimensional coordinate points of the test blank having displacement depth characteristics are generated along the direction of the acquisition device; wherein the hypothetical points of the test blank have the same depth direction as the three-dimensional coordinate points of the test blank.

5. The method according to claim 3, characterized in that The step of generating local structural features of the three-dimensional coordinate points of the test blank according to the three-dimensional coordinate points of the test blank and the hypothetical points corresponding to the three-dimensional coordinate points comprises: Constructing a directed edge between each hypothetical point corresponding to the three-dimensional coordinate point of the test blank and the three-dimensional coordinate point of the test blank; The three-dimensional coordinate points of the test blank are respectively subjected to edge convolution operations with the hypothetical points corresponding to the three-dimensional coordinate points of the test blank to generate local structural features of the three-dimensional coordinate points of the test blank.

6. The method according to claim 1, characterized in that For each of the hypothetical points, extracting the two-dimensional features of the hypothetical point from the feature image to generate the feature-enhanced point cloud data includes: S4c1: for each of the hypothetical points, performing variance calculation on the two-dimensional features of the hypothetical point identified from the feature images of the same size, to obtain two-dimensional variance features of the hypothetical points of different sizes; S4c2: for each of the hypothetical points, fusing the two-dimensional variance features of different sizes to obtain the two-dimensional features of the hypothetical point; S4c3: Combining the two-dimensional features with the spatial position features of the hypothetical points to generate feature-enhanced point cloud data.

7. The method according to claim 1, characterized in that The generating a depth residual map based on the probability scalar and the feature enhanced point cloud data comprises: For each of the hypothetical points, a displacement depth feature of the hypothetical point is identified to obtain a depth displacement of the hypothetical point, and a product of the depth displacement and the probability scalar is calculated to obtain an offset depth of the hypothetical point; Performing weighted sum calculation on the offset depth of each of the hypothetical points to obtain the displacement of the three-dimensional coordinate point; The displacement of the three-dimensional coordinate point is projected onto the initial depth map to obtain a depth residual map.

8. The method according to claim 1, characterized in that The pre-set acquisition parameters are determined in the following manner: Based on the acquisition parameters, an orthogonal table with 4 factors and 3 levels was constructed using an orthogonal test method; Based on the orthogonal table, 9 groups of factors and levels are selected to set acquisition parameters to acquire images of the test blank, and a three-dimensional model reconstruction test is performed to obtain test results; Based on the test results, three evaluation indicators are selected to evaluate the nine groups of three-dimensional model reconstruction tests to obtain evaluation results; The combination of factors and levels corresponding to the optimal evaluation result is determined as a preset acquisition parameter.

Images