Multi-view three-dimensional point cloud reconstruction method and device based on shape prior and medium

Abstract

The invention discloses a multi-view three-dimensional point cloud reconstruction method and device based on shape prior and a medium, and relates to the technical field of three-dimensional reconstruction, and the method comprises the steps: obtaining a plurality of images corresponding to a target range; inputting the plurality of images into a Transformers network to obtain a feature code output by a VIT encoder of the Transformers network, and obtaining a DPT feature output by the Transformers network; aligning and fusing the feature codes and the DPT features to obtain fused features, and extracting center features corresponding to the center points of the industrial parts from the fused features; performing attention mechanism processing on the fusion features and the center features to obtain target information of each industrial part; and based on the target information and the CAD model, reconstructing and obtaining a three-dimensional point cloud of each industrial part. The method and the device are used for solving the problem of poor accuracy during three-dimensional point cloud reconstruction based on multiple views in the prior art, and the reconstruction of the three-dimensional point cloud is quickly and accurately completed.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, in particular to a multi-viewing Figure 3 Method, device and medium for reconstructing three-dimensional point clouds. Background Art

[0002] Currently, most 3D point cloud reconstruction is based on multiple views. Compared to structured light, this approach offers advantages such as high cost-effectiveness and low environmental requirements. However, multi-view 3D point cloud reconstruction suffers from low image resolution. This lowers the accuracy of the 3D point cloud reconstruction, resulting in poor precision in the reconstructed 3D point cloud. Summary of the Invention

[0003] In response to the above problems and technical requirements, the applicant proposed a multi-view Figure 3 The invention discloses a method, device and medium for reconstructing three-dimensional point clouds, which are used to solve the problem of poor accuracy in the prior art when reconstructing three-dimensional point clouds based on multiple views, and to achieve rapid and accurate reconstruction of three-dimensional point clouds.

[0004] The embodiment of the present application provides a multi-view based on shape prior Figure 3 A method for reconstructing a 3D point cloud, the method comprising:

[0005] Acquire multiple images corresponding to a target range, wherein the target range includes one or more industrial parts, and the multiple images are obtained by photographing the target range at the same time using cameras located at different positions;

[0006] Input multiple images into the Transformers network to obtain feature codes output by the VIT encoder of the Transformers network and DPT features output by the Transformers network; align and fuse the feature codes and the DPT features to obtain fused features, and extract the central features corresponding to the center points of each industrial part from the fused features; process the fused features and the central features using an attention mechanism to obtain target information of each industrial part, wherein the target information includes: the position and posture of the industrial part and the scaling coefficient deformation parameters corresponding to the CAD model, wherein the CAD model is a three-dimensional model of the industrial part;

[0007] Based on the target information and the CAD model, a three-dimensional point cloud of each industrial part is reconstructed and obtained.

[0008] According to the shape prior-based multi-view Figure 3 The 3D point cloud reconstruction method extracts the central features corresponding to the center points of each industrial part from the fusion features, including:

[0009] The sigmoid function is used to process the fusion features to obtain the feature score corresponding to each block, and the first maximum feature score is obtained based on the comparison method;

[0010] The obtained feature scores will be subjected to maximum pooling processing to obtain the second largest feature score;

[0011] When it is determined that the first maximum feature score is consistent with the second maximum feature score and is greater than a preset threshold, the block corresponding to the second maximum feature score is determined as the center point, and the fusion feature corresponding to the center point is determined as the center feature.

[0012] According to the shape prior-based multi-view Figure 3 The method for reconstructing a dimensional point cloud aligns and fuses the feature code and the DPT feature to obtain a fused feature, including:

[0013] The DPT features are input into the convolution module to obtain the DPT features output by the convolution module that are aligned with the feature encoding size;

[0014] Input the aligned DPT features into MLP to obtain the DPT features with reduced dimensions output by MLP, and use the DPT features with reduced dimensions as the final DPT features;

[0015] The feature code and the final DPT feature are fused to obtain the fused feature.

[0016] According to the shape prior-based multi-view Figure 3 After reconstructing and obtaining the three-dimensional point cloud of each industrial part, the method also includes:

[0017] Get the original point clouds corresponding to multiple images;

[0018] Calculate the L1 loss and distance loss between the reconstructed 3D point cloud and the original point cloud;

[0019] When it is determined that the sum of the L1 loss and the distance damage is less than the preset loss value, it is determined that the reconstructed three-dimensional point cloud meets the standards.

[0020] According to the shape prior-based multi-view Figure 3 A three-dimensional point cloud reconstruction method, wherein different cameras correspond to different camera serial numbers, and the image carries the camera serial number code, and the camera serial number code represents the index number of the image taken by the camera;

[0021] Input multiple images into the Transformers network and obtain the DPT features output by the Transformers network, including:

[0022] Multiple images are input into the Transformers network, and the VIT encoder of the Transformers network generates a position code based on the index number, and the position code is added to the feature code of the corresponding image; the feature code with the added position code is input into multiple cross-attention modules, and the feature code of each image and the feature code of other images are cross-attention processed by the multiple cross-attention modules to obtain the processed feature code; the processed feature code is fused to obtain the DPT feature.

[0023] According to the shape prior-based multi-view Figure 3 The method for reconstructing the dimensional point cloud is to fuse the processed feature codes to obtain the DPT features, including:

[0024] A preset feature extraction layer determination method is used to determine a target cross-attention module from multiple cross-attention modules; features output by the target cross-attention module are extracted, and a fusion operation is performed on the extracted features to obtain and output the DPT features.

[0025] According to the shape prior-based multi-view Figure 3 The dimensional point cloud reconstruction method processes the fusion features and the central features using an attention mechanism to obtain target information of each industrial part, including:

[0026] Inputting the fused feature and the central feature into a plurality of cross-attention modules, performing cross-attention processing on the central feature and an image feature corresponding to the central feature by the plurality of cross-attention modules to obtain a processed central feature, wherein the image feature is obtained from the fused feature based on the central feature;

[0027] The processed central features are input into the MLP layer to obtain the target information output by the MLP layer.

[0028] According to the shape prior-based multi-view Figure 3 The 3D point cloud reconstruction method, before obtaining multiple images corresponding to the target range, also includes:

[0029] Get multiple image samples;

[0030] Input multiple image samples into the DPT model to obtain the predicted target information of each industrial part output by the DPT model, where the predicted target information includes: the predicted position and predicted posture of the industrial part and the predicted scaling coefficient deformation parameters corresponding to the CAD model. The DPT model includes a Transformers network;

[0031] Place the CAD model at the corresponding spatial position corresponding to the predicted target information, and rasterize the placed CAD model to obtain the predicted point cloud corresponding to each simulated camera;

[0032] Calculate the L1 loss and distance loss of the predicted point cloud and the point cloud samples corresponding to multiple image samples;

[0033] The network parameters of the DPT model are optimized based on the L1 loss and distance loss until the L1 loss and distance loss reach the preset loss values, and the DPT model optimization is determined to be complete.

[0034] The embodiment of the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-viewing system based on shape prior as described above is realized. Figure 3 Steps of the 3D point cloud reconstruction method.

[0035] The embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the shape prior-based multi-viewing method as described in any one of the above items. Figure 3 Steps of the 3D point cloud reconstruction method.

[0036] The multi-view based shape prior provided by the embodiment of the present application Figure 3 The invention relates to a method, device and medium for reconstructing a three-dimensional point cloud, which obtains multiple images corresponding to a target range, wherein the target range includes one or more industrial parts. The multiple images are obtained by photographing the target range at the same time by cameras located at different positions. The present application photographs the industrial parts within the target range from multiple angles to obtain a multi-view image of each industrial part, providing an effective data basis for the subsequent identification of industrial parts.

[0037] Furthermore, multiple images are input into the Transformers network to obtain the feature encoding output by the VIT encoder of the Transformers network, and the DPT features output by the Transformers network are obtained; the feature encoding and the DPT features are aligned and fused to obtain fused features, and the central features corresponding to the center points of each industrial part are extracted from the fused features; the fused features and the central features are processed by the attention mechanism to obtain the target information of each industrial part. This application extracts the central features and processes the fused features of multiple images and the extracted central features by the attention mechanism to accurately obtain the target information of each industrial part.

[0038] Finally, based on the target information and CAD model, the three-dimensional point cloud of each industrial part is reconstructed and obtained. The present invention accurately reconstructs the three-dimensional point cloud based on shape prior and precise target information, which solves the problem of poor accuracy in the existing technology when reconstructing three-dimensional point clouds based on multiple views, and realizes the rapid and accurate reconstruction of the three-dimensional point cloud of industrial parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 The embodiment of the present application provides a multi-view based on shape prior Figure 3 Flowchart of the 3D point cloud reconstruction method;

[0041] Figure 2 This is one of the structural diagrams of the model structure provided in the embodiments of the present application;

[0042] Figure 3 This is the second structural diagram of the model structure provided in the embodiment of the present application;

[0043] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] The embodiment of the present application provides a multi-view based on shape prior Figure 3 Dimensional point cloud reconstruction method. This method can be applied to smart terminals and servers. This application uses the method applied in a server as an example to illustrate. This is an example and is not used to limit the scope of protection of this application. Some other descriptions in the embodiments are also examples and will not be described one by one later. Figure 1 As shown, the method includes:

[0046] Step 101: Acquire multiple images corresponding to the target range.

[0047] The target range includes one or more industrial parts, and multiple images are obtained by shooting the target range at the same time by cameras located at different positions.

[0048] In this case, each camera captures one image at a time. The number of cameras and images captured at the same time is the same. Different cameras are assigned different camera serial numbers, and the images captured by the cameras are position-coded based on the camera serial numbers. Different images are assigned different position codes.

[0049] In step 102, multiple images are input into the Transformers network to obtain the feature encoding output by the VIT encoder of the Transformers network and the DPT features output by the Transformers network; the feature encoding and the DPT features are aligned and fused to obtain fused features, and the central features corresponding to the center points of each industrial part are extracted from the fused features; the fused features and the central features are processed using the attention mechanism to obtain the target information of each industrial part.

[0050] The target information includes: the position and posture of the industrial part and the corresponding scaling coefficient deformation parameters in the CAD model, wherein the CAD model is a three-dimensional model of the industrial part.

[0051] Step 103 : reconstruct and obtain the three-dimensional point cloud of each industrial part based on the target information and the CAD model.

[0052] Specifically, the point cloud and normal information of the CAD model have been obtained in advance. It is only necessary to combine the target information with the point cloud and normal information of the CAD model to obtain the three-dimensional point cloud of the industrial part.

[0053] The multi-view based shape prior provided by the embodiment of the present application Figure 3 A three-dimensional point cloud reconstruction method is proposed, which obtains multiple images corresponding to the target range, wherein the target range includes one or more industrial parts. The multiple images are obtained by shooting the target range at the same time with cameras located at different positions. This application performs multi-directional shooting of the industrial parts within the target range to obtain multi-view images of each industrial part, providing an effective data basis for subsequent identification of industrial parts.

[0054] Furthermore, multiple images are input into the Transformers network to obtain the feature encoding output by the VIT encoder of the Transformers network, and the DPT features output by the Transformers network are obtained; the feature encoding and the DPT features are aligned and fused to obtain fused features, and the central features corresponding to the center points of each industrial part are extracted from the fused features; the fused features and the central features are processed by the attention mechanism to obtain the target information of each industrial part. This application extracts the central features and processes the fused features of multiple images and the extracted central features by the attention mechanism to accurately obtain the target information of each industrial part.

[0055] Finally, based on the target information and CAD model, the three-dimensional point cloud of each industrial part is reconstructed and obtained. The present invention accurately reconstructs the three-dimensional point cloud based on shape prior and precise target information, which solves the problem of poor accuracy in the existing technology when reconstructing three-dimensional point clouds based on multiple views, and realizes the rapid and accurate reconstruction of the three-dimensional point cloud of industrial parts.

[0056] In a specific embodiment, the specific implementation of aligning and fusing the feature code and the DPT feature to obtain the fused feature includes:

[0057] The DPT feature is input into the convolution module to obtain the DPT feature output by the convolution module aligned with the feature encoding size; the aligned DPT feature is input into the MLP to obtain the DPT feature with reduced dimension output by the MLP, and the DPT feature with reduced dimension is used as the final DPT feature; the feature encoding and the final DPT feature are fused to obtain the fused feature.

[0058] The convolution module includes a multi-layer CNN network, and the convolution module has obtained the size of the feature encoding in advance.

[0059] Specifically, since the size of the feature code output by the VIT encoder is inconsistent with the size of the DPT feature, after obtaining the DPT feature, the size of the DPT feature is processed into a size consistent with the size of the feature code through the convolution module to achieve alignment between the feature code and the DPT feature.

[0060] In a specific embodiment, the specific implementation of extracting the central feature corresponding to the center point of each industrial part from the fused features includes:

[0061] The sigmoid function is used to process the fusion features to obtain the feature scores corresponding to each block, and the first maximum feature score is obtained based on the comparison method; the obtained feature scores are subjected to maximum pooling processing to obtain the second maximum feature score; when it is determined that the first maximum feature score and the second maximum feature score are consistent and greater than the preset threshold, the block corresponding to the second maximum feature score is determined as the center point, and the fusion feature corresponding to the center point is determined as the center feature.

[0062] Specifically, for each industrial part, the feature scores corresponding to each patch (equivalent to a block) are compared to obtain the first maximum feature; and the obtained feature scores are subjected to maximum pooling processing to obtain the second maximum feature score.

[0063] Among them, each industrial part can be identified based on the fusion features.

[0064] In a specific embodiment, different cameras correspond to different camera serial numbers, and the image carries the camera serial number code, which represents the index number of the image captured by the camera.

[0065] Among them, the position of one camera can be selected from multiple cameras to establish a reference coordinate system, and the position calibration between the cameras can be completed based on the reference coordinate system, and then the relationship between the captured images can be obtained based on the obtained calibration results.

[0066] Among them, in this application, the camera serial number code and the image index number can be simply set to be the same, that is, the camera and the image are one-to-one corresponding.

[0067] The specific implementation of inputting multiple images into the Transformers network and obtaining the DPT features output by the Transformers network includes:

[0068] Multiple images are input into the Transformers network, and the VIT encoder of the Transformers network generates a position code based on the index number, and the position code is added to the feature code of the corresponding image; the feature code with the added position code is input into multiple cross-attention modules, and the feature code of each image and the feature code of other images are cross-attention processed by multiple cross-attention modules to obtain the processed feature code; the processed feature code is fused to obtain the DPT feature.

[0069] Specifically, the VIT encoder adopts the form of absolute position encoding, and each patch has a unique encoding vector, which is generated by linear transformation and added with the generated position encoding to retain the position information.

[0070] In a specific embodiment, the processed feature codes are fused to obtain DPT features, which includes:

[0071] A preset feature extraction layer determination method is used to determine a target cross-attention module from multiple cross-attention modules; features output by the target cross-attention module are extracted, and the extracted features are fused to obtain and output DPT features.

[0072] Specifically, the feature extraction layer determination method includes: randomly selecting a preset number of cross-attention modules from multiple cross-attention modules as target cross-attention modules; selecting cross-attention modules from multiple cross-attention modules according to preset intervals as target cross-attention modules; determining the target cross-attention module from multiple cross-attention modules in the manner of 2*N / 4, 3*N / 4, N, where N represents the total number of cross-attention modules; etc.

[0073] Among them, regarding the method of determining the feature extraction layer, the user can choose according to his or her actual needs, and this application does not impose any restrictions.

[0074] In a specific embodiment, the fusion features and the center features are processed by the attention mechanism to obtain the target information of each industrial part. The specific implementation includes:

[0075] The fused features and the central features are input into multiple cross-attention modules, and the central features and the image features corresponding to the central features are cross-attention processed by multiple cross-attention modules to obtain the processed central features; the processed central features are input into the MLP layer to obtain the target information output by the MLP layer.

[0076] Among them, the image features are obtained from the fusion features based on the center features.

[0077] Specifically, the fused features have already fused the features of multiple images, so this process only performs cross-attention processing on the image features of the current image to obtain the processed central features.

[0078] In a specific embodiment, after reconstructing and obtaining the three-dimensional point cloud of each industrial part, the method further includes:

[0079] Obtain original point clouds corresponding to multiple images; calculate the L1 loss and distance loss of the reconstructed three-dimensional point cloud and the original point cloud; and determine that the reconstructed three-dimensional point cloud meets the standards when it is determined that the sum of the L1 loss and the distance loss is less than a preset loss value.

[0080] The distance loss is Chamfer Distance Loss, which is calculated by measuring the distance between two point clouds. By calculating the L1 loss and distance loss of the 3D point cloud and the original point cloud, and making the sum of these losses less than the preset loss value, we ensure consistency between the original point cloud and the reconstructed 3D point cloud, further verifying the accuracy of the 3D point cloud.

[0081] Specifically, through Figure 2 and Figure 3 The model structure of the DPT model of this application is schematically illustrated, and combined with Figure 2 and Figure 3 The model structure of the program is described in detail:

[0082] The DPT model of this application includes: an encoder part and a decoder part. The encoder part is implemented through the Transformers network and outputs DPT features. For details, see Figure 2 . Includes: VIT encoder and cross module. The cross module includes: self-attention module, multiple series cross attention modules and MLP. Among them, Figure 2 Two cross-attention modules are used as an example, and are not used to limit the number of cross-attention modules.

[0083] Specifically, multiple images are input into the Transformers network, and the feature encoding of each image is obtained through the Transformers network. The feature encoding of the current image is input into the self-attention module, and the feature encoding of other images is input into multiple series-connected cross-attention modules. The final DPT feature is obtained through cross-attention processing.

[0084] Specifically, the decoder structure can be found in Figure 3 , including: convolution module, fusion module, center point selection module and cross module (same Figure 2 ).

[0085] Specifically, the DPT feature is input into the convolution module to align the DPT feature with the feature encoding size, the aligned DPT feature and the feature encoding are input into the fusion module for feature fusion to obtain the fused feature, the fused feature is input into the center point selection module to obtain the center feature, the center feature and the image feature corresponding to the center feature are input into the cross module to obtain the target information.

[0086] In a specific embodiment, before acquiring multiple images corresponding to the target range, a Transformers network is pre-trained. The specific training process includes:

[0087] Acquire multiple image samples; input the multiple image samples into the DPT model to obtain predicted target information of each industrial part output by the DPT model; place the CAD model to a corresponding spatial position corresponding to the predicted target information, and rasterize the placed CAD model to obtain a predicted point cloud corresponding to each simulated camera; calculate the L1 loss and distance loss of the predicted point cloud and the point cloud samples corresponding to the multiple image samples; optimize the network parameters of the DPT model based on the L1 loss and distance loss until the L1 loss and distance loss reach a first preset loss value, and determine that the DPT model optimization is complete.

[0088] The predicted target information includes: the predicted position and predicted posture of the industrial part and the predicted scaling coefficient deformation parameters corresponding to the CAD model.

[0089] Among them, the DPT model includes the Transformers network.

[0090] Specifically, multiple image samples are input into the Transformers network, and the Transformers network outputs DPT prediction features.

[0091] The DPT features are converted into data to be used, and the L1 loss of the data to be used and the point cloud sample is calculated. The network parameters of the Transformers network are optimized based on the L1 loss until the L1 loss reaches a second preset loss value, and the Transformers network optimization is determined to be completed.

[0092] This application processes multiple images based on a pre-trained model to obtain target information of each industrial part, and then reconstructs the three-dimensional point cloud based on shape priors. It solves the problem of poor accuracy in the existing technology when reconstructing three-dimensional point clouds based on multiple views, and realizes the rapid and accurate reconstruction of the three-dimensional point cloud of industrial parts.

[0093] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 communicate with each other via the communication bus 404. The processor 401 may call the logic instructions in the memory 403 to execute the multi-viewing algorithm based on the shape prior. Figure 3 dimensional point cloud reconstruction method.

[0094] In addition, the logic instructions in the above-mentioned memory 403 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0095] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer can perform the shape prior based multi-viewing provided by the above methods. Figure 3 dimensional point cloud reconstruction method.

[0096] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is used to execute the shape prior-based multi-viewing algorithm provided in the above embodiments when the computer program is executed by a processor. Figure 3 dimensional point cloud reconstruction method.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0098] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0099] Finally, it should be noted that the above description is merely a preferred embodiment of the present application and the present application is not limited to the above embodiments. It is understood that other improvements and variations directly derived or conceived by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included within the scope of protection of the present application.

Claims

1. A multi-view 3D point cloud reconstruction method based on shape prior, characterized in that: The method comprises: Acquire multiple images corresponding to a target range, wherein the target range includes one or more industrial parts, and the multiple images are obtained by photographing the target range at the same time using cameras located at different positions; Input multiple images into the Transformers network to obtain feature codes output by the VIT encoder of the Transformers network and DPT features output by the Transformers network; align and fuse the feature codes and the DPT features to obtain fused features, and extract the central features corresponding to the center points of each industrial part from the fused features; process the fused features and the central features using an attention mechanism to obtain target information of each industrial part, wherein the target information includes: the position and posture of the industrial part and the scaling coefficient deformation parameters corresponding to the CAD model, wherein the CAD model is a three-dimensional model of the industrial part; Based on the target information and the CAD model, a three-dimensional point cloud of each industrial part is reconstructed and obtained.

2. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 1, characterized in that: Extract the central features corresponding to the center points of each industrial part from the fusion features, including: The sigmoid function is used to process the fusion features to obtain the feature score corresponding to each block, and the first maximum feature score is obtained based on the comparison method; The obtained feature scores will be subjected to maximum pooling processing to obtain the second largest feature score; When it is determined that the first maximum feature score is consistent with the second maximum feature score and is greater than a preset threshold, the block corresponding to the second maximum feature score is determined as the center point, and the fusion feature corresponding to the center point is determined as the center feature.

3. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 1 or 2, characterized in that: Aligning and fusing the feature code and the DPT feature to obtain a fused feature, including: The DPT features are input into the convolution module to obtain the DPT features output by the convolution module that are aligned with the feature encoding size; Input the aligned DPT features into MLP to obtain the DPT features with reduced dimensions output by MLP, and use the DPT features with reduced dimensions as the final DPT features; The feature code and the final DPT feature are fused to obtain the fused feature.

4. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 1 or 2, characterized in that: After reconstructing and obtaining the 3D point cloud of each industrial part, it also includes: Get the original point clouds corresponding to multiple images; Calculate the L1 loss and distance loss between the reconstructed 3D point cloud and the original point cloud; When it is determined that the sum of the L1 loss and the distance damage is less than the preset loss value, it is determined that the reconstructed three-dimensional point cloud meets the standards.

5. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 1 or 2, characterized in that: Different cameras correspond to different camera serial numbers, and the image carries the camera serial number code, which represents the index number of the image taken by the camera; Input multiple images into the Transformers network and obtain the DPT features output by the Transformers network, including: Multiple images are input into the Transformers network, and the VIT encoder of the Transformers network generates a position code based on the index number, and the position code is added to the feature code of the corresponding image; the feature code with the added position code is input into multiple cross-attention modules, and the feature code of each image and the feature code of other images are cross-attention processed by the multiple cross-attention modules to obtain the processed feature code; the processed feature code is fused to obtain the DPT feature.

6. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 5, characterized in that: The processed feature codes are fused to obtain the DPT features, including: A preset feature extraction layer determination method is used to determine a target cross-attention module from multiple cross-attention modules; features output by the target cross-attention module are extracted, and a fusion operation is performed on the extracted features to obtain and output the DPT features.

7. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 1 or 2, characterized in that: The fusion features and the central features are processed by the attention mechanism to obtain the target information of each industrial part, including: Inputting the fused feature and the central feature into a plurality of cross-attention modules, performing cross-attention processing on the central feature and an image feature corresponding to the central feature by the plurality of cross-attention modules to obtain a processed central feature, wherein the image feature is obtained from the fused feature based on the central feature; The processed central features are input into the MLP layer to obtain the target information output by the MLP layer.

8. The multi-view 3D point cloud reconstruction method based on shape prior according to claim 1 or 2, characterized in that: Before acquiring multiple images corresponding to the target range, it also includes: Get multiple image samples; Input multiple image samples into the DPT model to obtain the predicted target information of each industrial part output by the DPT model, where the predicted target information includes: the predicted position and predicted posture of the industrial part and the predicted scaling coefficient deformation parameters corresponding to the CAD model. The DPT model includes a Transformers network; Place the CAD model at the corresponding spatial position corresponding to the predicted target information, and rasterize the placed CAD model to obtain the predicted point cloud corresponding to each simulated camera; Calculate the L1 loss and distance loss of the predicted point cloud and the point cloud samples corresponding to multiple image samples; The network parameters of the DPT model are optimized based on the L1 loss and distance loss until the L1 loss and distance loss reach the preset loss values, and the DPT model optimization is determined to be complete.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the multi-view three-dimensional point cloud reconstruction method based on shape prior are implemented as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-view three-dimensional point cloud reconstruction method based on shape prior are implemented.

Images