An image super-resolution method for 3D reconstruction of automobile parts

Abstract

This invention provides an image super-resolution method for 3D reconstruction of automotive parts, belonging to the field of image enhancement. This method, based on multi-angle input automotive part images, introduces a cross-view residual collaborative enhancement mechanism and a multi-view consistency optimization module, combined with a neural radiation field method. This method achieves high-quality enhancement and structural consistency of multi-view images, ultimately producing high-definition images with continuous structure and clear textures. This method provides a highly reliable and accurate visual foundation for subsequent industrial applications such as automotive part modeling, inspection, and defect identification.

Description

Technical Field

[0001] The invention belongs to the field of image enhancement, and in particular relates to an image super-resolution method for three-dimensional reconstruction of automobile parts. Background Art

[0002] As the automotive industry develops towards intelligent manufacturing and high-precision inspection, higher requirements are placed on the 3D reconstruction and surface defect identification of automotive parts. High-quality 3D reconstruction results not only help to accurately restore the part's morphology, but also provide key support for subsequent defect analysis. Traditional 3D reconstruction technology often relies on multi-view images as input, and its reconstruction accuracy is limited by the resolution and consistency of the original image. In industrial environments, due to factors such as limited imaging device resolution, limited acquisition angles, and strong reflective surfaces on parts, the acquired images have problems such as insufficient clarity, inconsistent cross-view structures, and blurred textures, which seriously affect the accuracy and stability of 3D reconstruction. This is especially true for parts with rich details or complex surfaces, such as engine housings, gears, and fasteners. Traditional methods find it difficult to restore their complete spatial structure and surface texture.

[0003] In recent years, image super-resolution and neural rendering technologies have developed rapidly, providing new solutions for industrial visual inspection. The image diffusion model performs superior in detail recovery, while the neural radiance field method can construct a three-dimensional consistent implicit representation from multi-angle images, generate structurally continuous image views in three-dimensional space, and show good structural consistency. However, traditional neural radiance field methods usually rely on high-quality input images, and it is difficult to maintain image details and structure consistency in low-resolution or texture-blurred scenes, which limits its applicability in automobile parts inspection. To solve the above problems, the present invention proposes an automobile parts image enhancement method that integrates multi-view consistent diffusion super-resolution and three-dimensional modeling. This method combines the detail generation capability of the diffusion model with the structural modeling capability of the neural radiance field method, and achieves multi-view image consistent super-resolution for three-dimensional reconstruction of automobile parts by introducing a cross-view residual collaborative enhancement mechanism and a multi-view consistency optimization module. Summary of the Invention

[0004] The present invention provides an image super-resolution method for three-dimensional reconstruction of automobile parts. The method aims to improve the consistency and clarity of multi-angle automobile part images and three-dimensional modeling under low-resolution image input conditions by introducing a cross-view residual collaborative enhancement mechanism and a multi-view consistency optimization module, using a neural radiation field method. The method includes the following steps.

[0005] S1. Use an industrial camera array mounted on a bracket array to capture multi-angle, low-resolution images of automotive parts. At the same time, calibrate each camera with the help of a calibration plate to obtain the camera pose information corresponding to each image.

[0006] S2. Construct a cross-view collaborative enhancement module CVCEM, which includes linear projection, calculation of attention correlation score, residual fusion, and enhanced features. This module focuses on utilizing adjacent views to enhance the latent variable expression of each view.

[0007] S3. Construct a multi-view consistency module MVCM, which is divided into four parts: sparsity loss calculation, diffusion loss calculation, joint optimization of MultiViewUNet network and total loss and parameter update. Sparse loss calculation is used to sparsify the residual and reduce redundant noise. Diffusion loss calculation constructs reconstruction loss through forward diffusion and reverse noise prediction process. MultiViewUNet network is used for noise prediction. Total loss joint optimization and parameter update are used to jointly optimize the parameters of the entire module. This module focuses on updating the residual while ensuring the consistency of the enhanced latent vector expression under multiple perspectives.

[0008] S4. Construct a multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF, which is composed of a multi-view consistent diffusion super-resolution model MVCDSR based on CVCEM and MVCM and a NeRF reconstruction module.

[0009] Preferably, in said S1, firstly, a plurality of industrial cameras are installed on a preset automobile parts shooting platform to form a fixed-angle camera bracket array, and the industrial cameras are arranged around the automobile parts to be inspected, covering multiple viewing angles of their typical structural surfaces; in the image acquisition stage, a low-resolution image set of the automobile parts at different angles is obtained by synchronously triggering the camera group. To ensure geometric consistency in subsequent image enhancement and 3D modeling, a calibration plate is used to perform camera calibration. A multi-camera calibration tool is used to solve the intrinsic and extrinsic parameters of each camera to obtain the camera pose parameter set corresponding to the image. .

[0010] Preferably, in said S2, a cross-view collaborative enhancement module CVCEM is constructed, and the specific process includes: inputting the residual set corresponding to all views and a set of latent variables , each Corresponding to The potential vector representation obtained by encoding the low-resolution image of each perspective by the variational self-encoder is , this module updates the features through the cross-view residual enhancement mechanism. The specific process is: first, and its neighboring perspective sets , respectively, through the learnable linear mapping matrix and Transform to get the query vector and key vector , based on the dot product similarity between the query vector and each neighbor key vector, calculate the cross-view attention weight ,Then, based on the above attention weights, the total residual of the current perspective is calculated. The specific process is: ,in Perspective The residual, For the current perspective The residual, is the set self-residual fusion coefficient, and finally, updates the current perspective The potential vector expression of , the update process is , is a set of potential vector expressions after enhancing features, which is the output of this module.

[0011] Preferably, in S2, a strategy for enhancing the latent features of images based on cross-view residual guidance and attention fusion is proposed to improve the structural consistency and information integrity of multi-angle images of automotive parts in the latent expression stage; the representation of query vectors and key vectors is realized by constructing a learnable linear mapping matrix, and dynamic weights are generated under the dot product attention framework to achieve selective fusion of the residual information of each neighboring perspective; on this basis, the latent representation of each perspective is superimposed and updated with the fused residual information to obtain an enhanced latent expression with a more complete structure and richer features; while maintaining the semantic features of the original perspective, this module dynamically integrates supplementary information from neighboring perspectives, which can effectively alleviate the problem of missing image information caused by shooting angle deviation, reflection interference or low-resolution acquisition, and provide a stable and continuous expression basis for subsequent consistency optimization and image super-resolution reconstruction.

[0012] Preferably, in said S3, the specific process of constructing the multi-view consistency module MVCM is as follows:

[0013] S31, Multi-view consistency module MVCM is divided into four parts: sparse loss calculation, diffusion loss calculation, building MultiViewUNet network and total loss joint optimization and parameter update. For the sparse loss calculation part, the input is a set of perspective residuals , the specific process is: for each residual variable , calculate the residual sparsity loss corresponding to the perspective as ,in , , Corresponding to the number of channels, height and width of the residual features respectively;

[0014] S32, for building the MultiViewUNet network part, used to predict noise , the input of the network is the current view potential vector after noise addition , the set of adjacent view potential vectors and the set of adjacent view poses The specific process is: first and Perform flattening operations to obtain pixel sequence representation and ,in , Represents the height and width of the feature map, Represents the number of channels, Represents the flattening operation; secondly, the pose is broadcast to each pixel to form a pose feature matrix ,in representative posture Embedded coding, Represents a broadcast operation, The length is The unit vector of ; then the neighbor noise feature and the pose embedding are added element by element to obtain the fused key vector , the fusion key vectors of all neighboring views are stacked along the pixel dimension to form a complete multi-view key set ,in, is the number of neighboring views; based on the cross attention mechanism, the query vector of the current view is used Merge key vectors with neighbors The dot product relationship between them is used to obtain the fusion condition features As the final conditional input of UNet, The function represents the normalization operation, ; Finally, the noise potential vector of the current view is , time step , fusion conditional features Input to the UNet backbone network to generate the final noise prediction output ;

[0015] S33. For the diffusion loss calculation part, the input is a set of enhanced latent vectors and a set of corresponding pose information , for each ,pass Perform forward diffusion to obtain the noise-added vector and , For the The cumulative noise attenuation coefficient corresponding to the time step is generated by the diffusion scheduler and is calculated as follows: ,in Adds a preset percentage of noise per step to the diffusion schedule, Represents standard Gaussian noise; noise is predicted using the MultiViewUNet network ; Based on noise prediction results , calculate the forward latent vector , the calculation method is , and finally define the diffusion reconstruction loss as ;

[0016] S34. For the total loss joint optimization and parameter update part, the input is a set of diffusion reconstruction losses. and a set of sparsity losses , first calculate the total loss of the current perspective as ,in To preset hyperparameters, secondly, calculate the total loss of all view angles as , Third, update the residual based on the total loss gradient descent of the current perspective. The specific process is: ,in is the preset learning rate, Represents the gradient of the residual variable of the current view. Fourth, the learnable linear mapping matrix is ​​updated based on the total loss gradient descent of all views. and The specific process is: , ,in and is the linear mapping matrix and Gradient with respect to the overall loss.

[0017] Preferably, in the S3, a multi-view consistency module MVCM is constructed, which focuses on guiding the consistent expression between different perspectives in the potential feature enhancement stage. The module consists of four parts: sparsity loss calculation, diffusion reconstruction loss construction, MultiViewUNet network design, and total loss joint optimization. First, a sparse regularization term is constructed in the sparsity loss calculation part to encourage the network to generate sparse, clearly structured residual information and reduce redundant interference. Secondly, the MultiViewUNet network constructed based on the noisy latent vector introduces adjacent view images and pose information, and combines the cross-attention mechanism for feature fusion, thereby achieving high-precision prediction of noise. Thirdly, through the forward diffusion and reverse reconstruction process, the diffusion reconstruction loss in the latent space is constructed to effectively evaluate the consistency performance of the image in the step-by-step denoising process. Finally, the sparsity loss and the diffusion loss are jointly optimized, and the residual variables and the learnable attention mapping parameters are updated in a gradient descent manner to achieve consistency enhancement and residual adaptive adjustment between multiple perspectives. This module significantly improves the stability and coordination of the latent expression under multiple perspectives, and provides a high-quality feature foundation for structural alignment for subsequent processes.

[0018] Preferably, in S4, a multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF is constructed, and the specific process is as follows:

[0019] S41, Multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF includes MVCDSR module and NeRF reconstruction module. For MVCDSR module, the input is a set of low-resolution images. and its corresponding camera pose parameters ,go through , get the potential vector representation of the picture , Represents the variational autoencoder encoding process; the potential vector is expressed as Input to the cross-view collaborative enhancement module CVCEM module to obtain enhanced features ; Then the enhanced features are input to the multi-view consistency module MVCM module, the residual is updated, and the potential representation is optimized; the CVCEM and MVCM are jointly iterated Z times to output the final enhanced potential vector , and then passed through the variational autoencoder Decoding, where Represents the decoding process, is a set of output high-resolution multi-view images;

[0020] S42, for the NeRF reconstruction module, the high-resolution multi-view image The corresponding camera pose parameters The neural radiation field model is input for three-dimensional reconstruction, and the reconstructed output image is obtained as the input of the next round of MVCDSR module. The MVCDSR module and the NeRF reconstruction module are jointly iteratively executed M times, and finally output an image representation and its three-dimensional model with high-resolution details and three-dimensional consistent structure.

[0021] Preferably, in said S4, by inputting multi-angle high-resolution images and their camera poses into the neural radiation field model, three-dimensional modeling and consistent view reconstruction of the surface of automobile parts are achieved, and this is used as the input for a new round of image super-resolution enhancement, continuously optimizing the image expression and geometric consistency, and finally outputting an image representation and its three-dimensional model with high-resolution details and three-dimensional consistent structure.

[0022] Compared with the existing technology, the beneficial effects of the present invention are as follows: it combines the advantages of the image diffusion model in detail enhancement and the capabilities of the neural radiation field in three-dimensional structure modeling, and targets the pain points of low-resolution, multi-angle images in structural inconsistency and detail blurring. Through cross-perspective residual collaborative enhancement and multi-perspective consistency optimization strategy, it achieves fine reconstruction and structural alignment of the potential expression of the image; at the same time, it introduces the NeRF module as a three-dimensional geometric constraint to achieve joint iterative optimization of image enhancement and three-dimensional modeling; this method has significant advantages such as strong perspective consistency, high detail recovery accuracy, and stable and natural enhancement process, which can significantly improve the usability and accuracy of low-quality automotive parts images in three-dimensional reconstruction and subsequent inspection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of an image super-resolution method for three-dimensional reconstruction of automobile parts provided by the present invention.

[0024] Figure 2 This is a structural diagram of the cross-view collaborative enhancement module CVCEM provided by the present invention.

[0025] Figure 3 This is a structural diagram of the multi-view consistency module MVCM provided by the present invention.

[0026] Figure 4 This is the MultiViewUNet network structure diagram provided by the present invention.

[0027] Figure 5 This is a structural diagram of the MVCDSR module provided by the present invention.

[0028] Figure 6 This is a structural diagram of the multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF provided by the present invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] See also Figures 1 to 6 The present invention provides an image super-resolution method for three-dimensional reconstruction of automobile parts. It aims to generate high-resolution images and three-dimensional representations with clear details and structural consistency based on multi-view low-resolution images and camera pose information through a collaborative enhancement and consistency optimization mechanism and a neural radiation field method, thereby improving the detection accuracy and spatial structure restoration capability of automobile parts.

[0031] S1. Use an industrial camera array mounted on a bracket array to capture multi-angle, low-resolution images of automotive parts. At the same time, calibrate each camera with the help of a calibration plate to obtain the camera pose information corresponding to each image.

[0032] Furthermore, in S1, 8 industrial cameras are selected to form a ring bracket array installed around the inspection station. The brackets are distributed in a hemispherical shape, and each camera is arranged around the component at an evenly spaced 45° horizontal angle. At the same time, they are divided into two layers of height in the vertical direction. Before shooting, a flat calibration plate with precise marking points is placed in multiple spatial positions of the inspection station in turn, and each camera is used to collect images of the calibration plate. The intrinsic parameter matrix and extrinsic parameter matrix of each camera are calculated by industrial calibration software to obtain the corresponding pose information of each image. In actual acquisition, the target automotive component is fixed on the platform, and the low-resolution image corresponding to each perspective is acquired by synchronous shooting with multiple cameras. Eight images and their corresponding eight sets of camera pose information are obtained for each automotive component.

[0033] S2. Construct a cross-view collaborative enhancement module CVCEM, which includes linear projection, calculation of attention correlation score, residual fusion, and enhanced features. This module focuses on utilizing adjacent views to enhance the latent variable expression of each view.

[0034] Furthermore, in S2, a cross-view collaborative enhancement module CVCEM is constructed. The specific process includes: the input is the residual set corresponding to all views and a set of latent variables , each Corresponding to The potential vector representation obtained by encoding the low-resolution image of the perspectives through the variational autoencoder. In this embodiment, the number of perspectives is 8. The initial value is 0, for each input potential feature , this module updates the features through the cross-view residual enhancement mechanism. The specific process is: first, and its neighboring perspective sets , respectively, through the learnable linear mapping matrix and Transform to get the query vector and key vector In this embodiment, the neighbor perspective For and perspective A set of 4 spatially adjacent view numbers; the cross-view attention weight is calculated based on the dot product similarity between the query vector and each neighbor key vector ,Then, based on the above attention weights, the total residual of the current perspective is calculated. The specific process is: ,in Perspective The residual, For the current perspective The residual, is the set self-residual fusion coefficient. In this embodiment, Set to 0.3, and finally, update the current perspective The latent vector representation of , the update process is , is a set of potential vector expressions after enhancing features, which is the output of this module.

[0035] S3. Construct a multi-view consistency module MVCM, which is divided into four parts: sparsity loss calculation, diffusion loss calculation, joint optimization of MultiViewUNet network and total loss and parameter update. Sparse loss calculation is used to sparsify the residual and reduce redundant noise. Diffusion loss calculation constructs reconstruction loss through forward diffusion and reverse noise prediction process. MultiViewUNet network is used for noise prediction. Total loss joint optimization and parameter update are used to jointly optimize the parameters of the entire module. This module focuses on updating the residual while ensuring the consistency of the enhanced latent vector expression under multiple perspectives.

[0036] Furthermore, in S3, a multi-view consistency module MVCM is constructed, and the specific process includes:

[0037] S31, Multi-view consistency module MVCM is divided into four parts: sparse loss calculation, diffusion loss calculation, building MultiViewUNet network and total loss joint optimization and parameter update. For the sparse loss calculation part, the input is a set of perspective residuals , the specific process is: for each residual variable , calculate the residual sparsity loss corresponding to the perspective as ,in , , They correspond to the number of channels, height and width of the residual features. In this embodiment, the number of channels is set , feature map size .

[0038] S32, for building the MultiViewUNet network part, used to predict noise , the input of the network is the current view potential vector after noise addition , the set of adjacent view potential vectors and the set of adjacent view poses The specific process is: first and Perform flattening operations to obtain pixel sequence representation and ,in , Represents the height and width of the feature map, Represents the number of channels, Represents the flattening operation; secondly, the pose is broadcast to each pixel to form a pose feature matrix ,in representative posture Embedded coding, Represents a broadcast operation, The length is The unit vector of ; then the neighbor noise feature and the pose embedding are added element by element to obtain the fused key vector , the fusion key vectors of all neighboring views are stacked along the pixel dimension to form a complete multi-view key set ,in, is the number of neighbor perspectives. In this embodiment, ; Based on the cross attention mechanism, the query vector of the current perspective is used Merge key vectors with neighbors The dot product relationship between them is used to obtain the fusion condition features As the final conditional input of UNet, The function represents the normalization operation, ; Finally, the noise potential vector of the current view is , time step , fusion conditional features Input to the UNet backbone network to generate the final noise prediction output In this embodiment, the total number of steps in the diffusion process is set to , in each training iteration, randomly sample time steps Controls the current diffusion strength.

[0039] S33, for the diffusion loss calculation part, the input is a set of enhanced latent vectors And a set of corresponding pose information , for each ,pass Perform forward diffusion to obtain the noise-added vector and , For the The cumulative noise attenuation coefficient corresponding to the time step is generated by the diffusion scheduler and is calculated as follows: ,in Adds a preset percentage of noise per step to the diffusion schedule, represents standard Gaussian noise. In this embodiment, Linearly increasing ; Use MultiViewUNet network to predict noise ; Based on noise prediction results , calculate the forward latent vector , the calculation method is , and finally define the diffusion reconstruction loss as .

[0040] S34. For the total loss joint optimization and parameter update part, the input is a set of diffusion reconstruction losses. and a set of sparsity losses , first calculate the total loss of the current perspective as ,in To preset hyperparameters, in this embodiment, Second, calculate the total loss for all views as , Third, update the residual based on the total loss gradient descent of the current perspective. The specific process is: ,in is the preset learning rate. In this embodiment, , Represents the gradient of the residual variable of the current view. Fourth, the learnable linear mapping matrix is ​​updated based on the total loss gradient descent of all views. and The specific process is: , ,in and is the linear mapping matrix and Gradient with respect to the overall loss.

[0041] S4. Construct a multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF, which is composed of a multi-view consistent diffusion super-resolution model MVCDSR based on CVCEM and MVCM and a NeRF reconstruction module.

[0042] Furthermore, in S4, a multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF is constructed, and the specific process includes:

[0043] S41, Multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF includes MVCDSR module and NeRF reconstruction module. For MVCDSR module, the input is a set of low-resolution images. and its corresponding camera pose parameters ,go through , get the potential vector representation of the picture , Represents the variational autoencoder encoding process; the potential vector is expressed as Input to the cross-view collaborative enhancement module CVCEM module to obtain enhanced features ; Then the enhanced features are input to the multi-view consistency module MVCM module, the residual is updated, and the potential representation is optimized; Then the enhanced features are input to the multi-view consistency module MVCM module, the residual is updated, and the potential representation is optimized; The CVCEM and MVCM are executed Z times in a joint iterative manner, and the final enhanced potential vector is output , and then passed through the variational autoencoder Decoding, where Represents the decoding process, The resolution of a set of outputs is In this example, the image .

[0044] S42, for the NeRF reconstruction module, the high-resolution image The corresponding camera pose parameters The neural radiation field model is input for three-dimensional reconstruction. The neural radiation field model is trained for 5000 steps using dense sampling and ray integration technology, sampling 1024 rays per step, and the reconstructed output image is obtained as the input of the next round of MVCDSR module. The MVCDSR module and the NeRF reconstruction module are executed M times in a joint iterative manner, and finally output an image representation and a three-dimensional model with high-resolution details and three-dimensional consistent structure. In this embodiment, .

[0045] Furthermore, the image super-resolution model for 3D reconstruction of automobile parts was coded using the Pycharm application and Python language, using the Pytorch framework, and the model input was automobile parts images from 8 perspectives with a resolution of 128×128×3.

[0046] The above are only preferred embodiments of the present invention. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. An image super-resolution method for 3D reconstruction of automobile parts, characterized in that: The following steps are involved: S1. Use an industrial camera array mounted on a bracket array to capture multi-angle, low-resolution images of automotive parts. Simultaneously, calibrate each camera using a calibration plate to obtain the camera pose information corresponding to each image. S2. Construct a cross-view collaborative enhancement module (CVCEM), which includes linear projection, calculation of attention correlation scores, residual fusion, and feature enhancement. This module focuses on leveraging adjacent views to enhance the latent variable expression of each view. S3. Construct a multi-view consistency module (MVCM), which is divided into four parts: sparsity loss calculation, diffusion loss calculation, joint optimization of the MultiViewUNet network and total loss, and parameter update. The sparsity loss calculation is used to sparsify the residual and reduce redundant noise. The diffusion loss calculation constructs the reconstruction loss through the forward diffusion and reverse noise prediction process. The MultiViewUNet network is used for noise prediction. The total loss is jointly optimized with parameter update to jointly optimize the parameters of the entire module. This module focuses on updating the residual while ensuring the consistency of the enhanced latent vector expression under multiple views. S4. Construct a multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF, which is composed of a multi-view consistent diffusion super-resolution model MVCDSR based on CVCEM and MVCM and a NeRF reconstruction module.

2. According to the image super-resolution method for 3D reconstruction of automobile parts in claim 1, in step S2, a cross-view collaborative enhancement module (CVCEM) is constructed, characterized in that: The input is the residual set corresponding to all perspectives and a set of latent variables , each Corresponding to The potential vector representation obtained by encoding the low-resolution image of each perspective by the variational self-encoder is , this module updates the features through the cross-view residual enhancement mechanism. The specific process is: first, and its neighboring perspective sets , respectively, through the learnable linear mapping matrix and Transform to get the query vector and key vector , based on the dot product similarity between the query vector and each neighbor key vector, calculate the cross-view attention weight ,Then, based on the above attention weights, the total residual of the current perspective is calculated. The specific process is: ,in Perspective The residual, For the current perspective The residual, is the set self-residual fusion coefficient, and finally, updates the current perspective The potential vector expression of , the update process is , is a set of potential vector expressions after enhancing features, which is the output of this module.

3. According to the image super-resolution method for 3D reconstruction of automobile parts in claim 1, in step S3, a multi-view consistency module (MVCM) is constructed, characterized in that: S31, Multi-view consistency module MVCM is divided into four parts: sparse loss calculation, diffusion loss calculation, building MultiViewUNet network and total loss joint optimization and parameter update. For the sparse loss calculation part, the input is a set of perspective residuals , the specific process is: for each residual variable , calculate the residual sparsity loss corresponding to the perspective as ,in , , Corresponding to the number of channels, height and width of the residual features respectively; S32, for building the MultiViewUNet network part, used to predict noise , the input of the network is the current view potential vector after noise addition , the set of adjacent view potential vectors and the set of adjacent view poses The specific process is: first and Perform flattening operations to obtain pixel sequence representation and ,in , Represents the height and width of the feature map, Represents the number of channels, Represents the flattening operation; secondly, the pose is broadcast to each pixel to form a pose feature matrix ,in representative posture Embedded coding, Represents a broadcast operation, The length is The unit vector of ; then the neighbor noise feature and the pose embedding are added element by element to obtain the fused key vector , the fusion key vectors of all neighboring views are stacked along the pixel dimension to form a complete multi-view key set ,in, is the number of neighboring views; based on the cross attention mechanism, the query vector of the current view is used Merge key vectors with neighbors The dot product relationship between them is used to obtain the fusion condition features As the final conditional input of UNet, The function represents the normalization operation, ; Finally, the noise potential vector of the current view is , time step , fusion conditional features Input to the UNet backbone network to generate the final noise prediction output ; S33. For the diffusion loss calculation part, the input is a set of enhanced latent vectors and a set of corresponding pose information , for each ,pass Perform forward diffusion to obtain the noise-added vector and , For the The cumulative noise attenuation coefficient corresponding to the time step is generated by the diffusion scheduler and is calculated as follows: ,in Adds a preset percentage of noise per step to the diffusion schedule, Represents standard Gaussian noise; noise is predicted using the MultiViewUNet network ; Based on noise prediction results , calculate the forward latent vector , the calculation method is , and finally define the diffusion reconstruction loss as ; S34. For the total loss joint optimization and parameter update part, the input is a set of diffusion reconstruction losses. and a set of sparsity losses , first calculate the total loss of the current perspective as ,in To preset hyperparameters, secondly, calculate the total loss of all view angles as , Third, update the residual based on the total loss gradient descent of the current perspective. The specific process is: ,in is the preset learning rate, Represents the gradient of the residual variable of the current view. Fourth, the learnable linear mapping matrix is ​​updated based on the total loss gradient descent of all views. and The specific process is: , ,in and is the linear mapping matrix and Gradient with respect to the overall loss.

4. According to the image super-resolution method for 3D reconstruction of automobile parts in claim 1, in step S4, a multi-view uniform diffusion super-resolution reconstruction module MVCDSR-NeRF is constructed, characterized in that: S41, Multi-view consistent diffusion super-resolution reconstruction module MVCDSR-NeRF includes MVCDSR module and NeRF reconstruction module. For MVCDSR module, the input is a set of low-resolution images. and its corresponding camera pose parameters ,go through , get the potential vector representation of the picture , Represents the variational autoencoder encoding process; The latent vector is expressed as Input to the cross-view collaborative enhancement module CVCEM module to obtain enhanced features ; Then the enhanced features are input to the multi-view consistency module MVCM module, the residual is updated, and the potential representation is optimized; the CVCEM and MVCM are jointly iterated Z times to output the final enhanced potential vector , and then passed through the variational autoencoder Decoding, where Represents the decoding process, is a set of output high-resolution multi-view images; S42, for the NeRF reconstruction module, the high-resolution multi-view image The corresponding camera pose parameters The neural radiation field model is input for three-dimensional reconstruction, and the reconstructed output image is obtained as the input of the next round of MVCDSR module. The MVCDSR module and the NeRF reconstruction module are jointly iteratively executed M times, and finally output an image representation and its three-dimensional model with high-resolution details and three-dimensional consistent structure.

Images