Machine learning-based single-view 3D photometric reconstruction methods, systems, devices, and media

Abstract

This invention discloses a single-view 3D photometric reconstruction method, system, device, and medium based on machine learning, relating to the field of computer vision technology. The method includes acquiring single-view images of a target object under different light source conditions and light source direction information; preprocessing the images and light source direction information to generate standardized multi-dimensional input data; performing feature extraction and fusion to obtain a fused feature representation; performing preliminary 3D surface normal estimation; outputting preliminary normal vectors; iteratively optimizing the preliminary normal vectors; combining light source reliability assessment and material prior knowledge constraints to obtain accurate normal vectors; and constructing a 3D model of the target object using a 3D geometric reconstruction algorithm. This invention significantly improves the accuracy, robustness, and direct usability of 3D reconstruction in complex lighting and heterogeneous material scenarios by combining multi-scale feature learning with channel attention and incorporating physical optimization based on light source reliability assessment and material prior knowledge.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a single-view 3D photometric reconstruction method, system, device, and medium based on machine learning. Background Technology

[0002] Significant progress has been made in single-view-based 3D reconstruction technology in the field of computer vision, which has become a key technology path for non-contact measurement and digital twin construction. With the deepening of deep learning theory, data-driven methods have been widely introduced into photometric stereo vision. By constructing hierarchical neural networks, complex nonlinear feature maps are automatically extracted from image data, which significantly improves the robustness and reconstruction accuracy of traditional methods when dealing with non-Lambertian reflective surfaces and complex lighting environments, providing solid technical support for high-precision 3D digitization.

[0003] However, existing machine learning-based single-view photometric reconstruction methods still face many challenges in practical applications. On the one hand, traditional deep learning models often lack adaptive fusion mechanisms during feature extraction, making it difficult to effectively distinguish the contribution of different feature channels to geometric estimation. This results in insufficient feature discriminability when illumination conditions are unstable or the light source direction is deviated, making it impossible to accurately eliminate interference from unreliable light sources, thereby reducing the accuracy of the initial normal estimation. On the other hand, existing solutions typically do not deeply integrate physical prior knowledge and dynamic light source reliability assessment in the optimization stage, ignoring surface material property constraints. This leads to errors in the reconstruction results in terms of local high-frequency details and overall geometric consistency, making it difficult to meet the stringent requirements of high-precision industrial inspection and 3D modeling. Summary of the Invention

[0004] In view of the above-mentioned existing problems, the present invention provides a single-view 3D photometric reconstruction method, system, device and medium based on machine learning, to solve the problems of insufficient feature discrimination, inability to accurately eliminate interference from unreliable light sources and errors in reconstruction results in terms of local high-frequency details and overall geometric consistency in the prior art.

[0005] To address the aforementioned technical issues, a single-view 3D photometric reconstruction method based on machine learning is proposed, including: The system acquires single-view images of the target object under different light source conditions and light source direction information, and preprocesses the images and light source direction information to generate standardized multi-dimensional input data. It then extracts and fuses features from the standardized input data to obtain a discriminative fused feature representation, and performs preliminary 3D surface normal estimation based on the fused feature representation, outputting a preliminary normal vector. The preliminary normal vector is iteratively optimized, and combined with light source reliability assessment and material prior knowledge constraints, an optimized precise normal vector is obtained. Based on the precise normal vector, a 3D model of the target object is constructed using a 3D geometric reconstruction algorithm.

[0006] As a preferred embodiment of the single-view three-dimensional photometric reconstruction method based on machine learning described in this invention, the preprocessing includes converting the acquired single-view color image into a single-channel image, screening and evaluating all light sources based on the overall brightness level of the image area under each light source illumination, and retaining effective light sources that meet the preset reliability conditions. The direction vectors of effective light sources are normalized. The pixel brightness values ​​of a single-channel image are mapped to the normalized direction vectors of all effective light sources by normalizing their numerical ranges. The normalized pixel brightness values ​​are then combined with the direction vectors of all light sources in a preset order to form a fixed-dimensional vector.

[0007] As a preferred embodiment of the single-view 3D photometric reconstruction method based on machine learning described in this invention, the feature extraction and fusion includes: reconstructing a fixed-dimensional vector into a two-dimensional feature map, inputting it into a hierarchical feature extraction network, and the feature extraction network extracting and abstracting multiple scale features from local details to global structure from the input data through multiple layers. An adaptive feature fusion mechanism is adopted to analyze all feature channels output by the final layer of the feature extraction network. Based on the importance of each feature channel to the final 3D geometry estimation task, a weight coefficient is calculated for each channel. The original feature channels are then weighted and fused using the calculated weight coefficients to generate a fused feature representation.

[0008] As a preferred embodiment of the single-view 3D photometric reconstruction method based on machine learning described in this invention, the iterative optimization includes: constructing an observation model that integrates preliminary normal vectors and light source direction information; designing a light source reliability assessment mechanism; and assigning dynamically adjusted confidence weights to each light source in the observation model based on the actual performance of each light source during the imaging process. By integrating prior physical knowledge, constraints on changes in surface reflection properties are formed. Based on the weighted observation model and with the objective of satisfying the physical constraints, a joint optimization function is constructed. By iteratively solving the joint optimization function, the initial input normal vector is globally adjusted and corrected, and the accurate normal vector is output.

[0009] As a preferred embodiment of the single-view 3D photometric reconstruction method based on machine learning described in this invention, the hierarchical feature extraction network includes: a first convolutional layer performing a convolution operation on the input single-channel feature map to extract primary features of edges and textures; a second convolutional layer performing a convolution operation on the output of the first convolutional layer to fuse the primary features into intermediate features representing local contours; and a third convolutional layer performing a convolution operation on the output of the second convolutional layer to aggregate the intermediate features and generate high-level features containing semantic information of the target components. The formula for a three-layer convolution operation is expressed as follows: in, This is the feature map output by the first convolutional layer. For the input feature map, This is the weight matrix of the first convolutional layer. This refers to the bias term of the first convolutional layer. For convolution operations, For activation function, This is the feature map output by the second convolutional layer. This is the bias term for the second convolutional layer. This is the weight matrix of the second convolutional layer. The feature map output by the third convolutional layer. This is the weight matrix of the third convolutional layer. This refers to the bias term of the third convolutional layer; The adaptive feature fusion mechanism includes: performing global average pooling on the feature map output by the third convolutional layer along the spatial dimension to compress and obtain channel feature vectors; inputting the feature vectors into the first fully connected layer and processing them with the ReLU activation function; inputting them into the second fully connected layer and normalizing them with the Sigmoid activation function to obtain channel attention weight vectors; multiplying each weight value in the weight vector element-wise with all pixel values ​​of the corresponding channel in the original feature map to output the final fused feature map. The formula for obtaining the channel feature vector is expressed as: in, Let be the average of the feature vectors of the c channels. The height of the feature map, The width of the feature map. The output feature map of the third convolutional layer is used to determine the feature value at the c-th channel position (i, j). For channel indexing; The formula for a fully connected layer is expressed as follows: in, For intermediate feature vectors, This is the weight matrix of the first fully connected layer. The channel feature vectors are input to the first fully connected layer. For the bias term of the first fully connected layer, For activation function, Here is the channel attention weight vector. It is the Sigmoid activation function. This is the weight matrix of the second fully connected layer. For the bias term of the second fully connected layer, It is a natural constant. For input data, The fused feature map This is the channel-by-channel multiplication operator.

[0010] As a preferred embodiment of the single-view 3D photometric reconstruction method based on machine learning described in this invention, the step of obtaining the optimized accurate normal vector includes: calculating the light source weights based on the average gray intensity of the image under each light source, constructing a diagonal matrix with the light source weights as the main diagonal, multiplying the weight matrix with the light source direction matrix to obtain a weighted photometric matrix, assigning a priori reflectance values ​​to each pixel in the image based on the prior knowledge of the material of different parts of the target object, and constructing a diagonal reflectance constraint matrix; The formula for calculating average gray intensity is expressed as: in, The average gray intensity of the light source, The total number of effective light sources, Let be the average gray intensity of the image under the m-th light source; Singular value decomposition is performed on the weighted luminance matrix to obtain the left singular matrix, singular value matrix and right singular matrix. Combined with the reflectance constraint matrix, the orthogonal matrix is ​​solved. Using the solved orthogonal matrix and singular value matrix, the final optimized precise normal matrix and reflectance matrix are calculated. The optimized formula for the precise normal is expressed as follows: in, For the optimized precise normal matrix, For the weighted observation grayscale matrix, Here is the light source direction matrix. Let be the normal matrix to be solved. Here is the reflectivity matrix. The square of the L2 norm. For regularization parameters, This is the reflectivity constraint loss term.

[0011] As a preferred embodiment of the single-view 3D photometric reconstruction method based on machine learning described in this invention, the construction of the 3D model of the target object includes: using the optimized and corrected accurate normal vector field, using the Poisson reconstruction algorithm to approximate the normal field as the 3D surface gradient field, constructing and solving the Poisson equation to obtain the implicit function defined on the 3D voxel grid, and generating the 3D point cloud model of the target by extracting the isosurface of the implicit function. The Poisson reconstruction algorithm is expressed as follows: in, For the Laplace operator, Let be the implicit function to be found. Let x be the Laplace value of the implicit function at point x. For divergence operators, For the optimized precise normal vector field, Let be the divergence of the normal vector field. Let x be the second-order partial derivative with respect to the variable x. Let be the second-order partial derivative with respect to the variable y. Let z be the second-order partial derivative with respect to the variable z. To optimize the components of the normal in the x-direction, To optimize the components of the normal in the y-direction, To optimize the components of the normal in the z-direction, This is a transpose operation; Post-processing includes using a statistical filtering strategy to remove outliers that are more than three standard deviations away from the centroid of the point cloud to eliminate noise, and performing iterative nearest-point registration between the filtered reconstructed point cloud and the standard 3D model of the power grid facility to achieve accurate alignment of the point cloud.

[0012] The beneficial effects of this preferred technical solution are as follows: by preprocessing the image and light source information to improve robustness, by combining hierarchical feature fusion and iterative optimization of physical constraints to obtain accurate normal vectors, and by using Poisson reconstruction and registration algorithms, accurate reconstruction from a single view to a high-quality, high-precision three-dimensional model of power facilities can be achieved.

[0013] As a preferred embodiment of the single-view 3D photometric reconstruction system based on machine learning described in this invention, it is characterized by including an input preprocessing module, a feature extraction and fusion module, a preliminary normal estimation module, and an improved SVD optimization module.

[0014] The input preprocessing module is used to convert the color image into a single-channel grayscale image that reflects brightness information by weighted averaging, to filter light sources based on a preset intensity threshold, to retain effective light sources and to normalize the direction vector, and to map all data to a unified numerical range by linear normalization, thereby constructing an input vector with a unified format and fixed dimensions.

[0015] The feature extraction and fusion module is used to employ a hierarchical multi-scale convolutional neural network structure to abstract full-scale feature information from the input data through convolution operations, and integrates a channel attention fusion mechanism to assign weights to each channel and perform adaptive weighted fusion of features.

[0016] The preliminary normal estimation module is used to receive the fused feature map output by feature extraction and fusion, perform nonlinear transformation and dimensionality compression through a fully connected layer, output the three-dimensional coordinates of each pixel, and perform normalization processing on the output results.

[0017] The improved SVD optimization module is used to dynamically adjust the contribution of each light source in the calculation according to the imaging quality of each light source by introducing an adaptive weighting mechanism for the light source, and to integrate the physical prior constraints of material reflectivity, using the reflection characteristics of different materials in the power grid facilities as optimization boundary conditions, and outputting accurate normal vectors.

[0018] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a method for single-view 3D photometric reconstruction based on machine learning.

[0019] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for single-view 3D photometric reconstruction based on machine learning.

[0020] The beneficial effects of this invention are as follows: By preprocessing the acquired single-view image and light source direction information, this invention achieves standardization of input data and construction of a high signal-to-noise ratio, effectively eliminating the influence of ambient light noise and improving the robustness of the model to changes in illumination. Through a hierarchical feature extraction network and an adaptive feature fusion mechanism, it achieves adaptive focusing on key geometric features and generation of discriminative feature representations, improving the accuracy of preliminary normal estimation. By constructing an observation model and combining iterative optimization with light source reliability assessment and material prior knowledge constraints, it achieves global adjustment and physical consistency correction of the preliminary normal vector, obtaining an accurate normal vector that conforms to the laws of physical imaging. By using the Poisson reconstruction algorithm to solve the implicit function, it achieves accurate reconstruction from the normal field to the complete three-dimensional geometric surface. Through statistical filtering and iterative nearest-point registration, it achieves noise elimination and accurate spatial alignment of the reconstruction results, ultimately outputting a three-dimensional model of the power facility. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The overall flowchart of a machine learning-based single-view 3D photometric reconstruction method provided in one embodiment of the present invention is shown below.

[0023] Figure 2 A system flowchart of a single-view 3D photometric reconstruction system based on machine learning, provided as an embodiment of the present invention. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0025] Example 1, referring to Figure 1 As one embodiment of the present invention, a single-view 3D photometric reconstruction method based on machine learning is provided, comprising: S100: Acquire single-view images of the target object under different light source conditions and light source direction information, and preprocess the images and light source direction information to generate standardized multidimensional input data.

[0026] S200: The standardized input data is subjected to feature extraction and fusion to obtain a discriminative fused feature representation, and a preliminary 3D surface normal estimation is performed based on the fused feature representation to output a preliminary normal vector; S300: Iteratively optimize the initial normal vector, combine the light source reliability assessment and material prior knowledge constraints to obtain the optimized accurate normal vector, and construct the three-dimensional model of the target object based on the accurate normal vector through a three-dimensional geometric reconstruction algorithm.

[0027] It should be noted that this invention enhances robustness to complex lighting through data standardization preprocessing, improves geometric reasoning accuracy by combining multi-scale feature learning with channel attention, and solves the accuracy problem under heterogeneous materials and lighting fluctuations by incorporating physical constraint optimization with light source evaluation and material priors. Based on accurate normals, Poisson reconstruction and post-processing are performed to generate a high-quality 3D solid model.

[0028] Example 2, refer to Figure 1 This is a second embodiment of the present invention, which provides a single-view 3D photometric reconstruction method based on machine learning, including: In step S100, the preprocessing includes steps S101 to S103: S101: Obtain single-view images of the target object under multiple light source conditions and the corresponding light source direction information, and perform grayscale conversion, that is, convert the obtained single-view color image into a single-channel image that reflects the overall brightness distribution; Acquiring single-view images and corresponding light source direction information involves capturing single-view color images of target power grid facilities (such as transmission towers and insulators) at the work site using a high-definition camera, and collecting light source direction data from 96 different directions at the time of capture using a light sensor. To eliminate shadow interference, the light sources need to be screened, the average gray intensity of the image under each light source is calculated, and light sources with intensity normalized below a threshold of 0.1 are removed, ultimately retaining 76 effective light sources.

[0029] S102: Based on the overall brightness level of the image area under each light source illumination, all light sources are screened and evaluated, and effective light sources that meet the preset reliability conditions are retained. The direction vectors of the effective light sources are then normalized to ensure uniformity of the modulus.

[0030] S103: Normalize the numerical range of the pixel brightness value of the single-channel image and the normalized direction vectors of all effective light sources, and combine the normalized pixel brightness value and the direction vectors of all light sources in a preset order to form a fixed-dimensional vector.

[0031] Furthermore, in this embodiment of the application, in step S101, the grayscale conversion includes steps A1~A2: A1: Take the RGB three-channel image, and for each pixel, obtain the brightness value of each of the three channels.

[0032] A2: Calculate the grayscale value according to the weight coefficients 0.299, 0.587, and 0.114, and output a single-channel grayscale image; The formula for calculating grayscale values ​​is expressed as: in, To output the grayscale value of the image at pixel location (p, q), and Here are the row and column coordinates of the pixel. Given the red channel brightness value of the input image at position (p, q), The green channel brightness value of the input image at position (p, q) The value of the blue channel of the input image at position (p, q).

[0033] In an optional implementation, step S101, the grayscale conversion further includes converting the RGB image to the HSV color space, extracting the V channel (luminance channel), performing histogram equalization on the V channel to enhance image contrast, and outputting the processed V channel as a grayscale image.

[0034] In another optional implementation, in step S101, the grayscale conversion may further include using YUV color space conversion to extract the Y component (luminance component), calculating the local average luminance of the image block (16×16 pixel block), performing gamma correction on low-brightness areas, compressing high-brightness areas, and outputting a grayscale image with balanced luminance.

[0035] Furthermore, in this embodiment of the application, in step S102, the normalization process includes steps B1~B2: B1: After reading the three-dimensional direction vector of each light source, calculate the magnitude of the current vector. The calculation formula is as follows: in, This is the normalized direction vector of the m-th light source. Let be the original three-dimensional direction vector of the m-th light source. For vectors The length of the mold, Let be the x-component of the m-th light source direction vector in the three-dimensional Cartesian coordinate system. Let be the y-component of the direction vector of the m-th light source. Let be the component of the m-th light source direction vector along the z-axis.

[0036] B2: Divide each component of the vector by the current magnitude to normalize the direction vector, ultimately unifying the magnitude of all light source direction vectors to 1.

[0037] In an optional implementation, in step S102, the normalization process further includes converting the direction vector in the Cartesian coordinate system to spherical coordinates, fixing the radial distance to 1, and normalizing the polar angle and azimuth angle to the intervals of [0,π] and [0,2π], and then converting back to Cartesian coordinates.

[0038] In another optional implementation, in step S102, the normalization process may further include collecting all light source direction vectors to construct a data matrix, calculating the covariance matrix and performing eigenvalue decomposition, and performing whitening transformation through the eigenvector matrix to make each direction component uncorrelated and have a variance of 1, and finally performing normalization.

[0039] In step S200, the feature extraction and fusion includes steps S201 to S203: S201: Reconstruct a fixed-dimensional vector into a two-dimensional feature map, input it into a hierarchical feature extraction network, and the feature extraction network extracts and abstracts features of various scales from local details to global structure from the input data through multiple layers.

[0040] S202: An adaptive feature fusion mechanism is adopted to analyze all feature channels of the final layer output of the feature extraction network, and to calculate the weight coefficient for each channel according to the importance of each feature channel to the final 3D geometry estimation task.

[0041] S203: Using the calculated weight coefficients, the original feature channels are weighted and fused to generate a fused feature representation. The calculation formula is as follows: in, Here is the channel attention weight vector. The fused feature map For channel-by-channel multiplication, This is the feature map output by the third convolutional layer.

[0042] Furthermore, in this embodiment of the application, in step S201, the reconstruction of the fixed-dimensional vector into a two-dimensional feature map includes steps C1~C3: C1: Convolutional feature extraction includes three convolutional layers. The first convolutional layer receives a single-channel input feature map, performs a convolution operation with a stride of 1 using 32 3×3 convolutional kernels, and uses edge padding to maintain the size. After processing by the ReLU activation function, the output feature map is processed.

[0043] C2: The second convolutional layer uses the output of the previous layer as input and continues to perform convolution and ReLU activation using 64 3×3×32 convolutional kernels to further extract features and increase the number of channels.

[0044] C3: The third convolutional layer receives the output of the second layer, performs feature transformation through 128 3×3×64 convolutional kernels, and applies the ReLU activation function again, finally outputting a deep feature map with 128 channels; The formula for a three-layer convolution operation is expressed as follows: in, This is the feature map output by the first convolutional layer. For the input feature map, This is the weight matrix of the first convolutional layer. This refers to the bias term of the first convolutional layer. For convolution operations, For activation function, This is the feature map output by the second convolutional layer. This is the bias term for the second convolutional layer. This is the weight matrix of the second convolutional layer. The feature map output by the third convolutional layer. This is the weight matrix of the third convolutional layer. This is the bias term for the third convolutional layer.

[0045] In an optional implementation, in step S201, the reconstruction of the fixed-dimensional vector into a two-dimensional feature map further includes decomposing the 77-dimensional vector into low-frequency, mid-frequency, and high-frequency components in the frequency domain, reconstructing the low-frequency components into a low-resolution feature map (H / 4×W / 4), the mid-frequency components into a medium-resolution feature map (H / 2×W / 2), and the high-frequency components into an original-resolution feature map (H×W), and merging the multi-scale features through upsampling and a fusion layer.

[0046] In another optional implementation, in step S201, the reconstruction of the fixed-dimensional vector into a two-dimensional feature map may further include grouping the 77-dimensional vector according to physical meaning into a gray-level channel group (1 dimension), a main light source group (first 38 dimensions), and an auxiliary light source group (last 38 dimensions), each group being independently reconstructed into a two-dimensional feature map, and the three feature maps are spliced ​​together in the channel dimension to form a feature representation of H×W×(1+38+38).

[0047] It should be noted that, in the embodiments of this application, the analysis in step S202 includes steps D1 to D3: D1: Calculate global average pooling along the spatial dimension for the 128-channel feature map output by the third convolutional layer, compressing it into a 1×1×128 channel feature vector. The formula for calculating global average pooling is expressed as: in, Let be the average of the feature vectors of the c channels. The height of the feature map, The width of the feature map. The output feature map of the third convolutional layer is used to determine the feature value at the c-th channel position (i, j). For channel indexing.

[0048] D2: The importance of each channel is learned through two fully connected layers (dimension change: 128→64→128), and the weights are normalized to the [0,1] interval using the Sigmoid function; The formula for a fully connected layer is expressed as follows: in, For intermediate feature vectors, This is the weight matrix of the first fully connected layer. The channel feature vectors are input to the first fully connected layer. For the bias term of the first fully connected layer, For activation function, Here is the channel attention weight vector. It is the Sigmoid activation function. This is the weight matrix of the second fully connected layer. For the bias term of the second fully connected layer, It is a natural constant. For input data, The fused feature map This is the channel-by-channel multiplication operator.

[0049] D3: The learned 128-dimensional weight vector is multiplied channel by channel with the original feature map to achieve adaptive weighted fusion.

[0050] In an optional implementation, in step S202, the analysis further includes constructing a feature pyramid, extracting multi-scale feature maps at different depths of the convolutional network, fusing deep semantic features with shallow detail features through lateral connections and top-down paths, and adding them element by element after adjusting the number of channels using 1×1 convolution.

[0051] In another optional implementation, in step S202, the analysis may further include adding offset learning on the basis of standard convolution to adapt the receptive field to the shape of the object, performing group convolution on the deformed feature map, and controlling the fusion ratio of different groups of features through a gating mechanism.

[0052] Furthermore, in step S200, the output of the preliminary normal vector includes steps S211~S213: S211: The first fully connected layer maps the high-dimensional vector to 64 dimensions and uses the hyperbolic tangent activation function for nonlinear processing, as expressed in the formula: in, This is the 64-dimensional feature vector output by the first fully connected layer. The hyperbolic tangent activation function is used. This is the weight matrix of the first fully connected layer. The flattened feature vector is input to the fully connected network. This is the bias term for the first fully connected layer.

[0053] S212: The second fully connected layer reduces the feature dimension to 32, and further extracts features using the hyperbolic tangent activation function. The calculation formula is as follows: in, This is the 32-dimensional feature vector output by the second fully connected layer. This is the weight matrix of the second fully connected layer. This is the bias term for the second fully connected layer.

[0054] S213: The third fully connected layer maps the 32-dimensional vector to the corresponding 3-dimensional output of the normal vector coordinates, and uses a linear activation function to obtain the preliminary estimation result. To prevent numerical instability, the preliminary estimated normal vector is normalized by dividing by the magnitude to obtain the preliminary unit normal vector, which is expressed by the following formula: in, This is the 3D vector output by the third fully connected layer. It is a linear activation function. This is the weight matrix of the third fully connected layer. For the bias term of the third fully connected layer, This is the initial normal vector after normalization. For vectors The length of the mold, It is a very small positive number.

[0055] In step S300, obtaining the optimized accurate normal vector includes steps S301 to S303: S301: Based on improved singular value decomposition (SVD), optimization correction is performed. It receives the initial unit normal vector and preprocessed light source data to output accurate normal vector and reflectivity. A weighted luminance matrix is ​​constructed. Adaptive weights are determined by calculating the ratio of the average gray intensity of each effective light source to the global average gray intensity. A diagonal light source weight matrix is ​​constructed and multiplied with the light source direction matrix. The formula for calculating average gray intensity is expressed as: in, The average gray intensity of the light source, The total number of effective light sources, Let be the average gray intensity of the image under the m-th light source; Assigning adaptive weights to each light source is represented as follows: in, Let m be the adaptive weighting coefficient for the m-th light source. Let be the average gray intensity of the image under the m-th light source.

[0056] S302: Introduce a priori reflection constraints, assign a priori reflection values ​​to each pixel based on the material of the power grid facilities, and construct a diagonal reflectivity constraint matrix.

[0057] S303: Enter the improved SVD solution stage, perform singular value decomposition on the weighted photometric matrix, and construct an optimization model containing photometric error terms and reflectance a priori deviation terms in combination with reflectance constraints. Use the preliminary normal as the initial value for iterative optimization or constrained least squares solution, and finally obtain the optimized accurate normal matrix and reflectance vector, thereby achieving accurate normal vector correction for each pixel. The optimized formula for the precise normal is expressed as follows: in, For the optimized precise normal matrix, For the weighted observation grayscale matrix, Here is the light source direction matrix. Let be the normal matrix to be solved. Here is the reflectivity matrix. The square of the L2 norm. For regularization parameters, This is the reflectivity constraint loss term.

[0058] Furthermore, in step S300, the construction of the three-dimensional model of the target object using the three-dimensional geometric reconstruction algorithm includes steps S311~S313: S311: Treat the normal vector field as an approximation of the gradient field of the three-dimensional surface, find the implicit function that matches the gradient and the normal field by solving the Poisson equation, and extract the isosurface as the reconstructed surface. The Poisson reconstruction algorithm is expressed as follows: in, For the Laplace operator, Let be the implicit function to be found. Let x be the Laplace value of the implicit function at point x. For divergence operators, For the optimized precise normal vector field, Let be the divergence of the normal vector field. Let x be the second-order partial derivative with respect to the variable x. Let be the second-order partial derivative with respect to the variable y. Let z be the second-order partial derivative with respect to the variable z. To optimize the components of the normal in the x-direction, To optimize the components of the normal in the y-direction, To optimize the components of the normal in the z-direction, This is a transpose operation.

[0059] S312: Post-processing is performed, using statistical filtering to remove outliers that are more than 3 standard deviations away from the centroid of the point cloud in order to eliminate noise.

[0060] S313: The reconstructed point cloud is then iteratively registered with the standard 3D model of the power grid facility using the Intermediate Point (ICP) method to achieve precise alignment, laying the foundation for subsequent applications such as size measurement and defect comparison.

[0061] Furthermore, in step S300, the model training and fine-tuning strategy includes steps S321 to S323: S321: Training samples were constructed, including the DiLiGenT dataset and a supplementary dataset for power scenarios. By removing shadow light sources and dividing the training and test sets proportionally, rotation, scaling, adding Gaussian noise, and data augmentation operations were implemented to improve generalization ability.

[0062] S322: During training, a composite weighted loss function is used, assigning weights of 0.7, 0.2, and 0.1 to the normal estimation loss, photometric consistency loss, and reflectivity constraint loss, respectively, to comprehensively guide the network learning; The loss function is expressed as: in, The total loss function value optimized during training. Estimate the loss term for the normal. For the photometric consistency loss term, For reflectivity constraint loss term, This represents the total number of pixels in the current training batch. Let be the normal vector predicted by the model for the nth pixel. Let be the true normal vector corresponding to the nth pixel. This is the reflectance value predicted by the model for the nth pixel. Let n be the minimum reflectivity allowed for the nth pixel based on its corresponding material category. This represents the maximum reflectivity allowed for the nth pixel based on its corresponding material category.

[0063] The weights of the convolutional and fully connected layers are initialized using Xavier and He, respectively. The initial learning rate and batch size are set using a conjugate gradient optimizer and iterated. An early stopping mechanism is introduced to prevent overfitting. The optimizer uses a conjugate gradient optimizer with an initial learning rate of 0.001, a batch size of 32, and 500 training iterations. Training stops when the validation set loss does not decrease for 20 consecutive iterations.

[0064] S323: Using a transfer learning strategy, the model pre-trained on the DiLiGenT dataset is fine-tuned on the power scenario dataset with a low learning rate, so that the model can be adapted to the specific characteristics of power grid facilities.

[0065] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0066] Example 3, referring to Figure 2 This is the third embodiment of the present invention, which provides a single-view 3D photometric reconstruction system based on machine learning, including an input preprocessing module, a feature extraction and fusion module, a preliminary normal estimation module, and an improved SVD optimization module.

[0067] The input preprocessing module is used to convert the color image into a single-channel grayscale image that reflects brightness information by weighted averaging, to filter light sources based on a preset intensity threshold, to retain effective light sources and to normalize the direction vector, and to map all data to a unified numerical range by linear normalization, thereby constructing an input vector with a unified format and fixed dimensions.

[0068] The feature extraction and fusion module is used to employ a hierarchical multi-scale convolutional neural network structure to abstract full-scale feature information from the input data through convolution operations, and integrates a channel attention fusion mechanism to assign weights to each channel and perform adaptive weighted fusion of features.

[0069] The preliminary normal estimation module is used to receive the fused feature map output by feature extraction and fusion, perform nonlinear transformation and dimensionality compression through a fully connected layer, output the three-dimensional coordinates of each pixel, and perform normalization processing on the output results.

[0070] The improved SVD optimization module is used to dynamically adjust the contribution of each light source in the calculation according to the imaging quality of each light source by introducing an adaptive weighting mechanism for the light source, and to integrate the physical prior constraints of material reflectivity, using the reflection characteristics of different materials in the power grid facilities as optimization boundary conditions, and outputting accurate normal vectors.

[0071] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0072] Example 4, the fourth embodiment of the present invention, differs from the previous three embodiments in that: If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0074] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0075] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

Claims

1. A single-view 3D photometric reconstruction method based on machine learning, characterized in that: include, Acquire single-view images and light source direction information of the target object under different light source conditions, and preprocess the images and light source direction information to generate standardized multidimensional input data; The standardized input data is subjected to feature extraction and fusion to obtain a discriminative fused feature representation. Based on the fused feature representation, a preliminary 3D surface normal is estimated, and a preliminary normal vector is output. The initial normal vector is iteratively optimized, and combined with the reliability assessment of the light source and the constraints of prior material knowledge, the optimized accurate normal vector is obtained. Based on the accurate normal vector, a three-dimensional model of the target object is constructed through a three-dimensional geometric reconstruction algorithm.

2. The single-view 3D photometric reconstruction method based on machine learning as described in claim 1, characterized in that: The preprocessing includes converting the acquired single-view color image into a single-channel image, screening and evaluating all light sources based on the overall brightness level of the image area under each light source illumination, and retaining effective light sources that meet the preset reliability conditions. The direction vectors of effective light sources are normalized. The pixel brightness values ​​of a single-channel image are mapped to the normalized direction vectors of all effective light sources by normalizing their numerical ranges. The normalized pixel brightness values ​​are then combined with the direction vectors of all light sources in a preset order to form a fixed-dimensional vector.

3. The single-view 3D photometric reconstruction method based on machine learning as described in claim 2, characterized in that: The feature extraction and fusion process includes reconstructing a fixed-dimensional vector into a two-dimensional feature map and inputting it into a hierarchical feature extraction network. The feature extraction network extracts and abstracts features of various scales, from local details to global structure, from the input data through multiple layers. An adaptive feature fusion mechanism is adopted to analyze all feature channels output by the final layer of the feature extraction network. Based on the importance of each feature channel to the final 3D geometry estimation task, a weight coefficient is calculated for each channel. The original feature channels are then weighted and fused using the calculated weight coefficients to generate a fused feature representation.

4. The single-view 3D photometric reconstruction method based on machine learning as described in claim 3, characterized in that: The iterative optimization includes constructing an observation model that integrates the preliminary normal vector and the light source direction information, designing a light source reliability assessment mechanism, and assigning dynamically adjusted confidence weights to each light source in the observation model based on the actual performance of each light source during the imaging process. By integrating prior physical knowledge, constraints on changes in surface reflection properties are formed. Based on the weighted observation model and with the objective of satisfying the physical constraints, a joint optimization function is constructed. By iteratively solving the joint optimization function, the initial input normal vector is globally adjusted and corrected, and the accurate normal vector is output.

5. The single-view 3D photometric reconstruction method based on machine learning as described in claim 4, characterized in that: The hierarchical feature extraction network includes: a first convolutional layer performing a convolution operation on the input single-channel feature map to extract primary features of edges and textures; a second convolutional layer performing a convolution operation on the output of the first convolutional layer to fuse the primary features into intermediate features representing local contours; and a third convolutional layer performing a convolution operation on the output of the second convolutional layer to aggregate the intermediate features and generate high-level features containing semantic information of the target component. The formula for a three-layer convolution operation is expressed as follows: in, This is the feature map output by the first convolutional layer. For the input feature map, This is the weight matrix of the first convolutional layer. This refers to the bias term of the first convolutional layer. For convolution operations, For activation function, This is the feature map output by the second convolutional layer. This is the bias term for the second convolutional layer. This is the weight matrix of the second convolutional layer. The feature map output by the third convolutional layer. This is the weight matrix of the third convolutional layer. This refers to the bias term of the third convolutional layer; The adaptive feature fusion mechanism includes: performing global average pooling on the feature map output by the third convolutional layer along the spatial dimension to compress and obtain channel feature vectors; inputting the feature vectors into the first fully connected layer and processing them with the ReLU activation function; inputting them into the second fully connected layer and normalizing them with the Sigmoid activation function to obtain channel attention weight vectors; multiplying each weight value in the weight vector element-wise with all pixel values ​​of the corresponding channel in the original feature map to output the final fused feature map. The formula for obtaining the channel feature vector is expressed as: in, Let be the average of the feature vectors of the c channels. The height of the feature map, The width of the feature map. The output feature map of the third convolutional layer is used to determine the feature value at the c-th channel position (i, j). For channel indexing; The formula for a fully connected layer is expressed as follows: in, For intermediate feature vectors, This is the weight matrix of the first fully connected layer. The channel feature vectors are input to the first fully connected layer. For the bias term of the first fully connected layer, For activation function, Here is the channel attention weight vector. It is the Sigmoid activation function. This is the weight matrix of the second fully connected layer. For the bias term of the second fully connected layer, It is a natural constant. For input data, The fused feature map This is the channel-by-channel multiplication operator.

6. The single-view 3D photometric reconstruction method based on machine learning as described in claim 5, characterized in that: The process of obtaining the optimized accurate normal vector includes: calculating the light source weights based on the average gray intensity of the image under each light source; constructing a diagonal matrix with the light source weights as the main diagonal; multiplying the weight matrix with the light source direction matrix to obtain a weighted photometric matrix; assigning a priori reflectance values ​​to each pixel in the image based on the prior knowledge of the material of different parts of the target object; and constructing a diagonal reflectance constraint matrix. The formula for calculating average gray intensity is expressed as: in, The average gray intensity of the light source, The total number of effective light sources, Let be the average gray intensity of the image under the m-th light source; Singular value decomposition is performed on the weighted luminance matrix to obtain the left singular matrix, singular value matrix and right singular matrix. Combined with the reflectance constraint matrix, the orthogonal matrix is ​​solved. Using the solved orthogonal matrix and singular value matrix, the final optimized precise normal matrix and reflectance matrix are calculated. The optimized formula for the precise normal is expressed as follows: in, For the optimized precise normal matrix, For the weighted observation grayscale matrix, Here is the light source direction matrix. Let be the normal matrix to be solved. Here is the reflectivity matrix. The square of the L2 norm. For regularization parameters, This is the reflectivity constraint loss term.

7. The single-view 3D photometric reconstruction method based on machine learning as described in claim 6, characterized in that: The construction of the three-dimensional model of the target object includes: using the optimized and corrected accurate normal vector field, using the Poisson reconstruction algorithm to approximate the normal field as the gradient field of the three-dimensional surface, constructing and solving the Poisson equation to obtain the implicit function defined on the three-dimensional voxel mesh, and generating the three-dimensional point cloud model of the target by extracting the isosurface of the implicit function. The Poisson reconstruction algorithm is expressed as follows: in, For the Laplace operator, Let be the implicit function to be found. Let x be the Laplace value of the implicit function at point x. For divergence operators, For the optimized precise normal vector field, Let be the divergence of the normal vector field. Let x be the second-order partial derivative with respect to the variable x. Let be the second-order partial derivative with respect to the variable y. Let z be the second-order partial derivative with respect to the variable z. To optimize the components of the normal in the x-direction, To optimize the components of the normal in the y-direction, To optimize the components of the normal in the z-direction, This is a transpose operation; Post-processing includes using a statistical filtering strategy to remove outliers that are more than three standard deviations away from the centroid of the point cloud to eliminate noise, and performing iterative nearest-point registration between the filtered reconstructed point cloud and the standard 3D model of the power grid facility to achieve accurate alignment of the point cloud.

8. A machine learning-based single-view 3D photometric reconstruction system, employing the machine learning-based single-view 3D photometric reconstruction method as described in any one of claims 1 to 7, characterized in that, It includes an input preprocessing module, a feature extraction and fusion module, a preliminary normal estimation module, and an improved SVD optimization module; The input preprocessing module is used to convert the color image into a single-channel grayscale image that reflects brightness information by weighted averaging, to filter the light source based on a preset intensity threshold, to retain the effective light source and to normalize the direction vector, and to map all data to a unified numerical range by linear normalization, thereby constructing an input vector with a unified format and fixed dimensions. The feature extraction and fusion module is used to employ a hierarchical multi-scale convolutional neural network structure, abstract full-scale feature information from the input data through convolution operations, and integrate a channel attention fusion mechanism to assign weights to each channel and perform adaptive weighted fusion of features. The preliminary normal estimation module is used to receive the fused feature map output by feature extraction and fusion, perform nonlinear transformation and dimensionality compression through a fully connected layer, output the three-dimensional coordinates of each pixel, and perform normalization processing on the output results. The improved SVD optimization module is used to dynamically adjust the contribution of each light source in the calculation according to the imaging quality of each light source by introducing an adaptive weighting mechanism for the light source, and to integrate the physical prior constraints of material reflectivity, using the reflection characteristics of different materials in the power grid facilities as optimization boundary conditions, and outputting accurate normal vectors.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the single-view three-dimensional photometric reconstruction method based on machine learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the single-view three-dimensional photometric reconstruction method based on machine learning as described in any one of claims 1 to 7.

Images