A high-precision normal vector solution method based on deep learning and polarization physics guidance

Abstract

The present invention belongs to the field of polarization three-dimensional technology, and in particular, relates to a high-precision normal vector solution method based on deep learning and polarization physics guidance. It includes: S1: collecting the normal vector truth map and polarization image data set of the target object, and making an augmented polarization image data set; S2: building a deep neural network based on the polarization characteristics of the reflected light; S3: building an accuracy-structure consistency loss function, and training the deep neural network in combination with the accuracy-structure consistency loss function to obtain a trained deep neural network; S4: processing the target object to be calculated according to step S1, obtaining the augmented polarization image data of the target object to be calculated, inputting the augmented polarization image data of the target object to be calculated into the trained deep neural network, and outputting the surface normal vector. The normal vector solution method of the present invention has the characteristics of high precision, strong generalization, strong illumination and robustness.

Description

Technical Field

[0001] The present invention belongs to the field of polarization three-dimensional technology, and in particular relates to a high-precision normal vector solution method based on deep learning and polarization physics guidance. Background Art

[0002] Polarization 3D imaging has attracted extensive attention due to its unique advantages in low-texture objects (polarization 3D imaging does not rely on the texture features of the object surface and can effectively suppress glare. For low-texture and highly reflective objects, polarization 3D imaging can obtain complete 3D surface information). However, due to factors such as detector performance, optical system errors, and normal direction uncertainty, the accuracy of the actual solution algorithm vector is poor. With the development of deep learning technology, deep learning technology has provided a new solution for polarization 3D imaging. However, the existing deep learning-based polarization 3D imaging methods still have deficiencies in algorithm accuracy and texture details. Specifically, (1) Insufficient data training: The performance of deep learning models depends to a large extent on the quantity and quality of training data. At present, there are few dedicated training data sets for polarization 3D imaging, which limits the improvement of model performance. (2) Model complexity and computational cost: In order to improve imaging accuracy and detail expression, deep learning models often have high complexity, which will lead to increased computational cost and reduced real-time performance. (3) Insufficient algorithm optimization: In view of the particularity of polarization 3D imaging, how to design a reasonable network structure, loss function, and optimization algorithm still needs further research and exploration. Summary of the invention

[0003] In view of this, the present invention aims to provide a high-precision normal vector solution method based on deep learning and polarization physics guidance to solve the problems of low accuracy and strong light source dependence of traditional polarization three-dimensional imaging algorithms. The present invention utilizes the powerful feature extraction and data fitting capabilities of deep learning, and designs a FMR_ResUnet network (deep neural network) suitable for polarization three-dimensional imaging based on the Unet framework. It combines the physical prior conditions of traditional deep learning and uses a precision-structure comprehensive loss function to constrain training, thereby achieving high-precision normal vector solution.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0005] A high-precision normal vector solution method based on deep learning and polarization physics guidance specifically includes the following steps:

[0006] S1: Collect the normal vector truth map and polarization image dataset of the target object, and create an augmented polarization image dataset;

[0007] S2: Building a deep neural network based on the polarization characteristics of reflected light;

[0008] S3: constructing an accuracy-structure consistency loss function, and training the deep neural network in combination with the accuracy-structure consistency loss function to obtain a trained deep neural network;

[0009] S4: Process the target object to be calculated according to step S1 to obtain augmented polarization image data of the target object to be calculated, input the augmented polarization image data of the target object to be calculated into a trained deep neural network, and output a surface normal vector.

[0010] Furthermore, step S1 specifically includes the following steps:

[0011] S11: Use a polarization camera to collect images of the target object to obtain a polarization image dataset, which includes a polarization image , polarization image , polarization image and polarization images ;

[0012] S12: using a high-precision scanner to collect a three-dimensional topography image of the target object, and processing the three-dimensional topography image based on a central finite difference format to obtain a normal vector true value image;

[0013] S13: calibrating the full-field Mueller matrix of the current lens of the polarization camera, calculating the original Stokes vector of the incident light incident to the polarization camera based on the full-field Mueller matrix, and simulating the polarization images under different lenses based on the distribution law of the full-field Mueller matrix, so as to realize one-time augmentation of the polarization image data set;

[0014] S14: Record the full well charge and A2D bit number of the polarization camera, add Gaussian noise and Poisson noise to the area with a zenith angle less than 20° in the normal vector truth image, and realize secondary augmentation of the polarization image data set;

[0015] S15: Polarization image based on secondary augmentation , polarization image , polarization image and polarization images , calculate the polarization degree diagram ;

[0016] S16: Calculate the diffuse reflection zenith angle map using the polarization map and diffuse reflection model , and use the polarization degree diagram and the specular reflection model to calculate the first specular reflection zenith angle diagram and the second specular reflection zenith angle diagram ;

[0017] S17: Polarization image dataset and polarization degree map based on secondary augmentation , Diffuse reflection zenith angle map , First specular reflection zenith angle diagram and the second specular reflection zenith angle diagram Construct tensor input:

[0018] ;

[0019] S18: Divide the tensor into n images, perform three augmentations on the polarization image dataset, and obtain an augmented polarization image dataset.

[0020] Furthermore, in the normal vector truth map, a spatial rectangular coordinate system is constructed with the exit pupil center of the polarization camera as the origin, the XOY plane coincides with the target surface of the polarization camera, and the Z axis is perpendicular to the XOY plane.

[0021] Furthermore, the deep neural network includes an input module, a first encoder, a second encoder, a third encoder, a fourth encoder, an intermediate module, a first decoder, a second decoder, a third decoder, a fourth decoder and an output module, wherein the augmented polarization image data set is divided into a physical prior and a polarization image, and the physical prior includes a polarization degree map, a diffuse reflection zenith angle map , First specular reflection zenith angle diagram and the second specular reflection zenith angle diagram , polarization image includes polarization image , polarization image , polarization image and polarization images , the physical prior and polarization images are spliced ​​by channel to obtain integrated features, the integrated features are input into the input module for 1×1 convolution processing to obtain feature A1, feature A1 is input into the first encoder for processing to obtain feature A2, feature A2 is input into the second encoder for processing to obtain feature A3, feature A3 is input into the third encoder for processing to obtain feature A4, feature A4 is input into the fourth encoder for processing to obtain feature A5, feature A5 is input into the intermediate module for processing to obtain feature A6;

[0022] After feature A5 is processed by the feature modification module, feature B1 is obtained. After the polarization image is down-sampled by 1 / 16, feature C1 is obtained. Feature A6, feature B1 and feature C1 are input to the first decoder for processing to obtain feature A7. After feature A4 is processed by the feature modification module, feature B2 is obtained. After the polarization image is down-sampled by 1 / 8, feature C2 is obtained. Feature A7, feature B2 and feature C2 are input to the second decoder for processing to obtain feature A8. After feature A3 is processed by the feature modification module, feature B3 is obtained. After the polarization image is down-sampled by 1 / 4, feature C3 is obtained. Feature A8, feature B3 and feature C3 are input to the third decoder for processing to obtain feature A9. After feature A2 is processed by the feature modification module, feature B4 is obtained. After the polarization image is down-sampled by 1 / 2, feature C4 is obtained. Feature A9, feature B4 and feature C4 are input to the fourth decoder for processing to obtain feature A10.

[0023] Feature A10, integrated feature and feature A1 are input into the output module for processing to obtain the surface normal vector.

[0024] Furthermore, the feature modification module processes the input features in the following steps:

[0025] Perform 1×1 convolution on the input feature F1 to obtain the first tensor F2, and flatten the first tensor F2 to obtain the second tensor F3;

[0026] For the second tensor F3 Softmax Transform to obtain feature map F4:

[0027] ;

[0028] in, is the i-th layer feature map of the second tensor F3;

[0029] Flatten the input feature F1 to obtain the third tensor F5;

[0030] Multiply the feature map F4 by the third tensor F5 to obtain the autocorrelation map F6;

[0031] Perform expansion and contraction processing on the autocorrelation graph F6 to obtain the scale factor and bias factor ;

[0032] The input feature F1 and the scaling factor The feature map F7 and the bias factor obtained after multiplication Add together to obtain the modified output feature map F8.

[0033] Furthermore, in step S3, the accuracy-structure consistency loss function is:

[0034] ;

[0035] in, is the accuracy-structure consistency loss function, is the cosine consistency loss function, is the multi-scale structure loss function.

[0036] Furthermore, the cosine consistency loss function is:

[0037] ;

[0038] in, is the pixel point in the normal vector truth map The true value of the normal vector, is the pixel point in the normal vector truth map The predicted value of the normal vector is is the symbol for the dot product of two vectors, and are the length and width of the normal vector truth map respectively.

[0039] Furthermore, the multi-scale structure loss function is:

[0040] ;

[0041] in, is the size of the window, is the weight under each window, SSIM is the function for calculating the structural similarity of the target object, N is the normal vector truth map, Normal vector prediction map output by the deep neural network.

[0042] Furthermore, the function for calculating the structural similarity of the target object is:

[0043] ;

[0044] in, is the pixel mean of the true normal vector image of the target object, is the pixel mean of the normal vector prediction image of the target object, is the covariance between the true normal vector image and the predicted normal vector image of the target object, is the pixel variance of the true value image of the normal vector of the target object, The pixel variance of the normal vector prediction image of the target object, C1 and C2 are constants close to 0.

[0045] Compared with the prior art, the invention can achieve the following beneficial effects:

[0046] (1) The high-precision normal vector solution method based on deep learning and polarization physics guidance created by the present invention utilizes the third augmentation of the data set to expand multiple sets of data based on a limited real-world data set, simulate polarization three-dimensional imaging at different distances and in different optical systems, and enhance the generalization performance of the network.

[0047] (2) The feature modification module created by the present invention starts from the characteristics of the polarized reflection light field and adjusts the feature map layer by layer in a weighted manner to achieve the purpose of separating the specular reflection features and the diffuse reflection features, thereby enhancing the feature extraction capability of the network.

[0048] (3) The precision-structure consistency loss function created by the present invention comprehensively considers the overall precision and local structure of the output normal vector, so that the output normal vector has fewer artifacts and the structural continuity is enhanced, which facilitates the restoration of the three-dimensional morphology of the surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0050] Figure 1 A flowchart of a high-precision normal vector solution method based on deep learning and polarization physics guidance described in an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a coordinate system of a normal vector true value diagram according to an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the network structure of a deep neural network according to an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of the network structure of the feature modification module described in the embodiment of the present invention.

[0054] Description of reference numerals:

[0055] 1. Polarization camera; 2. Input module; 3. First encoder; 4. Second encoder; 5. Third encoder; 6. Fourth encoder; 7. Intermediate module; 8. First decoder; 9. Second decoder; 10. Third decoder; 11. Fourth decoder; 12. Output module. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0057] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0058] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0059] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0060] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0061] like Figure 1 As shown, the high-precision normal vector solution method based on deep learning and polarization physics guidance provided by the present invention specifically includes the following steps:

[0062] S1: Collect the normal vector truth map and polarization image dataset, record the imaging system information, and produce the augmented polarization image dataset.

[0063] S11: Use polarization camera 1 to collect images of the target object to obtain a polarization image dataset, wherein the polarization image dataset includes polarization images collected at four angles: 0°, 45°, 90°, and 135°. , polarization image , polarization image and polarization images .

[0064] S12: using a high-precision scanner to collect a three-dimensional topography image of the target object, and processing the three-dimensional topography image based on a central finite difference format (obtaining a gradient image in the x direction and a gradient image in the y direction of the surface of the target object through the central finite difference format), and obtaining a normal vector true value image N;

[0065] like Figure 2 As shown in the normal vector true value diagram, a spatial rectangular coordinate system is constructed with the exit pupil center of the polarization camera 1 (or a reference point near the target object) as the origin. Figure 2 In the figure, the normal vector origin O is the optical center of the polarization camera 1, the XoY plane is the detection target surface of the polarization camera 1, the Z axis is perpendicular to the XoY plane, and the XYZ axes are mutually perpendicular, satisfying the right-hand criterion.

[0066] S13: Calibrate the full-field Mueller matrix of the current lens of the polarization camera 1, calculate the original Stokes vector of the incident light incident to the polarization camera 1 based on the full-field Mueller matrix, and simulate the polarization images under different lenses based on the distribution law of the full-field Mueller matrix (the polarization image here is the Stokes vector obtained after simulation), so as to achieve an augmentation of the polarization image data set.

[0067] The distribution law of the full-field Mueller matrix is ​​recorded in the patent "Calculation Method of Mueller Matrix of Optical System" with publication number CN114295212B.

[0068] S14: Record the full well charge and A2D bit number of polarization camera 1, add Gaussian noise and Poisson noise to the area where the zenith angle of the normal vector in the normal vector truth image is less than 20°, and realize the secondary augmentation of the polarization image data set.

[0069] The method of adding Gaussian noise and Poisson noise is consistent with the method described in the paper "Analysis of Polarization Detector Performance Parameters on Polarization 3D Imaging Accuracy" from the journal "Sensors".

[0070] S15: Polarization image based on secondary augmentation , polarization image , polarization image and polarization images , calculate the polarization degree diagram .

[0071] S16: Calculate the diffuse reflection zenith angle map using the polarization map and diffuse reflection model , and use the polarization degree diagram and the specular reflection model to calculate the first specular reflection zenith angle diagram and the second specular reflection zenith angle diagram ;

[0072] Both the diffuse reflection model and the specular reflection model are traditional polarization 3D imaging models.

[0073] S17: Polarization image dataset, polarization degree map, diffuse reflection zenith angle map, first specular reflection zenith angle map based on secondary augmentation and the second specular reflection zenith angle diagram Construct tensor input:

[0074] .

[0075] S18: Divide the tensor into n (n≥100) small-resolution images, realize the third augmentation of the polarization image dataset, and obtain the augmented polarization image dataset.

[0076] S2: Build a deep neural network based on the polarization characteristics of reflected light.

[0077] S3: Construct an accuracy-structure consistency loss function, and train the deep neural network in combination with the accuracy-structure consistency loss function to obtain a trained deep neural network.

[0078] In step S3, the accuracy-structure consistency loss function is:

[0079] ;

[0080] in, is the accuracy-structure consistency loss function, is the cosine consistency loss function, is the multi-scale structure loss function.

[0081] The cosine consistency loss function is:

[0082] ;

[0083] in, is the pixel point in the normal vector truth map The true value of the normal vector, is the pixel point in the normal vector truth map The predicted value of the normal vector is is the symbol for the dot product of two vectors, and are the length and width of the normal vector truth map respectively.

[0084] The sign of the dot product of two vectors is calculated as:

[0085] ;

[0086] A and B are space vectors, and is the magnitude of the vector, and are the first and second order space vectors A and B respectively. i elements, eps is a minimum value, in order to ensure that there is no division by 0, it is generally taken .

[0087] The multi-scale structure loss function is:

[0088] ;

[0089] in, is the size of the window, typical window values ​​are 3, 5, 7, 9, and 11, is the weight under each window, SSIM is the function for calculating the structural similarity of the target object, N is the normal vector truth map, Normal vector prediction map output by the deep neural network.

[0090] The function for calculating the structural similarity of the target object is:

[0091] .

[0092] in, is the pixel mean of the true normal vector image of the target object, is the pixel mean of the normal vector prediction image of the target object, is the covariance between the true normal vector image and the predicted normal vector image of the target object, is the pixel variance of the true value image of the normal vector of the target object, The pixel variance of the normal vector prediction image of the target object, C1 and C2 are constants close to 0.

[0093] The augmented polarization image data set of step S1 is divided into a training set and a validation set at a ratio of 9:1. The deep neural network is first trained using the training set, and then the deep neural network is converged using the validation set. The learning rate of the deep neural network is dynamically adjusted, and the gradient is adjusted to be large at the beginning and small at the end. An early stopping parameter is set, and the training is stopped when the accuracy of the deep neural network output is not optimized within the specified number of training rounds.

[0094] S4: Process the target object to be calculated according to step S1 to obtain augmented polarization image data of the target object to be calculated, input the augmented polarization image data of the target object to be calculated into a trained deep neural network, and output a surface normal vector.

[0095] like Figure 3 As shown, the deep neural network includes an input module 2, a first encoder 3, a second encoder 4, a third encoder 5, a fourth encoder 6, an intermediate module 7, a first decoder 8, a second decoder 9, a third decoder 10, a fourth decoder 11 and an output module 12, wherein the augmented polarization image data set is divided into a physical prior and a polarization image, and the physical prior includes a polarization degree map, a diffuse reflection zenith angle map , First specular reflection zenith angle diagram and the second specular reflection zenith angle diagram , polarization image includes polarization image , polarization image , polarization image and polarization images , the physical prior and polarization images are spliced ​​by channel to obtain integrated features, the integrated features are input to the input module 2 for 1×1 convolution processing, and feature A1 is obtained, feature A1 is input to the first encoder 3 for processing, and feature A2 is obtained, feature A2 is input to the second encoder 4 for processing, and feature A3 is obtained, feature A3 is input to the third encoder 5 for processing, and feature A4 is obtained, feature A4 is input to the fourth encoder 6 for processing, and feature A5 is obtained, feature A5 is input to the intermediate module 7 for processing, and feature A6 is obtained.

[0096] After feature A5 is processed by the feature modification module, feature B1 is obtained. After the polarization image is down-sampled by 1 / 16, feature C1 is obtained. Feature A6, feature B1 and feature C1 are input to the first decoder 8 for processing to obtain feature A7. After feature A4 is processed by the feature modification module, feature B2 is obtained. After the polarization image is down-sampled by 1 / 8, feature C2 is obtained. Feature A7, feature B2 and feature C2 are input to the second decoder 9 for processing to obtain feature A8. After feature A3 is processed by the feature modification module, feature B3 is obtained. After the polarization image is down-sampled by 1 / 4, feature C3 is obtained. Feature A8, feature B3 and feature C3 are input to the third decoder 10 for processing to obtain feature A9. After feature A2 is processed by the feature modification module, feature B4 is obtained. After the polarization image is down-sampled by 1 / 2, feature C4 is obtained. Feature A9, feature B4 and feature C4 are input to the fourth decoder 11 for processing to obtain feature A10.

[0097] The feature A10, the integrated feature and the feature A1 are input to the output module 12 for processing to obtain the surface normal vector.

[0098] The workflow of the deep neural network is as follows: the tensor data set input passes through the input module 2, and several types of data are stacked in the channel dimension; the stacked data passes through each encoder for feature extraction, and the features output by each encoder are reshaped by the feature modification module and recorded as E i , where i represents the i-th encoder; the feature map D obtained by upsampling the feature map output by the previous decoder by modulating each decoder using the downsampled polarization image i ; D i and E i The data are spliced ​​by channel and used as one of the input data of the next decoder; finally, the output of the normal vector is completed through the output module 12.

[0099] like Figure 4 As shown, the feature modification module processes the input features in the following steps:

[0100] The original size is The input feature F1 is processed by 1×1 convolution to obtain a size of The first tensor F2 is flattened to obtain a size of The second tensor F3;

[0101] For the second tensor F3 Softmax Transform to obtain feature map F4:

[0102] ;

[0103] in, is the i-th layer feature map of the second tensor F3;

[0104] The original size is The input feature F1 is flattened to a size of The third tensor F5;

[0105] Multiply the feature map F4 with the third tensor F5 to obtain a size of Autocorrelation plot F6;

[0106] Perform expansion and contraction processing on the autocorrelation graph F6 to obtain the scale factor and bias factor ;

[0107] The input feature F1 and the scaling factor The feature map F7 and the bias factor obtained after multiplication Add together to obtain the modified output feature map F8.

[0108] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0109] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A high-precision normal vector solution method based on deep learning and polarization physics guidance, characterized by: The specific steps include: S1: Collect the normal vector truth map and polarization image dataset of the target object, and create an augmented polarization image dataset; S2: Building a deep neural network based on the polarization characteristics of reflected light; The deep neural network includes an input module, a first encoder, a second encoder, a third encoder, a fourth encoder, an intermediate module, a first decoder, a second decoder, a third decoder, a fourth decoder and an output module, wherein the augmented polarization image data set is divided into a physical prior and a polarization image, and the physical prior includes a polarization degree map, a diffuse reflection zenith angle map , First specular reflection zenith angle diagram and the second specular reflection zenith angle diagram , polarization image includes polarization image , polarization image , polarization image and polarization images , the physical prior and polarization images are spliced ​​by channel to obtain integrated features, the integrated features are input into the input module for 1×1 convolution processing to obtain feature A1, feature A1 is input into the first encoder for processing to obtain feature A2, feature A2 is input into the second encoder for processing to obtain feature A3, feature A3 is input into the third encoder for processing to obtain feature A4, feature A4 is input into the fourth encoder for processing to obtain feature A5, feature A5 is input into the intermediate module for processing to obtain feature A6; After feature A5 is processed by the feature modification module, feature B1 is obtained. After the polarization image is down-sampled by 1 / 16, feature C1 is obtained. Feature A6, feature B1 and feature C1 are input to the first decoder for processing to obtain feature A7. After feature A4 is processed by the feature modification module, feature B2 is obtained. After the polarization image is down-sampled by 1 / 8, feature C2 is obtained. Feature A7, feature B2 and feature C2 are input to the second decoder for processing to obtain feature A8. After feature A3 is processed by the feature modification module, feature B3 is obtained. After the polarization image is down-sampled by 1 / 4, feature C3 is obtained. Feature A8, feature B3 and feature C3 are input to the third decoder for processing to obtain feature A9. After feature A2 is processed by the feature modification module, feature B4 is obtained. After the polarization image is down-sampled by 1 / 2, feature C4 is obtained. Feature A9, feature B4 and feature C4 are input to the fourth decoder for processing to obtain feature A10. Input feature A10, integrated feature and feature A1 to the output module for processing to obtain the surface normal vector; The feature modification module processes the input features in the following steps: Perform 1×1 convolution on the input feature F1 to obtain the first tensor F2, and flatten the first tensor F2 to obtain the second tensor F3; For the second tensor F3 Softmax Transform to obtain feature map F4: ； in, is the i-th layer feature map of the second tensor F3, and are the length and width of the normal vector truth map respectively; Flatten the input feature F1 to obtain the third tensor F5; Multiply the feature map F4 by the third tensor F5 to obtain the autocorrelation map F6; Perform expansion and contraction processing on the autocorrelation graph F6 to obtain the scale factor and bias factor ; The input feature F1 and the scaling factor The feature map F7 and the bias factor obtained after multiplication Add together to obtain the modified output feature map F8; S3: constructing an accuracy-structure consistency loss function, and training the deep neural network in combination with the accuracy-structure consistency loss function to obtain a trained deep neural network; The precision-structure consistency loss function is: ； in, is the accuracy-structure consistency loss function, is the cosine consistency loss function, is the multi-scale structure loss function; S4: Process the target object to be calculated according to step S1 to obtain augmented polarization image data of the target object to be calculated, input the augmented polarization image data of the target object to be calculated into a trained deep neural network, and output a surface normal vector.

2. The high-precision normal vector solution method based on deep learning and polarization physics guidance according to claim 1 is characterized in that: Step S1 specifically includes the following steps: S11: Use a polarization camera to collect images of the target object to obtain a polarization image dataset, wherein the polarization image dataset includes a polarization image , polarization image , polarization image and polarization images ; S12: using a high-precision scanner to collect a three-dimensional topography image of the target object, and processing the three-dimensional topography image based on a central finite difference format to obtain a normal vector true value image; S13: calibrating the full-field Mueller matrix of the current lens of the polarization camera, calculating the original Stokes vector of the incident light incident to the polarization camera based on the full-field Mueller matrix, and simulating the polarization images under different lenses based on the distribution law of the full-field Mueller matrix, so as to realize one-time augmentation of the polarization image data set; S14: Record the full well charge and A2D bit number of the polarization camera, add Gaussian noise and Poisson noise to the area with a zenith angle less than 20° in the normal vector truth image, and realize secondary augmentation of the polarization image data set; S15: Polarization image based on secondary augmentation , polarization image , polarization image and polarization images , calculate the polarization degree diagram ; S16: Calculate the diffuse reflection zenith angle map using the polarization map and diffuse reflection model , and use the polarization degree diagram and the specular reflection model to calculate the first specular reflection zenith angle diagram and the second specular reflection zenith angle diagram ; S17: Polarization image dataset and polarization degree map based on secondary augmentation , Diffuse reflection zenith angle map , First specular reflection zenith angle diagram and the second specular reflection zenith angle diagram Construct tensor input: ； S18: Divide the tensor into n images, perform three augmentations on the polarization image dataset, and obtain an augmented polarization image dataset.

3. The high-precision normal vector solution method based on deep learning and polarization physics guidance according to claim 2 is characterized in that: In the normal vector truth value map, a spatial rectangular coordinate system is constructed with the exit pupil center of the polarization camera as the origin, the XOY plane coincides with the target surface of the polarization camera, and the Z axis is perpendicular to the XOY plane.

4. The high-precision normal vector solution method based on deep learning and polarization physics guidance according to claim 1, characterized in that: The cosine consistency loss function is: ； in, is the pixel point in the normal vector truth map The true value of the normal vector, is the pixel point in the normal vector truth map The predicted value of the normal vector is is the symbol for the dot product of two vectors, and are the length and width of the normal vector truth map respectively.

5. The high-precision normal vector solution method based on deep learning and polarization physics guidance according to claim 1 is characterized in that: The multi-scale structure loss function is: ； in, is the size of the window, is the weight under each window, SSIM is the function for calculating the structural similarity of the target object, N is the normal vector truth map, Normal vector prediction map output by the deep neural network.

6. The high-precision normal vector solution method based on deep learning and polarization physics guidance according to claim 5, characterized in that: The function for calculating the structural similarity of the target object is: ； in, is the pixel mean of the true normal vector image of the target object, is the pixel mean of the normal vector prediction image of the target object, is the covariance between the true normal vector image and the predicted normal vector image of the target object, is the pixel variance of the true value image of the normal vector of the target object, The pixel variance of the normal vector prediction image of the target object, C1 and C2 are non-zero constants.

Images