Polarization generative adversarial network method based on physical consistency constraint

Abstract

The application discloses a polarization generative adversarial network method based on physical consistency constraints, and aims to solve the problem that existing monocular RGB image polarization information generation seriously depends on pure data driving and is easy to generate distorted characteristics contrary to optical common sense. A physical guided generative adversarial network framework is provided, a learnable Sobel edge module is introduced to extract high-frequency gradient features, a hybrid attention channel decoupling mechanism is combined to isolate different waveband polarization responses, and a regional refractive index modeling module and a Fresnel reflection physical modulation layer are embedded, forward physical inversion is realized by explicitly calculating Stokes parameters through prediction of an incident angle; meanwhile, a physical discriminator is proposed to determine whether the generated result conforms to the Fresnel reflection surface, the Brewster angle gradient relationship and the AoP spatial periodic consistency; a joint physical loss function is supervised, which contains a multi-channel energy conservation constraint and an AoP sine and cosine periodic loss, the network solution space is narrowed, high-robustness polarization perception is realized, and the polarization perception is deployed in visual tasks such as transparent obstacle detection and automatic driving at low cost.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method for extracting and generating high-fidelity polarization optical features from conventional monocular RGB images using deep neural networks, particularly a polarization generative adversarial network method based on physical consistency constraints. Background Technology

[0002] Polarization, besides intensity, wavelength (corresponding to color), and phase, is another crucial intrinsic physical property of light waves. It records the vibration of the electric field vector of the light wave in a plane perpendicular to the direction of propagation. When light interacts with the surface of an object (e.g., through reflection, refraction, and scattering), its polarization state changes significantly. These changes contain high-dimensional physical information completely missing from conventional RGB images, such as the object's microscopic geometric normal vectors, material refractive index properties, and surface roughness. In complex vision tasks such as autonomous driving and robot navigation (e.g., detecting transparent obstacles like glass, or losing targets due to strong specular reflection), pure RGB images are highly susceptible to drastic drops in local information entropy due to highly uniform background camouflage, uneven lighting, or the transparency of the material itself, leading to the complete failure of conventional vision algorithm systems. Polarization imaging technology, through the joint analysis of the angle of polarization (AoP) and the degree of polarization (DoP), can provide physical dimensional clues orthogonal to texture features, effectively overcoming the aforementioned visual perception bottlenecks.

[0003] However, a review of current mainstream technologies and cutting-edge research approaches for acquiring polarization information reveals significant and insurmountable technical limitations and developmental bottlenecks. Through in-depth online literature retrieval (including Google Scholar, IEEE Xplore, and major patent databases) and systematic technology novelty searches, existing technologies are mainly categorized into three major technical routes, each with its own pain points analyzed below:

[0004] 1. Traditional polarization imaging hardware methods Focal plane micropolarization array (DoFP) camera, time-division rotating polarizer system It relies heavily on expensive, dedicated optical hardware. DoFP cameras cover the CMOS pixel sensor with periodically arranged... The four-directional tiny metal wire grid polarization filter. This physical blockage causes a severe attenuation of light intensity of up to 50%, resulting in an exponential deterioration of the signal-to-noise ratio (SNR) of the imaging system in low-light or dim lighting environments. In addition, the mosaic arrangement inherently reduces spatial resolution by half, and the hardware cost is extremely high, which seriously hinders its large-scale deployment in consumer devices and low-cost autonomous driving fields. 2. A pure deep learning mapping method based on RGB to polarization generation Purely data-driven regression networks (such as TransLab, TOSQ, and other CNN-based networks in the literature). These methods attempt to establish a "black box" regression model that directly maps RGB pixels to AoP / DoP values. However, because deep convolutional layers themselves lack any optical prior knowledge or theoretical constraints, the model merely memorizes the statistical regularities of surface textures in the training set images, resulting in extremely poor generalization ability. The polarization features it predicts are highly prone to violating fundamental optical physics equations: for example, the model might predict abrupt polarization noise on a physically uniform and smooth surface, or generate parameters that severely violate Stokes' law of conservation of energy, causing downstream tasks (such as 3D reconstruction) to fail. 3. Traditional GAN ​​Image Generative Adversarial Techniques Standard image translation GANs (such as Pix2Pix, conditional GAN, and other cross-modal transformation frameworks) While traditional GANs excel at generating visually extremely realistic textures and patterns, their core discriminator optimization is limited to determining the distribution distance between the generated image domain and the real image domain in a low-dimensional feature manifold space. When faced with polarization generation tasks that involve deep-seated optical physics information, conventional discriminators are completely unable to understand what "physical plausibility" means, often leading to "mode collapse" or "physical hallucination"—that is, the generated polarization map looks normal at the pixel level, but in reality, the polarization states of adjacent pixels within it severely violate Snell's law of refraction and Fresnel's law of reflection in terms of numerical correlation.

[0005] As the above analysis shows, exploring an innovative technological path that can completely break free from the constraints of expensive and inefficient physical polarization hardware, and also eradicate the drawbacks of "black-box blind prediction" in deep networks from the deep algorithm architecture, has become a major demand in the field of computational optics imaging. There is an urgent need for an intelligent generation method that deeply integrates prior physical theorems. Summary of the Invention

[0006] To address the aforementioned problems, this invention discloses a polarization generative adversarial network method based on physical consistency constraints.

[0007] The specific technical solution is as follows:

[0008] This invention proposes a polarization generative adversarial network (GAN) method based on physical consistency constraints. By constructing a GAN framework that deeply integrates the optical reflection theorem and high-order feature mapping, it achieves high-fidelity physical mapping from monocular RGB images to polarization feature matrices. This technical solution mainly comprises three core components: a generator network with an explicit physical embedding computation layer, a rule-based physical discriminator network, and a multi-dimensional joint physical loss function.

[0009] I. Generator Networks with Explicitly Embedded Physical Computation Layers

[0010] The generator network abandons the traditional pure data-driven scalar regression model. After receiving the monocular RGB image tensor, it sequentially performs forward physical inversion of the feature stream through the following differentiable physical embedding computation layers:

[0011] The learnable Sobel-type gradient edge extraction module addresses the optical characteristic of drastic polarization changes at object contours and heterogeneous material interfaces. It reconstructs the X- and Y-direction gradient convolution kernel parameters of the traditional fixed-weight Sobel edge detection operator into learnable parameter tensors within the neural network. During training, an orthogonal gradient structure is applied to maintain regularization, constraining the X-direction kernel to maintain horizontal gradient sensitivity (alternating positive and negative polarities along the X-axis) and the Y-direction kernel to maintain vertical gradient sensitivity (alternating positive and negative polarities along the Y-axis). This allows the learnable kernel to retain the Sobel-type edge direction selection capability while adaptively enhancing the gradient magnitude response to polarization-specific material boundaries. During model backpropagation, the learnable parameters are adaptively fine-tuned using a local polarization gradient loss to enhance feature capture capabilities for high-frequency polarized edge regions such as transparent corners and highly reflective areas.

[0012] Hybrid Attention Channel Decoupling Module: To eliminate coupling interference from polarization responses of different wavelength bands, a hybrid attention mechanism with spatial and channel interleaving is used for feature isolation.

[0013] Spatial attention branch: Utilizing a depthwise separable convolution with a group number half that of the input channels, the spatial distribution response of physical features in the image plane is extracted, and then normalized and activated with a sigmoid to generate a spatially aware weight map;

[0014] Channel attention branch: The spatial information is compressed into a one-dimensional vector by global adaptive average pooling, and nonlinear dimensionality reduction and reconstruction are performed through multiple 1×1 convolutions to generate channel importance weights;

[0015] Feature fusion: Perform channel-wise tensor Hadamard product on the spatially perceived weight map and the channel importance weights, and use the fused joint weight matrix to filter the original features and remove mutually interfering spectral polarization characterization channels.

[0016] Regional refractive index modeling and Fresnel physical modulation layer:

[0017] Refractive index modeling: Multiple sets of learnable initial refractive index parameter constants are predefined, and an activation pruning function is used to forcibly truncate all predicted refractive indices participating in the forward calculation within a reasonable range of 1.3 to 2.0 for the physical medium; a built-in lightweight region allocator network maps the high-dimensional feature map to a probability distribution map of the material region. After the initial refractive index constant set is reparameterized by Gumbel-Softmax and transformed into a hard allocation decision, the corresponding single medium refractive index value is selected for each pixel from the refractive index constant set according to the decision, generating a dense pixel-by-pixel refractive index feature map;

[0018] Physical modulation: Substitute the predicted incident angle tensor and refractive index characteristic map into the differentiable Fresnel reflection equation, calculate the reflection coefficients of the vertical polarization component and the parallel polarization component respectively, and then derive the upper limit of the theoretical physical polarization reflection polarization degree to physically modulate the predicted polarization characteristics.

[0019] The Stokes parameter calculation and polar coordinate analysis module: To avoid phase reversal and gradient tearing caused by direct regression of polarization angle, the independent components of the Stokes vector (total intensity component S0, horizontal / vertical polarization difference component S1, and diagonal polarization difference component S2) are first generated. After L2 norm normalization, the sine and cosine distribution maps of the polarization double angle are calculated through trigonometric function transformation. Finally, the continuous and smooth polarization angle and degree of polarization are output through the arctangent analytical function that supports multiple quadrants.

[0020] II. Rule-based physical discriminator network

[0021] The physics discriminator network goes beyond the traditional GAN's function of merely determining the authenticity of textures, and is endowed with the function of verifying physical laws. Its core is a learnable multi-scale convolutional discriminator network.

[0022] This framework is used to determine the probability of authenticity of the generated polarization image and the real polarization image in terms of texture distribution. Furthermore, two rule-level verification modules based on deterministic physical equations are configured in parallel around the discriminator to perform Fresnel reflection algebra inverse verification and polarization derivative field correlation determination, respectively. These two verification modules do not participate in the update of learnable parameters in adversarial games; instead, they serve as computational units embedding prior physical knowledge. The resulting verification error signals are directly fed into the joint physical loss function, using rule-level gradient constraints to guide the parameter optimization direction of the generator.

[0023] Fresnel physical algebra verification branch: Receives the polarization degree, refractive index and incident angle output by the generator, substitutes them into the differentiable Fresnel reflection theory model for reverse derivation, and determines whether the physical parameters satisfy the nonlinear polarization degree attenuation relationship.

[0024] Polarization constraint derivative correlation branch: Based on Brewster's angle theorem, determine the direction of the gradient of the partial derivative of the generated degree of polarization with respect to the incident angle, and whether it exhibits an extreme value reversal law near the theoretical Brewster angle; at the same time, determine the periodic consistency and smooth continuity of the generated polarization angle in the two-dimensional spatial field through the discrete Laplace operator.

[0025] Data distribution true / false game branch: Based on the receptive field of multi-scale convolution kernel, determine the true / false probability of generated polarized images and real polarized images in terms of micro-texture distribution and pixel manifold mapping.

[0026] III. Multi-dimensional Joint Physical Loss Function

[0027] To drive the aforementioned physical computation network to perform smooth parameter optimization, a multidimensional physical constraint joint loss function covering multiple theoretical equations was constructed. Its overall expression is a weighted sum of the losses of each component, specifically including:

[0028] Adversarial loss: Employs a least-squares GAN approach to stabilize the game-like training process between the generator and the discriminator;

[0029] Polarization angle period consistency loss: After converting the linear angle error into sine and cosine components on the unit circle, the Euclidean distance error is calculated to solve the phase folding and gradient jump problems caused by the polarization angle at 0 and π.

[0030] Polarization degree constraint loss: Set hard boundary penalty logic, and apply nonlinear ultra-high gradient reverse penalty when the predicted polarization degree jumps out of the physical range of [0.0, 1.0].

[0031] Fresnel physical consistency loss: calculates the L1 norm distance between the generator's predicted polarization degree and the theoretical polarization degree calculated based on the Fresnel reflection equation, forcing the network's latent tensor values ​​to approach classical physical laws.

[0032] Multi-channel energy conservation constraint loss: Based on the Stokes vector optical inequality logic, a penalty mechanism is set by activating the threshold gate function ReLU. Non-zero gradient penalty is generated only when the sum of squares of the predicted linear polarization component energies is greater than the square of the first Stokes component of the total light intensity, thus forcing the network feature distribution to follow the law of conservation of optical energy.

[0033] The advantages of this invention are: compared to traditional micro-polarization hardware (such as DoFP cameras)—it achieves a paradigm breakthrough of "algorithm-driven replacement of expensive hardware": This invention, through precise physical law algorithmic modeling and differentiable calculation, enables the system to achieve or even surpass the reconstruction effect of a dedicated polarization sensor with tens of millions of pixels by requiring only ordinary, inexpensive, and high-light-throughput monocular RGB image input. It acquires a high-fidelity, multi-dimensional physical feature matrix with zero additional hardware cost, completely overcoming the hardware structural weakness of traditional filter arrays where the signal-to-noise ratio drops drastically in low-light scenarios.

[0034] Compared to traditional RGB-to-polarization generation deep learning algorithms—a leap from "surface black-box fitting" to "understanding physical laws": Traditional methods merely perform statistically significant data representation mapping, and their generalization ability collapses completely when the test set environment changes slightly; this scheme creatively embeds regional refractive index clustering assignment and precise Fresnel reflection modulation into the deep latent space of the network. It transforms complex 3D microscopic optical prior information into a differentiable forward propagation tensor operator that the neural network can process, forcing the deep model to provide answers based on "physical causality."

[0035] Compared to conventional GAN ​​image generation techniques—which aim to overcome "model collapse" and achieve absolute optical rigor—conventional image synthesis discriminators are prone to model collapse or misclassification of high-quality real samples due to the generation of deceptive high-frequency noise. This invention's "physical discriminator" employs deterministic nonlinear algebraic equations and differential relationships for "rule-based dimensionality reduction verification." This approach forcibly shrinks the boundless solution space of the neural network into a reasonable subspace that conforms to optical common sense, using mathematical constraints. This not only greatly stabilizes the convergence process of the generator and discriminator's game-like training but also ensures that the final polarization diagram generated by inference possesses impeccable rigor in terms of mathematical logic and energy conservation. Attached Figure Description

[0036] Figure 1 The overall structure and data flow of the polarization generation system based on physical consistency constraints are shown in the global block diagram.

[0037] Figure 2 To generate a detailed topology diagram of a Generative Adversarial Network (GAN);

[0038] Figure 3 Flowchart for physical constraint verification and judgment;

[0039] Figure 4 Flowchart for forward physical generation of polarization features;

[0040] Figure 5 This is a diagram of the backpropagation structure of the multidimensional joint physical loss function. Detailed Implementation

[0041] Implementation Step 1: System initialization of multimodal data and decoupling of attention-based channel physical features

[0042] In optical imaging tasks, the generative network of this invention first receives a standard monocular RGB image tensor, processed with wide dynamic range correction (WDR) and nonlinear normalization, as initial input. The tensor dimension is H×W×3. Because the refractive index of different media in nature exhibits a dispersion effect with the incident light wavelength, light waves of different bands (corresponding to each RGB color channel) will exhibit different polarization response characteristics after reflection from the same interface. That is, the degree of polarization modulation of different color channels by the same material is not the same. To enable the network to accurately capture this difference in spectral-polarization correlation caused by material dispersion, this invention places a spectral-polarization channel feature decoupling module at the front end of the feature decoder.

[0043] Unlike conventional deep learning architectures that use 1×1 convolutions for linear separation, this module employs a hybrid spatial and channel attention mechanism to extract and decouple features because the spectral response of polarized light in the physical world exhibits nonlinear dynamic differences.

[0044] Spatial physical dimension response analysis: The network first utilizes depthwise separable convolutional layers, where the number of groups is set to half the number of input feature channels. This operation extracts the polarization physical reflection response of light waves of various frequency bands at different two-dimensional pixel spatial locations while maintaining low computational parameter complexity. Then, after activation and sigmoid nonlinear mapping, a probability weight map representing the importance of physical space is output.

[0045] Channel causal dimension compression and correlation: In another parallel branch, the module synchronously employs a global adaptive average pooling function to compress and fuse the entire high-resolution spatial features into a one-dimensional feature description vector. Subsequently, a multi-layered cascaded 1×1 convolutional array is used to perform nonlinear channel dimensionality reduction and reconstruction, generating channel weight scaling coefficients that reflect the importance of polarization correlations within each convolutional channel.

[0046] At the end of the two branches, feature fusion is performed through element-wise tensor Hadamard product operations. After the above operations, the band features of the original broadband optical signal of the model, which exhibit differentiated polarization responses due to material dispersion, are decoupled and separated according to physical wavelength response characteristics, and reconstructed into a deep polarization latent characterization channel matrix constructed from 64-dimensional specific wavelength physical properties.

[0047] To further enhance the feature response, a learnable Sobel edge differential physics module is introduced in this implementation phase. Based on optical experimental observations, regions with drastic changes in polarization state are often accompanied by abrupt changes in the surface normal geometry, fold lines in transparent materials, or boundaries between heterogeneous materials. Unlike traditional fixed-weight learnable Sobel gradient edge extraction modules, this invention encapsulates and declares it as a built-in tensor attribute parameter of the neural network that supports dynamic gradient descent fine-tuning. This allows the deep network to actively optimize and determine a set of edge difference filter banks that enhance the ability to capture polarization gradient information of specific material surfaces during adversarial training backpropagation.

[0048] Step 2: Controlled Regional Refractive Index Physical Simulation and Multidimensional Fresnel Curvature Modulation Layer

[0049] After completing the channel-level polarization feature decoupling and extraction, the deep feature matrix is ​​transferred to the core region of polarization calculation. The polarization degree physical quantity of the target surface depends on the intrinsic dielectric constant and refractive index properties of the target material itself. If the network architecture ignores the underlying physical differences based on the material properties of the object, it will lead to a decrease in performance during the generalization stage. Therefore, this embodiment constructs a regional refractive index physical modeling module in the deep network structure.

[0050] Initialization and safety clamping constraints of physical constants: The system predefines multiple sets of refractive index constant parameters representing the a priori distribution of reflection of basic materials in the real physical world in the initial memory area of ​​the network. For example, the system sets the init_n=[1.333,1.338,1.343] constant matrix for liquid water and water vapor media, and the init_n=[1.580,1.586,1.592] constant matrix for solid polymer and transparent glass surfaces. To ensure the stability of the gradient descent process, a safety clamping boundary is set at the network framework level: all media refractive index tensor values ​​n that are self-learned and fine-tuned by the network layer and participate in the next forward derivative calculation are required to be truncated to a reasonable buffer closed interval 1.3≤n≤2.0.

[0051] Semantic-aware region adaptive allocation network: For scenarios where multiple mixed media may exist within the field of view, this module internally incorporates a lightweight receptive field sensing sequence network. This sequence maps a highly abstract 3D input feature map onto several probability density distribution slices representing different material cluster categories along the output channel dimension. Subsequently, after the Softmax function transforms these slices into confidence scalars with a probability sum of 1, a differentiable Gumbel-Softmax reparameterization sampling mechanism transforms the material category probability distribution at each pixel location into a one-hot hard allocation decision. Based on this hard allocation result, the corresponding single-medium refractive index value is selected for each pixel from multiple pre-initialized sets of refractive index constant tensors. This generates a dense pixel-wise refractive index feature map that maintains physical abrupt changes at material boundaries and preserves gradient continuity and differentiability during backpropagation.

[0052] After this refractive index map is formed, the system, together with the light geometric incident angle parameter θ map derived by the independent theta_predictor (whose output has been compressed and activated by the nonlinear Sigmoid function to the physically allowed incident angle range [0,π / 2]), is input into the Fresnel physical equation modulation interaction layer built on the fundamental laws of optics.

[0053] This computational layer reimplements the classical optical Fresnel electromagnetic reflection derivation equations using tensor algebra within the network topology. The Fresnel reflection model used in this implementation is based on the physical assumption that the incident medium is air (with a refractive index approximately 1.0), meaning it is applicable to the general case where the object surface in the target scene is directly exposed to the atmospheric environment. The system calculates the reflection coefficients Rs (vertical polarization component) and Rp (parallel polarization component) present in the target reflected light wavefront. For special scenarios involving multi-layered media or underwater imaging, where the refractive index of the incident medium significantly deviates from 1.0, the refractive index of the incident medium can be used as an additional input variable to extend this computational framework. The calculation formula is as follows:

[0054] Based on the obtained optical physics core parameter matrix, the network forward derives and calculates the expected upper limit of the theoretical physical polarization reflection polarization degree of the target scene in the specular reflection-dominant region. It should be noted that this theoretical polarization degree is derived based on the Fresnel reflection model of a smooth, ideal mirror surface, and is applicable to pixel regions with low surface roughness and dominant specular reflection components. For pixel locations experiencing diffuse reflection or volume scattering due to surface roughness, the actual polarization degree will be lower than this theoretical upper limit. This method uses the Fresnel theoretical calculation value as a physical reference upper limit for the generated network rather than a strict equality constraint, to accommodate the diversity of surface micro-roughness in real-world scenes. The calculation formula is:

[0055] Furthermore, since the matrix denominator may experience numerical decay during network computation, leading to division by zero overflow, this implementation module sets a very small tolerance constant threshold eps=1e-7 in the forward tensor computation link to avoid non-numeric (NaN) computational pollution as a protection for the underlying computation. Substituting the above physical equations into the computation process maps the abstract latent variables extracted by the deep residual network to real optical physical measures.

[0056] Step 3: Transformation and Calculation Logic Reconstruction of the Stokes Parametric Coordinate System Based on the Law of Energy

[0057] After completing the complex physical modulation feature interaction, the next step is to convert these features into image output. This invention's network framework does not directly establish a linear regression path from high-dimensional features to the polarization angle (AoP) scalar and polarization degree (DoP) scalar. The fundamental reason is that scalars such as the polarization angle in polar coordinates cannot directly satisfy the linear homogeneous superposition rules in vector space, and there is a phase reversal abrupt change at the boundaries of 0° and 180° (or 0 and π). Direct regression can easily lead to abnormal network gradients.

[0058] To circumvent this problem, this implementation architecture is based on the polarization energy metric law—Stokes vector theory system—and reconstructs the decoding projection system mapping mechanism.

[0059] The high-order channel feature matrix output from the feature decoder backbone is separated and transformed into three Stokes-independent physical components (corresponding to linear polarization description systems). First, the predicted total optical field reference intensity channel is defined as the first intrinsic Stokes parameter S0 = I. Following this, physical components S1, representing the difference in intensity between horizontal and vertical polarization directions, and S2, representing the difference in intensity between diagonally orthogonal polarization directions, are independently decoded and output through dual parallel branches of the network. It should be noted that most reflection scenarios in nature are dominated by linear polarization, and the circular polarization component S3 contributes very little to reflections from conventional media surfaces. Therefore, this implementation focuses on linear polarization characterization, defaulting to zero for S3. Complete circular polarization recovery can be considered an extension of this framework. Subsequently, L2 norm normalized mathematical projection operations are used to eliminate scale energy differences.

[0060] Within this characteristic space, which is normalized and mapped onto the ideal unit circle hypersphere, the physical expression related to the polarization azimuth angle follows a trigonometric function transformation in Euclidean space. The calculation formulas for the pre-calculated polarization double-angle cosine distribution map and sine distribution map are as follows:

[0061] This transformation system resolves the function discontinuities and first-derivative singularities caused by crossing a 180-degree period in polar coordinate space from the underlying logical architecture of the algorithm, delivering smooth, continuously differentiable, and high-precision orthogonal polar coordinate vector feature surfaces to the next level of the network. At the final node of the generation process, the system calculates the final polarization orientation angle parameter for each pixel by calling the inverse arctangent analytical function calculation engine that supports multi-quadrant recognition: AoP = 0.5 × arctan2(S2, S1).

[0062] Implementation Step Four: Physical Discriminator Architecture Design and Multi-channel Constraint Joint Verification Rule

[0063] The combination of the above three steps ensures the simulation of polarization physics within the generator. However, based on the characteristics of adversarial networks, in order to suppress the possibility of uncontrolled "physical illusions" arising within the GAN system from the adversarial side, this invention constructs a central review mechanism—the "physical discriminator"—with mutually independent internal structures but deeply coupled logical rules.

[0064] This discriminator goes beyond the traditional binary classification of image texture authenticity based on large receptive field segmentation using PatchGAN. The system incorporates a learnable texture authenticity discrimination branch and two deterministic nonlinear algebraic optical rule constraint verification modules.

[0065] Fresnel reflection equation algebraic inverse derivation verification flow: The discriminator's data input probe synchronously extracts the current polarization distribution map DoP_pred, the surface incident light angle matrix θ, and the local absolute refractive index coefficient n calculated and output by the generator during the inference phase. The discriminator uses an independent computing thread to perform the inverse derivation verification of the aforementioned nonlinear Fresnel electromagnetic emission equation matrix. It performs residual difference hedging by comparing the calculated theoretical polarization extrema DoP_theoretical with the image-level prediction tensor DoP_pred provided by the generator. If the system comparison finds that the absolute difference between the two exceeds the set optical tolerance redundancy threshold, the underlying algorithm module of the physical discriminator will generate and return a low-confidence true / false probability scoring matrix.

[0066] Brewster Angle Physical Sensitive Region Second-Order Partial Derivative Constraint Decision Flow: In this review process, the physical discriminator is activated as the optical field differential calculation module. It monitors the coupling correlation coefficient between the gradient change field vector of the entire image and the incident angle gradient vector through high-frequency edge operators. According to the optical reflection theorem, when external light is incident with a Brewster polarization angle θ_b=arctan(n), the reflected light will approach the state of a purely linearly polarized light wave. That is, at this time, the DoP measured by the system's physics will increase in response and form a standing wave at a local maximum point. In adversarial training, if the physical discriminator detects that the polarization degree numerical surface generated by the network and the corresponding incident angle gradient vector show divergence or abnormal trends when near this characteristic region, the physical discriminator will activate the rejection mechanism, determine that these fake samples violate spatial geometric optics, and reduce their weight.

[0067] AoP Spatial Manifold Topological Periodicity Verification Network: To prevent defects in the model caused by forced fitting in the pure data domain in two-dimensional space, this discriminant network uses operators such as the discrete Laplacian matrix to detect and check the generated AoP distribution field pixel by pixel across the entire full-dimensional two-dimensional spatial cross-section to ensure it exhibits sufficient smoothness and structural continuity. Its primary monitoring objective is to identify and suppress large-area phase wrapping and tearing errors caused by neglecting the periodicity of polarization.

[0068] Implementation Step 5: Mathematical Derivation of the End-to-End Multidimensional Physical Joint Loss Function Driving Global Convergence

[0069] The physical verification signal generated by the physical discriminator needs to be used to perform inverse gradient iterative updates and physical penetration on the network generator through a multi-dimensional joint set of physical constraint target loss functions. The overall architecture expression of the loss function driving global system optimization in this invention is constructed as follows: In the formula, the coefficients λ represent the proportion of the hyperparameter weights of the corresponding loss terms in backpropagation.

[0070] The specific implementation mechanism and gradient descent driving rule of this system of equations, which includes five penalty mechanisms, are analyzed as follows:

[0071] Adversarial probability generation loss ( The system's main framework employs a distance metric formula in the form of LeastSquaresGAN (LSGAN), which can alleviate the gradient vanishing problem. Mean squared error is used to optimize the distance distribution, and the physical discriminator is stabilized through asymmetric multi-round game training on pixel-level real / fake data pairs.

[0072] Polarization angle multiple times periodic manifold uniformity smoothing loss ( Since the values ​​of the polarization angle are inherently limited by the physical geometric domain, this loss term is calculated by converting its linear angular error into sine and cosine components on the unit circle and then calculating the Euclidean distance error. The residual area between the sine and cosine components and the true sine and cosine components is then calculated in three-dimensional Euclidean space coordinates. This reconstructs and restores the unique smooth manifold periodic topological consistency characteristics of the polarization angle, especially when crossing poles, at the underlying model training level. The specific calculation formula is as follows: in, and 1 / hw represents the polarization angle matrix predicted by the generator and the true polarization angle matrix provided by the dataset, respectively. 1 / hw is the overall spatial resolution size of the input feature map, and the coefficient 2 is used to map the physical polarization angle to the complete 2π Stokes vector azimuth space.

[0073] Polarization Degree Exceedance Hard Constraint Penalty Loss: At the macro-control level of adversarial training, in addition to using standard L1 norm distance loss to force a closer numerical similarity between the system's predicted polarization degree curve and the test real label under normal settings, the system also introduces a physical safety hard isolation penalty logic. The physical common sense of polarization degree is clear: the proportion of polarized light energy in a region is between [0.0, 1.0]. Once the network predicts a value that jumps out of this closed physical range, the loss function kernel will apply a nonlinear high gradient backward penalty. The specific calculation formula is as follows:

[0074] Fresnel physical equation closed-loop consistency fitting loss: This physical constraint mechanism mainly functions to force the numerical flow of latent tensors within the generator network to converge towards classical physical laws. A further round of formula verification and validation is performed using the intermediate physical parameter network matrix, which is autonomously implicitly mined and combined in the aforementioned model generator. The specific calculation formula is as follows: ;

[0075] This penalty term uses explicit analytical physical equations to examine the parameters of deep networks. This hard constraint eliminates interfering values ​​that ignore the optical physics meaning, thereby significantly compressing the multidimensional parameter optimization exploration space of the system and improving training stability and convergence speed.

[0076] The bottom line constraint of the energy conservation theorem for multi-physics channel matrices: This represents the optical fundamental energy supervision and verification system of this invention. According to the Stokes polarization metric, for the Stokes full-element feature vector component sequence S0 to S3 matrix decoded from predicted features, representing the total energy of the unpolarized light intensity, it should be greater than or equal to the sum of the absolute values ​​of the energies of all its split micro-polarized light state components. For the linearly polarized dominant scenario (circular polarization component S3 is approximately zero) targeted by this method, the loss constraint equation is defined as follows: Here, Ŝ0 represents the first Stokes component of the predicted total light intensity, and Ŝ1 and Ŝ2 represent the predicted horizontal / vertical polarization components and diagonal polarization components, respectively. The ReLU activation function ensures that a non-zero gradient penalty is generated only when the sum of the linear polarization component energies predicted by the generator exceeds the total light intensity energy. If the generator network exhibits divergent behavior during the game-theoretic process, causing the geometric sum of the polarization feature energy of the generated microscopic vertical parallel channel differences, or the physical feature energy in the diagonal difference direction, to exceed the intrinsic first component light intensity feature representing the overall base brightness intensity of this pixel, the hidden ReLU nonlinear rectified neurons in this module will be activated, triggering the generation of an error penalty gradient for physical structure correction. This penalty gradient propagates backward along the computation graph, prompting the neural network to perform gradient backtracking and parameter updates. It continuously supervises the entire training process, ensuring that the deep feature matrix constructed by this automatically generated model adheres to the constraints of physical thermodynamic laws and the conservation of photoelectric and optical energy, within a mathematical framework.

Claims

1. A polarization-based generative adversarial network method based on physical consistency constraints, characterized in that, A generative adversarial network framework is constructed to generate a mapping from monocular RGB images to polarization feature matrices. The generative adversarial network framework includes: a generator network configured with a Fresnel reflection equation forward computation layer, used to extract deep features of the input monocular RGB image and predict the polarization angle, polarization degree and corresponding Stokes parameter components based on the internally embedded differentiable reflection physical model; and a physical discriminator network, used to distinguish the differences between the generated data and the real data distribution, and specifically evaluate whether the generated result strictly obeys the physical laws of optical reflection. The method employs a multidimensional physical constraint joint loss function to perform alternating adversarial training on the generator network and the physical discriminator network, forcing the generated results to approximate the texture of real data while conforming to the laws of optical conservation and reflection.

2. The method according to claim 1, characterized in that, The generator network is composed of multiple physically associated sub-modules cascaded sequentially, specifically including: A learnable Sobel-type gradient edge extraction module is used to extract the physical edge gradient difference features of the input image in different directions; The channel decoupling module is used to nonlinearly isolate and filter the spatial and channel dimensions of the gradient features in order to decouple the polarization response channel dominated by a specific wavelength. The regional refractive index modeling module is used to dynamically assign and predict pixel-level surface refractive index feature maps for different semantic regions in an image within a continuous and bounded range of physical values. A Fresnel reflection physical modulation layer is used to receive the predicted incident angle tensor and the pixel-level surface refractive index feature map, and to physically modulate the predicted polarization features to a theoretical upper limit using a differentiable Fresnel reflection equation. The parameter calculation module is used to calculate the independent components of the Stokes vector based on the modulated deep features, and to use polar coordinate transformation logic to parse them into the final continuous polarization angle and polarization degree output.

3. The method according to claim 2, characterized in that, In the learnable edge extraction module, the X and Y gradient kernel parameters of the traditional fixed-weight Sobel edge operator are reconstructed into learnable parameters in the neural network, and an orthogonal gradient structure preservation constraint is applied to maintain gradient direction selectivity. During the backpropagation optimization process of the model, the learnable parameters are adaptively weighted by combining local polarization gradient loss to enhance the feature capture capability for high-frequency edge regions with polarization characteristics.

4. The method according to claim 2, characterized in that, The channel decoupling module is implemented based on a hybrid attention mechanism that interweaves spatial and channel elements, specifically including: Spatial attention extraction branch: Depthwise separable convolution is used to extract the spatial distribution response of physical features in the image plane and generate a spatially aware weight map; Channel attention extraction branch: Spatial information is compressed into a one-dimensional channel vector through global adaptive average pooling, and nonlinear channel dimensionality reduction and reconstruction are performed using multi-layer convolution to generate channel importance weights; Joint feature fusion: The spatially perceived weight map and the channel importance weights are multiplied by a channel-by-channel tensor. The fused joint weight matrix is ​​used to filter and isolate the original input features in the channel dimension, and to remove mutually interfering spectral polarization characterization channels.

5. The method according to claim 2, characterized in that, In the region refractive index modeling module, multiple sets of learnable initial refractive index parameter constants are predefined. An activation pruning function is used to forcibly truncate all predicted refractive indices participating in the forward calculation within a reasonable range of 1.3 to 2.0 for the physical medium. The module contains a lightweight region allocator network that maps the input high-dimensional feature map to a probability distribution map corresponding to a specific material region category. After the probability distribution map is reparameterized by Gumbel-Softmax into a hard allocation decision, the corresponding single medium refractive index value is selected for each pixel from the set of learnable initial refractive index parameter constants, generating a dense pixel-by-pixel refractive index physical feature map.

6. The method according to claim 2, characterized in that, The physical discriminator network includes a learnable multi-scale texture authenticity discrimination branch and two rule-level verification modules based on deterministic physical equations. The verification error signal is fed into the joint physical loss function. Data Distribution True / False Game Branch: Determine the true / false probability of generated polarized images and real polarized images in terms of micro-texture distribution and pixel manifold mapping based on the receptive field of multi-scale convolutional kernels; Fresnel physical algebra verification branch: Receives the predicted values ​​of polarization degree, refractive index and incident angle of light output by the generator network, substitutes them into the differentiable Fresnel reflection theory model, and determines whether the physical parameters satisfy the nonlinear polarization degree attenuation relationship; Polarization constraint derivative correlation branch: Based on the Brewster angle theorem, it is determined whether the gradient direction of the partial derivative of the generated degree of polarization with respect to the change of the incident angle of the light shows an extreme value reversal law around the expected angle. At the same time, the periodic consistency and smooth continuity of the generated polarization angle in the two-dimensional spatial field are determined by the discrete Laplace operator.

7. The method according to claim 2, characterized in that, The global optimization mathematical expression of the multidimensional physical constraint joint loss function is a weighted sum of multiple losses, including: adversarial loss in the stable game process, polarization angle period consistency loss to eliminate phase reversal, polarization degree constraint loss in the normalized numerical range, Fresnel physical consistency loss for fitting the reflection equation, and multi-channel energy conservation constraint loss following the thermodynamic framework.

8. The method according to claim 7, characterized in that, The polarization angle period consistency loss calculates the Euclidean distance error by converting the linear angle error into sine and cosine components on the unit circle, thus solving the gradient jump problem caused by phase folding of the polarization angle due to the coordinate system being defined at 0 and π. The multi-channel energy conservation constraint loss is based on the premise that the linear polarization characterization of this method is focused and the circular polarization Stokes component S3 is zero. It is constructed according to the logic of the Stokes vector optical inequality and sets the penalty through a threshold gating function. A non-zero gradient penalty is generated only when the sum of the squares of the predicted linear polarization component energies is greater than the square of the first Stokes component of the predicted total light intensity, so as to force the network feature distribution to follow the optical energy conservation law.

9. The method according to claim 1, characterized in that, The generator network does not directly regress the polarization angle scalar, but first predicts the independent components S0, S1, and S2 of the Stokes vector. Since this method focuses on linear polarization characterization, the circularly polarized Stokes component S3 is set to zero. After L2 norm normalization, the sine and cosine distribution maps of the polarization double angle are calculated. Then, the continuous polarization angle is output through the arctangent analytical function AoP = 0.5 × arctan2(S2, S1) that supports multi-quadrant recognition, so as to avoid the gradient jump problem caused by phase folding at the 0 and π boundary in the polar coordinate system.

10. A computer device or a non-volatile computer-readable storage medium, characterized in that, The computer device includes a memory and a processor, and a computer program is stored on the non-volatile computer-readable storage medium. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images