A deep learning-based photonic display image enhancement method and system

Abstract

This application provides a deep learning-based method and system for enhancing photonic display images, belonging to the field of image enhancement technology. The method includes: identifying the viewpoint-channel correlation response feature map when a preset uniform reference pattern is displayed on a photonic display medium; determining the medium modulation component characterizing the spatial non-uniform modulation effect of the medium and the content intrinsic component characterizing the display content signal in the photonic display image to be enhanced; generating constrained enhancement features characterizing the non-uniform distribution across viewpoints and color channels from the medium modulation component and the correlation response feature map; generating a corresponding compensation mapping map based on the constrained enhancement features combined with a spatial-color channel joint compensation network; and performing joint compensation and reconstruction on the content intrinsic components based on the compensation mapping map to obtain the enhanced photonic display image. The technical solution provided in this application can enhance photonic display images even when there is non-uniform crosstalk between the observation viewpoint and color channels.

Description

Technical Field

[0001] This application relates to the field of image enhancement technology, and more specifically, to a deep learning-based method and system for enhancing photonic display images. Background Technology

[0002] In high-end visual detection such as photon counting imaging, the original image is often limited by extremely low signal-to-noise ratio, sparsity of weak photon events, and inherent dark noise of the detector, causing details to be submerged in the background and making it difficult to meet the requirements of high-precision observation. Traditional enhancement methods such as histogram equalization or linear filtering can improve contrast, but they are prone to amplifying noise or destroying the Poisson statistical properties of photon signals. They are particularly lacking in robustness in high dynamic scenes. With the integration of deep learning and computational imaging, photon image enhancement technology based on neural networks has emerged. By learning the spatiotemporal correlation of photon events and the prior of noise distribution, it can adaptively recover weak edge and texture information while suppressing Poisson-Gaussian mixed noise.

[0003] In photonic display image enhancement, a Poisson-Gaussian mixture model is first used to model and adaptively suppress quantum shot noise and dark current noise during photon counting to identify real photon events. Then, nonlinear grayscale mapping stretches the photon density distribution compressed at low grayscale levels to the full dynamic range, significantly improving visual contrast. Finally, gradient domain edge detection or high-frequency information gain techniques are used to suppress residual noise while restoring submerged fine structures and texture boundaries, thus completing the enhancement of the photonic display image. The photonic display image is the optical response image acquired when the photonic display medium displays the content to be enhanced at the observed angle. However, in photonic displays (such as novel display technologies based on photonic crystals, diffractive optical elements, or angle-selective scattering media), the display medium typically exhibits microstructure-induced angle-selective light... In terms of spectral response, existing technologies mostly employ uniformity correction under a fixed viewing angle or compensate only for a single color channel independently. When the viewing angle changes, non-uniform crosstalk exists between the viewing angle and the color channels. That is, after the incident white light is modulated by the medium microstructure, the spectral power distribution of its output light is a function of the viewing angle. When the viewing angle changes, not only does the brightness change, but the spectral peak wavelength of the pixel also drifts. This is directly reflected in the fact that the attenuation coefficients of the RGB three channels are not a fixed ratio, but rather exhibit a cross-channel spatial coupling relationship. This non-linear chromaticity-angle crosstalk will cause the photonic display image to exhibit brightness unevenness, color shift, and structural artifacts that vary with the combined viewing angle and color channels. Therefore, how to enhance the photonic display image under the condition of non-uniform crosstalk between the viewing angle and the color channels has become a difficult problem for the industry. Summary of the Invention

[0004] This application provides a deep learning-based method and system for enhancing photonic display images, which can enhance photonic display images even when there is non-uniform crosstalk between the viewing angle and the color channel.

[0005] In a first aspect, this application provides a deep learning-based photonic display image enhancement method, comprising the following steps: When displaying a preset uniform reference pattern on a photonic display medium, multiple sets of optical response images are acquired under different observation angles and different color channels, and a viewpoint-channel correlation response feature map is constructed based on the optical response images. A photonic display image to be enhanced is acquired, wherein the photonic display image is an optical response image acquired by an imaging device when the content is displayed on the photonic display medium, and the photonic display image is subjected to medium effect separation to obtain a medium modulation component characterizing the non-uniform modulation effect of the medium space and a content intrinsic component characterizing the display content signal. The medium modulation component and the associated response feature map are fused and encoded to generate constrained enhancement features that characterize the non-uniform distribution across viewpoints and color channels. Based on the constrained enhancement features, a corresponding compensation map is generated through a pre-trained space-color channel joint compensation network. Based on the compensation mapping, the intrinsic components of the content are jointly compensated and reconstructed using spatial and color channels to obtain an enhanced photonic display image.

[0006] In some embodiments, when a preset uniform reference pattern is displayed on a photonic display medium, multiple sets of optical response images are acquired at different viewing angles and in different color channels, and a viewpoint-channel correlated response feature map is constructed based on the optical response images. Specifically, this includes: Under multiple preset discrete observation angles and discrete color channels, the optical response images of the photonic display medium to a preset uniform reference pattern are acquired to obtain a set of multidimensional observation data cubes; Spatial normalization is performed on the multidimensional observation data cube to eliminate the self-uniform distribution prior of the preset uniform reference pattern, and a normalized multi-view-multi-channel response tensor is obtained. The normalized multi-view-multi-channel response tensor is decomposed along the view dimension and the channel dimension to extract the view-channel joint intrinsic modes. The view-channel joint intrinsic modes are reconstructed according to the original spatial dimensions to obtain the view-channel associated response feature map.

[0007] In some embodiments, under multiple preset discrete observation angles and discrete color channels, optical response images of the photonic display medium to a preset uniform reference pattern are acquired to obtain a set of multidimensional observation data cubes, specifically including: Display a pre-defined uniform reference pattern with a known uniform spatial intensity distribution on a photonic display medium; Multiple discrete observation angles are set within the observation hemisphere of the photonic display medium. At each discrete observation angle, a multispectral imaging device sequentially acquires the uniform reference pattern through multiple preset discrete color channels to obtain a set of original multi-view-multi-channel optical response image sequences. The original multi-view, multi-channel optical response image sequence is organized and arranged according to three dimensions: observation angle index, color channel index, and spatial pixel coordinates, forming a multi-dimensional observation data cube.

[0008] In some embodiments, performing medium effect separation on the photonic display image to obtain a medium modulation component characterizing the non-uniform modulation effect of the medium space and a content intrinsic component characterizing the display content signal specifically includes: The photonic display image is input into a pre-trained medium effect separation network, which includes a parallel content coding branch and a medium coding branch, to extract features from the photonic display image to obtain content intrinsic features and medium modulation features. The content intrinsic features and the medium modulation features are subjected to deentanglement training under the constraint of minimizing mutual information, so that the content coding branch and the medium coding branch output semantically decoupled feature representations, thus obtaining deentangled content features and deentangled medium features; The unentangled content features and the unentangled medium features are reconstructed and mapped through the content decoding branch and the medium decoding branch, respectively, to obtain the medium modulation component characterizing the non-uniform modulation effect of the medium space and the content intrinsic component characterizing the display content signal.

[0009] In some embodiments, fusing and encoding the medium modulation component with the associated response feature map to generate constrained enhancement features characterizing cross-viewpoint and cross-color channel non-uniform distribution specifically includes: The medium modulation components are expanded in the color channel dimension to obtain a multi-channel medium modulation tensor; The multi-channel medium modulation tensor and the associated response feature map are concatenated along the channel dimension to obtain the joint input tensor; The joint input tensor is input into a pre-trained feature fusion coding network. The multi-layer convolution operation of the feature fusion coding network is used to extract the correlation mapping relationship between the spatial non-uniform modulation mode in the medium modulation component and the viewpoint-color channel coupled distribution mode in the correlation response feature map, and the constraint enhancement feature is output.

[0010] In some embodiments, generating a corresponding compensation map based on the constraint enhancement features using a pre-trained space-color channel joint compensation network specifically includes: The constraint enhancement features are input into the pre-trained spatial-color channel joint compensation network. After forward inference of the spatial-color channel joint compensation network, a pixel-wise compensation coefficient map with the same spatial resolution as the medium modulation component and the same number of channels as the preset discrete color channels is output. The pixel-by-pixel compensation coefficient map is used as the compensation mapping map.

[0011] In some embodiments, the uniform reference pattern is a standard grayscale test image with equal pixel values ​​everywhere. Secondly, this application provides a deep learning-based photonic display image enhancement system for performing a deep learning-based photonic display image enhancement method. The system includes: The feature recognition module is used to acquire multiple sets of optical response images under different observation angles and different color channels when displaying a preset uniform reference pattern on a photonic display medium, and to construct a viewpoint-channel correlation response feature map based on the optical response images. The medium effect separation module is used to acquire the photonic display image to be enhanced, wherein the photonic display image is an optical response image acquired by an imaging device when the photonic display medium displays content, and to perform medium effect separation on the photonic display image to obtain the medium modulation component characterizing the non-uniform modulation effect of the medium space and the content intrinsic component characterizing the display content signal. The compensation map generation module is used to fuse and encode the medium modulation component and the associated response feature map to generate constrained enhancement features that characterize the non-uniform distribution across viewpoints and color channels. Based on the constrained enhancement features, a corresponding compensation map is generated through a pre-trained space-color channel joint compensation network. The compensation and reconstruction module is used to perform joint compensation and reconstruction of the content intrinsic components based on the compensation mapping map, using spatial and color channels, to obtain an enhanced photon display image.

[0012] Thirdly, this application provides a computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described deep learning-based photonic display image enhancement method.

[0013] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned deep learning-based photonic display image enhancement method.

[0014] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The deep learning-based photonic display image enhancement method and system provided in this application firstly acquires multiple sets of optical response images under different viewing angles and different color channels when a preset uniform reference pattern is displayed on the photonic display medium, and constructs a viewpoint-channel correlation response feature map based on the optical response images; secondly, the photonic display image to be enhanced is acquired, which is an optical response image acquired by an imaging device when the content is displayed on the photonic display medium, and the photonic display image is subjected to medium effect separation to obtain a medium modulation component characterizing the spatial non-uniform modulation effect of the medium and a content intrinsic component characterizing the display content signal; then, the medium modulation component and the correlation response feature map are fused and encoded to generate constrained enhancement features characterizing the non-uniform distribution across viewpoints and across color channels, and a corresponding compensation mapping map is generated based on the constrained enhancement features through a pre-trained space-color channel joint compensation network; finally, the content intrinsic component is jointly compensated and reconstructed based on the compensation mapping map to obtain the enhanced photonic display image.

[0015] Therefore, this application can enhance photonic display images even when there is non-uniform crosstalk between the viewing angle and color channel. First, by pre-collecting the optical response of a uniform reference pattern under different viewing angles and color channels and identifying the associated response feature maps, the inherent cross-viewing angle and cross-color channel coupling non-uniformity distribution of the photonic display medium can be extracted as physical prior features. This provides global view-channel coupling constraint information for subsequent compensation, avoiding compensation deviations caused by relying solely on single-viewing angle or single-channel calibration. Second, by separating the medium effect in the photonic display image, the non-uniform modulation effect of the medium space is decoupled from the display content signal into independent medium modulation components and content intrinsic components. This allows the compensation network to specifically correct the medium modulation components without damaging the details and structure of the display content itself, solving the problem of mutual interference between content information and distortion information in traditional enhancement methods. Then, the medium modulation components and associated response feature maps are fused and encoded to generate constrained enhancement features, achieving... This paper proposes an adaptive correlation matching between the instantiated modulation information of the image to be compensated and the inherent physical prior mode of the medium. A pre-trained spatial-color channel joint compensation network generates an accurate spatial and color channel joint compensation map, enabling the compensation coefficients to be dynamically adjusted according to different display content while maintaining consistency with the physical characteristics of the medium. Finally, the intrinsic components of the content are reconstructed through spatial and color channel joint compensation using the compensation map. Non-uniform distortion is eliminated through pixel-by-pixel and channel-by-channel multiplicative compensation, resulting in an enhanced photonic display image while fully preserving the display content signal information. This significantly improves the uniformity and color fidelity of the photonic display medium under different viewing angles and color channels, avoiding non-linear chromaticity-angle crosstalk that causes brightness unevenness, color shift, and structural artifacts in the photonic display image that vary with viewing angle and color channels. In summary, the technical solution provided in this application can enhance photonic display images even when there is non-uniform crosstalk between the viewing angle and color channels. Attached Figure Description

[0016] Figure 1 This is an exemplary flowchart of a deep learning-based photonic display image enhancement method according to some embodiments of this application; Figure 2 This is a schematic flowchart illustrating the separation of photonic display image media effects according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the determination of constraint enhancement features according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a deep learning-based photonic display image enhancement system according to some embodiments of this application; Figure 5This is a schematic diagram of the structure of a computer device that implements a deep learning-based photonic display image enhancement method according to some embodiments of this application. Detailed Implementation

[0017] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] refer to Figure 1 This figure is an exemplary flowchart of a deep learning-based photonic display image enhancement method according to some embodiments of this application. The figure mainly includes the following steps: In step S101, when a preset uniform reference pattern is displayed on the photonic display medium, multiple sets of optical response images are acquired under different observation angles and different color channels, and a viewpoint-channel correlation response feature map is constructed based on the optical response images.

[0019] It should be noted that, in this application, photonic display medium refers to a type of functional device that uses the interaction between photons and the micro / nano structure of matter to generate, regulate, or present visual information. Its core feature is that the optical effect of the display does not depend on the chemical properties of traditional dyes or luminescent materials, but is precisely controlled by the diffraction, interference, scattering, or resonance effects of light through physical structures, such as photonic crystals, diffraction gratings, metasurfaces, and angle-selective scattering films. Photonic display media can selectively enhance or suppress light of specific wavelengths and incident / outgoing angles, thereby achieving unique functions such as structural color, dynamic viewing angle effects, or transparent display.

[0020] In some embodiments, when a preset uniform reference pattern is displayed on a photonic display medium, multiple sets of optical response images are acquired at different viewing angles and in different color channels, and a viewpoint-channel correlated response feature map is constructed based on the optical response images using the following steps: Under multiple preset discrete observation angles and discrete color channels, the optical response images of the photonic display medium to a preset uniform reference pattern are acquired to obtain a set of multidimensional observation data cubes; Spatial normalization is performed on the multidimensional observation data cube to eliminate the self-uniform distribution prior of the preset uniform reference pattern, and a normalized multi-view-multi-channel response tensor is obtained. The normalized multi-view-multi-channel response tensor is decomposed along the view dimension and the channel dimension to extract the view-channel joint intrinsic modes. The view-channel joint intrinsic modes are reconstructed according to the original spatial dimensions to obtain the view-channel associated response feature map.

[0021] In some embodiments, the following steps are used to acquire an optical response image of the photonic display medium to a preset uniform reference pattern under multiple preset discrete observation angles and discrete color channels to obtain a set of multidimensional observation data cubes: Display a pre-defined uniform reference pattern with a known uniform spatial intensity distribution on a photonic display medium; Multiple discrete observation angles are set within the observation hemisphere of the photonic display medium. At each discrete observation angle, a multispectral imaging device sequentially acquires the uniform reference pattern through multiple preset discrete color channels to obtain a set of original multi-view-multi-channel optical response image sequences. The original multi-view, multi-channel optical response image sequence is organized and arranged according to three dimensions: observation angle index, color channel index, and spatial pixel coordinates, forming a multi-dimensional observation data cube.

[0022] Specifically, firstly, a preset uniform reference pattern with a known uniform spatial intensity distribution is displayed on a photonic display medium. The uniform reference pattern adopts a standard grayscale test image with equal pixel values ​​everywhere, and its spatial intensity distribution is known and constant.

[0023] Subsequently, multiple discrete observation angles are set within the observation hemisphere of the photonic display medium. These discrete observation angles can be determined using an equal-angle interval division strategy, for example, in the azimuth angle. to Within the range Divide the step size into 12 azimuth angles, at the polar angle to Within the range Five polar angles are divided for the step size, which are then combined to obtain 60 discrete observation angles.

[0024] At each discrete observation angle, a uniform reference pattern is sequentially acquired through multiple preset discrete color channels using a multispectral imaging device. These preset discrete color channels are passbands within a specific wavelength range defined by the narrowband filter array built into the multispectral imaging device. For example, the red channel corresponds to a passband with a center wavelength of 620 nm and a full width at half maximum (FWHM) of 20 nm, the green channel corresponds to a passband with a center wavelength of 532 nm and a FWHM of 20 nm, and the blue channel corresponds to a passband with a center wavelength of 470 nm and a FWHM of 20 nm. The multispectral imaging device acquires images channel by channel at each discrete observation angle, resulting in a set of original multi-view-multi-channel optical response image sequences. Each frame in these original multi-view-multi-channel optical response image sequences reflects the spatial distribution response of the radiative exitance of the photonic display medium to the uniform reference pattern at a specific observation angle and in a specific color channel.

[0025] Finally, the original multi-view, multi-channel optical response image sequence is organized and arranged according to three dimensions: observation angle index, color channel index, and spatial pixel coordinates, thus establishing a three-dimensional coordinate index system. The first dimension is the observation angle index, which is the calibrated discrete observation angle number. The second dimension is the color channel index, which is the band identifier of the preset discrete color channel. The third dimension is the spatial pixel coordinates, which is the row and column position of the image pixels. Each frame of the image is filled into the three-dimensional array according to the correspondence of the above three dimensions to form a multi-dimensional observation data cube.

[0026] It should be noted that, in this embodiment, the multidimensional observation data cube refers to a three-dimensional data structure that organizes the optical response of the photonic display medium to a uniform reference pattern under multiple discrete observation angles and multiple discrete color channels according to three dimensions: observation angle index, color channel index, and spatial pixel coordinates. Each element value of this three-dimensional data structure represents the light radiation response value at a certain spatial pixel position under a specific observation angle and a specific color channel.

[0027] Specifically, firstly, optical response images of the photonic display medium to a preset uniform reference pattern are acquired under multiple preset discrete observation angles and discrete color channels, resulting in a set of multidimensional observation data cubes. Secondly, the multidimensional observation data cubes are spatially normalized, specifically by using a pixel-by-pixel, color-by-color-channel, and observation-by-observation-angle approach. The response value of each pixel position in the multidimensional observation data cube is divided by the known standard intensity value of the uniform reference pattern at the corresponding pixel position. Since the intensity values ​​of each pixel in the uniform reference pattern are equal and known, this division operation eliminates the interference of the prior information of the uniform spatial distribution of the uniform reference pattern itself on the subsequent decomposition, resulting in a normalized multi-viewpoint-multi-channel response tensor. This normalized multi-viewpoint-multi-channel response tensor characterizes the relative response distribution under the combined effect of the spatial non-uniform modulation effect of the photonic display medium and the viewpoint-color channel coupling effect.

[0028] Subsequently, the normalized multi-view-multi-channel response tensor is decomposed along the view dimension and channel dimension. Specifically, the Tucker decomposition algorithm is used to decompose the normalized multi-view-multi-channel response tensor into a modular multiplication of a core tensor and three factor matrices. The three factor matrices correspond to the observation angle mode, color channel mode, and spatial pixel mode, respectively. The core tensor encodes the interaction relationship between the modes. By retaining the significant singular values ​​related to the joint variation of view-channel in the core tensor and their corresponding factor matrix column vectors, the joint intrinsic mode of view-channel is extracted. The joint intrinsic mode of view-channel is a low-dimensional feature representation that characterizes the cooperative modulation law of the spatial non-uniform response to the changes in observation angle and color channel.

[0029] Finally, the view-channel joint intrinsic modes are reconstructed according to the original spatial dimensions. Specifically, the extracted view-channel joint intrinsic modes are reconstructed by modular multiplication with the retained spatial pixel mode factor matrix to restore the same spatial resolution as the original multidimensional observation data cube, thus obtaining the view-channel correlation response feature map.

[0030] It should be noted that the correlation response feature map in this application is a multi-channel feature map that records the spatial non-uniform response coupling distribution mode of the photonic display medium under different observation angles and different color channels. In this embodiment, the high-order coupling relationship between the observation angle change, color channel change and the spatial non-uniform response of the photonic display medium, which was originally mixed in the original multi-view-multi-channel data, is explicitly decoupled and compressed into a low-dimensional core representation of the view-channel joint intrinsic mode through tensor decomposition. Then, this low-dimensional core representation is reconstructed back to the original spatial dimension, so that the obtained correlation response feature map completely preserves the spatial non-uniform distribution details of the photonic display medium in the spatial dimension, thereby providing a highly condensed and physically interpretable cross-view-cross-color channel non-uniformity distribution prior constraint for the compensation network.

[0031] In step S102, a photonic display image to be enhanced is acquired. The photonic display image is an optical response image acquired by an imaging device when the photonic display medium displays content. The photonic display image is then subjected to medium effect separation to obtain a medium modulation component characterizing the non-uniform modulation effect of the medium space and a content intrinsic component characterizing the display content signal.

[0032] Specifically, the process involves acquiring a photonic display image to be enhanced. This image is obtained by an imaging device acquiring the original display content presented by the photonic display medium under preset observation conditions. After radiometric calibration, the device response value is mapped to a linear optical radiometric space consistent with the multidimensional observation data cube, resulting in a digital image to be processed. The photonic display image is an optical response image acquired by the imaging device when the photonic display medium displays content. The optical response image refers to a two-dimensional image acquired by the imaging device at a given observation angle and a given color channel, capturing the pattern presented by the photonic display medium. This image contains both the ideal radiation distribution information characterizing the display content signal and the brightness and color distortion caused by the spatial non-uniform modulation effect of the photonic display medium itself.

[0033] In some embodiments, the dielectric effect separation of the photonic display image to obtain the dielectric modulation component characterizing the non-uniform modulation effect of the dielectric space and the content intrinsic component characterizing the display content signal is achieved through the following steps: The photonic display image is input into a pre-trained medium effect separation network, which includes a parallel content coding branch and a medium coding branch, to extract features from the photonic display image to obtain content intrinsic features and medium modulation features. The content intrinsic features and the medium modulation features are subjected to deentanglement training under the constraint of minimizing mutual information, so that the content coding branch and the medium coding branch output semantically decoupled feature representations, thus obtaining deentangled content features and deentangled medium features; The unentangled content features and the unentangled medium features are reconstructed and mapped through the content decoding branch and the medium decoding branch, respectively, to obtain the medium modulation component characterizing the non-uniform modulation effect of the medium space and the content intrinsic component characterizing the display content signal.

[0034] In practice, the photon display image to be enhanced is first input into a pre-trained medium effect separation network. This medium effect separation network adopts a dual-branch encoder-decoder architecture, which includes a parallel content encoding branch and a medium encoding branch. Both the content encoding branch and the medium encoding branch adopt a fully convolutional network structure composed of multiple residual convolutional blocks stacked together. The number of convolutional layer channels in the content encoding branch is set to 64, 128, and 256, increasing layer by layer, to extract the structural and semantic information of the image to obtain the intrinsic features of the content. The number of convolutional layer channels in the medium encoding branch is set to 32, 64, and 128, increasing layer by layer, to capture the spatially slowly changing modulation mode to obtain the medium modulation features.

[0035] The intrinsic content features refer to the high-dimensional distributed vector representation obtained in the latent space after feature extraction of the photonic display image by the content coding branch in the medium effect separation network. These features are used to characterize the structural, texture, and semantic information related to the display content signal itself in the photonic display image, that is, to reflect the ideal spatial distribution and color composition of the original display pattern. The medium modulation features refer to the high-dimensional distributed vector representation obtained in the latent space after feature extraction of the photonic display image by the medium coding branch in the medium effect separation network. These features are used to characterize the non-uniform modulation mode of slow spatial changes caused by the physical properties of the photonic display medium itself in the photonic display image, that is, to reflect the spatial distortion law of the medium on the brightness and chromaticity of each pixel position and each color channel.

[0036] Secondly, during the deentanglement training under the mutual information minimization constraint, a mutual information estimator is introduced between the intrinsic content features and the medium modulation features. This mutual information estimator adopts a variational mutual information estimation method based on the KL divergence approximation. An auxiliary discriminant network scores the feature sample pairs from the two branches, and the log-likelihood ratio output by the discriminant network is used as the estimated value of the lower bound of mutual information. This estimated value of the lower bound of mutual information is added as a mutual information minimization regularization term to the total loss function of the medium effect separation network. The parameters of the content encoding branch, the medium encoding branch, and the mutual information estimator are updated simultaneously through gradient backpropagation, so that the content encoding branch and the medium encoding branch output statistically independent and semantically decoupled feature representations. After training convergence, deentangled content features and deentangled medium features are obtained respectively. The deentangled content features are feature maps that only encode the display content signal information and are unrelated to the medium modulation effect, and the deentangled medium features are feature maps that only encode the non-uniform modulation effect of the medium space and are unrelated to the display content signal.

[0037] Finally, the deentangled content features and deentangled medium features are reconstructed and mapped through the content decoding branch and the medium decoding branch, respectively. The content decoding branch uses a stacked structure of transposed convolutional layers symmetrical to the content encoding branch to upsample and reconstruct the deentangled content features layer by layer. The medium decoding branch uses a stacked structure of transposed convolutional layers symmetrical to the medium encoding branch to upsample and reconstruct the deentangled medium features layer by layer. This yields the medium modulation component, which characterizes the non-uniform modulation effect of the medium space, and the content intrinsic component, which characterizes the display content signal.

[0038] refer to Figure 2 This figure is a schematic diagram of the process of separating the medium effect of a photonic display image according to some embodiments of this application. The photonic display image to be enhanced is input on the left. The image signal is sent to the content encoder and the medium encoder in two separate paths. The intrinsic content features and the medium modulation features are extracted in sequence. The two sets of features are de-entangled by minimizing mutual information constraints. Then, the intrinsic content component and the medium modulation component on the right are decoded and output respectively. The intrinsic content component completely preserves the texture and color information of the image objects and has no medium distortion interference. The medium modulation component only presents the smooth spatial brightness and darkness distortion distribution caused by the photonic display medium and does not contain the details of the image content. The complete working logic of the dual-branch de-entanglement network to separate the image display content and the medium optical distortion is fully and intuitively demonstrated.

[0039] It should be noted that, in this application, the medium modulation component is a spatial variation map recording the relative modulation weights of the photonic display medium at each pixel position to each color channel. The content intrinsic component is an image that reflects only the ideal radiation distribution of the original display content signal after removing the medium modulation effect. The role of the medium modulation component is to explicitly separate and quantize the non-uniform modulation effect of the photonic display medium on the brightness and chromaticity of each color channel at each pixel position due to its own physical characteristics from the original photonic display image into an independent component. This provides input information that directly reflects the spatial non-uniform modulation mode distribution of the medium under the current display content for subsequent fusion encoding with the associated response feature map, enabling the compensation network to identify the spatial position and channel range to be compensated based on this medium modulation component. The role of the content intrinsic component is to retain the image that reflects only the ideal radiation distribution of the original display content signal after removing the medium modulation effect as an independent component, providing a clean content signal source without medium distortion for subsequent joint spatial and color channel compensation reconstruction based on the compensation map. This ensures that the compensation operation only corrects the non-uniformity introduced by the medium without changing the information of the display content itself, ultimately obtaining an enhanced photonic display image after compensation and reconstruction.

[0040] In step S103, the medium modulation component and the associated response feature map are fused and encoded to generate constrained enhancement features that characterize the non-uniform distribution across viewpoints and color channels. Based on the constrained enhancement features, a corresponding compensation map is generated through a pre-trained space-color channel joint compensation network.

[0041] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart of determining constraint enhancement features according to some embodiments of this application. In this embodiment, the constraint enhancement features characterizing the non-uniform distribution across viewpoints and color channels are generated by fusing and encoding the medium modulation component and the associated response feature map using the following steps: In step S1031, the medium modulation component is expanded in the color channel dimension to obtain a multi-channel medium modulation tensor; In step S1032, the multi-channel medium modulation tensor and the associated response feature map are concatenated along the channel dimension to obtain a joint input tensor; In step S1033, the joint input tensor is input into a pre-trained feature fusion coding network. The correlation mapping relationship between the spatial non-uniform modulation mode in the medium modulation component and the viewpoint-color channel coupled distribution mode in the correlation response feature map is extracted through the multi-layer convolution operation of the feature fusion coding network, and the constraint enhancement feature is output.

[0042] In practice, the medium modulation components are first expanded in the color channel dimension. If the medium modulation components are in a multi-channel image format, each color channel is separated into an independent two-dimensional spatial distribution map and stacked in channel order to obtain a multi-channel medium modulation tensor. This multi-channel medium modulation tensor is a three-dimensional data structure with dimensions of the number of color channels, spatial height, and spatial width, respectively. It records the relative modulation weight of each pixel position of the photonic display medium to each color channel under the current display content.

[0043] Then, the multi-channel medium modulation tensor and the associated response feature map are concatenated along the channel dimension. Specifically, each channel of the multi-channel medium modulation tensor and each channel of the associated response feature map are connected end to end along the channel dimension to form a joint input tensor with the number of channels equal to the sum of the number of channels of the multi-channel medium modulation tensor and the number of channels of the associated response feature map. This joint input tensor contains, at the same spatial location, both the medium modulation amplitude information of the current image to be enhanced and the inherent cross-viewing angle-cross-color channel coupling response mode information of the photonic display medium.

[0044] Finally, the joint input tensor is fed into a pre-trained feature fusion encoding network. This network employs a fully convolutional structure consisting of multiple convolutional layers with a kernel size of 3×3 and a stride of 1, alternating with batch normalization layers and ReLU activation functions. The network depth is set to 4 to 6 layers. The number of convolutional channels in each layer is gradually compressed from the number of channels in the joint input tensor to the preset number of constraint enhancement feature channels. Through layer-by-layer convolution operations, the lower layers of the network learn the correlation between the local spatial modulation mode of the medium modulation component and the local viewpoint-channel coupling distribution mode of the associated response feature map. The higher layers of the network aggregate the local correlations layer by layer into a global cross-viewpoint and cross-color channel non-uniform distribution correlation mapping relationship. Finally, the network outputs constraint enhancement features.

[0045] It should be noted that the constraint enhancement feature in this application is a multi-channel feature map obtained by adaptively fusing and encoding the spatial non-uniform modulation information to be compensated in the medium modulation component with the prior view-color channel coupling distribution law in the associated response feature map. The processing method of this embodiment enables the constraint enhancement feature to simultaneously encode the correspondence between the instantiated modulation information that the current image to be compensated needs to compensate for and the physical prior mode of how the non-uniformity of the medium changes under different viewpoints and color channels. That is, it realizes the precise positioning and conditional constraint of the medium modulation effect in the joint space of viewpoint and color channel under specific display content. In the prior art, the compensation for display non-uniformity usually only relies on the fixed compensation coefficient calibrated by a single uniform reference pattern, or directly estimates the compensation parameters from a single image to be enhanced. It does not explicitly fuse and encode the prior feature map reflecting the physical response law of the medium across viewpoints and channels with the actual modulation component of the current image to be processed. Therefore, it is impossible to dynamically generate constraint enhancement features consistent with the physical characteristics of the medium for different display content and different observation conditions, resulting in limited compensation accuracy and cross-viewpoint and cross-channel consistency.

[0046] In some embodiments, the spatial-color channel joint compensation network is trained using the following steps: A training dataset is constructed, which contains pairs of constraint-enhanced feature samples and target compensation map samples. The target compensation map samples are space-color channel joint non-uniformity inverse maps obtained by spatial normalization of standard uniform reference patterns collected at each discrete observation angle and at each discrete color channel. The constraint enhancement feature samples are input into the space-color channel joint compensation network to be trained, and the space-color channel joint compensation network outputs a predicted compensation map. A loss function is constructed using the reconstruction error between the predicted compensation map and the target compensation map samples. The network parameters of the spatial-color channel joint compensation network are iteratively updated through backpropagation until the loss function converges, thus obtaining the pre-trained spatial-color channel joint compensation network.

[0047] In specific implementation, firstly, a training dataset is constructed, which contains pairs of constraint enhancement feature samples and target compensation map samples. The constraint enhancement feature samples are generated by acquiring multiple original photonic display images with different display content, and performing medium effect separation and feature fusion encoding operations on each original photonic display image in sequence to obtain the constraint enhancement feature samples corresponding to each original photonic display image. The target compensation map samples are generated by acquiring optical response images of the photonic display medium against a standard uniform reference pattern at each discrete observation angle and each discrete color channel, performing spatial normalization on the optical response image at each pixel position, each color channel, and each observation angle, and taking the reciprocal of the normalized response value to obtain a space-color channel joint non-uniformity inverse map. The value of each pixel position in the inverse map represents the multiplicative compensation coefficient that restores the position to the ideal uniform response. The inverse map is used as a target compensation map sample and paired with the constraint enhancement feature samples under the corresponding display content to form training data sample pairs.

[0048] Subsequently, the constrained enhancement feature samples are input into the spatial-color channel joint compensation network to be trained. This spatial-color channel joint compensation network adopts an encoder-decoder architecture. The encoder consists of multiple stacked convolutional layers with a stride of 2 to progressively compress the spatial resolution and extract high-level semantic features. The decoder consists of a corresponding number of stacked transposed convolutional layers with a stride of 2 to progressively restore the mapping to the target compensation. Figure 1 To achieve the desired spatial resolution, the number of input channels to the network is equal to the number of channels for the constrained enhancement features, and the number of output channels is equal to the preset number of discrete color channels. After forward inference, the network outputs a prediction compensation map, which is the network's estimation result of the distribution of the joint compensation coefficients of the spatial and color channels under the current input.

[0049] Finally, a loss function is constructed based on the reconstruction error between the predicted and target compensation map samples. This loss function uses the mean squared error loss function to calculate the mean of the sum of squares of the differences in the pixel-by-pixel compensation coefficients between the predicted and target compensation map samples. Simultaneously, a perceptual loss function can be introduced as an auxiliary loss to improve the recovery accuracy of the compensation map's edge regions. After calculating the loss value, gradient backpropagation is performed using the Adam optimizer with an initial learning rate of 1× The weight and bias parameters of all convolutional and transposed convolutional layers in the space-color channel joint compensation network are iteratively updated with a batch size of 16 and a training epoch of 200 until the loss function value no longer decreases in multiple consecutive training epochs. This is considered convergence, and the pre-trained space-color channel joint compensation network is obtained.

[0050] It should be noted that the spatial-color channel joint compensation network in this application refers to a pre-trained deep convolutional neural network with an encoder-decoder architecture. It takes constraint enhancement features as input, and the encoder extracts the high-level semantic association between the medium modulation amplitude information encoded in the constraint enhancement features and the viewpoint-color channel coupling distribution pattern layer by layer. Then, the decoder maps and restores the high-level semantic association to the same spatial resolution as the original photon display image. Finally, it outputs a pixel-wise compensation coefficient map with the same number of channels as the preset discrete color channels. The role of this network is to adaptively generate a compensation mapping map for a specific medium spatial location and a specific color channel based on the constraint enhancement features corresponding to different display content, so as to provide accurate multiplicative compensation coefficients for subsequent joint compensation and reconstruction of the content intrinsic components in terms of spatial and color channels.

[0051] In some embodiments, the generation of the corresponding compensation map based on the constraint enhancement features using a pre-trained spatial-color channel joint compensation network is achieved through the following steps: The constraint enhancement features are input into the pre-trained spatial-color channel joint compensation network. After forward inference of the spatial-color channel joint compensation network, a pixel-wise compensation coefficient map with the same spatial resolution as the medium modulation component and the same number of channels as the preset discrete color channels is output. The pixel-by-pixel compensation coefficient map is used as the compensation mapping map.

[0052] Specifically, the constraint enhancement features are first input into a pre-trained spatial-color channel joint compensation network. This network has all its parameters optimized and fixed during pre-training. The number of channels and spatial resolution of the constraint enhancement features are kept consistent with those used during network training. If inconsistent, bilinear interpolation is used to adjust them to the standard size defined during training. The encoder part of the spatial-color channel joint compensation network performs convolutions with a stride of 2 and ReLU activation layer by layer, progressively compressing the spatial resolution of the constraint enhancement features to a preset minimum resolution bottleneck layer to extract globally related semantic features. Then, the decoder part performs layer-by-layer... A transposed convolution with a stride of 2 and a ReLU activation operation progressively upsample the globally correlated semantic features of the bottleneck layer to restore the feature map size to the same spatial resolution as the medium modulation component. Finally, a 1×1 convolutional layer maps the number of feature channels to the output channels, which are equivalent to the preset number of discrete color channels. The output values ​​are then normalized to a reasonable range of compensation coefficient values ​​by a Sigmoid activation function, resulting in a pixel-wise compensation coefficient map. This pixel-wise compensation coefficient map is a multi-channel image that provides a multiplicative compensation coefficient for each preset color channel at each pixel position. Finally, this pixel-wise compensation coefficient map is directly output as the compensation map to the subsequent joint compensation and reconstruction step.

[0053] It should be noted that the compensation map in this application encodes a joint compensation scheme for spatial and color channels for the spatial non-uniform modulation effect of the photonic display medium under the current display content. The purpose of determining the compensation map is to provide a set of precise multiplicative compensation coefficients for each pixel position and each color channel of the content intrinsic components. This allows the content intrinsic components to be multiplied pixel by pixel and channel by channel in the subsequent joint compensation and reconstruction steps. Based on these compensation coefficients, the spatial non-uniform brightness attenuation and the imbalance in response between color channels caused by the photonic display medium can be reversed. This eliminates the medium modulation distortion while preserving the information of the display content signal itself, and finally obtains an enhanced photonic display image that presents a uniform visual effect under different viewing angles.

[0054] In step S104, the content intrinsic components are jointly compensated and reconstructed based on the compensation mapping map to obtain an enhanced photonic display image.

[0055] It should be noted that, in this application, joint compensation reconstruction refers to the process of performing pixel-by-pixel and channel-by-channel multiplicative correction on the intrinsic components of the content, which reflect only the ideal radiation distribution of the original display content signal after the medium modulation effect has been removed, and the compensation mapping map generated by the space-color channel joint compensation network based on the constraint enhancement features. Then, the corrected multi-channel content tensors are merged and mapped to the target display color gamut. This is used to achieve integrated correction and reconstruction of the non-uniform modulation effect of the photonic display medium in space and the non-uniform modulation effect of the color channels.

[0056] In some embodiments, the enhanced photonic display image is obtained by performing joint spatial and color channel compensation reconstruction on the content intrinsic components based on the compensation map using the following steps: The content intrinsic components are expanded in the color channel dimension to obtain a multi-channel content intrinsic tensor; The multi-channel content intrinsic tensor is multiplied by the pixel-by-pixel compensation coefficient at the corresponding spatial position in the compensation mapping map to obtain the compensated multi-channel content tensor. The compensated multi-channel content tensor is combined in the color channel dimension and mapped to the target display color gamut to obtain the enhanced photonic display image.

[0057] Specifically, firstly, the content intrinsic components are expanded in the color channel dimension. If the content intrinsic components are in RGB three-channel image format, they are separated into three independent two-dimensional spatial distribution matrices: R channel, G channel, and B channel. These matrices are then stacked in channel order to obtain a multi-channel content intrinsic tensor. This multi-channel content intrinsic tensor is a three-dimensional data structure with dimensions 3×H×W. H refers to the size value of the multi-channel content intrinsic tensor in the three-dimensional data structure representing the spatial height dimension, which is equal to the number of pixel rows in the vertical direction of the content intrinsic component image. W refers to the size value of the multi-channel content intrinsic tensor in the three-dimensional data structure representing the spatial width dimension, which is equal to the number of pixel columns in the horizontal direction of the content intrinsic component image. The multi-channel content intrinsic tensor is used to reflect the ideal radiance distribution of the original display content signal in each color channel.

[0058] Secondly, a multiplication operation is performed on the multi-channel content intrinsic tensor and the pixel-wise compensation coefficients at the corresponding spatial positions in the compensation map. Specifically, for each color channel c and each spatial coordinate position (i,j), the value of the multi-channel content intrinsic tensor at (c,i,j) is multiplied by the compensation coefficient value at (c,i,j) in the compensation map. This compensation coefficient value is a floating-point number greater than 0. For pixel positions with low brightness, the compensation coefficient is greater than 1 to increase brightness, and for pixel positions with high brightness, the compensation coefficient is less than 1 to suppress brightness. After the multiplication operation, the compensated multi-channel content tensor is obtained. The compensated multi-channel content tensor maintains the same dimensional structure as the multi-channel content intrinsic tensor, but the radiance values ​​at each pixel position have been corrected for spatial non-uniformity and tend to be uniform under different observation angles.

[0059] Finally, the compensated multi-channel content tensor is merged along the color channel dimension and mapped to the target display color gamut. Specifically, the color channels of the compensated multi-channel content tensor are merged into a multi-channel image in the original channel order. Then, the multi-channel image is mapped from the linear radiometric space of the acquisition device to the device-dependent color space of the target display device through a color gamut conversion matrix. For example, the standard ITU-R BT.709 color gamut conversion matrix is ​​used to convert the linear RGB values ​​to sRGB non-linear encoded values ​​to adapt to conventional display devices, resulting in an enhanced photonic display image.

[0060] It should be noted that the enhanced photonic display image in this application refers to the final output image after eliminating the spatial non-uniform modulation effect of the photonic display medium.

[0061] Furthermore, in another aspect of this application, in some embodiments, this application provides a deep learning-based photonic display image enhancement system, referencing... Figure 4The figure is a schematic diagram of the structure of a deep learning-based photonic display image enhancement system according to some embodiments of this application. The deep learning-based photonic display image enhancement system includes: a feature recognition module 201, a medium effect separation module 202, a compensation map generation module 203, and a compensation reconstruction module 204, which are described below: Feature recognition module 201, in this application, is mainly used to collect multiple sets of optical response images under different observation angles and different color channels when displaying a preset uniform reference pattern on a photonic display medium, and to construct a view-channel correlation response feature map based on the optical response images; The medium effect separation module 202 in this application is mainly used to acquire the photonic display image to be enhanced. The photonic display image is an optical response image acquired by an imaging device when displaying content in a photonic display medium. The medium effect separation module 202 is used to separate the photonic display image to obtain the medium modulation component characterizing the non-uniform modulation effect of the medium space and the content intrinsic component characterizing the display content signal. The compensation map generation module 203 in this application is mainly used to fuse and encode the medium modulation component and the associated response feature map to generate constrained enhancement features that characterize the non-uniform distribution across viewpoints and color channels. Based on the constrained enhancement features, a corresponding compensation map is generated through a pre-trained space-color channel joint compensation network. The compensation and reconstruction module 204 in this application is mainly used to perform joint compensation and reconstruction of the spatial and color channels of the content intrinsic components based on the compensation mapping map to obtain an enhanced photon display image.

[0062] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described deep learning-based photonic display image enhancement method.

[0063] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a deep learning-based photonic display image enhancement method according to some embodiments of this application. The deep learning-based photonic display image enhancement method in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0064] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the deep learning-based photonic display image enhancement method in this application.

[0065] The communication bus 302 can be used to transmit information between the aforementioned components.

[0066] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0067] The memory 303 stores program code for executing the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the deep learning-based photonic display image enhancement method can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0068] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0069] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0070] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0071] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described deep learning-based photonic display image enhancement method.

[0072] Although preferred embodiments of this application have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0073] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application.

Claims

1. A deep learning-based method for enhancing photonic display images, characterized in that, Includes the following steps: When displaying a preset uniform reference pattern on a photonic display medium, multiple sets of optical response images are acquired under different observation angles and different color channels, and a viewpoint-channel correlation response feature map is constructed based on the optical response images. A photonic display image to be enhanced is acquired, wherein the photonic display image is an optical response image acquired by an imaging device when the content is displayed on the photonic display medium, and the photonic display image is subjected to medium effect separation to obtain a medium modulation component characterizing the non-uniform modulation effect of the medium space and a content intrinsic component characterizing the display content signal. The medium modulation component and the associated response feature map are fused and encoded to generate constrained enhancement features that characterize the non-uniform distribution across viewpoints and color channels. Based on the constrained enhancement features, a corresponding compensation map is generated through a pre-trained space-color channel joint compensation network. Based on the compensation mapping, the intrinsic components of the content are jointly compensated and reconstructed using spatial and color channels to obtain an enhanced photonic display image.

2. The method as described in claim 1, characterized in that, When displaying a preset uniform reference pattern on a photonic display medium, multiple sets of optical response images are acquired under different observation angles and different color channels. A viewpoint-channel correlated response feature map is constructed based on these optical response images, specifically including: Under multiple preset discrete observation angles and discrete color channels, the optical response images of the photonic display medium to a preset uniform reference pattern are acquired to obtain a set of multidimensional observation data cubes; Spatial normalization is performed on the multidimensional observation data cube to eliminate the self-uniform distribution prior of the preset uniform reference pattern, and a normalized multi-view-multi-channel response tensor is obtained. The normalized multi-view-multi-channel response tensor is decomposed along the view dimension and the channel dimension to extract the view-channel joint intrinsic modes. The view-channel joint intrinsic modes are reconstructed according to the original spatial dimensions to obtain the view-channel associated response feature map.

3. The method as described in claim 2, characterized in that, Under multiple preset discrete observation angles and discrete color channels, optical response images of the photonic display medium to a preset uniform reference pattern are acquired, resulting in a set of multidimensional observation data cubes, specifically including: Display a pre-defined uniform reference pattern with a known uniform spatial intensity distribution on a photonic display medium; Multiple discrete observation angles are set within the observation hemisphere of the photonic display medium. At each discrete observation angle, a multispectral imaging device sequentially acquires the uniform reference pattern through multiple preset discrete color channels to obtain a set of original multi-view-multi-channel optical response image sequences. The original multi-view, multi-channel optical response image sequence is organized and arranged according to three dimensions: observation angle index, color channel index, and spatial pixel coordinates, forming a multi-dimensional observation data cube.

4. The method as described in claim 1, characterized in that, Dielectric effect separation is performed on the photonic display image to obtain the dielectric modulation component characterizing the non-uniform modulation effect in the dielectric space and the content intrinsic component characterizing the display content signal. Specifically, this includes: The photonic display image is input into a pre-trained medium effect separation network, which includes a parallel content coding branch and a medium coding branch, to extract features from the photonic display image to obtain content intrinsic features and medium modulation features. The content intrinsic features and the medium modulation features are subjected to deentanglement training under the constraint of minimizing mutual information, so that the content coding branch and the medium coding branch output semantically decoupled feature representations, thus obtaining deentangled content features and deentangled medium features; The unentangled content features and the unentangled medium features are reconstructed and mapped through the content decoding branch and the medium decoding branch, respectively, to obtain the medium modulation component characterizing the non-uniform modulation effect of the medium space and the content intrinsic component characterizing the display content signal.

5. The method as described in claim 1, characterized in that, The process of fusing and encoding the medium modulation component with the associated response feature map to generate constrained enhancement features characterizing the non-uniform distribution across viewpoints and color channels specifically includes: The medium modulation components are expanded in the color channel dimension to obtain a multi-channel medium modulation tensor; The multi-channel medium modulation tensor and the associated response feature map are concatenated along the channel dimension to obtain the joint input tensor; The joint input tensor is input into a pre-trained feature fusion coding network. The multi-layer convolution operation of the feature fusion coding network is used to extract the correlation mapping relationship between the spatial non-uniform modulation mode in the medium modulation component and the viewpoint-color channel coupled distribution mode in the correlation response feature map, and the constraint enhancement feature is output.

6. The method as described in claim 1, characterized in that, Based on the aforementioned constraint enhancement features, the generation of the corresponding compensation map through a pre-trained space-color channel joint compensation network specifically includes: The constraint enhancement features are input into the pre-trained spatial-color channel joint compensation network. After forward inference of the spatial-color channel joint compensation network, a pixel-wise compensation coefficient map with the same spatial resolution as the medium modulation component and the same number of channels as the preset discrete color channels is output. The pixel-by-pixel compensation coefficient map is used as the compensation mapping map.

7. The method as described in claim 1, characterized in that, The uniform reference pattern uses a standard grayscale test image where pixel values ​​are equal everywhere.

8. A deep learning-based photonic display image enhancement system, used to perform the deep learning-based photonic display image enhancement method as described in any one of claims 1 to 7, characterized in that, The system includes: The feature recognition module is used to acquire multiple sets of optical response images under different observation angles and different color channels when displaying a preset uniform reference pattern on a photonic display medium, and to construct a viewpoint-channel correlation response feature map based on the optical response images. The medium effect separation module is used to acquire the photonic display image to be enhanced, wherein the photonic display image is an optical response image acquired by an imaging device when the photonic display medium displays content, and to perform medium effect separation on the photonic display image to obtain the medium modulation component characterizing the non-uniform modulation effect of the medium space and the content intrinsic component characterizing the display content signal. The compensation map generation module is used to fuse and encode the medium modulation component and the associated response feature map to generate constrained enhancement features that characterize the non-uniform distribution across viewpoints and color channels. Based on the constrained enhancement features, a corresponding compensation map is generated through a pre-trained space-color channel joint compensation network. The compensation and reconstruction module is used to perform joint compensation and reconstruction of the content intrinsic components based on the compensation mapping map, using spatial and color channels, to obtain an enhanced photon display image.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the deep learning-based photonic display image enhancement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based photonic display image enhancement method as described in any one of claims 1 to 7.

Images