Naked eye 3D transparent display imaging system

Abstract

The invention discloses a naked eye 3D transparent display imaging system, and relates to the technical field of 3D transparent imaging. The method comprises the steps that a multi-scale parallax layer construction module is used for decomposing a multi-scale parallax layer and dividing regional images; the dynamic viewpoint prediction module is used for generating two-eye three-dimensional space coordinates through human eye parameters; the parallax scheduling mapping module is used for performing space mapping according to the binocular three-dimensional space coordinates to generate a target visual angle image; the background compensation module is used for carrying out brightness correction on the target view angle image to obtain a compensated view angle image; the directional light field regulation and control module is used for processing the compensation view angle image and generating directional light field information; and the full-layer fusion module is used for processing directional light field information, eliminating interface artifacts between layers and generating a naked-eye 3D transparent display image. Through multi-scale parallax construction, dynamic viewpoint tracking and multi-layer light field fusion, the visual angle adaptability and image fusion quality of naked eye 3D display are improved, and a transparent three-dimensional imaging effect with high immersion, high consistency and high definition is realized.

Description

Technical Field

[0001] The present invention relates to the field of 3D transparent imaging technology, and specifically to a naked-eye 3D transparent display imaging system. Background Art

[0002] With the rapid development of technologies such as augmented reality, mixed reality, and spatial visualization, the naked-eye 3D display technology is becoming the key display form for the next-generation human-computer interaction and spatial perception because it can present a stereoscopic visual experience without wearing auxiliary devices. Among them, the transparent display screen has broad prospects in applications such as in-vehicle head-up displays, smart glass, and wearable devices due to its dual capabilities of information display and background perspective.

[0003] However, there are still many bottlenecks in the existing naked-eye 3D transparent display systems in terms of parallax construction accuracy, dynamic viewpoint prediction stability, image compensation authenticity, and directional light field regulation ability. For example, the accuracy of parallax layer construction is insufficient. The traditional parallax division method based on depth estimation relies on single-scale image processing and is difficult to maintain the edge accuracy of depth-consistent regions in multi-scale structures, resulting in discontinuous parallax transitions and spatial illusions or depth drifts after image fusion; the viewpoint tracking response has a large delay and error. Most systems rely on a single infrared tracker or eye movement camera and lack the joint modeling of iris deformation, pupil movement, and coordinate drift, making it difficult to achieve high-precision prediction of the real-time spatial position of the observer and affecting the timely update of perspective images; the transparent background interference is serious, the compensation mechanism is imperfect, the light field regulation granularity is limited, the directional adjustment is inaccurate, the layer fusion algorithm has poor robustness and is prone to artifacts. The existing layer fusion mechanisms usually rely on linear superposition or fixed template synthesis and lack the perception ability of the feature differences and boundary mutations between different depth layers, often resulting in significant artifacts at the boundaries and destroying the transparency and continuity of the naked-eye 3D imaging.

[0004] Therefore, there is an urgent need for a naked-eye 3D transparent display imaging system that integrates multi-scale structure tensor modeling, dynamic iris and eye movement prediction, real physical transmission compensation, directional light field response control, and multi-feature depth fusion mechanism to break through the current technical bottlenecks and improve the spatial realism, transparent background adaptability, and directional visual quality of the naked-eye 3D imaging. Summary of the Invention

[0005] Based on the above-mentioned disadvantages of the existing technology, the purpose of the present invention is to provide a naked-eye 3D transparent display imaging system to solve the above technical problems.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A naked-eye 3D transparent display imaging system, comprising: Multi-scale parallax layer construction module, which decomposes the input image sequence into multi-scale parallax layers. The deep structure factor mapping method is used to construct the parallax layers. The region images with similar depth information are divided through the structure-oriented tensor template and the soft region gating function; Dynamic viewpoint prediction module, which captures the iris deformation parameters and eye movement tracking data of the observer in real time through an infrared camera, and maps and generates the binocular three-dimensional space coordinates based on the linear discriminant model; Parallax scheduling mapping module, which performs spatial mapping on the parallax layers according to the binocular three-dimensional space coordinates to generate the target perspective image. The dynamic parallax scheduling is performed through non-linear resampling during the spatial mapping process; Background compensation module, which simulates the background penetration effect through the environmental transmittance map, combines the multi-channel irradiation model to generate the visual compensation map, corrects the brightness and contrast of the target perspective image, and obtains the compensated perspective image; Directional light field regulation module, which encodes the light propagation angle of the compensated perspective image through the predefined directional response basis function to generate the directional light field information containing the light intensity regulation coefficient; Full-layer fusion module, which non-linearly superimposes all the directional light field information based on the dynamic fusion weight tensor. After fusion, the interface artifacts between layers are eliminated through depth correction to generate the naked-eye 3D transparent display image.

[0007] The present invention is further configured that the multi-scale parallax layer construction module includes: The input image sequence is decomposed into different resolution levels through the multi-scale Gaussian pyramid, and the spatio-temporal structure tensor analysis is performed on each level to extract the principal components representing depth consistency and generate the preliminary parallax response; Based on the principal component responses of each level, the structure-oriented tensor template corresponding to the depth region is generated through depth clustering and spatial correlation matrix feature mapping;

[0008] Pixels are dynamically assigned to each parallax layer through the normalized probability model. The gating probability is generated by combining the preliminary parallax response and the structure-oriented tensor template, and the preliminary parallax response is weighted by the gating probability to generate the final parallax layer.

[0009] The present invention is further configured that the dynamic viewpoint prediction module includes: The dynamic deformation parameters of the iris region are extracted through infrared image segmentation and deformation registration; The coordinate system conversion and motion noise filtering are performed on the pupil position data to generate smooth eye movement tracking data; A joint feature vector is constructed based on the iris deformation parameters and the eye movement tracking data;

[0010] The joint feature vector is mapped into the binocular three-dimensional space coordinates through the linear discriminant model.

[0011] The present invention is further configured such that the parallax scheduling and mapping module includes: Generate horizontal and vertical parallax vectors based on the binocular three-dimensional space coordinates and the depth scaling factor of the parallax layer, and optimize the parallax offset through a depth residual network; Perform pixel displacement mapping on the parallax layer based on deformable convolutional kernels, and dynamically mask invalid sampling areas in combination with a gated mask mechanism; Generate the target perspective image by fusing mapping results of different scales through optical flow alignment and depth confidence weighting.

[0012] The present invention is further configured such that the calculation of the depth scaling factor includes: Based on the calibrated physical depth value and spatial gradient distribution of the parallax layer, in combination with the dynamic parallax correction term and the amplitude of the binocular parallax vector, dynamically adjust the scaling intensity of each parallax layer through linear ratio operations.

[0013] The present invention is further configured such that the background compensation module includes: Generate a transmittance map through a physical attenuation model based on the real-time ambient light irradiance distribution and the display panel transmittance; Decompose the target perspective image into diffuse and specular reflection components, and generate a synthetic irradiance distribution by combining the transmittance map and background irradiation superposition; Analyze the brightness contrast difference between the target irradiance and the synthetic irradiance, and generate an adaptive compensation weight map through edge-preserving filtering; Perform non-linear brightness adjustment on the target perspective image according to the compensation weight map to obtain the compensated perspective image.

[0014] The present invention is further configured such that the calculation of the display panel transmittance includes: Generate a basic attenuation component of the transmittance based on the driving voltage and the wavelength-dependent electro-optical modulation coefficient, and the driving voltage is dynamically adjusted according to the ambient light intensity to control the non-linear attenuation intensity of the transmittance; Based on the intrinsic absorption spectral width parameter and thermal vibration frequency of the display panel material, correct the dispersion deviation of the electro-optical modulation term through an error function to generate a wavelength-dependent absorption compensation component; Perform wavelength-by-wavelength product fusion on the basic attenuation component and the absorption compensation component, and output the display panel transmittance that dynamically adapts to the ambient light.

[0015] The present invention is further configured such that the directional light field regulation module includes: Construct an orthogonality directional basis function family based on the physical characteristics of the light field regulation unit, satisfying the preset angular resolution and coverage range; Decompose the compensated perspective image into a linear combination of basis functions pixel by pixel to generate directional light field regulation coefficients; Dynamically optimize the directional light field regulation coefficient through sparse constraints and energy conservation conditions, and retain the high-frequency details in the target direction; Map the optimized directional light field regulation coefficient to a light field regulation coefficient matrix matching the physical drive unit to generate directional light field information.

[0016] The present invention is further configured that the full-layer fusion module includes: Construct a pixel-level feature vector based on the depth confidence, light field intensity, and edge gradient of the directional light field information, and generate a dynamic fusion weight through a lightweight convolutional network; Perform pixel-by-pixel weighted fusion on the directional light field information and retain the depth-level features; Detect the depth jump region of adjacent layers, smooth the interface artifacts through edge-aware joint bilateral filtering, and generate a naked-eye 3D transparent display image.

[0017] The present invention provides a naked-eye 3D transparent display imaging system. The method includes a multi-scale parallax layer construction module that decomposes the input image sequence into multi-scale parallax layers. The parallax layers are constructed using a deep structure factor mapping method, and the region images with similar depth information are divided through a structure-guided tensor template and a soft region gating function; a dynamic viewpoint prediction module that captures the iris deformation parameters and eye movement tracking data of the observer in real time through an infrared camera, and maps and generates the binocular three-dimensional space coordinates based on a linear discriminant model; a parallax scheduling mapping module that performs spatial mapping on the parallax layers according to the binocular three-dimensional space coordinates to generate a target perspective image. The spatial mapping process performs dynamic parallax scheduling through non-linear resampling; a background compensation module that simulates the background penetration effect through an environmental transmittance map, combines a multi-channel irradiance model to generate a visual compensation map, and corrects the brightness and contrast of the target perspective image to obtain a compensated perspective image; a directional light field regulation module that encodes the light propagation angle of the compensated perspective image through a predefined directional response basis function to generate directional light field information including light intensity regulation coefficients; a full-layer fusion module that non-linearly superimposes all the directional light field information based on a dynamic fusion weight tensor, and after fusion, eliminates the interface artifacts between layers through depth correction to generate a naked-eye 3D transparent display image. The beneficial effects produced include: 1. Achieve a realistic naked-eye 3D display experience: Through multi-scale parallax layer construction and structure-guided tensor division, effectively extract the regions with consistent image depth, significantly enhance the three-dimensional sense of stereoscopy, and avoid the ghosting problem caused by parallax blur or depth mismatch in traditional methods; 2. Have high-precision dynamic viewpoint tracking ability: Use a combined feature of iris deformation and eye movement to construct a model, and use a linear discriminant method to realize real-time prediction of binocular three-dimensional space coordinates. Compared with traditional eye movement tracking technologies, it has higher accuracy and faster response, enhancing the user interaction immersion; 3. Achieve seamless fusion of multiple layers and artifact suppression: Construct a dynamic fusion weight tensor, combine edge perception and bilateral filtering strategies, achieve per-pixel weighted fusion of directional light field layers and interface smoothing, effectively eliminate fusion artifacts in depth jump regions, and improve the overall coherence and realism of the displayed image.

[0018] The above description is only an overview of the technical solution of this application. In order to be able to understand the technical means of this application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the following specifically illustrates the specific implementation manners of this application. Brief Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings: Figure 1 It is a flowchart of a naked-eye 3D transparent display imaging system shown in an exemplary embodiment of the present invention. Detailed Embodiment

[0020] The following will illustrate the embodiments of the present invention with reference to the drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for explaining the present invention, rather than for limiting the protection scope of the present invention.

[0021] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The type, number and ratio of each component in its actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0022] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0023] Embodiment 1

[0024] A naked-eye 3D transparent display imaging system, as Figure 1 shown, includes: A multi-scale parallax layer construction module that decomposes the input image sequence into multi-scale parallax layers. The deep structure factor mapping method is used to construct the parallax layers. The region images with similar depth information are divided through the structure-oriented tensor template and the soft region gating function; A dynamic viewpoint prediction module that captures the iris deformation parameters and eye movement tracking data of the observer in real time through an infrared camera, and maps and generates the three-dimensional space coordinates of both eyes based on a linear discriminant model; A parallax scheduling mapping module that performs spatial mapping on the parallax layers according to the three-dimensional space coordinates of both eyes to generate a target perspective image. The dynamic parallax scheduling is performed through non-linear resampling during the spatial mapping process; A background compensation module that simulates the background penetration effect through the environmental transmittance map, combines with a multi-channel irradiance model to generate a visual compensation map, corrects the brightness and contrast of the target perspective image, and obtains a compensated perspective image; A directional light field regulation module that encodes the light propagation angle of the compensated perspective image through a predefined directional response basis function to generate directional light field information including light intensity regulation coefficients; A full-layer fusion module that non-linearly superimposes all directional light field information based on a dynamic fusion weight tensor, and eliminates the interface artifacts between layers through depth correction after fusion to generate a naked-eye 3D transparent display image.

[0025] The present invention is further configured such that the multi-scale parallax layer construction module includes: The input image sequence is decomposed into different resolution levels through a multi-scale Gaussian pyramid, and spatio-temporal structure tensor analysis is performed on each level to extract the principal components representing depth consistency and generate a preliminary parallax response. Specifically, the image sequence , is the number of images, , is the coordinate, and each image is decomposed into different resolution levels through a multi-scale Gaussian pyramid , where is the number of scales; for each scale-level image , its three-dimensional structure tensor is calculated, and the principal components are extracted through high-order singular value decomposition HOSVD, , is the number of principal components, is the high-order singular value, , , are the space-time direction basis vectors after HOSVD decomposition. According to sorting, the first Generate a preliminary disparity response map from the principal components ; Based on the principal component responses at each level, generate a structure-oriented tensor template corresponding to the depth region through deep clustering and spatial correlation matrix feature mapping. Specifically, for each disparity layer Generate a spatial structure constraint template , suppressing cross-layer interference; for perform deep clustering on the pixel points, and divide the pixels into classes based on motion consistency, with each class corresponding to a disparity layer; learn the spatial correlation matrix for each class region, and generate a structure template through feature mapping , where is the activation function, is the learnable convolutional kernel, optimized through backpropagation for extracting depth-related structural features, is the spatial correlation matrix, calculated through the neighborhood gray covariance of the pixels in the k-th layer after clustering, is the bias term, jointly trained with ; Dynamically allocate pixels to each disparity layer through a normalized probability model, combine the preliminary disparity response with the structure-oriented tensor template to generate a gating probability, and weight the preliminary disparity response by the gating probability to generate the final disparity layer. Specifically, the gating probability combines the depth response and the structure template to calculate the probability that pixel belongs to the th layer: , and weight the preliminary disparity response by the gating probability to generate the final disparity layer: .

[0026] The present invention is further configured such that the dynamic viewpoint prediction module includes: Extract the dynamic deformation parameters of the iris region through infrared image segmentation and deformation registration. Specifically, perform pixel-level segmentation on the iris region based on the U-Net network to output the iris mask ; calculate the displacement and local rotation angle of each pixel through a non-rigid registration algorithm; compress the deformation field into a low-dimensional vector through principal component analysis (PCA) to characterize the dynamic deformation features of the iris ; Perform coordinate transformation and motion noise filtering on the pupil position data to generate smooth eye movement tracking data. Specifically, input the pupil center coordinates and the eye ball attitude angle , convert the pupil position in the camera coordinate system to the world coordinate system to generate a normalized position vector , is a 3×3 affine transformation matrix obtained through calibration of a calibration board; the noise caused by head micro-movement is eliminated through Kalman filtering, and the smoothed eye movement trajectory is output ; Construct a joint feature vector based on iris deformation parameters and eye movement tracking data. Specifically, for and perform Hadamard product to generate cross features ; Map the joint feature vector to the three-dimensional coordinates of both eyes through a linear discriminant model, and learn the projection matrix based on the training data to map the joint features to the three-dimensional coordinates of both eyes: , where , are the three-dimensional coordinates of the left and right eyes, and the projection matrix is obtained through supervised learning training.

[0027] The present invention is further configured such that the disparity scheduling mapping module includes: Generate horizontal and vertical disparity vectors according to the three-dimensional coordinates of both eyes and the depth scaling factor of the disparity layer, and optimize the disparity offset through a depth residual network. Specifically, for each disparity layer , based on its depth scaling factor , calculate its horizontal and vertical disparity vectors relative to the reference viewpoint: , ; Predict the dynamic correction term of the disparity offset through the depth residual network ResNet, ; Perform pixel displacement mapping on the disparity layer based on a deformable convolutional kernel, and dynamically mask the invalid sampling area in combination with a gated mask mechanism. Specifically, use the deformable convolutional kernel to perform dynamic sampling: , where is the disparity layer image, is the weight of the deformable convolutional kernel; Dynamically mask the invalid sampling area through the gated mask ; Generate the target perspective image by fusing the mapping results of different scales through optical flow alignment and depth confidence weighting. Specifically, input the mapping results of each scale layer, is the number of multi-scale levels, align the mapping results of different scales through optical flow estimation, eliminate the displacement error caused by the resolution difference, and generate a fusion weight map based on the depth confidence to generate the target perspective image: .

[0028] The present invention is further configured such that the calculation of the depth scaling factor includes: Based on the calibrated physical depth values and the spatial gradient distribution of the parallax layers, combining the dynamic parallax correction term and the magnitude of the binocular parallax vector, dynamically adjust the scaling intensity of each parallax layer through linear ratio operations. Specifically, the calculation formula for the depth scaling factor is: , where is the calibrated physical depth value of the th parallax layer, obtained through offline calibration, is the spatial gradient of the parallax layer , is the L1 norm of the gradient, measuring the geometric complexity of the parallax layer, is the dynamic parallax correction term scalar, obtained by global average pooling after being predicted by ResNet, is the horizontal and vertical parallax vectors of the binocular coordinates, , is the dynamic weight factor, calibrated through experiments.

[0029] The present invention is further configured such that the background compensation module includes: Based on the real-time ambient light irradiance distribution and the display panel transmittance, generate a transmittance map through a physical attenuation model. Specifically, for the penetration effect modeling, calculate the attenuation distribution after the background light penetrates the display layer: , where is the multi-spectral irradiation intensity captured by the ambient light sensor, is the wavelength, is the display panel transmittance, is the panel material absorption coefficient (calibrated through laboratory measurements; Decompose the target perspective image into diffuse reflection and specular reflection components, and combine the transmittance map and the background irradiation to generate a synthetic irradiation distribution. Specifically, based on the Retinex theory, separate the diffuse reflection component and the specular reflection component , , , where is the target perspective image; synthesize the total irradiance based on the physical rendering equation, , where is the synthetic irradiance, is the transmittance map, is the viewing angle-dependent specular transmittance, , is the angle between the observer's line of sight and the panel normal; Analyze the brightness contrast difference between the target irradiation and the synthetic irradiation, and generate an adaptive compensation weight map through edge-preserving filtering. Specifically, calculate the difference map, , where is the difference map, is the ideal irradiance, is the synthetic irradiance, which is edge-preserving smoothed by a bilateral filter for to generate a compensation weight map ; Nonlinear brightness adjustment is performed on the target perspective image according to the compensation weight map to obtain a compensated perspective image. Specifically, the nonlinear brightness adjustment formula is: , where is the compensated perspective image, is the ambient light intensity adaptive adjustment factor.

[0030] The present invention is further configured such that the display panel transmittance calculation includes: Generating a basic attenuation component of the transmittance based on the driving voltage and the wavelength-dependent electro-optic modulation coefficient. The driving voltage is dynamically adjusted according to the ambient light intensity to control the nonlinear attenuation intensity of the transmittance, which is the electro-optic modulation dominant term of the display panel transmittance: , where is the wavelength-dependent electro-optic modulation coefficient, which is related to the energy band structure of the panel material, is the nonlinear attenuation exponent, is the panel driving voltage; Based on the intrinsic absorption spectral width parameter and the thermal vibration frequency of the display panel material, the dispersion deviation of the electro-optic modulation term is corrected by an error function to generate a wavelength-dependent absorption compensation component, which is the dispersion absorption correction term of the display panel transmittance , where is the dispersion correction factor, which compensates for the wavelength-dependent phase delay, is the material absorption spectral width parameter, is the lattice thermal vibration frequency, is the Gauss error function; The basic attenuation component and the absorption compensation component are multiplied and fused wavelength by wavelength to output the display panel transmittance that dynamically adapts to the ambient light. The display panel transmittance calculation formula is: .

[0031] The present invention is further configured such that the directional light field control module includes: Constructing an orthonormal directional basis function family based on the physical characteristics of the light field control unit, satisfying the preset angular resolution and coverage range. Specifically, the initial basis function is corrected by Gram-Schmidt orthogonalization to ensure no energy coupling between different basis functions; Decomposing the compensated perspective image into a linear combination of basis functions pixel by pixel to generate directional light field control coefficients. Specifically, for each pixel of the light propagation direction , Let the polar angle represent the angle between the light propagation direction and the normal direction of the display panel, and the azimuth angle represents the projection direction of the light on the display panel plane. Perform basis function expansion: , where is the basis function index, is the light field coefficient to be solved, is the basis function used to describe the light propagation direction; Dynamically optimize the directional light field modulation coefficient through sparse constraint and energy conservation conditions, and retain the high-frequency details in the target direction. Specifically, minimize the reconstruction error through L1 regularization optimization, and use the ADMM alternating direction multiplier method for iterative solution to balance the reconstruction accuracy and sparsity; through dynamic response optimization, dynamically adjust the basis function weights and preferentially retain the high-frequency details in the target direction: , is the optimized directional light field modulation coefficient, is the set of basis function indices corresponding to the target direction, is the enhancement factor, is the indicator function, which takes 1 when the basis function index n belongs to the target direction set and takes 0 otherwise; enforce the total light intensity to satisfy through the energy conservation constraint, is the maximum light intensity of a single pixel of the display device, to avoid local overexposure or underexposure; Map the optimized directional light field modulation coefficient to a light field modulation coefficient matrix matching the physical driving unit to generate directional light field information. Specifically, generate a light field modulation coefficient matrix corresponding to the physical driving unit of the display device , is the spatial resolution of the display panel, that is, the width and height of the image, is the number of directional basis functions, and convert the coefficient matrix into a driving signal to adapt to the response characteristics of the nano-grating or liquid crystal lens.

[0032] The present invention is further configured such that the full-layer fusion module includes: Construct a pixel-level feature vector based on the depth confidence, light field intensity, and edge gradient of the directional light field information, and generate a dynamic fusion weight through a lightweight convolutional network. Specifically, input each directional light field information and its depth confidence map , is the total number of light field layers, and construct a pixel-level feature vector , where is the depth confidence of the th light field layer, is the amplitude of the light field intensity gradient, is the maximum light intensity value within the pixel neighborhood; generate dynamic fusion weights through a lightweight convolutional neural network (CNN): , where is a 3-layer lightweight convolution with a 3×3 convolution kernel, and the training data is the ground truth of multi-scene light field fusion, normalize and constrain the weight range to [0,1], and the dynamic fusion weights satisfy ; perform pixel-by-pixel weighted fusion on the directional light field information and retain the depth level features. Specifically, the pixel-by-pixel fusion formula is: , where is the non-linear interaction term coefficient to enhance the inter-layer correlation; obtain the enhanced image by strengthening the edge sharpness of the fused image through the Laplacian operator ; detect the depth jump regions of adjacent layers, smooth the interface artifacts through edge-aware joint bilateral filtering, and generate a naked-eye 3D transparent display image. Specifically, the depth jump detection formula is: , where , is the depth confidence of adjacent spatial positions, is the depth jump threshold to mark the significant discontinuous regions; generate a naked-eye 3D transparent display image through edge-aware joint bilateral filtering: , where is the naked-eye 3D transparent display image, is the normalization factor, is the neighborhood pixel coordinates within the filtering window centered on the current pixel , is the standard deviation in the spatial domain to control the size of the filtering kernel, is the standard deviation in the color domain to control the color similarity weight.

[0033] It should be noted that for the naked-eye 3D transparent display imaging system provided in the above embodiments, the specific manners in which each module and unit perform operations have been described in detail in the method embodiments and will not be elaborated here. In practical applications, for the naked-eye 3D transparent display imaging system provided in the above embodiments, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above, and this is not limited here either.

[0034] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0035] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0036] In the present application, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following items (or a similar expression)" refers to any combination of these items, including any combination of single items (or a single item) or plural items (or plural items). For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0037] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0038] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0039] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0040] In several embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be electrical, mechanical, or other forms.

[0041] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0042] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0043] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0044] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A naked-eye 3D transparent display imaging system, characterized in that, Including: A multi-scale disparity layer construction module that decomposes the input image sequence into multi-scale disparity layers. The deep structure factor mapping method is used to construct the disparity layers, and the region images with similar depth information are divided through a structure-guided tensor template and a soft region gating function; A dynamic viewpoint prediction module that captures the iris deformation parameters and eye movement tracking data of the observer in real time through an infrared camera, and maps and generates the binocular three-dimensional space coordinates based on a linear discriminant model; A disparity scheduling mapping module that performs spatial mapping on the disparity layers according to the binocular three-dimensional space coordinates to generate the target perspective image. The dynamic disparity scheduling is performed through non-linear resampling during the spatial mapping process; A background compensation module that simulates the background penetration effect through the environmental transmittance map, combines with a multi-channel irradiance model to generate a visual compensation map, corrects the brightness and contrast of the target perspective image, and obtains the compensated perspective image; A directional light field regulation module that encodes the light propagation angle of the compensated perspective image through a predefined directional response basis function to generate directional light field information containing light intensity regulation coefficients; A full-layer fusion module that non-linearly superimposes all directional light field information based on a dynamic fusion weight tensor, and eliminates the interface artifacts between layers through depth correction after fusion to generate a naked-eye 3D transparent display image.

2. The naked-eye 3D transparent display imaging system according to claim 1, wherein The multi-scale disparity layer construction module includes: Decompose the input image sequence into levels with different resolutions through a multi-scale Gaussian pyramid, and perform spatio-temporal structure tensor analysis on each level, extract the principal components representing depth consistency, and generate a preliminary disparity response; Based on the principal component responses of each level, generate a structure-guided tensor template corresponding to the depth region through depth clustering and spatial correlation matrix feature mapping; Dynamically allocate pixels to each disparity layer through a normalized probability model, combine the preliminary disparity response with the structure-guided tensor template to generate a gating probability, and weight the preliminary disparity response through the gating probability to generate the final disparity layer.

3. A naked-eye 3D transparent display imaging system according to claim 1, characterized in that The dynamic viewpoint prediction module includes: Extract the dynamic deformation parameters of the iris region through infrared image segmentation and deformation registration; Perform coordinate transformation and motion noise filtering on the pupil position data to generate smooth eye movement tracking data; Construct a joint feature vector based on the iris deformation parameters and eye movement tracking data; Map the joint feature vector to binocular three-dimensional space coordinates through a linear discriminant model.

4. A naked-eye 3D transparent display imaging system according to claim 2 or 3, characterized in that, The disparity scheduling mapping module includes: Generate horizontal and vertical disparity vectors according to the binocular three-dimensional space coordinates and the depth scaling factor of the disparity layer, and optimize the disparity offset through a depth residual network; Perform pixel displacement mapping on the disparity layer based on a deformable convolutional kernel, and dynamically mask the invalid sampling region in combination with a gating mask mechanism; Generate the target perspective image by fusing the mapping results of different scales through optical flow alignment and depth confidence weighting.

5. The naked-eye 3D transparent display imaging system according to claim 4, wherein The calculation of the depth scaling factor includes: Based on the calibrated physical depth value and spatial gradient distribution of the disparity layer, combine the dynamic disparity correction term and the amplitude of the binocular disparity vector, and dynamically adjust the scaling intensity of each disparity layer through linear ratio operation.

6. The naked-eye 3D transparent display imaging system according to claim 4, characterized in that, The background compensation module includes: Generate a transmittance map through a physical attenuation model based on the real-time ambient light irradiance distribution and the display panel transmittance; Decompose the target perspective image into diffuse and specular reflection components, and generate a synthetic irradiance distribution by combining the transmittance map and background irradiance superposition; Analyze the brightness contrast difference between the target irradiance and the synthetic irradiance, and generate an adaptive compensation weight map through edge-preserving filtering; Perform non-linear brightness adjustment on the target perspective image according to the compensation weight map to obtain a compensated perspective image.

7. The naked-eye 3D transparent display imaging system according to claim 6, wherein, The display panel transmittance calculation includes: Generate the basic attenuation component of the transmittance based on the driving voltage and the wavelength-dependent electro-optic modulation coefficient. The driving voltage is dynamically adjusted according to the ambient light intensity to control the non-linear attenuation intensity of the transmittance; Based on the intrinsic absorption spectral width parameter and thermal vibration frequency of the display panel material, correct the dispersion deviation of the electro-optic modulation term through the error function to generate a wavelength-dependent absorption compensation component; Perform pixel-by-wavelength product fusion on the basic attenuation component and the absorption compensation component, and output the display panel transmittance that dynamically adapts to the ambient light.

8. A naked-eye 3D transparent display imaging system according to claim 6, characterized in that, The directional light field control module includes: Construct an orthonormal directional basis function family based on the physical characteristics of the light field control unit, meeting the preset angular resolution and coverage; Decompose the compensated perspective image into a linear combination of basis functions by pixel to generate directional light field control coefficients; Dynamically optimize the directional light field control coefficients through sparse constraints and energy conservation conditions, and retain the high-frequency details in the target direction; Map the optimized directional light field control coefficients to a light field control coefficient matrix that matches the physical drive unit to generate directional light field information.

9. A naked-eye 3D transparent display imaging system according to claim 8, characterized in that The full-layer fusion module includes: Construct a pixel-level feature vector based on the depth confidence, light field intensity, and edge gradient of the directional light field information, and generate a dynamic fusion weight through a lightweight convolutional network; Perform pixel-by-pixel weighted fusion on the directional light field information and retain the depth-level features; Detect the depth jump region of adjacent layers, and smooth the interface artifacts through edge-aware joint bilateral filtering to generate a naked-eye 3D transparent display image.

Images