Three-dimensional light field screen

Abstract

The invention provides a three-dimensional light field screen which comprises the following steps: a processing module acquires and processes original display data to generate first projection data and second projection data; the first projection module generates a first projection beam according to the first projection data; the second projection module generates a second projection beam according to the second projection data; the movable end of the displacement module is connected with the second projection module to drive the second projection module to move; the first half-transparent and half-reflecting mirror reflects the first projection beam to obtain first reflection projection light and transmits the second projection beam to obtain second transmission projection light; the second semi-transparent and semi-reflective mirror reflects the first reflection projection light and the second transmission projection light; the concave mirror reflects the first reflection projection light and the second transmission projection light to obtain first imaging light and second imaging light; the first imaging light and the second imaging light are transmitted by the second semi-transparent and semi-reflective mirror to realize imaging. According to the invention, switching of multiple display modes is realized, and diversified visual demands in a complex scene are met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional display, in particular to a three-dimensional light field screen. BACKGROUND

[0002] The current vehicle-mounted light field display screen technology is gradually applied to the intelligent cockpit field due to its advantages of breaking through the physical space limit, realizing long-distance imaging and super large frame. Compared with the traditional liquid crystal screen, it can provide a more immersive experience and effectively reduce visual fatigue and car sickness. The existing technology can adapt to different user heights, sitting postures and use scenarios by adjusting the spatial position (such as stretching, flipping) of the light field screen. However, these solutions still focus on optimizing the display effect in a single dimension (mainly two-dimensional plane) in essence, and have significant limitations: they cannot flexibly switch between multiple display modes in the same system. Specifically, the existing technology lacks support for naked-eye three-dimensional display mode with large depth of field and immersion, and cannot provide integration and switching capabilities between two-dimensional and three-dimensional display modes. This limits its potential to provide the most suitable visual experience for users in complex driving scenarios. Therefore, there is room for improvement. SUMMARY

[0003] The present application provides a three-dimensional light field screen to solve the technical problem that the existing technology cannot support large depth of field and immersive naked-eye three-dimensional display, and lacks integration and switching capabilities between two-dimensional and three-dimensional modes, limiting the potential for visual experience adaptation in complex driving scenarios.

[0004] The three-dimensional light field screen of the present application comprises:

[0005] A processing module for obtaining original display data and processing the original display data to generate first projection data and second projection data;

[0006] A first projection module for obtaining and generating first projection light beams according to the first projection data;

[0007] A second projection module for obtaining and generating second projection light beams according to the second projection data; the propagation direction of the first projection light beams is perpendicular to the propagation direction of the second projection light beams;

[0008] A displacement module with a movable end connected to the second projection module; the processing module is further configured to control the displacement module to drive the second projection module to move;

[0009] A first semi-transparent semi-reflective mirror located at the intersection of the propagation direction of the first projection light beams and the propagation direction of the second projection light beams, for reflecting the first projection light beams to obtain first reflected projection light rays, and for transmitting the second projection light beams to obtain second transmitted projection light rays; the first reflected projection light rays are parallel to the second transmitted projection light rays;

[0010] a second half-transmission half-reflection mirror, parallel to the first half-transmission half-reflection mirror, for reflecting the first reflected projection light and the second transmitted projection light;

[0011] a concave mirror, with its reflecting surface facing the second half-transmission half-reflection mirror and its optical axis being perpendicular to the first reflected projection light, for receiving and reflecting the first reflected projection light and the second transmitted projection light reflected by the second half-transmission half-reflection mirror, to obtain first imaging light and second imaging light; the first imaging light and the second imaging light are transmitted through the second half-transmission half-reflection mirror to realize imaging.

[0012] In one embodiment of the present application, the first half-transmission half-reflection mirror is a polarization multiplexing type half-transmission half-reflection mirror, the first projection module is located on the S polarization light side of the first half-transmission half-reflection mirror, and the second projection module is located on the P polarization light side of the first half-transmission half-reflection mirror.

[0013] In one embodiment of the present application, the top of the concave mirror is in contact with the top of the second half-transmission half-reflection mirror.

[0014] In one embodiment of the present application, when the processing module controls the displacement module to drive the second projection module to move to a preset first position, the second projection module is symmetrical to the first projection module with respect to the first half-transmission half-reflection mirror.

[0015] When the processing module controls the displacement module to drive the second projection module to move to a preset second position, the equivalent object distance of the second projection module with respect to the concave mirror is greater than the equivalent object distance of the first projection module with respect to the concave mirror.

[0016] In one embodiment of the present application, the original display data is a stereoscopic image picture; the processing module is further configured to control the second projection module to move to the first position and perform binocular view separation processing on the original display data to generate corresponding left eye view data and right eye view data.

[0017] The processing module is further configured to generate the first projection data and the second projection data according to the left eye view data and the right eye view data, respectively.

[0018] The first projection module and the second projection module respectively receive the first projection data and the second projection data and synchronously image to perform fusion three-dimensional display.

[0019] In one embodiment of the present application, the processing module is further configured to control the second projection module to move to the second position and control the first projection module and the second projection module to alternately image according to a preset switching time interval to perform non-fatigue two-dimensional display.

[0020] In one embodiment of the present application, the processing module is further configured to control the second projection module to move to the second position, and perform scene parsing on the original display data to generate corresponding foreground image data and background image data.

[0021] The processing module is further configured to generate first projection data according to the foreground image data, and generate second projection data according to the background image data.

[0022] The first projection module and the second projection module respectively receive the first projection data and the second projection data and synchronously image to perform double-plane layered stereoscopic display.

[0023] In one embodiment of the present application, the processing module is further configured to, for each image frame in the original display data:

[0024] generate a seed point set of a foreground image of the current image frame;

[0025] perform neighborhood pixel iterative growth processing on each seed point in the seed point set according to a preset similarity measure function and region growing condition to generate a corresponding foreground image mask;

[0026] extract corresponding foreground image data from the current image frame according to the foreground image mask, and obtain background image data according to the extracted current image frame.

[0027] In one embodiment of the present application, the original display data includes multiple observation angle target object views; the processing module is further configured to control the second projection module to move to the second position, and perform light field layering processing on the original display data to generate corresponding first layering data and second layering data.

[0028] The processing module is further configured to generate first projection data according to the first layering data, and generate second projection data according to the second layering data.

[0029] The first projection module and the second projection module respectively receive the first projection data and the second projection data and synchronously image to perform naked-eye three-dimensional display.

[0030] In one embodiment of the present application, the processing module is further configured to, based on a ray reverse tracing algorithm, perform processing on each target object view in the original display data to generate a target ray set expected to reach an imaging plane at each observation angle.

[0031] Based on the initial pixel value of the first projection data and the second projection data, the first projection light beam and the second projection light beam are superimposed at each observation angle to form a reconstructed light set on the imaging plane;

[0032] Based on a hybrid loss function combined with mean square error and perceptual image similarity, a brightness loss value between each target light set and the corresponding reconstructed light set is calculated;

[0033] The brightness loss value is input into an Adam optimizer, and the pixel value is iteratively updated through gradient back propagation, and the brightness loss value is recalculated after each update until the brightness loss value converges to a preset threshold, the optimized pixel value corresponding to the first projection module is taken as first layered data, and the optimized pixel value corresponding to the second projection module is taken as second layered data.

[0034] The three-dimensional light field screen provided by the application can realize flexible switching of multiple display modes on a single hardware platform by controlling the movement of the second projection module, combining the parallelization processing of the reflection / transmission of the vertically propagating double light beams by the polarization multiplexing type half-transmission half-reflection mirror, and the cooperative reflection imaging of the half-transmission half-reflection mirror and the concave mirror. The application solves the problems of lack of three-dimensional immersive experience and single mode of traditional vehicle-mounted displays, and seamlessly switches between two-dimensional display, three-dimensional display, double-plane stacked stereoscopic display, and naked-eye three-dimensional display in a compact structure. At the same time, the application uses a folded light path to integrate the double light beams into parallel light and reflect them by a concave mirror to generate virtual images with depth differences in a limited space, which not only retains the advantages of long distance and large frame of the vehicle-mounted light field screen, but also provides a stereoscopic visual experience with large depth of field and low dizziness. In addition, the vertical light path design and displacement control of the double projection modules ensure stable optical path and accurate adjustable virtual image position during multi-mode switching, meeting the diversified visual needs in complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0035] The drawings incorporated into the specification and forming part of the specification, show embodiments consistent with the present application, and together with the specification, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the application, and those skilled in the art can obtain other drawings from these drawings without creative labor.

[0036] In the drawings:

[0037] Figure 1 The structure of the three-dimensional light field screen provided by an embodiment of the application is shown in the structure schematic view of the three-dimensional light field screen.

[0038] Figure 2A light path schematic diagram of a three-dimensional light field screen in an embodiment of the present application is shown.

[0039] Figure 3 Another light path schematic diagram of a three-dimensional light field screen in an embodiment of the present application is shown.

[0040] Reference signs are as follows:

[0041] 100, first projection module 101, first projection light beam; 200, second projection module; 201, second projection light beam; 300, displacement module; 400, first half-transmission half-reflection mirror; 401, first reflected projection light; 402, second transmitted projection light; 500, second half-transmission half-reflection mirror; 600, concave mirror; 601, first imaging light; 602, second imaging light. DETAILED DESCRIPTION

[0042] The present application is described in detail below with specific reference being made to certain specific examples. It is to be understood that the examples are merely illustrative of the present application and that other embodiments of the present application can be used to advantage. The many variations and modifications of the present application will be apparent to those of ordinary skill in the art once the application has been described by way of illustration and example, as outlined above. The scope of the present application is limited only by the claims that follow.

[0043] It is to be understood that the drawings are to be used only for illustrating the specific embodiments of the present application and not for limiting the present application, and the actual implementation of the components in the drawings can be changed in number, shape and size, and the layout of the components can be more complicated.

[0044] In the following description, numerous specific details are discussed in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to one of ordinary skill in the art that the embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid obscuring the embodiments of the present application.

[0045] Please refer to Figure 1The three-dimensional light field screen of the present application can comprise a processing module (not shown in the figure), a first projection module 100, a second projection module 200, a displacement module 300 (not shown in the figure), a first half-transmission half-reflection mirror 400, a second half-transmission half-reflection mirror 500, and a concave mirror 600. The processing module can be electrically connected with the first projection module 100, the second projection module 200, and the displacement module, which can be used to obtain original display data in various formats, and process the original display data to generate first projection data and second projection data. The first projection module 100 can be used to obtain and generate first projection light beam 101 according to the first projection data. The second projection module 200 can be used to obtain and generate second projection light beam 201 according to the second projection data. In the embodiment, the propagation direction of the first projection light beam 101 is perpendicular to the propagation direction of the second projection light beam 201. The active end of the displacement module 300 can be connected to the second projection module 200, which can be used to drive the second projection module 200 to move according to the control of the processing module. The first half-transmission half-reflection mirror 400 can displace the intersection of the propagation direction of the first projection light beam 101 and the propagation direction of the second projection light beam 201, which can be used to reflect the first projection light beam 101 to obtain the first reflected projection light beam 401, and can also be used to transmit the second projection light beam 201 to obtain the second transmitted projection light beam 402. The second half-transmission half-reflection mirror 500 can be connected with the first half-transmission half-reflection mirror 400, which can be used to reflect the first reflected projection light beam 401 and the second transmitted projection light beam 402. The reflecting surface of the concave mirror 600 can face the second half-transmission half-reflection mirror 500, and the direction of its optical axis is perpendicular to the first reflected projection light beam 401, which can be used to receive and reflect the first reflected projection light beam 401 and the second transmitted projection light beam 402 reflected by the second half-transmission half-reflection mirror, to obtain the first imaging light beam 601 and the second imaging light beam 602. In the embodiment, the first imaging light beam 601 and the second imaging light beam 602 can realize imaging after being transmitted by the second half-transmission half-reflection mirror 400.

[0046] In an embodiment of the present application, the first projection module 100 can include a liquid crystal panel and a corresponding backlight module. The liquid crystal panel is the core imaging device of the projection, which can be a small-sized transmissive liquid crystal panel. In this embodiment, the light transmittance of each pixel point on the liquid crystal panel can be independently controlled (turned on, turned off or adjusted in gray scale). When light passes through it, the liquid crystal panel forms a "light valve" pattern corresponding to the image on the panel according to the input video signal. The backlight module can be used to provide uniform light covering the entire surface of the liquid crystal panel to provide sufficient light intensity to meet the projection brightness requirements and match the color filter of the liquid crystal panel (if it is a single panel LCD) or the light splitting and combining system (if it is a 3LCD) in spectral characteristics. As can be seen, the backlight module emits strong light, and the strong light is uniformly irradiated onto the liquid crystal panel. The liquid crystal panel accurately controls the light transmittance of each pixel point according to the first projection data input by the processing module, thereby forming an image in the light beam passing through the liquid crystal panel to generate a first projection light beam. The first projection light beam is S-polarized light (vertical polarization).

[0047] In this embodiment, the second projection module 200 has the same structure as the first projection module 100. The second projection light beam is P-polarized light (parallel polarization).

[0048] In an embodiment of the present application, the first half-transmission half-reflection mirror 400 is a polarization multiplexing type half-transmission half-reflection mirror (PBS, Polarizing Beam Splitter). The first projection module 100 can be located on the S-polarized light side of the first half-transmission half-reflection mirror 400, and the second projection module 200 can be located on the P-polarized light side of the first half-transmission half-reflection mirror 400. It can be understood that the PBS surface is coated with a special polarization beam splitting film, which has high reflection for S-polarized light and high transmission for P-polarized light.

[0049] In an embodiment of the present application, the second half-transmission half-reflection mirror 500 is a half-mirror (HW, Half-Mirror). It can be understood that the HW is a non-polarization beam splitter, which can partially reflect and partially transmit the incident light at a fixed ratio.

[0050] Please refer to Figure 2 , Figure 3In one embodiment of the present application, the first half-transmission half-reflection mirror 400 is erected above the horizontal reference plane with a 45° inclination angle. The second half-transmission half-reflection mirror 500 is arranged in parallel above the first half-transmission half-reflection mirror, so as to form a coplanar optical group. The first projection module 100 is perpendicular to the horizontal reference plane and arranged at the S-polarized light side of the first half-transmission half-reflection mirror 400. The second projection module 200 is parallel to the horizontal reference plane and arranged at the P-polarized light side of the first half-transmission half-reflection mirror 400. The optical axis of the concave mirror 600 can be parallel to the horizontal reference plane, and it can be arranged at the same side of the second half-transmission half-reflection mirror 500 as the first half-transmission half-reflection mirror 400. In this embodiment, the top of the concave mirror 600 forms a point contact with the top edge of the second half-transmission half-reflection mirror 500.

[0051] In this embodiment, the first projection light beam 101 is incident on the upper surface of the first half-transmission half-reflection mirror 400 in a direction parallel to the horizontal reference plane, and is totally reflected to turn upward to form the first reflected projection light 401. The first reflected projection light 401 propagates upward in a direction perpendicular to the horizontal reference plane, and is transmitted to the concave mirror 600 in a direction parallel to the horizontal reference plane after being reflected twice on the lower surface of the second half-transmission half-reflection mirror 500, and is reflected by the concave mirror 600 to form the first imaging light 601. The first imaging light 601 is transmitted through the second half-transmission half-reflection mirror 500 and forms an image at a preset observation position.

[0052] In this embodiment, the second projection light beam 201 propagates upward in a direction perpendicular to the horizontal reference plane, is transmitted through the lower surface of the first half-transmission half-reflection mirror 400 to form the second transmitted projection light 402. The second transmitted projection light 402 propagates upward in a direction perpendicular to the horizontal reference plane, and is transmitted to the concave mirror 600 in a direction parallel to the horizontal reference plane after being reflected twice on the lower surface of the second half-transmission half-reflection mirror 500, and is reflected by the concave mirror 600 to form the second imaging light 602. The second imaging light 602 is transmitted through the second half-transmission half-reflection mirror 500 and forms an image at a preset observation position in coaxial transmission with the first imaging light 601.

[0053] Please refer to Figure 2 In one embodiment of the present application, when the processing module controls the displacement module 300 to drive the second projection module 200 to move to a preset first position, the second projection module 200 is symmetrical to the first projection module 100 about the first half-transmission half-reflection mirror 400, and at this time, the equivalent object distance of the second projection module 200 relative to the equivalent of the concave mirror 600 is equal to the equivalent object distance of the first projection module 100 relative to the concave mirror 600.

[0054] Please refer to Figure 3In an embodiment of the present application, when the processing module controls the displacement module 300 to drive the second projection module 200 to move to the preset second position, the equivalent object distance of the second projection module 200 relative to the concave mirror 600 is greater than the equivalent object distance of the first projection module 100 relative to the concave mirror 600.

[0055] Based on the concave mirror imaging geometry, when the equivalent object distances u1, u2 of the first projection module 100 and the second projection module 200 satisfy 0 < u1 < u2, the virtual image of the first projection module 100 is located in the space behind the concave mirror 600, and the virtual image of the second projection module 200 is located in the space in front of the concave mirror 600. i (f (f is the mirror focal length), a positive upright magnified virtual image can be generated in the space behind the mirror. According to the Gaussian imaging formula It can be obtained that the virtual image position v of the i-th layer projection picture is i The corresponding magnification is Wherein 1 ≤ i ≤ 2, by dynamically adjusting the difference of the equivalent object distances of the first projection module 100 and the second projection module 200, the smooth switching of various imaging modes such as virtual image superposition and layered display can be realized.

[0056] Please refer to Figure 2 In an embodiment of the present application, when the three-dimensional light field screen of the present application needs to provide a three-dimensional display experience that requires wearing auxiliary glasses, it enters the visual aid three-dimensional display mode. Specifically, the processing module first receives the original display data. In this mode, the original display data specifically refers to an image or a video picture containing stereoscopic information, that is, each frame actually contains two views with parallax corresponding to the same moment, but respectively for the left eye and the right eye of the viewer to watch.

[0057] After the processing module identifies that the visual aid three-dimensional display mode needs to be entered, it issues a control instruction to the displacement module 300. The instruction requires the displacement module 300 to drive the second projection module 200 to move to the first position. When the second projection module 200 moves to the first position, its spatial position forms a mirror symmetry relationship with the first projection module 100 about the plane of the first half-transmission half-reflection mirror 400. At the same time, the processing module starts to perform binocular view separation processing on the received original display data. Specifically, the processing module analyzes the input original display data to obtain left eye view data and right eye view data, that is, effectively disassembles the composite stereoscopic picture into two independent view data streams respectively corresponding to the left eye and the right eye.

[0058] Then, the processing module generates first projection data for driving the first projection module 100 and second projection data for driving the second projection module 200 based on the separated left eye view data and right eye view data. This step needs to ensure that the two projection data are strictly synchronized in time, that is, each frame of left eye view data and the right eye view data at the same time form a complete stereoscopic frame pair.

[0059] The generated first projection data and second projection data are sent to the first projection module and the second projection module respectively. The first projection module receives the first projection data (i.e. left-eye view data) and generates first projection light beams according to the data. The second projection module receives the second projection data (i.e. right-eye view data) and generates second projection light beams according to the data. The two projection modules work synchronously and display the respective view contents simultaneously, so that the first projection light beams and the second projection light beams are generated synchronously. Since the second projection module is located at the first position at this time, its equivalent object distance relative to the concave mirror is equal to the equivalent object distance of the first projection module relative to the concave mirror when the first projection module is located at the fixed position. According to the Gaussian imaging formula, the virtual image distance formed by the light path of the first projection light beams is necessarily equal to the virtual image distance formed by the light path of the second projection light beams. Therefore, the first imaging light rays and the second imaging light rays accurately form two virtual images that are erect, magnified and completely spatially coincident. That is, the left-eye view content and the right-eye view content are imaged at the same position in space and are superimposed together.

[0060] In order for the viewer to finally perceive the three-dimensional effect, the user needs to wear a special polarized 3D glasses. The left-eye lens of the glasses is designed to only transmit S-polarized light (or light corresponding to the polarization state after reflection of the first projection light beams), and the right-eye lens is designed to only transmit P-polarized light (or light corresponding to the polarization state after transmission of the second projection light beams). In this way, the user's left eye can only see the virtual image formed by the light rays from the first projection module (displaying the left-eye view), and the right eye can only see the virtual image formed by the light rays from the second projection module (displaying the right-eye view). Since the two views have parallax and are transmitted to the left and right eyes respectively, the user's brain fuses them, thereby visually producing a three-dimensional image with depth and stereoscopic effect, thereby assisting three-dimensional display.

[0061] Please refer to Figure 3 In an embodiment of the present application, when the three-dimensional light field screen of the present application needs to provide a two-dimensional display that can alleviate visual fatigue, i.e. enters a non-fatigue two-dimensional display mode. Specifically, in this mode, the original display data is a conventional two-dimensional plane video or image content, and does not need to contain stereoscopic parallax information. The data contains a sequence of continuous images that need to be displayed to the viewer.

[0062] First, the processing module sends an accurate control instruction to the displacement module 300 after determining that the non-fatigue two-dimensional display mode is entered. The instruction requires the displacement module 300 to drive the second projection module 200 to move to the second position. When the second projection module 200 is located at the second position, its equivalent object distance relative to the concave mirror 600 is greater than the equivalent object distance of the first projection module 100 relative to the concave mirror.

[0063] Meanwhile, the processing module processes the received raw display data, and generates two copies of display data with the same content from the same frame of raw display data: the first projection data and the second projection data. That is, the content of the first projection data is the same as the content of the second projection data at each moment, and both correspond to the current frame content of the raw display data.

[0064] The generated first projection data and second projection data are respectively sent to the first projection module 100 and the second projection module 200. The processing module controls the two projection modules to display alternately at a preset time interval t. The specific control logic is as follows: in the first time period t, the first projection module 100 is driven to generate a first projection light beam according to the first projection data and display the corresponding picture, while the second projection module 200 is controlled to remain closed or display a black picture (i.e. no effective light output is generated); in the next time period t, the second projection module 200 is driven to generate a second projection light beam according to the second projection data and display the corresponding picture, while the first projection module 100 is controlled to remain closed or display a black picture. This cycle is repeated, and the first projection module and the second projection module alternately display the same frame content with a switching period of 2t.

[0065] Since the equivalent object distance of the first projection module 100 is u1, and the equivalent object distance of the second projection module 200 is u2, u2 is greater than u1. According to the Gaussian imaging principle, the virtual image distance calculated is necessarily different due to the different object distances. Therefore, in space, the first projection module 100 will form a positive and enlarged virtual image at a depth v1 when it is working; the second projection module 200 will form another positive and enlarged virtual image at a depth v2 when it is working. The two virtual images display the same picture content, but they are located at different depth positions in front of the viewer.

[0066] Through accurate control, it is ensured that the first projection module 100 and the second projection module 200 strictly display alternately at a time interval t. Therefore, the user will observe that the display picture alternately switches and refreshes between the two different positions of depth v1 and depth v2. This mechanism of actively and periodically changing the display depth of the picture forces the viewer's eyes to constantly adjust the focal length slightly, avoiding focusing on the same fixed depth plane for a long time. This dynamic change effectively relieves the visual accommodation fatigue caused by gazing at a single distance screen for a long time, thereby providing a two-dimensional viewing experience without visual fatigue.

[0067] Please refer to Figure 3 In an embodiment of the present application, when the three-dimensional light field screen of the present application needs to provide a stereoscopic display effect with depth layering and without the need to wear auxiliary glasses, it enters a double-plane stacked stereoscopic display mode. Specifically, in this mode, the raw display data is a conventional two-dimensional video or image sequence, and its content contains potential foreground objects and background information.

[0068] After determining to enter the dual-plane layered stereoscopic display mode, the processing module sends a control instruction to the displacement module 300. The instruction requires the displacement module 300 to drive the second projection module 200 to move to the second position.

[0069] Meanwhile, the processing module performs scene parsing processing on each frame of image in the original display data to intelligently separate the foreground part and the background part in the image.

[0070] In this embodiment, the scene parsing processing can include the following steps:

[0071] Step S11, a seed point set of the foreground image of the current image frame is generated.

[0072] In an embodiment of the present application, when step S11 is performed, specifically, for the image frame currently being processed, an initial seed point set of the foreground image is generated. The set contains the coordinates of one or more pixel points. The selection strategy of the initial seed points is determined according to the specific scene and target setting, and is usually based on prior knowledge (such as preset coordinates of the region of interest), automatic detection (such as salient object detection) or user interactive input, aiming to mark the starting pixel position belonging to the foreground object.

[0073] Step S12, according to a preset similarity measurement function and region growing condition, each seed point in the seed point set is subjected to neighborhood pixel iterative growing processing to generate a corresponding foreground image mask.

[0074] In an embodiment of the present application, when step S12 is performed, specifically, the similarity measurement function and the region growing condition are first defined. The similarity measurement function is used to calculate the difference degree of each seed point pixel in the initial seed point set and its surrounding 8 neighborhood pixels in color or grayscale features. Specifically, the function is usually based on the RGB three-channel color value of the pixel. For a seed point pixel P seed and a neighborhood pixel P neighbori , the similarity measurement function S(P seed , P neighbori ) can be calculated as the Euclidean distance, Manhattan distance or cosine similarity between the RGB vectors of the two. The function quantifies the proximity of the seed point and its neighbor pixels in visual features.

[0075] The region growing condition is a standard for judging whether a neighborhood pixel should be classified into the same category (i.e. foreground) as the seed point. The most core growing condition is usually expressed as: if the similarity measurement value S(P neighbori of the neighborhood pixel P seed and the seed point pixel P seed , P neighbori) less than or equal to a preset growth threshold T, the neighborhood pixel is considered similar enough to the seed point and belongs to the same region (foreground). The value of the growth threshold T can be adjusted according to the image content complexity, noise level and the desired foreground extraction precision. A smaller T value will result in more strict growth, only merging very similar pixels; a larger T value allows merging pixels with slightly larger differences, and the region grows larger.

[0076] The processing module initiates the iterative growth process of the neighborhood pixels based on the defined seed point set, the similarity measure function and the region growth condition. The specific process is as follows: first, traverse each seed point in the current seed point set, and check all 8 neighborhood pixels of the seed point; then, for each neighborhood pixel, calculate its similarity measure value with the current seed point, and determine whether it meets the region growth condition (i.e. S≤T); next, if a neighborhood pixel meets the condition, mark it as belonging to the foreground category, and add its coordinates to the seed point set to become a new seed point for subsequent growth; finally, remove the original seed point that has completed the current processing. This process is continuously iterated: using the updated seed point set (containing newly added pixels) to repeat the above steps of traversal, neighborhood checking, similarity calculation, condition judgment and new seed point addition. The iteration continues until there are no new pixels that can be added to the seed point set, meaning that under the current growth condition, no more adjacent pixels similar enough to the grown region can be found. At this time, the growth for the current seed point set (or initial set) is completed, generating a connected foreground region, the boundary of which is defined by pixels that no longer meet the growth condition. All pixels corresponding to this connected region constitute a binary foreground image mask, in which pixels belonging to the foreground are marked as 1 and background pixels are marked as 0.

[0077] After the extraction of the first foreground region (i.e. generating the foreground image mask) is completed, the processing module checks whether there are still pixel regions in the current image frame that have not been marked as foreground and may belong to other independent foreground objects. If so, new seed points are set in the image for these unextracted parts (i.e. remaining background regions). The selection of new seed points is also based on a preset strategy or detection. After setting new seed points, the processing module repeats the above process. Through repeated hierarchical iterations, all significant foreground objects in the image are identified and extracted, or the preset extraction layer limit is reached. Finally, all extracted foreground regions are combined to form a complete foreground image mask.

[0078] Step S13, according to the foreground image mask, extract the corresponding foreground image data from the current image frame, and obtain the background image data according to the extracted current image frame.

[0079] In one embodiment of the present application, when step S13 is performed, specifically, the processing module extracts the corresponding foreground image data from the original current image frame according to the finally generated foreground image mask. The specific operation is: the foreground image mask is applied to the original image frame, the positions marked as foreground in the mask are kept as the original pixel values, and the rest positions (i.e. the background) are set as transparent or a specific background color (such as black), so as to obtain the image data containing only the foreground objects. At the same time, the background image data is obtained according to the extracted current image frame. After the scene analysis processing is completed, the processing module generates the first projection data for driving the first projection module according to the generated foreground image data, and generates the second projection data for driving the second projection module according to the generated background image data. That is, the first projection data corresponds to the foreground content, and the second projection data corresponds to the background content.

[0080] The generated first projection data and second projection data are respectively sent to the first projection module and the second projection module. The processing module controls the two projection modules to synchronously display the respective data. The first projection module receives the first projection data (foreground data) and generates the first projection light beam according to the data. The second projection module receives the second projection data (background data) and generates the second projection light beam according to the data. The two projection modules are activated at the same time and simultaneously display the respective assigned picture layers.

[0081] Since the equivalent object distance of the first projection module 100 is u1, and the equivalent object distance of the second projection module 200 is u2. According to the Gaussian imaging principle, since the object distances are different, the calculated virtual image distances must be different. Therefore, in space, the virtual image formed by the foreground content displayed by the first projection module 100 is located at a relatively near depth v1, and the virtual image formed by the background content displayed by the second projection module 200 is located at a relatively far depth v2. The two erect and enlarged virtual images are stacked in front and back in space.

[0082] When a viewer observes with naked eyes, since there is a clear depth difference between the foreground virtual image (depth v1) and the background virtual image (depth v2), and the foreground objects usually partially block the background, the natural parallax blocking relationship is clearly presented. The human visual system interprets this depth level and blocking relationship as stereoscopic effect and spatial effect, thereby significantly enhancing the stereoscopic effect of the played picture and realizing simple stereoscopic display without wearing special glasses.

[0083] Please refer to Figure 3 In one embodiment of the present application, when the three-dimensional light field screen of the present application needs to provide a large depth of field, an immersive naked-eye three-dimensional viewing experience without wearing auxiliary equipment, i.e. enters the naked-eye three-dimensional display mode. Specifically, in this mode, the original display data contains views of target objects collected from multiple observation angles.

[0084] After determining to enter the naked-eye three-dimensional display mode, the processing module sends a control instruction to the displacement module. The instruction requires the displacement module to drive the second projection module 200 to move to a preset second position. At this time, the equivalent object distance of the first projection module 100 relative to the concave mirror 600 is v1, and the equivalent object distance of the second projection module 200 relative to the concave mirror 600 is v2. The first projection module 100, the second projection module 200, and the second projection module 200 together constitute a superimposed compression light field structure.

[0085] Meanwhile, the processing module performs light field layering processing on the original display data to generate corresponding first layering data and second layering data.

[0086] In this embodiment, the light field layering processing includes the following steps:

[0087] Step S21, based on the initial pixel values of the first projection data and the second projection data, the first projection light beam and the second projection light beam are calculated under each observation angle. The reconstructed light set formed by the superposition of the imaging plane (referring to the system composed of the first projection module and the second projection module. Under the same specific observation angle, the actual predicted light information reaching the imaging plane).

[0088] In one embodiment of the present application, when step S21 is performed, specifically,

[0089] First, the spatial coordinates of the viewer position are preset. For each observation angle of the target object view in the original display data, the spatial coordinates of the object points of the three-dimensional object are selected. According to the object point position and the preset viewer position, the ideal light path (i.e. target light) from the object point to the viewer's pupil is calculated. The intersection coordinates of the light path after passing through the virtual imaging plane and the physical plane where the first projection module 100 and the second projection module 200 are located are determined by backtracking the light path. The set of light paths generated by all object points under each observation angle constitutes the target light set of the angle, which fully characterizes the light direction, intensity and polarization information that should reach the imaging plane under ideal conditions. In this embodiment, the light intensity of each target light of the target light set can satisfy the following formula:

[0090]

[0091] Wherein, represents the light intensity value of the light ray at the kth layer LCD screen coordinate pixel point, β represents the transmittance of the LCD, β is a constant, β k-1 represents the (k-1) power of β, that is, the transmittance of the light ray emitted by the pixel point on the kth layer LCD after passing through the k-1 layer LCD, and v is the number of viewpoints. In this embodiment, the value of k is 2.

[0092] The processing module initializes the pixel values of the first projection module 100 and the second projection module 200. Based on the current pixel values, the actual light path propagation process is simulated: the first projection light beam 101 is reflected by the first half-transmission half-reflection mirror 400, the second projection light beam 201 is transmitted by the first half-transmission half-reflection mirror 400, and the two light beams are transmitted to the surface of the second half-transmission half-reflection mirror 500 after the propagation direction is changed to be parallel. The light beam reflected by the second half-transmission half-reflection mirror 500 reaches the concave mirror 600. According to the curved surface reflection characteristics of the concave mirror 600 and the Gaussian imaging law, the distribution of the superimposed light field formed by the light beam after amplification in the virtual imaging plane is calculated. Through this geometric optical model, the direction and intensity of the actual light ray reaching the imaging plane at each observation angle are predicted, and the corresponding reconstructed light ray set is generated.

[0093] In step S22, a brightness loss value between each target light ray set and the corresponding reconstructed light ray set is calculated based on a hybrid loss function combined by a mean square error and a perceptual image similarity.

[0094] In an embodiment of the present application, when step S22 is performed, specifically, the processing module uses a hybrid loss function to quantify the difference between the target light ray set and the reconstructed light ray set at each observation angle. The hybrid loss function consists of two parts: a mean square error loss (MSE) and a perceptual image similarity loss (LPIPS). The mean square error loss is used to calculate the sum of the squares of the differences between all corresponding light intensities in the two sets, which can reflect the pixel-level brightness difference. The perceptual image similarity loss extracts the feature maps generated by the two sets through a pre-trained deep neural network, calculates the distance in the feature space, and quantifies the visual perception difference of the human eye. In this embodiment, the mean square error loss (MSE) can satisfy the following formula:

[0095]

[0096] The perceptual image similarity loss (LPIPS) can satisfy the following formula:

[0097]

[0098] In step S23, the brightness loss value is input to the Adam optimizer, and the pixel values are updated iteratively through gradient backpropagation. After each update, the brightness loss value is recalculated until the brightness loss value converges to a preset threshold. The optimized pixel values corresponding to the first projection module are taken as the first layered data, and the optimized pixel values corresponding to the second projection module are taken as the second layered data.

[0099] In an embodiment of the present application, when step S23 is performed, specifically, the processing module inputs the mixed loss value calculated in step S22 into the Adam optimizer. The optimizer calculates the gradient of the mixed loss function with respect to each pixel value of the first projection module and the second projection module, and iteratively updates the pixel values through the gradient back propagation algorithm. After each update, the reconstruction ray set simulation and the mixed loss calculation are re-executed to form a closed-loop optimization process. Steps S21 to S23 are repeated until the mixed loss value converges to a preset threshold. At this time, the optimized first projection module pixel value is output as the first layered data, and the optimized second projection module pixel value is output as the second layered data.

[0100] The processing module generates first projection data according to the first layered data and generates second projection data according to the second layered data. The first projection data is sent to the first projection module to drive it to generate light beams carrying the first layered information, and the second projection data is sent to the second projection module to drive it to generate light beams carrying the second layered information. The two projection modules are strictly synchronized to activate the display.

[0101] The first projection light beam 101 is reflected by the first half-mirror 400, and the second projection light beam 201 is transmitted by the first half-mirror 400. The two light beams pass through the second half-mirror 500 to the concave mirror 600. The concave mirror 600 enlarges the double light beams to form two layers of virtual image arrays separated in position in space, where the virtual image distance of the first projection module 100 is v1, and the virtual image distance of the second projection module 200 is v2. Since the light field layered data is generated by the human eye perception optimization algorithm, the viewer does not need to wear any auxiliary equipment to observe the three-dimensional scene with continuous depth information and no resolution loss at a preset viewing position, realizing a large depth of field immersive experience.

[0102] The above embodiments are only illustrative of the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea disclosed by the present application should be covered by the claims of the present application.

Claims

1. A three-dimensional light field screen, characterized in that: include: a processing module, configured to acquire original display data and process the original display data to generate first projection data and second projection data; a first projection module, configured to acquire and generate a first projection beam according to the first projection data; a second projection module, configured to acquire and generate a second projection beam according to the second projection data; wherein a propagation direction of the first projection beam is perpendicular to a propagation direction of the second projection beam; a displacement module, a movable end of which is connected to the second projection module; the processing module is further configured to control the displacement module to drive the second projection module to move; a first semi-transparent and semi-reflective mirror, located at the intersection of the propagation direction of the first projection light beam and the propagation direction of the second projection light beam, for reflecting the first projection light beam to obtain a first reflected projection light ray, and for transmitting the second projection light beam to obtain a second transmitted projection light ray; the first reflected projection light ray is parallel to the second transmitted projection light ray; a second semi-transparent and semi-reflective mirror, parallel to the first semi-transparent and semi-reflective mirror, for reflecting the first reflected projection light and the second transmitted projection light; A concave mirror, whose reflective surface faces the second semi-transparent and semi-reflective mirror, and whose optical axis is perpendicular to the first reflected projection light, is used to receive and reflect the first reflected projection light and the second transmitted projection light reflected by the second semi-transparent and semi-reflective mirror to obtain a first imaging light and a second imaging light; the first imaging light and the second imaging light are imaged after being transmitted through the second semi-transparent and semi-reflective mirror.

2. The three-dimensional light field screen according to claim 1, characterized in that: The first semi-transparent mirror is a polarization multiplexing semi-transparent mirror, the first projection module is located on the S-polarized light side of the first semi-transparent mirror, and the second projection module is located on the P-polarized light side of the first semi-transparent mirror.

3. The three-dimensional light field screen according to claim 2, characterized in that: The top of the concave mirror contacts the top of the second semi-transparent and semi-reflective mirror.

4. The three-dimensional light field screen according to claim 3, characterized in that: When the processing module controls the displacement module to drive the second projection module to move to a preset first position, the second projection module and the first projection module are symmetrical with respect to the first semi-transparent and semi-reflective mirror; When the processing module controls the displacement module to drive the second projection module to move to a preset second position, an equivalent object distance of the second projection module relative to the concave mirror is greater than an equivalent object distance of the first projection module relative to the concave mirror.

5. The three-dimensional light field screen according to claim 4, characterized in that: The original display data is a stereoscopic image; the processing module is further used to control the second projection module to move to the first position, and perform binocular view separation processing on the original display data to generate corresponding left-eye view data and right-eye view data; The processing module is further configured to generate the first projection data and the second projection data according to the left-eye view data and the right-eye view data, respectively; The first projection module and the second projection module respectively receive the first projection data and the second projection data and synchronously generate images to perform fused three-dimensional display.

6. The three-dimensional light field screen according to claim 4, characterized in that: The processing module is further configured to control the second projection module to move to the second position, and control the first projection module and the second projection module to perform imaging in turn according to a preset switching time interval, so as to perform fatigue-free two-dimensional display.

7. The three-dimensional light field screen according to claim 4, characterized in that: The processing module is further configured to control the second projection module to move to the second position, and perform scene analysis processing on the original display data to generate corresponding foreground image data and background image data; The processing module is further configured to generate first projection data based on the foreground image data, and generate second projection data based on the background image data; The first projection module and the second projection module respectively receive the first projection data and the second projection data and synchronously form images to perform dual-plane stacked stereoscopic display.

8. The three-dimensional light field screen according to claim 7, characterized in that: The processing module is further configured to: Generate a set of seed points of the foreground image of the current image frame; According to a preset similarity measurement function and region growing conditions, performing neighborhood pixel iterative growth processing on each seed point in the seed point set to generate a corresponding foreground image mask; According to the foreground image mask, corresponding foreground image data is extracted from the current image frame, and background image data is obtained according to the extracted current image frame.

9. The three-dimensional light field screen according to claim 4, characterized in that: The original display data includes views of the target object from multiple observation angles; the processing module is further configured to control the second projection module to move to the second position and perform light field layering processing on the original display data to generate corresponding first layered data and second layered data; The processing module is further configured to generate first projection data based on the first layered data, and generate second projection data based on the second layered data; The first projection module and the second projection module respectively receive the first projection data and the second projection data and synchronously form images to perform naked-eye three-dimensional display.

10. The three-dimensional light field screen according to claim 9, characterized in that: The processing module is further configured to process each target object view in the original display data based on a ray back tracing algorithm to generate a target ray set expected to reach the imaging plane at each observation angle; Calculating, based on initial pixel values ​​of the first projection data and the second projection data, a reconstructed light ray set formed by superimposing the first projection beam and the second projection beam on an imaging plane at each observation angle; Calculating a brightness loss value between each target light set and a corresponding reconstructed light set based on a hybrid loss function combining mean square error and perceived image similarity; The brightness loss value is input into the Adam optimizer, and the pixel value is iteratively updated through gradient back propagation. The brightness loss value is recalculated after each update until the brightness loss value converges to a preset threshold. The optimized pixel value corresponding to the first projection module is used as the first layered data, and the optimized pixel value corresponding to the second projection module is used as the second layered data.

Images