Image processing method, device, equipment and system for naked-eye three-dimensional display

Abstract

The application provides an image processing method, device, equipment and system for naked-eye three-dimensional display. The method comprises the following steps: acquiring a reference interlaced picture of a pure color background scene under a first number of viewing angles and a first resolution, wherein the reference interlaced picture does not contain texture; rendering a shadow of a foreground object under the pure color background scene at a second number of viewing angles and a second resolution to obtain a plurality of viewing angle pictures corresponding to the second number of viewing angles and containing the shadow of the foreground object; wherein the second number of viewing angles is less than the first number of viewing angles, and the second resolution is lower than or equal to the first resolution; processing the plurality of viewing angle pictures containing the shadow of the foreground object to obtain a shadow interlaced picture having the same number of viewing angles and the same resolution as the reference interlaced picture; performing differential processing on the shadow interlaced picture and the reference interlaced picture to obtain a differential interlaced picture; and mixing the differential interlaced picture into a multi-view interlaced picture containing texture to obtain a target interlaced picture.

Description

Technical Field

[0001] This application relates to the field of naked-eye 3D display technology, and in particular to an image processing method, apparatus, device and system for naked-eye 3D display. Background Technology

[0002] Glasses-free 3D display technology is widely used in digital signage, advertising displays, medical imaging, and entertainment. In these applications, to achieve realistic stereoscopic fusion, it is usually necessary to render dynamic shadows for foreground objects in real time against a glasses-free 3D background.

[0003] In related technologies, dynamic shadow rendering typically employs a method of independently calculating a complete shadow map for each viewpoint. However, when the number of viewpoints is large and the rendering resolution is high, a significant amount of texture sampling and depth testing must be performed per frame. Even with GPU hardware acceleration, it is difficult to achieve the frame rate required for real-time interaction; this problem is even more pronounced on resource-constrained embedded or mobile devices. Therefore, the excessive computational load of shadow rendering in glasses-free 3D displays negatively impacts the real-time interactive experience of glasses-free 3D displays. Summary of the Invention

[0004] In view of the above, this application provides an image processing method, apparatus, device and system for naked-eye 3D display to solve at least one problem existing in the background art.

[0005] In a first aspect, an image processing method for glasses-free 3D display is provided, the method comprising: Obtain a reference interlacing map of a solid-color background scene at a first-viewpoint number and a first-resolution, wherein the reference interlacing map does not contain texture; In the solid color background scene, the shadow of the foreground object is rendered with a second viewpoint number and a second resolution to obtain multiple viewpoint images containing the shadow of the foreground object corresponding to the second viewpoint number; wherein, the second viewpoint number is less than the first viewpoint number, and the second resolution is lower than or equal to the first resolution. The images containing the shadows of foreground objects are processed to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map. The shadow interlacing map and the reference interlacing map are differentially processed to obtain a differential interlacing map; The differential interlacing map is blended into a multi-view interlacing map containing texture to obtain the target interlacing map.

[0006] In conjunction with the first aspect, in an optional implementation, processing the multiple viewpoint images containing the shadows of the foreground object to obtain a shadow interlacing map having the same number of viewpoints and the same resolution as the reference interlacing map includes: During the interleaving calculation, the multiple viewpoint images containing the shadows of foreground objects are interpolated to obtain a shadow interleaving map with the same number of viewpoints and the same resolution as the reference interleaving map.

[0007] In conjunction with the first aspect, in an optional implementation, the step of performing differential processing on the shadow interlacing map and the reference interlacing map to obtain a differential interlacing map includes: In the linear or logarithmic domain of luminance, the shadow interlacing image and the reference interlacing image are differentially processed pixel by pixel to obtain the differential interlacing image.

[0008] In conjunction with the first aspect, in an optional implementation, the step of blending the differential interlacing map to a multi-view interlacing map containing texture to obtain a target interlacing map includes: The ratio of the pixel value of the differential interlacing map to the pixel value of the reference interlacing map at the corresponding position is calculated for each pixel to obtain the normalization coefficient of each pixel; For each pixel in the multi-view interlacing map containing texture, the pixel value of the pixel is multiplied by an attenuation coefficient to obtain the pixel value of the corresponding pixel in the target interlacing map; wherein, the attenuation coefficient is equal to 1 minus the product of the shadow intensity factor and the normalization coefficient of the pixel, and the shadow intensity factor has a value range of 0 to 1.

[0009] In conjunction with the first aspect, in an optional implementation, rendering the shadows of foreground objects in the solid-color background scene using a second viewpoint and a second resolution to obtain multiple viewpoint images containing the shadows of the foreground objects corresponding to the second viewpoint includes: In the solid color background scene, for the multiple frames of foreground images containing foreground objects, the shadows of the foreground objects are rendered frame by frame with the second viewpoint number and the second resolution to obtain viewpoint image groups corresponding to the multiple frames of foreground images respectively. Each view image group contains multiple view images corresponding to the second view number, including the shadows of foreground objects.

[0010] In conjunction with the first aspect, in an optional implementation, the method is applied to a glasses-free 3D display system, the glasses-free 3D display system comprising a first 2D display panel and a 3D display panel stacked together; The first 2D display panel is used to display an image of the foreground object; The textured multi-view interlacing map is a textured background interlacing map. The 3D display panel is used to display a three-dimensional background based on the target interlacing map and to provide backlight illumination for the first 2D display panel.

[0011] In conjunction with the first aspect, in an alternative implementation, the method further includes: Obtain an interlacing map sequence of multiple out-of-screen objects with a texture complexity lower than that of the foreground object. Each interlacing map sequence corresponds to a preset motion trajectory of an out-of-screen object and includes a multi-frame interlacing map corresponding to multiple trajectory points in the preset motion trajectory. Based on the layout information of each frame of the foreground object in the first 2D display panel, determine the exclusion bounding box corresponding to each frame of the image. Select the target out-of-screen object whose motion trajectory does not intersect with the exclusion bounding box corresponding to each frame image from the plurality of out-of-screen objects; The interlacing maps of multiple frames in the interlacing map sequence of the target out-of-screen object are superimposed frame by frame onto the target interlacing map.

[0012] In conjunction with the first aspect, in one optional implementation, the spatial position corresponding to the out-of-screen object is located on one side of the out-of-screen direction of the zero parallax plane; the higher the texture complexity of the out-of-screen object, the closer its spatial position is to the zero parallax plane; the lower the texture complexity of the out-of-screen object, the farther its spatial position is from the zero parallax plane. Different spatial locations correspond to different parallax amplitude ranges. The parallax amplitude is the pixel offset of the same out-of-screen object in images from different viewing angles. The further the spatial location of the out-of-screen object is from the zero parallax plane, the greater the corresponding parallax amplitude.

[0013] In conjunction with the first aspect, in an optional implementation, determining the exclusion bounding box corresponding to each frame of the foreground object based on the layout information of each frame of the foreground object in the first 2D display panel includes: Obtain the set of pixel coordinates of each frame of the foreground object in the first 2D display panel; For each frame of the foreground object, the minimum bounding rectangle corresponding to the foreground object is calculated based on the set of pixel coordinates of the frame image, and the minimum bounding rectangle is expanded according to a preset dilation coefficient to obtain the repulsion bounding box corresponding to the frame image.

[0014] In conjunction with the first aspect, in an optional implementation, the interlacing map corresponding to each trajectory point of the out-of-screen object includes a paired generated object interlacing map and an object mask interlacing map; the step of superimposing the multi-frame interlacing maps from the interlacing map sequence of the target out-of-screen object onto the target interlacing map frame by frame includes: Each frame of the object interlacing map of the target out-of-screen object is fused onto the target interlacing map using the corresponding object mask interlacing map; Wherein, the opacity value in the inner region of the target object is a first preset value, and the opacity value in the edge region of the target object gradually decreases from the first preset value to a second preset value; the transparency corresponding to the first preset value is lower than the transparency corresponding to the second preset value.

[0015] Secondly, an image processing apparatus for glasses-free 3D display is provided, the apparatus comprising: The first acquisition module is used to acquire a reference interlacing map of a solid color background scene under a first viewpoint and a first resolution, wherein the reference interlacing map does not contain texture. The rendering module is used to render the shadow of the foreground object in the solid color background scene with a second viewpoint number and a second resolution to obtain multiple viewpoint images containing the shadow of the foreground object corresponding to the second viewpoint number; wherein the second viewpoint number is less than the first viewpoint number, and the second resolution is lower than or equal to the first resolution. An interlacing module is used to process the multiple viewpoint images containing the shadows of foreground objects to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map. The difference module is used to perform difference processing on the shadow interlacing map and the reference interlacing map to obtain a difference interlacing map; The first mixing module is used to mix the differential interlacing map into a multi-view interlacing map containing texture to obtain a target interlacing map.

[0016] Thirdly, a glasses-free 3D display system is provided, the glasses-free 3D display system comprising: The first 2D display panel is used to display an image of the foreground object; A 3D display panel is stacked on top of the first 2D display panel to present a three-dimensional background and provide backlight illumination for the first 2D display panel. The processing unit, connected to the first 2D display panel and the 3D display panel, is configured to perform the steps of the image processing method as described in the first aspect or any optional implementation thereof.

[0017] Fourthly, an electronic device is provided, including a processor, a memory, and an executable program stored in the memory and executable by the processor, wherein the processor, when executing the executable program, performs steps of an image processing method for naked-eye 3D display as provided in the first aspect or any alternative implementation thereof.

[0018] Fifthly, a storage medium is provided having an executable program stored thereon, which, when executed by a processor, implements the steps of the image processing method for naked-eye 3D display provided by the first aspect or any alternative implementation thereof.

[0019] The technical solution provided in this application renders dynamic shadows of foreground objects at a second viewpoint and a second resolution, where the second viewpoint is less than the first viewpoint and the second resolution is less than or equal to the first resolution. This reduces the computational load of dynamic shadow rendering, thus helping to meet the real-time interactive requirements of naked-eye 3D displays. Furthermore, since both the reference interlacing map and the multiple viewpoint images of the shadows are obtained from a solid-color background scene without texture, after processing the multiple viewpoint images of the shadows to obtain the shadow interlacing map, the differential interlacing map obtained by differential processing between the reference interlacing map and the shadow interlacing map is independent of the specific texture of the background scene. This allows the differential interlacing map to be independently superimposed on any background interlacing map containing texture, without needing to recalculate the shadows for different background scenes, thereby improving the versatility and practicality of the solution. Attached Figure Description

[0020] Figure 1 This illustration shows one of the schematic flowcharts of an image processing method for naked-eye 3D display provided in an embodiment of this application; Figure 2 It shows Figure 1 A flowchart illustrating step S105. Figure 3 This paper shows a schematic diagram of the architecture of the naked-eye 3D display system provided in an embodiment of this application; Figure 4 This illustration shows a schematic diagram of an interlacing diagram provided in an embodiment of this application; Figure 5 This is a second schematic flowchart of an image processing method for naked-eye 3D display provided in an embodiment of this application; Figure 6 This illustration shows a schematic diagram of the out-of-screen enhancement effect provided in an embodiment of this application; Figure 7 It shows Figure 5 A flowchart illustrating step S202; Figure 8 This is a third schematic flowchart of an image processing method for naked-eye 3D display provided in an embodiment of this application; Figure 9 This invention provides a schematic diagram of the structure of an image processing apparatus for naked-eye 3D display according to an embodiment of the present application. Figure 10This is a second schematic diagram of the structure of an image processing device for naked-eye 3D display provided in an embodiment of this application. Detailed Implementation

[0021] To make the technical solution and beneficial effects of this application more apparent and understandable, a detailed description is provided below by listing specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application pertains.

[0022] The embodiments in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0023] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0024] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0025] Glasses-free 3D display technology is widely used in digital signage, advertising displays, medical imaging, and entertainment. However, to present multi-view stereoscopic images, the effective pixels of the display panel need to be distributed across various viewpoints, resulting in a significant decrease in clarity at a single viewpoint and thus a serious resolution loss problem. A dual-layer display architecture can be used to solve this resolution loss problem. In a dual-layer display architecture, the rear display panel is responsible for presenting the glasses-free 3D stereoscopic background (such as simulating the interior space of a box) and serves as the light source for the front display panel; the high-resolution front panel is responsible for displaying the main objects (such as text, logos, high-definition product images, etc.). With this setup, the system can maintain the stereoscopic effect while ensuring the clarity of the foreground subject.

[0026] In the aforementioned dual-layer display architecture, to achieve realistic stereoscopic fusion, dynamic shadows need to be rendered for foreground objects in the rear naked-eye 3D background. If a complete shadow map is independently calculated for each viewpoint (e.g., 24 or 30 viewpoints), the computational complexity is directly proportional to the number of viewpoints and the rendering resolution. In real-time interactive scenarios (such as object movement and light source changes), texture sampling and shadow depth testing are required for all viewpoints every frame. This makes it difficult to achieve the frame rate required for real-time interaction even with GPU hardware acceleration; this problem is even more pronounced on resource-constrained embedded or mobile devices.

[0027] To facilitate understanding, a brief explanation of some of the terms used in this application will be provided first.

[0028] Dual-layer display system: refers to a display system consisting of a front high-resolution 2D display panel and a rear naked-eye 3D display panel stacked together. The rear display panel is used to display stereoscopic background images and also serves as a transmissive backlight source for the front display panel.

[0029] Naked-eye 3D display panel: refers to a display panel that can present stereoscopic vision without wearing auxiliary glasses. It is usually composed of a 2D display substrate and a lenticular grating or parallax barrier. Through multi-view interlacing technology, images from different perspectives are directed to the left and right eyes of the viewer.

[0030] Multi-view interlaced image: This refers to an encoded form that resamples and merges multiple view images from different perspectives according to a specific arrangement rule (such as a sub-pixel mapping table corresponding to the raster tilt angle) into a single image. After the image is split by a raster, the corresponding view images can be reconstructed in different spatial directions.

[0031] View image: A two-dimensional image rendered from a three-dimensional scene at a preset viewing position (viewpoint).

[0032] Masked / Obscured Area: This refers to the pixel area in the rear naked-eye 3D panel that spatially corresponds to the front 2D display content. This area requires special image processing (such as filling with specific brightness values ​​or low-frequency light components) to prevent the background image from penetrating into the foreground object area.

[0033] Asymmetric viewpoint allocation: This refers to a strategy of rendering different visual information using different numbers of viewpoints. High-information content such as the background is rendered using the full viewpoint (e.g., 30 viewpoints) to ensure a sense of depth, while low-frequency information such as shadows is rendered using only a few viewpoints (e.g., 10 viewpoints) to significantly reduce computational overhead.

[0034] Exclusion bounding box: This refers to the safe region formed by morphologically expanding the smallest bounding rectangle of all non-transparent pixels in the preceding 2D display content. This region serves as a no-entry zone for out-of-screen enhanced objects, ensuring that pre-calculated semi-transparent 3D materials do not obscure key foreground information during movement.

[0035] Figure 1 This illustration shows a flowchart of an image processing method for glasses-free 3D display provided in an embodiment of this application. This image processing method can be applied to various glasses-free 3D display systems employing multi-viewpoint interlaced display, such as a single-layer glasses-free 3D display architecture, or a dual-layer display architecture including a front-layer 2D display panel and a rear-layer glasses-free 3D display panel. In both the rear-layer glasses-free 3D display panel and the single-layer glasses-free 3D display panel, a multi-viewpoint interlaced display method is used, where the interlaced image is formed by interlacing multiple viewpoint images according to the pixel arrangement rules of the display panel.

[0036] like Figure 1 As shown, the image processing method for naked-eye 3D display provided in this application includes the following steps: S101: Obtain the baseline interlacing map of the solid color background scene at the first viewpoint and first resolution. The baseline interlacing map does not contain textures. S102: In a solid color background scene, render the shadow of the foreground object using a second viewpoint and a second resolution to obtain multiple viewpoint images containing the shadow of the foreground object corresponding to the second viewpoint; wherein, the second viewpoint is less than the first viewpoint, and the second resolution is lower than or equal to the first resolution. S103: Process multiple view images containing the shadows of foreground objects to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map. S104: Perform differential processing on the shaded interlacing map and the reference interlacing map to obtain the differential interlacing map; S105: Blend the differential interlacing map into a multi-view interlacing map containing texture to obtain the target interlacing map.

[0037] In this embodiment, a solid color background scene refers to a scene in which all surfaces are monochromatic surfaces without texture, such as a pure white diffuse reflective surface.

[0038] Here, the number of first-viewpoints refers to the total number of viewpoints displayed in naked-eye 3D. The number of first-viewpoints can be 30, and the first resolution can be 1920×1080 (i.e., full HD resolution). This embodiment does not specifically limit the number of first-viewpoints and the first resolution.

[0039] In step S101 above, the reference interlacing map can be obtained by pre-rendering multiple viewpoint images of a solid-color background scene and interlacing them. Specifically, a solid-color background scene is pre-rendered to obtain multiple viewpoint images corresponding to the first number of viewpoints, each viewpoint image containing no texture. These multiple viewpoint images are then interlaced according to the interlacing rules of the naked-eye 3D display panel to obtain the reference interlacing map. The reference interlacing map represents the uniform illumination base of the background scene at each viewpoint when there is no shadow occlusion.

[0040] In this embodiment, the baseline interleaving graph can be generated and stored only once during the initialization phase, and then repeatedly called upon subsequently.

[0041] In some examples, the number of second-viewpoints can be the first percentage of the number of first-viewpoints. The first percentage can be set according to the actual application needs. For example, the first percentage can be set to 1 / 3, that is, if the number of first-viewpoints is 30, then the number of second-viewpoints is 10.

[0042] Considering that shadows are low-frequency spatial information in vision, the human eye's sensitivity to the outline accuracy and parallax between viewpoints of shadows is much lower than its sensitivity to the texture and color of the object itself. Therefore, the second resolution can be set lower than the first resolution. For example, the horizontal and vertical dimensions of the second resolution are both the second percentage of the first resolution. For example, the second percentage is set to 1 / 3, that is: if the first resolution is 1920×1080, then the second resolution is set to 640×360.

[0043] In some examples, the first viewpoint number can be called the full viewpoint number (or high viewpoint number), and the first resolution can be called the high resolution; the first viewpoint number can be called the low viewpoint number, and the second resolution can be called the low resolution (if set to be less than the first resolution).

[0044] It is understood that this embodiment does not limit the specific ratio between the number of second viewpoints and the number of first viewpoints. Furthermore, the second resolution can also be set to be equal to the first resolution.

[0045] In step S102 above, for dynamic foreground objects, in order to achieve real-time updates of stereoscopic shadows, the shadows of the foreground objects can be rendered frame by frame in a solid color background scene for multiple foreground images containing foreground objects, using the second viewpoint number and the second resolution, to obtain viewpoint image groups corresponding to the multiple foreground images respectively; wherein, each viewpoint image group contains multiple viewpoint images containing the shadows of the foreground objects corresponding to the second viewpoint number.

[0046] Specifically, for each frame of a foreground image containing a dynamic foreground object, the shadow of that dynamic foreground object is rendered once in a solid-color background scene at a second viewpoint and a second resolution, resulting in multiple viewpoint images containing the shadow of the dynamic foreground object, the same number as the second viewpoint. When the position of the foreground object or the lighting conditions in the scene change, the rendering results of the foreground object shadows in adjacent frames are also updated accordingly, thus ensuring the continuity of dynamic shadows during real-time interaction.

[0047] It should be noted that during the rendering of foreground object shadows, the foreground object generates shadows by occluding the light source. The occlusion relationship between the foreground object and the light source determines the projection area and shape of its shadow on the solid-color background scene. However, the rendered images from various viewpoints only contain the solid-color background scene and the shadows cast by the foreground object, not the foreground object itself.

[0048] Foreground objects can be, for example, people, animals, or goods, which are the main subjects that need to be displayed in the foreground.

[0049] In this embodiment, under a textureless, solid-color background scene, the shadows of foreground objects are rendered using a second number of viewpoints and a second resolution. The rendered result is multiple viewpoint images with the same number of viewpoints as the second number of viewpoints, each with the second resolution and containing dynamic shadows. Rendering under a solid-color background scene reduces texture sampling overhead during rendering. Furthermore, since the number of second viewpoints is less than the number of first viewpoints, and the second resolution is lower than or equal to the first resolution, the computational load required for shadow rendering per frame is reduced compared to rendering with the first number of viewpoints and the first resolution, thereby reducing the computational burden of dynamic shadow rendering.

[0050] For example, compared to shadow rendering using the full number of viewpoints (e.g., 30) and full resolution, if the number of viewpoints in real-time shadow rendering is reduced from 30 to 10, and the rendering resolution (total number of pixels) is reduced to 1 / 9 of the full resolution, then the computational load of shadow rendering per frame is about 1 / 27 of the original shadow rendering computational load.

[0051] In step S103, multiple viewpoint images containing the shadow of the foreground object can be interpolated to obtain multiple viewpoint images with the same number of viewpoints as the first viewpoint, and the interpolated viewpoint images have the same resolution as the first resolution; then the multiple interpolated viewpoint images are interwoven to obtain a shadow interwoven image with the same number of viewpoints and the same resolution as the reference interwoven image.

[0052] For example, the interpolation process may include: arranging multiple view images rendered under a second view number in the view space according to their corresponding view number. Each view image under the second view number corresponds to a portion of view positions under the first view number. For example, when the second view number is 10 and the first view number is 30, the 10 view images under the second view number can correspond to the 1st, 4th, 7th, ..., 28th view positions under the first view number (i.e., selecting one every two views). For the remaining 20 view positions under the first view number that are not directly rendered, interpolation can be performed based on two adjacent rendered view images in the view space to obtain the view image corresponding to that view position. The interpolation method can be linear interpolation, weighted average, or disparity-based interpolation, etc., and this embodiment does not limit this.

[0053] If the second resolution (e.g., 640×360) used for shadow rendering is lower than the first resolution (e.g., 1920×1080), in order to align each viewpoint image containing the shadow of the foreground object with the reference interlacing map in terms of spatial resolution, it is necessary to interpolate the spatial resolution of each viewpoint image so that each interpolated viewpoint image has the same resolution as the first resolution. For example, linear interpolation, bicubic interpolation, and other algorithms can be used to enlarge the viewpoint image with the second resolution to the first resolution.

[0054] It should be noted that the shadow interlacing map and the reference interlacing map use the same subpixel interlacing mapping relationship; the subpixel interlacing mapping relationship is used to map the image samples from each viewing angle to the corresponding subpixel positions on the display panel. Each pixel in the reference interlacing map and the shadow interlacing map contains R, G, and B color components; in the reference interlacing map and the shadow interlacing map, corresponding color components within the same pixel are mapped to the same physical subpixel on the display panel and assigned to the same viewing angle.

[0055] In step S104, the differential processing can be performed pixel-by-pixel, that is, differential processing is performed on pixels corresponding to the same position in the shadow interlacing map and the reference interlacing map to obtain a differential interlacing map. Specifically, each pixel position is traversed, and differential processing is performed on the corresponding color components of the same pixel position in the reference interlacing map and the shadow interlacing map to obtain a differential interlacing map; the difference of the color components corresponds to the amount of brightness attenuation caused by the shadow under the viewing angle allocated to the color component.

[0056] In some examples, a differential interlacing map can be obtained by performing pixel-by-pixel difference processing on the shadow interlacing image and the reference interlacing map in the linear domain of brightness. Specifically, the brightness values ​​of pixels at the same position in the shadow interlacing image and the reference interlacing map are subtracted to obtain the brightness difference value of each pixel, thereby generating the differential interlacing map.

[0057] Taking the linear domain as an example, for each pixel position (x, y), the pixel value (i.e., the shadow intensity coefficient) of the corresponding pixel in the difference interlacing map is calculated using the following formula:

[0058] in, y is the pixel value at pixel position (x, y) in the differential interleaving image; The pixel value at pixel position (x,y) in the baseline interlacing diagram represents the baseline brightness of a solid color background scene without shadow occlusion. The pixel value at pixel position (x,y) in the shadow interlacing diagram represents the brightness of the shadow cast by the foreground object at that position.

[0059] In other examples, a differential interlacing image can be obtained by performing pixel-by-pixel difference processing on the shadow interlacing image and the reference interlacing image in the logarithmic domain. For example, the logarithm of the brightness value of each pixel in the shadow interlacing image and the reference interlacing image can be taken separately, and then the logarithm results can be subtracted to obtain the differential interlacing image.

[0060] In this embodiment, the shadow interlacing image is obtained by interlacing multiple viewpoint images containing the shadows of foreground objects, while the reference interlacing image is obtained by interlacing multiple viewpoint images of a solid-color background scene. Both viewpoint images are rendered based on a solid-color background scene and maintain consistency in the number of viewpoints and spatial resolution. Therefore, the pixel content at corresponding viewpoints and positions in both images only differs in brightness due to shadows. After differential processing, the values ​​in non-shadow areas approach 0, while the values ​​in shadow areas are positive, thus enabling the extraction of shadow intensity information.

[0061] When applied to a naked-eye 3D display system based on a dual-layer display architecture, since both the shadow interlacing map and the reference interlacing map are obtained under a solid-color background scene, the foreground content does not participate in the illumination difference calculation. This avoids interference from the foreground content with the difference results, ensuring that the difference results only reflect the shadow distribution in the background scene. During subsequent blending, this difference result is applied to the background interlacing map, not to the foreground content, thus ensuring that the foreground content is not affected by shadows.

[0062] In step S105, the multi-view interlaced map containing texture is, for example, an interlaced map of a textured background scene, which can be pre-rendered or rendered in real time and then interlaced. The multi-view interlaced map is obtained by rendering the textured background scene at a first viewpoint and a first resolution and then interlacing it.

[0063] Specifically, the pixel values ​​(i.e., shadow intensity coefficients) in the differential interlacing map can be used as modulation factors to modulate the brightness of corresponding pixels in the multi-view interlacing map containing texture, thus obtaining the target interlacing map. In this way, when generating the target interlacing map, the shadows only reduce the background brightness without introducing additional color shifts.

[0064] In the aforementioned image processing method for naked-eye 3D display, the rendering of dynamic shadows for foreground objects is performed at a second number of viewpoints and a second resolution, where the second number of viewpoints is less than the first number of viewpoints and the second resolution is lower than or equal to the first resolution. This reduces the computational load of dynamic shadow rendering compared to rendering at the first number of viewpoints and the first resolution, thus helping to meet the real-time interactive requirements of naked-eye 3D display. Furthermore, since both the reference interlacing map and the multiple viewpoint images of the shadows are obtained based on a solid-color background scene and do not contain textures, after processing the multiple viewpoint images of the shadows to obtain the shadow interlacing map, the differential interlacing map obtained by performing differential processing between the reference interlacing map and the shadow interlacing map is independent of the specific texture of the background scene. This allows the differential interlacing map to be independently superimposed on any background interlacing map containing textures without recalculating the shadows for different background scenes, thereby improving the versatility and practicality of the solution.

[0065] In some embodiments, step S103 above, processing multiple viewpoint images containing the shadows of foreground objects to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map, may include: During the interleaving calculation, multiple viewpoint images containing the shadows of foreground objects are interpolated to obtain a shadow interleaving map with the same number of viewpoints and the same resolution as the reference interleaving map.

[0066] The generation of the interlacing map requires determining the viewpoint number corresponding to each pixel position based on the pixel arrangement rules of the naked-eye 3D display panel, and sampling pixel values ​​from the viewpoint images corresponding to the viewpoint numbers. Specifically, during the interlacing calculation process, when sampling is required from a certain viewpoint image, if there is no directly corresponding viewpoint image for the viewpoint at the sampling position, the viewpoint dimensions can be interpolated using the viewpoint images of adjacent viewpoints. In addition, if the resolution of multiple viewpoint images containing the shadows of foreground objects is a lower second resolution, the sampling coordinates are not integer pixel positions in the low-resolution viewpoint images, and the spatial dimensions can be interpolated directly using neighboring pixels. Thus, after the interlacing calculation is completed, a shadow interlacing map with the same number of viewpoints and the same resolution as the baseline interlacing map is directly obtained.

[0067] For example, when the number of first viewpoints is 30 and the number of second viewpoints is 10, the viewpoint image only covers a portion of the 30 viewpoints. During the interlacing calculation of the shadow interlacing map, for pixels that need to be sampled from an uncovered target viewpoint, linear interpolation can be performed based on the viewpoint images adjacent to the target viewpoint on both sides to obtain approximate image data for that target viewpoint. If the second resolution is lower than the first resolution (e.g., 1 / 3 of the first resolution), the non-integer coordinates of each target pixel in the spatial dimension corresponding to the low-resolution viewpoint image can be obtained by performing bilinear interpolation on the four neighboring pixels to obtain the sampled value.

[0068] In this embodiment, by using the synchronous interpolation method described above during the interpolation calculation process, compared to first interpolating to generate a complete intermediate high-resolution, high-view-number image and then performing interpolation processing, the computational and storage overhead of the intermediate image can be saved.

[0069] In some embodiments, such as Figure 2 As shown, in step S105 above, blending the differential interlacing map into a multi-view interlacing map containing texture to obtain the target interlacing map may include: S1051: Calculate the ratio of the pixel value of the differential interlacing map to the reference interlacing map at the corresponding position for each pixel to obtain the normalization coefficient of each pixel.

[0070] Specifically, for each pixel position (x, y), its normalization coefficient S(x, y) can be calculated using the following formula:

[0071] in, y is the pixel value at pixel position (x, y) in the differential interleaving image; The pixel value of the corresponding pixel in the baseline interlacing diagram represents the baseline brightness when there is no shadow. This is a preset, extremely small constant used to prevent the denominator from being zero; for example, it can be 0.0001. Through this normalization process, the shadow intensity coefficient is converted into a proportional value relative to the reference brightness, thereby suppressing the influence of the absolute value of the background brightness and making the normalized shadow intensity coefficient independent of the specific brightness level of the background scene.

[0072] S1052: For each pixel in the multi-view interlacing map containing texture, multiply the pixel value of the pixel by a decay coefficient to obtain the pixel value of the corresponding pixel in the target interlacing map; wherein, the decay coefficient is equal to 1 minus the product of the shadow intensity factor and the normalization coefficient of the pixel, and the shadow intensity factor ranges from 0 to 1.

[0073] Specifically, for each pixel location (x, y), the pixel value in the target interlacing map Calculated using the following formula:

[0074] in, is the normalization coefficient for the pixel position (x, y); For an adjustable shadow intensity factor, preferably, The value can be 0.5 or 0.6; The pixel value at the pixel position (x, y) in the multi-view interlaced graph containing texture.

[0075] In this embodiment, by performing the above steps, it can be ensured that the shadow only reduces the background brightness without introducing any additional color shift. Furthermore, due to the normalization characteristics of the difference operation, the shadow intensity is independent of the specific texture of the background scene and can be universally superimposed.

[0076] In some embodiments, the method is applied to a naked-eye 3D display system, the naked-eye 3D display system including a first 2D display panel and a 3D display panel stacked together; the first 2D display panel is used to display an image of the foreground object; the textured multi-view interlacing map is a textured background interlacing map, the 3D display panel is used to display a 3D background based on the target interlacing map, and to provide backlight illumination for the first 2D display panel.

[0077] The naked-eye 3D display system in this embodiment adopts a dual-layer display architecture. In this architecture, the rear display panel is responsible for presenting the naked-eye 3D stereoscopic background (such as simulating the interior space of a box) and serves as the light source for the front display panel; the front high-resolution panel is responsible for displaying the main objects (such as text, trademarks, high-definition product images, etc.). With this setup, the system can maintain the naked-eye stereoscopic effect while ensuring the clarity of the foreground subject.

[0078] like Figure 3As shown, the naked-eye 3D display system includes a first 2D display panel (which may be referred to as the front display panel) 10 and a 3D display panel 20 (which may be referred to as the rear display panel) stacked together. It is understood that... Figure 3 The skirt shown on the first 2D display panel 10 is merely illustrative and does not constitute a limitation on the foreground display content.

[0079] The first 2D display panel 10 employs a transmissive liquid crystal display (LCD) front panel with high pixel density (PPI) or a similar light valve type display device, with a higher resolution than the rear display panel. The first 2D display panel 10 is positioned at the very front of the entire display system and is used to display foreground images, including but not limited to: high-definition text, trademarks, product images, UI elements, or video images. This panel has adjustable pixel-level transmittance, with the transmittance of each pixel independently controllable to compensate for the light distribution from the rear display panel, thereby accurately reproducing the detail information of the foreground image.

[0080] The 3D display panel 20 is used to present a three-dimensional background and serves as the backlight source for the first 2D display panel 10. The first 2D display panel 10 itself does not emit light, and its display brightness depends on the selective transmission of light through to the back layer.

[0081] A 3D display panel 20 is disposed on the back side of the front display panel and stacked parallel to the front panel. It includes a second 2D display panel 21 and an optical beam splitter 22. The second 2D display panel 21 is a transmissive display panel used to display multi-viewpoint interlaced images of a 3D background. The optical beam splitter 22 is disposed on the light-emitting side of the second 2D display panel 21 and is used to guide the light emitted by the second 2D display panel 21 to different viewpoints. As an example, the 3D display panel 20 is formed by bonding the second 2D display panel with a lenticular lens grating or a parallax barrier. As a glasses-free 3D display component, the 3D display panel 20 can present a background scene with stereoscopic depth (such as simulating a Cornell box 3D space) and project corresponding viewpoint images into different spatial directions in the form of a multi-view interlaced image through spatial beam splitting.

[0082] In this application, the rear 3D display panel is used not only to present a stereoscopic background (such as a Cornell box), but also as the sole physical backlight source for the front high-definition 2D display panel, enabling it to display foreground images for blending with the stereoscopic background. A Cornell box is a standard 3D scene used in computer graphics to test global illumination and rendering effects.

[0083] The optical beam splitter 22 can be a grating, such as a lenticular lens grating or a parallax grating.

[0084] In this embodiment, the glasses-free 3D display system further includes a backlight source 30, which is disposed on the non-display side of the second 2D display panel 21 and is used to provide illumination to the second 2D display panel 21. The light emitted by the backlight source 30 illuminates the second 2D display panel 21, is modulated by the multi-view interlaced image displayed on the second 2D display panel 21, and is then guided by an optical beam splitter 22 to different viewpoints, presenting background images of corresponding viewpoints at different viewpoint positions, thereby forming a 3D background. Simultaneously, the light emitted by the 3D display panel 20 illuminates the first 2D display panel 10 as backlight, enabling it to display foreground images. The glasses-free 3D display system also includes a processing unit, such as an image processor or display driver chip, for executing the image processing method provided in this embodiment.

[0085] Taking the naked-eye 3D display system with this dual-layer display architecture as an example, the foreground object is displayed independently by the front 2D display panel, while the rear naked-eye 3D display panel displays the 3D background. Figure 4 A schematic diagram of an interlacing diagram provided in an embodiment of this application is shown. For example... Figure 4 As shown, the leftmost image is the baseline interlacing map obtained from rendering a solid color background; the middle image is the shadow interlacing map generated based on the occlusion of foreground objects. This shadow interlacing map does not include the foreground objects themselves (i.e., the foreground objects are not visible in this interlacing map), only displaying the solid color background and the shadows of the foreground objects; the rightmost image shows the effect of extracting shadows (by differential processing of the shadow interlacing map and the baseline interlacing map) and blending them into a complex high-definition background, while simultaneously overlaying a 2D foreground. Blending the shadow map into the background makes the scene more three-dimensional.

[0086] In a naked-eye 3D display system employing a dual-layer display architecture, the front 2D display panel is used to display foreground objects, such as text, people, or high-definition product images; the rear naked-eye 3D display panel is used to provide a stereoscopic background, such as a Cornell box scene. However, the front 2D display content always rests on the physical surface of the screen, neither protruding nor recessed, resulting in insufficient overall stereoscopic depth. When rendering the rear 3D background, even if there are some out-of-screen elements, the lack of significant variations in the frame or box structure makes it difficult to provide effective out-of-screen parallax, and users still find it difficult to perceive a stereoscopic effect. If out-of-screen objects are added directly to the 3D background area, these objects are highly likely to overlap spatially with the front 2D display content, causing visual occlusion and confusion, and even leading to the loss of crucial information.

[0087] To enhance the stereoscopic depth of naked-eye 3D displays without obscuring foreground 2D content, embodiments of this application provide an image processing method for naked-eye 3D displays, such as... Figure 5 As shown, the method may include: S201: Obtain an interlacing sequence of multiple out-of-screen objects with a texture complexity lower than that of the foreground object. Each interlacing sequence corresponds to a preset motion trajectory of an out-of-screen object and contains multiple frames of interlacing corresponding to multiple trajectory points in the preset motion trajectory. S202: Determine the exclusion bounding box corresponding to each frame of the foreground object based on the layout information of each frame of the foreground object in the first 2D display panel; S203: Select the target out-of-screen object whose motion trajectory does not intersect with the exclusion bounding box corresponding to each frame image from multiple out-of-screen objects; S204: Overlay the multi-frame interlaced graphs from the interlaced graph sequence of the target out-of-screen object onto the target interlaced graph frame by frame.

[0088] In this embodiment, the out-of-screen object can be a low-detail, content-independent object, such as a floating water droplet or a sphere, which is different from the foreground object. The texture complexity of the out-of-screen object is lower than that of the foreground object, thus reducing the likelihood of significant crosstalk due to increased parallax when displayed out of the screen. It should be noted that texture complexity can be characterized by texture spatial frequency and / or gradient peak value; this embodiment does not limit the specific measurement method.

[0089] In step S201, multiple interlacing image sequences can be pre-calculated for each out-of-screen object. Each interlacing image sequence corresponds to a preset motion trajectory of an out-of-screen object. This preset motion trajectory contains multiple trajectory points, and each trajectory point corresponds to one frame of interlacing image. The interlacing image is a composite image formed by interlacing the multi-view images of the out-of-screen object at that trajectory point position. The interlacing image sequences of each out-of-screen object can be pre-calculated by the game engine or 3D rendering tool in the offline stage and directly called during runtime.

[0090] In this embodiment, the foreground object is the main object currently displayed on the front 2D display panel, such as a person, animal, or product. The layout information of each frame of the foreground object in the first 2D display panel is used to characterize the distribution of the currently displayed content on the display screen, such as the area occupied by the displayed content and its shape outline. Since the foreground display content can change dynamically over time, its layout information in the first 2D display panel will also change accordingly frame by frame.

[0091] In step S202, the exclusion bounding box is used to identify screen areas where off-screen objects are not allowed to enter, in order to avoid visual obstruction of foreground content. For example, the exclusion bounding box can be obtained by expanding a preset margin outward from the smallest rectangular area containing the currently displayed content of the front 2D display panel.

[0092] When there are multiple discrete foreground objects, the exclusion bounding box of each foreground object in the current frame can be determined separately, or a total exclusion bounding box covering all foreground objects can be determined. This embodiment does not limit this.

[0093] In step S203, for each out-of-screen object's preset motion trajectory, it is determined whether all trajectory points on the motion trajectory do not spatially overlap with the exclusion bounding box corresponding to each frame of the foreground object. If any trajectory point in a motion trajectory falls inside or is located on the exclusion bounding box, the out-of-screen object to which the motion trajectory belongs may obstruct the foreground content, and the out-of-screen object is not selected. Conversely, if all trajectory points of a motion trajectory are outside the exclusion bounding box, the out-of-screen object to which the motion trajectory belongs is selected as the target out-of-screen object, and the interlacing map sequence corresponding to the out-of-screen object is obtained for subsequent frame-by-frame overlay onto the background interlacing map containing texture.

[0094] It should be noted that this embodiment performs the above-mentioned selection operation of the out-of-screen object independently for each frame. When the relative relationship between the motion trajectory of the out-of-screen object and the exclusion bounding box corresponding to each frame changes, the system automatically switches the selected material, so that the out-of-screen object always moves within an area that does not interfere with the foreground content. This process only involves spatial judgment of trajectory and bounding box and does not consume 3D rendering resources.

[0095] In step S204, for each selected target out-of-screen object, the object interlacing image frame corresponding to the current display frame in the interlacing image sequence of the target out-of-screen object is obtained frame by frame, and the object interlacing image frame is superimposed on the target interlacing image to obtain an enhanced display image.

[0096] In some examples, the target interlacing map is a background interlacing map containing texture, which may include a background interlacing map containing the shadow of a foreground object, for example, by performing step S105 in the foregoing embodiments to obtain the background interlacing map containing the shadow of the foreground object.

[0097] In other examples, the target interlacing map can be a background interlacing map that does not contain the shadows of foreground objects. This background interlacing map can be mixed with the differential interlacing map obtained in step S104 of the foregoing embodiments to obtain a background interlacing map that contains the shadows of foreground objects.

[0098] The interlacing map corresponding to each trajectory point of the out-of-screen object includes a pair of generated object interlacing maps and an object mask interlacing map. That is, the interlacing map sequence of each out-of-screen object includes not only the object interlacing map of the out-of-screen object, but also its corresponding mask interlacing map, and the two correspond one-to-one on a frame-by-frame basis.

[0099] The object interlacing map of the out-of-screen object is obtained by rendering and interlacing the out-of-screen object from multiple perspectives; the mask interlacing map of the out-of-screen object is obtained by interlacing the mask of the out-of-screen object in each perspective; each out-of-screen object has a corresponding mask interlacing map, and both the mask interlacing map and the object interlacing map of the out-of-screen object can be generated in advance.

[0100] In some examples, the step of superimposing multiple frames of interlaced maps from the interlaced map sequence of the target out-of-screen object onto the target interlaced map frame by frame includes: fusing each frame of the object interlaced map of the target out-of-screen object onto the target interlaced map through the corresponding object mask interlaced map.

[0101] Here, the mask values ​​of each pixel in the object mask interlacing map are used as fusion coefficients. Pixel values ​​at the same pixel position in the corresponding object interlacing map and target interlacing map are weighted and fused pixel by pixel to obtain the fused interlacing map. Specifically, in the object mask interlacing map, pixel positions within object regions take the pixel values ​​from the object interlacing map, while pixel positions outside object regions retain the pixel values ​​from the target interlacing map.

[0102] In some examples, the opacity value within the inner region of the target out-of-screen object can be a first preset value, and the opacity value at the edge region of the target out-of-screen object can gradually decrease from the first preset value to a second preset value; the transparency corresponding to the first preset value is lower than the transparency corresponding to the second preset value.

[0103] Specifically, a transparency blending algorithm can be used to set the opacity coefficient α to 1 inside the out-of-screen object, and gradually change the opacity coefficient α from 1 to 0 at the edge of the out-of-screen object, thus achieving a smooth transition and avoiding jagged edge artifacts. Here, α is usually taken from the alpha channel of the foreground image. It should be noted that if the alpha channel stores transparency, it needs to be converted to opacity before participating in the blending.

[0104] In this embodiment, an opacity value of 1 indicates that the pixel fully utilizes the pixel values ​​of the out-of-screen object, while an opacity value of 0 indicates that the pixel fully utilizes the pixel values ​​of the target interlacing map. By gradually decreasing the opacity value from 1 to 0 in the edge region of the target out-of-screen object, edge jagged artifacts can be avoided. In real-time operation, only one transparency blending operation needs to be performed on the interlacing map per frame, requiring no additional 3D rendering computing power, reducing the real-time computation load and improving real-time performance.

[0105] In some examples, the speed of out-of-screen objects can be limited to below a preset threshold to avoid reducing the 3D visual experience or distracting the user from the foreground content due to rapid movement.

[0106] Figure 6 A schematic diagram illustrating the out-of-screen enhancement effect provided in an embodiment of this application is shown. For example... Figure 6 As shown, this example uses multiple spheres of varying sizes as out-of-screen objects. These spheres are superimposed on the background interlacing pattern of the rear naked-eye 3D display panel, and together with the high-definition foreground content displayed on the front 2D display panel, form a 2D-3D hybrid display effect. Figure 6 The leftmost image shows the interlaced pattern displayed on the rear naked-eye 3D display panel, which includes a high-resolution 3D background, superimposed out-of-screen objects (i.e., spheres), and shadows of foreground objects. The middle image shows the high-definition foreground image (i.e., a high-definition image of the foreground objects) displayed on the front 2D display panel. The rightmost image shows the 2D-3D display effect after mixing the leftmost interlaced pattern with the middle foreground image. It can be clearly seen that the spheres of varying sizes are distributed in a staggered manner along the out-of-screen direction, enhancing the three-dimensional depth of the image, while the spheres do not obstruct the foreground content.

[0107] This embodiment, through the aforementioned steps, obtains a sequence of interlaced maps of multiple out-of-screen objects with lower texture complexity than the foreground objects. It then uses foreground exclusion bounding boxes to filter out target out-of-screen objects. The interlaced map of the target out-of-screen object is then overlaid frame-by-frame onto the background interlaced map. This ensures that the out-of-screen enhancement elements do not overlap with key foreground content, effectively enhancing the stereoscopic effect and visual richness of the naked-eye 3D image while maintaining information integrity. Thus, a secure overlay of out-of-screen enhancement layers is achieved in a dual-layer display architecture, improving the stereoscopic depth while avoiding occlusion interference with foreground 2D content, thus helping to meet the real-time interactive needs of naked-eye 3D displays.

[0108] It should be noted that steps S201 to S204 described above can be implemented individually to constitute an independent embodiment, thereby enhancing the off-screen effect of naked-eye 3D display, or they can be combined with... Figure 1 Some or all of the steps shown constitute an independent embodiment.

[0109] To further optimize the stereoscopic enhancement effect, in some embodiments, the spatial position of the out-of-screen object is located on one side of the out-of-screen direction of the zero parallax plane; the higher the texture complexity of the out-of-screen object, the closer its spatial position is to the zero parallax plane; the lower the texture complexity of the out-of-screen object, the farther its spatial position is from the zero parallax plane; wherein, different spatial positions correspond to different parallax amplitude ranges, and the parallax amplitude is the pixel offset of the same out-of-screen object in images from different viewing angles. The farther the spatial position of the out-of-screen object is from the zero parallax plane, the greater the corresponding parallax amplitude.

[0110] Here, the zero parallax plane is a spatial plane in which the lateral parallax of each viewpoint is zero, and it is used as the boundary between the out-of-screen area and the in-screen area.

[0111] The further forward the virtual image of an object is on the screen, the greater the corresponding parallax amplitude. Increased parallax amplitude exacerbates crosstalk between viewing angles caused by imperfect grating beam splitting. When the texture of an out-of-screen object contains high spatial frequency components (such as dense stripes, sharp edges, or fine textures), crosstalk will be perceived by the human eye as high-frequency ghosting or rainbow patterns, leading to severe image quality degradation. Conversely, for objects with smooth textures and soft color transitions, even if they extend significantly out of the screen, crosstalk will only appear as a blurry halo, making it difficult to detect. In this embodiment, the spatial position of the out-of-screen object is predetermined based on its texture complexity. This allows objects with solid colors or low-frequency textures to extend significantly out of the screen without significant crosstalk, while highly detailed out-of-screen objects remain close to the screen, thereby enhancing the sense of depth without visual loss.

[0112] In some examples, the spatial frequency characteristics of out-of-screen objects can be quantified first, and their maximum permissible disparity amplitude can be calculated to ensure that crosstalk artifacts caused by disparity are below the visual perception threshold (e.g., the just-notifiable difference (JND) threshold), thus ensuring that the observer has no obvious crosstalk perception. Then, a mapping from spatial location to disparity amplitude is performed. The screen space is divided into multiple depth levels, with different levels corresponding to different disparity amplitude ranges, so that multiple out-of-screen objects form a staggered spatial distribution in the horizontal, vertical, and depth directions.

[0113] To simplify calculations, the depth of field can be divided into multiple intervals. The gradient of the texture of the object off-screen is calculated, and the peak gradient is mapped to the corresponding depth of field interval. For example, a solid-color sphere can be placed in the interval furthest off-screen; a highly detailed object with a texture map needs to be placed in the interval closer to the edge of the screen. The movable position of the object along the x and y axes also differs in different intervals. This is because, after perspective projection, 3D points with the same x and y coordinates have different z values, which in turn map to different 2D pixel positions on the screen.

[0114] In some embodiments, such as Figure 7 As shown, in step S202 above, determining the exclusion bounding box corresponding to each frame of the foreground object based on the layout information of each frame in the first 2D display panel may include: S2021: Obtain the set of pixel coordinates of each frame of the foreground object in the first 2D display panel.

[0115] For example, the position parameters of the foreground object in the current frame image within the front 2D display panel can be obtained, such as the bounding box vertex coordinates of the foreground object; or image analysis can be performed directly on the display content of the front 2D display panel to extract the pixel area covered by the foreground object. All pixel positions covered by the foreground object within the current frame are then merged to obtain the set of pixel coordinates for that frame.

[0116] S2022: For each frame of the foreground object, calculate the minimum bounding rectangle of the foreground object based on the set of pixel coordinates of the frame, and expand the minimum bounding rectangle according to a preset dilation coefficient to obtain the repulsion bounding box corresponding to the frame.

[0117] For example, the minimum and maximum values ​​of all pixel coordinates in the set of pixel coordinates are calculated in the horizontal and vertical directions to determine a minimum bounding rectangle containing all pixels of the foreground object. To avoid visual interference caused by subsequent out-of-screen objects being too close to the edge of the foreground object, the boundary can be expanded based on the minimum bounding rectangle according to a preset expansion coefficient. For example, each boundary of the rectangle can be expanded outward by a certain pixel distance or proportion to obtain a repulsive bounding box. The expansion coefficient can be preset according to actual display requirements, such as expanding by 5% or 10%, which is not limited in this embodiment.

[0118] The image processing method for naked-eye 3D display provided in this application will be further explained below with specific examples.

[0119] like Figure 8 As shown, an image processing method for naked-eye 3D display is provided, the method may include: S1: Pre-render a textureless white-framed interlacing map of the background scene. This interlacing map does not contain foreground content. This textureless white-framed interlacing map is the reference interlacing map in the aforementioned embodiment.

[0120] S2: Render a textureless white-framed scene with dynamic shadows at low resolution and low viewpoint settings to obtain a real-time shadow interlacing map. This shadow interlacing map includes the shadows of foreground objects.

[0121] S3: Perform differential processing on the textureless white frame interlaced map and the real-time shadow interlaced map to obtain the differential shadow. Since the foreground content is masked, the foreground pixels do not participate in the differential shadow calculation, so the differential shadow result does not contain the foreground content.

[0122] S4: Pre-render a high-resolution, high-viewpoint 3D interlaced background frame. This background frame 3D interlaced image is obtained by rendering and interlacing the textured background scene texture.

[0123] S5: Renders a high-resolution main view image of the foreground content and the corresponding main view mask image.

[0124] The main view mask does not contain shadows. This main view mask is located in the pixel area of ​​the rear naked-eye 3D panel that spatially corresponds to the front 2D display content. This area requires image processing (such as filling with specific brightness values ​​or low-frequency lighting components) to prevent the background image from penetrating into the foreground object area.

[0125] S6: Renders the 3D interlacing map of the out-of-screen objects and the corresponding mask interlacing map of the out-of-screen objects.

[0126] S7: Perform blending processing on the high-definition main view image of the foreground content and the corresponding main view mask area, the 3D interlaced image of the out-of-screen object and the corresponding out-of-screen object mask interlaced image to generate a shadowless 2D-3D blended display result.

[0127] S8: Perform shadow weighted blending on the differential shadow obtained in step S3 and the shadowless 2D-3D blending display result obtained in step S7 to generate a 2D-3D blending display effect with stereoscopic shadows.

[0128] In summary, the image processing method for naked-eye 3D display provided in this application utilizes the visual characteristics of shadows being low-frequency spatial information and the human eye having low sensitivity to shadow contour accuracy and parallax between viewpoints. It employs an asymmetric allocation strategy for the number of viewpoints and resolution. The background scene is pre-rendered at full viewpoint and full resolution, while dynamic shadows are rendered in real-time only at a lower resolution (e.g., reduced to 1 / 3 of the original resolution) and fewer viewpoints (e.g., only 10 viewpoints). Pure shadow intensity is then extracted through differential processing and blended into the full-view background interlacing map. This scheme significantly reduces the computational load of real-time shadow rendering. For example, when the number of viewpoints is reduced from 30 to 10 and the resolution is reduced to 1 / 9 of the original resolution, the computational load is approximately 1 / 27 of the original scheme. Therefore, real-time shadow updates can be achieved even under the CPU OpenGL rendering pipeline, significantly improving the practicality of the scheme on embedded and mobile platforms.

[0129] For the out-of-screen enhancement, this embodiment pre-calculates a sequence of interlaced images containing transparency information and performs motion trajectory collision detection using a foreground exclusion bounding box to ensure that the enhanced out-of-screen elements do not spatially overlap with key foreground content during display. In real-time operation, each frame only requires overlaying the selected interlaced image frame onto the background interlaced image through transparency blending, without requiring additional 3D rendering calculations. Thus, while ensuring the integrity of foreground information, it effectively enhances the out-of-screen stereoscopic effect and visual richness of the naked-eye display.

[0130] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0131] It is worth noting that, without contradiction, the foregoing method embodiments can be combined arbitrarily. For example, some or all of the steps of different embodiments can be combined arbitrarily, and one embodiment can be combined arbitrarily with the optional implementations of other embodiments.

[0132] Figure 9 A schematic diagram of the structure of the image processing device for naked-eye 3D display provided in an embodiment of this application is shown, such as... Figure 9 As shown, the image processing device 100 includes: The first acquisition module 101 is used to acquire a reference interlacing map of a solid color background scene under a first viewpoint and a first resolution, wherein the reference interlacing map does not contain texture. The rendering module 102 is used to render the shadow of the foreground object in the solid color background scene with a second viewpoint number and a second resolution to obtain multiple viewpoint images containing the shadow of the foreground object corresponding to the second viewpoint number; wherein the second viewpoint number is less than the first viewpoint number, and the second resolution is lower than or equal to the first resolution. Interlacing module 103 is used to process the multiple view images containing the shadows of foreground objects to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map. Differential module 104 is used to perform differential processing on the shadow interlacing map and the reference interlacing map to obtain a differential interlacing map; The first mixing module 105 is used to mix the differential interlacing map into a multi-view interlacing map containing texture to obtain a target interlacing map.

[0133] In some embodiments, the interleaving module 103 is used for: During the interleaving calculation, the multiple viewpoint images containing the shadows of foreground objects are interpolated to obtain a shadow interleaving map with the same number of viewpoints and the same resolution as the reference interleaving map.

[0134] In some embodiments, the differential module 104 is used for: In the linear or logarithmic domain of luminance, the shadow interlacing image and the reference interlacing image are differentially processed pixel by pixel to obtain the differential interlacing image.

[0135] In some embodiments, the first mixing module 105 is used to: The ratio of the pixel value of the differential interlacing map to the pixel value of the reference interlacing map at the corresponding position is calculated for each pixel to obtain the normalization coefficient of each pixel; For each pixel in the multi-view interlacing map containing texture, the pixel value of the pixel is multiplied by an attenuation coefficient to obtain the pixel value of the corresponding pixel in the target interlacing map; wherein, the attenuation coefficient is equal to 1 minus the product of the shadow intensity factor and the normalization coefficient of the pixel, and the shadow intensity factor has a value range of 0 to 1.

[0136] In some embodiments, the rendering module 102 is used for: In the solid color background scene, for the multiple frames of foreground images containing foreground objects, the shadows of the foreground objects are rendered frame by frame with the second viewpoint number and the second resolution to obtain viewpoint image groups corresponding to the multiple frames of foreground images respectively. Each viewpoint image group contains multiple viewpoint images corresponding to the second viewpoint number, including the shadows of foreground objects.

[0137] In some embodiments, the image processing device is applied to a glasses-free 3D display system, the glasses-free 3D display system including a first 2D display panel and a 3D display panel stacked together; The first 2D display panel is used to display an image of the foreground object; The textured multi-view interlacing map is a textured background interlacing map. The 3D display panel is used to display a three-dimensional background based on the target interlacing map and to provide backlight illumination for the first 2D display panel.

[0138] In some embodiments, based on Figure 9 ,like Figure 10 As shown, the image processing device 100 further includes: The second acquisition module 106 is used to acquire a sequence of interlaced graphs of multiple out-of-screen objects with a texture complexity lower than that of the foreground object. Each sequence of interlaced graphs corresponds to a preset motion trajectory of an out-of-screen object and includes a multi-frame interlaced graph corresponding to multiple trajectory points in the preset motion trajectory. The determining module 107 is used to determine the exclusion bounding box corresponding to each frame image based on the layout information of each frame image of the foreground object in the first 2D display panel. The selection module 108 is used to select, from the plurality of out-of-screen objects, a target out-of-screen object whose motion trajectory does not intersect with the exclusion bounding box corresponding to each frame image; The second mixing module 109 is used to superimpose the multi-frame interlaced map in the interlaced map sequence of the target out-of-screen object onto the target interlaced map frame by frame.

[0139] In some embodiments, the spatial position of the out-of-screen object is located on one side of the out-of-screen direction of the zero parallax plane; the higher the texture complexity of the out-of-screen object, the closer its spatial position is to the zero parallax plane; the lower the texture complexity of the out-of-screen object, the farther its spatial position is from the zero parallax plane. Different spatial locations correspond to different parallax amplitude ranges. The parallax amplitude is the pixel offset of the same out-of-screen object in images from different viewing angles. The further the spatial location of the out-of-screen object is from the zero parallax plane, the greater the corresponding parallax amplitude.

[0140] In some embodiments, the determining module 107 is configured to: Obtain the set of pixel coordinates of each frame of the foreground object in the first 2D display panel; For each frame of the foreground object, the minimum bounding rectangle corresponding to the foreground object is calculated based on the set of pixel coordinates of the frame image, and the minimum bounding rectangle is expanded according to a preset dilation coefficient to obtain the repulsion bounding box corresponding to the frame image.

[0141] In some embodiments, the interlacing map corresponding to each trajectory point of the out-of-screen object includes a paired generated object interlacing map and an object mask interlacing map; the second mixing module 109 is used for: Each frame of the object interlacing map of the target out-of-screen object is fused onto the target interlacing map using the corresponding object mask interlacing map; Wherein, the opacity value in the inner region of the target object is a first preset value, and the opacity value in the edge region of the target object gradually decreases from the first preset value to a second preset value; the transparency corresponding to the first preset value is lower than the transparency corresponding to the second preset value.

[0142] The image processing apparatus for naked-eye 3D display provided in this application belongs to the same concept as the image processing method for naked-eye 3D display provided in the above embodiments. It can execute the image processing method for naked-eye 3D display provided in any of the above embodiments and has the corresponding functional modules and beneficial effects for executing the image processing method for naked-eye 3D display. Technical details not described in detail in this embodiment can be found in the specific content of the image processing method for naked-eye 3D display provided in the above embodiments, and will not be repeated here.

[0143] It should be understood that the various modules in the image processing device used for naked-eye 3D display can be implemented in the form of a processor calling software, or the various modules can be implemented in the form of hardware circuits. The functions of some or all modules can be realized through the design of the hardware circuits, which can be understood as one or more processors.

[0144] This application embodiment also provides a glasses-free 3D display system, the glasses-free 3D display system comprising: The first 2D display panel is used to display an image of the foreground object; A 3D display panel is stacked on top of the first 2D display panel to present a three-dimensional background and provide backlight illumination for the first 2D display panel. The processing unit, connected to the first 2D display panel and the 3D display panel, is configured to perform the steps of the image processing method for naked-eye 3D display provided in any of the foregoing embodiments.

[0145] This application also provides an electronic device, including a processor, a memory, and an executable program stored in the memory and executable by the processor. When the processor runs the executable program, it performs the steps of the image processing method for naked-eye 3D display provided in any of the foregoing embodiments.

[0146] This application also provides a storage medium storing an executable program, which, when executed by a processor, implements the steps of the image processing method for naked-eye 3D display provided in any of the foregoing embodiments.

[0147] For ease of understanding, the following focuses on explaining the terminology used in this embodiment: In this application embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a type of microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented using an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), a Deep Learning Processing Unit (DPU), etc.

[0148] The computer-readable storage medium provided in this embodiment can execute the image processing method of the above embodiment. Its implementation principle and technical effect are similar to those of the above embodiment, and will not be repeated here.

[0149] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0150] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0151] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0152] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0153] In the description of this specification, references to "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method for naked-eye 3D display, characterized in that, The method includes: Obtain a reference interlacing map of a solid-color background scene at a first-viewpoint number and a first-resolution, wherein the reference interlacing map does not contain texture; In the solid color background scene, the shadow of the foreground object is rendered with a second viewpoint number and a second resolution to obtain multiple viewpoint images containing the shadow of the foreground object corresponding to the second viewpoint number; wherein, the second viewpoint number is less than the first viewpoint number, and the second resolution is lower than or equal to the first resolution. The images containing the shadows of foreground objects are processed to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map. The shadow interlacing map and the reference interlacing map are differentially processed to obtain a differential interlacing map; The differential interlacing map is blended into a multi-view interlacing map containing texture to obtain the target interlacing map.

2. The image processing method according to claim 1, characterized in that, The process of processing the multiple viewpoint images containing the shadows of foreground objects to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map includes: During the interleaving calculation, the multiple viewpoint images containing the shadows of foreground objects are interpolated to obtain a shadow interleaving map with the same number of viewpoints and the same resolution as the reference interleaving map.

3. The image processing method according to claim 1, characterized in that, The step of performing differential processing on the shadow interlacing map and the reference interlacing map to obtain a differential interlacing map includes: In the linear or logarithmic domain of luminance, the shadow interlacing image and the reference interlacing image are differentially processed pixel by pixel to obtain the differential interlacing image.

4. The image processing method according to claim 1, characterized in that, The step of blending the differential interlacing map into a multi-view interlacing map containing texture to obtain a target interlacing map includes: The ratio of the pixel value of the differential interlacing map to the pixel value of the reference interlacing map at the corresponding position is calculated for each pixel to obtain the normalization coefficient of each pixel; For each pixel in the multi-view interlacing map containing texture, the pixel value of the pixel is multiplied by an attenuation coefficient to obtain the pixel value of the corresponding pixel in the target interlacing map; wherein, the attenuation coefficient is equal to 1 minus the product of the shadow intensity factor and the normalization coefficient of the pixel, and the shadow intensity factor has a value range of 0 to 1.

5. The image processing method according to claim 1, characterized in that, In the solid color background scene, the shadows of the foreground objects are rendered using a second viewpoint and a second resolution to obtain multiple viewpoint images containing the shadows of the foreground objects corresponding to the second viewpoint, including: In the solid color background scene, for the multiple frames of foreground images containing foreground objects, the shadows of the foreground objects are rendered frame by frame with the second viewpoint number and the second resolution to obtain viewpoint image groups corresponding to the multiple frames of foreground images respectively. Each view image group contains multiple view images corresponding to the second view number, including the shadows of foreground objects.

6. The image processing method according to any one of claims 1 to 5, characterized in that, The method is applied to a glasses-free 3D display system, which includes a first 2D display panel and a 3D display panel stacked together. The first 2D display panel is used to display an image of the foreground object; The textured multi-view interlacing map is a textured background interlacing map. The 3D display panel is used to display a three-dimensional background based on the target interlacing map and to provide backlight illumination for the first 2D display panel.

7. The image processing method according to claim 6, characterized in that, The method further includes: Obtain an interlacing map sequence of multiple out-of-screen objects with a texture complexity lower than that of the foreground object. Each interlacing map sequence corresponds to a preset motion trajectory of an out-of-screen object and includes a multi-frame interlacing map corresponding to multiple trajectory points in the preset motion trajectory. Based on the layout information of each frame of the foreground object in the first 2D display panel, determine the exclusion bounding box corresponding to each frame of the image. Select the target out-of-screen object whose motion trajectory does not intersect with the exclusion bounding box corresponding to each frame image from the plurality of out-of-screen objects; The interlacing maps of multiple frames in the interlacing map sequence of the target out-of-screen object are superimposed frame by frame onto the target interlacing map.

8. The image processing method according to claim 7, characterized in that, The spatial position of the out-of-screen object is located on one side of the out-of-screen direction of the zero parallax plane; the higher the texture complexity of the out-of-screen object, the closer its spatial position is to the zero parallax plane; the lower the texture complexity of the out-of-screen object, the farther its spatial position is from the zero parallax plane. Different spatial locations correspond to different parallax amplitude ranges. The parallax amplitude is the pixel offset of the same out-of-screen object in images from different viewing angles. The further the spatial location of the out-of-screen object is from the zero parallax plane, the greater the corresponding parallax amplitude.

9. The method according to claim 7, characterized in that, The step of determining the exclusion bounding box corresponding to each frame of the foreground object based on the layout information of each frame of the foreground object in the first 2D display panel includes: Obtain the set of pixel coordinates of each frame of the foreground object in the first 2D display panel; For each frame of the foreground object, the minimum bounding rectangle corresponding to the foreground object is calculated based on the set of pixel coordinates of the frame image, and the minimum bounding rectangle is expanded according to a preset dilation coefficient to obtain the repulsion bounding box corresponding to the frame image.

10. The image processing method according to claim 7, characterized in that, The interlacing map corresponding to each trajectory point of the out-of-screen object includes a pair of generated object interlacing maps and an object mask interlacing map. The step of superimposing multiple frames of interlaced maps from the interlaced map sequence of the target out-of-screen object onto the target interlaced map frame by frame includes: Each frame of the object interlacing map of the target out-of-screen object is fused onto the target interlacing map using the corresponding object mask interlacing map; Wherein, the opacity value in the inner region of the target object is a first preset value, and the opacity value in the edge region of the target object gradually decreases from the first preset value to a second preset value; the transparency corresponding to the first preset value is lower than the transparency corresponding to the second preset value.

11. An image processing device for naked-eye 3D display, characterized in that, The device includes: The first acquisition module is used to acquire a reference interlacing map of a solid color background scene under a first viewpoint and a first resolution, wherein the reference interlacing map does not contain texture. The rendering module is used to render the shadow of the foreground object in the solid color background scene with a second viewpoint number and a second resolution to obtain multiple viewpoint images containing the shadow of the foreground object corresponding to the second viewpoint number; wherein the second viewpoint number is less than the first viewpoint number, and the second resolution is lower than or equal to the first resolution. An interlacing module is used to process the multiple viewpoint images containing the shadows of foreground objects to obtain a shadow interlacing map with the same number of viewpoints and the same resolution as the reference interlacing map. The difference module is used to perform difference processing on the shadow interlacing map and the reference interlacing map to obtain a difference interlacing map; The first mixing module is used to mix the differential interlacing map into a multi-view interlacing map containing texture to obtain a target interlacing map.

12. A glasses-free 3D display system, characterized in that, The naked-eye 3D display system includes: The first 2D display panel is used to display an image of the foreground object; A 3D display panel is stacked on top of the first 2D display panel to present a three-dimensional background and provide backlight illumination for the first 2D display panel. The processing unit, connected to the first 2D display panel and the 3D display panel, is configured to perform the steps of the image processing method as described in any one of claims 1 to 10.

13. An electronic device, characterized in that, It includes a processor, a memory, and an executable program stored in the memory and capable of being run by the processor, wherein the processor, when running the executable program, performs the steps of the image processing method as described in any one of claims 1 to 10.

Images