Naked-eye 3D display method based on parallax compensation

Abstract

The present application proposes a naked-eye 3D display method based on parallax compensation, which relates to the technical field of parallax compensation. The parallax deviation value of each pixel is calculated based on the angular relationship between the reference viewpoint and each viewing angle, so as to realize global parallax modeling and preliminary compensation of the image content. A neighborhood window search mechanism is introduced, and pixels with the smallest grayscale gradient change and consistent direction are selected as target pixels in a local area to perform image synthesis and perspective generation, thereby realizing local parallax optimization based on the image content. When it is detected that the parallax deviation value is too large or the user comfort level decreases, the system will dynamically adjust the parallax deviation value and perform depth perception correction on the target pixel points that exceed the preset depth range, forming a closed-loop feedback control mechanism. Through global modeling, local optimization, and a multiple compensation mechanism driven by user feedback, the parallax of the multi-view image sequence is finely adjusted, effectively alleviating the visual fatigue problem caused by excessive parallax without sacrificing image details.

Description

Technical Field

[0001] The present application relates to the technical field of parallax compensation, and in particular to a naked-eye 3D display method based on parallax compensation. Background Art

[0002] In the field of glasses-free 3D display, parallax compensation is a key technology for achieving multi-view image synthesis and spatial stereoscopic presentation. Existing parallax compensation methods typically adjust the overall parallax range of the image by multiplying all pixels by a uniform scaling factor, thereby alleviating viewing discomfort caused by excessive parallax. This method primarily addresses visual fatigue or double vision caused by the difference in left and right perspectives in stereoscopic images exceeding the human eye's ability to fuse them. It also offers a degree of adaptability to varying display sizes and viewing distances.

[0003] However, existing technologies have significant limitations. First, this method uses a global parallax compression strategy that ignores the depth information of each pixel in the image and cannot distinguish the depth differences between foreground and background areas. This unified processing often results in the compression of both foreground objects and background scenes, destroying the spatial hierarchy of the original scene and even causing image distortion, resulting in a visually unnatural effect.

[0004] Secondly, existing parallax compensation methods lack understanding and adaptability of image content, and fail to optimize processing based on local image features, such as edge structure, texture distribution, and other key visual information. This makes compressed images prone to problems such as loss of detail or blurred boundaries, affecting overall image quality.

[0005] Furthermore, traditional parallax compensation mechanisms are open-loop systems that completely disregard the user's real-time viewing status or comfort feedback, and are unable to dynamically adjust parallax compression strategies. Even when users are experiencing noticeable fatigue, the system still operates according to fixed parameters, lacking a user-friendly design and making it difficult to meet the needs of extended viewing.

[0006] In summary, while existing parallax compensation methods can alleviate the visual discomfort caused by excessive parallax to a certain extent, they lack depth perception, ignore image content characteristics, and fail to incorporate user feedback mechanisms, making it difficult to strike a balance between image quality and viewing comfort in practical applications. Therefore, there is an urgent need to propose a new parallax compensation method that can fully consider depth information, image content, and user status during the multi-view image synthesis process, achieving a more natural and comfortable naked-eye 3D display effect. Summary of the Invention

[0007] The present application aims to solve one of the technical problems in the related art at least to a certain extent.

[0008] To this end, the present application proposes a naked-eye 3D display method based on parallax compensation to strike a balance between image quality and viewing comfort.

[0009] To achieve the above objectives, the present invention proposes a naked-eye 3D display method based on parallax compensation, including:

[0010] Step 1: Obtain original images and depth images of multiple viewing angles under the target scene and perform preprocessing operations;

[0011] Step 2: determining the disparity offset value of each pixel in the pre-processed original image and the depth image, and obtaining a disparity offset matrix;

[0012] Step 3: Based on the disparity offset matrix, perform multi-view image synthesis on the pre-processed original image to obtain a multi-input view image sequence;

[0013] Step 4: spatially modulating the multi-view image sequence through a cylindrical lens array, and projecting the multi-view images in the multi-view image sequence after spatial light modulation onto the human eye area for 3D display;

[0014] Step 5: Dynamically adjust the disparity deviation value of the multi-input view images in the human eye area and update the multi-input view image sequence to optimize the display content.

[0015] In some implementations, determining the disparity offset value of each pixel in the preprocessed original image and the depth image includes the following steps:

[0016] Step 21: Set the reference viewpoint to be on the central axis of the display, and define the angle between the i-th viewing angle and the reference viewpoint as θ i ;

[0017] Step 22: Substitute the angle between the i-th viewing angle and the reference viewpoint into the parallax offset model to obtain the parallax deviation values ​​of the pixels in the original image of the i-th viewing angle at different depths.

[0018] In some implementations, performing multi-view image synthesis on the pre-processed original image based on the disparity offset matrix to obtain a multi-input view image sequence includes the following steps:

[0019] Step 31: Obtain the disparity deviation value of the current pixel in the pre-processed original image of the current viewing angle from the disparity offset matrix; and obtain the current input preset pixel according to the disparity deviation value of the current pixel and the current pixel.

[0020] Step 32: In each pre-processed original image of each viewing angle, select the current pixel as the center and construct a k×k neighborhood window from the center;

[0021] Step 33, searching for a neighborhood pixel point in the neighborhood window as a target pixel point, and replacing the current input preset pixel point with the target pixel point;

[0022] Step 34, repeatedly executing steps 31 to 33, replacing all current preset pixel points to obtain the current input perspective image;

[0023] Step 35: Repeat step 34 to obtain a multi-view image sequence.

[0024] In some implementations, the pixel point whose disparity deviation value between the neighborhood window and the current pixel point is in the same direction and whose grayscale gradient change is the smallest is taken as the target pixel point; a positive disparity deviation value represents a direction to the right, and a negative disparity deviation value represents a direction to the left.

[0025] In some implementations, dynamically adjusting the disparity deviation value of the multi-view input images within the human eye region and updating the multi-view image sequence to optimize display content includes:

[0026] Step 51: Collect feature point information of the user's face and perform three-dimensional reconstruction to obtain the spatial position coordinates of the user's face;

[0027] Step 52: determine whether the user is in the valid viewing area based on the spatial position coordinates; if not, execute step 53; if yes, execute step 54;

[0028] Step 53: Switch the multi-view image sequence to 2D display, and simultaneously generate a user prompt message to guide the user to return to the valid area. When it is detected that the user has returned to the valid area, the 3D display mode is automatically restored, and step 54 is executed;

[0029] Step 54: determine whether the current input view image in the multi-view image sequence requires disparity compression. If so, compress the disparity deviation value of the current target pixel in the current input view image to obtain an adjusted deviation value. If not, proceed to step 55.

[0030] Step 55: Obtain the depth value of the current target pixel in the current input perspective image, preset a reference depth value and a target depth range, calculate the absolute difference between the depth value and the reference depth value, and compare the absolute difference with the target depth range. If the absolute difference is greater than the target depth range, return to step 54 to compress the parallax deviation value of the current target pixel to obtain an adjusted deviation value; if not, correct the adjusted deviation value to obtain a corrected deviation value.

[0031] Step 56 : Based on the correction deviation value, return to step 3 and step 4 to obtain a multi-input viewing angle updated image sequence and display the optimized display content in 3D in the human eye area.

[0032] In some implementations, it is determined whether a current input view image in a multi-view image sequence requires disparity compression. If so, the disparity deviation value of the current target pixel in the current input view image is compressed to obtain an adjusted deviation value. If not, step 55 is executed, including:

[0033] Step 541, obtaining the maximum disparity deviation value of all target pixels in the current input perspective image;

[0034] Step 542: Obtain the user's viewing comfort level;

[0035] Step 543: Preset a first threshold and a second threshold;

[0036] In step 544 , if the maximum disparity deviation value is greater than the first threshold, or the viewing comfort is greater than the second threshold, the disparity deviation value of the current target pixel in the current input perspective image is compressed to obtain an adjusted deviation value; if not, step 55 is executed.

[0037] In some implementations, the adjusted deviation value is obtained by the disparity deviation value of the current target pixel point in the current input perspective image, the first threshold, the second threshold, and a maximum disparity deviation value.

[0038] In some implementations, the target depth range is k times the maximum depth value, 0 <k<1。

[0039] In some implementations, the adjusted offset value is corrected according to the target depth range, the adjusted offset value, and the absolute difference value to obtain a corrected offset value.

[0040] In some implementations, the first threshold is 1% times the width of the display.

[0041] Compared with the existing technology, the naked-eye 3D display method based on parallax compensation provided by this application integrates a dual compensation mechanism combining global parallax adjustment and local pixel optimization in the image processing process, effectively solving the problems of image distortion and viewing discomfort existing in traditional parallax compression methods, and significantly improving the picture quality and visual comfort of naked-eye 3D display.

[0042] After acquiring multi-view original images and corresponding depth images, this application calculates the disparity deviation value of each pixel based on the angular relationship between the reference viewpoint and each viewing angle, and constructs a disparity offset matrix to achieve global disparity modeling and preliminary compensation for the image content. This process not only ensures spatial consistency between multi-view images, but also provides basic data support for subsequent local optimization.

[0043] This application introduces a neighborhood window search mechanism, selecting pixels with the smallest grayscale gradient change and consistent direction within a local area as target pixels for image synthesis and perspective generation, thereby achieving local disparity optimization based on image content. This approach avoids the foreground / background synchronization compression problem caused by traditional unified scaling strategies, preserves key structural information in the image, and enhances the realism and three-dimensionality of the image.

[0044] This application also performs 3D reconstruction based on the user's facial feature points to determine whether they are in the effective viewing area and obtain the user's viewing comfort in real time. When the parallax deviation value is detected to be too large or the user's comfort level decreases, the system will dynamically adjust the parallax deviation value and perform depth perception correction on target pixels outside the preset depth range, forming a closed-loop feedback control mechanism. This adaptive parallax compensation strategy, combined with user status, enables the system to proactively optimize displayed content while maintaining image quality, enhancing the comfortable viewing experience for extended periods of time.

[0045] In summary, this application achieves refined parallax adjustment of multi-perspective image sequences through a multi-compensation mechanism of global modeling + local optimization + user feedback-driven, effectively alleviating the visual fatigue problem caused by excessive parallax without sacrificing image details, and has good practicality and promotion value.

[0046] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0048] Figure 1 A schematic flow chart of a naked-eye 3D display method based on parallax compensation provided in an embodiment of the present application;

[0049] Figure 2 A structural diagram of a naked-eye 3D display system based on parallax compensation provided in an embodiment of the present application;

[0050] Figure 3 A block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0052] The following describes a naked-eye 3D display method based on parallax compensation according to an embodiment of the present application with reference to the accompanying drawings.

[0053] Figure 1 This is a flow chart of a naked-eye 3D display method based on parallax compensation provided in an embodiment of the present application.

[0054] It should be noted that the executor of the naked-eye 3D display method based on parallax compensation in the embodiment of the present application is the naked-eye 3D display system based on parallax compensation in the embodiment of the present application. The naked-eye 3D display system based on parallax compensation can be configured in an electronic device so that the electronic device can perform the naked-eye 3D display function based on parallax compensation.

[0055] like Figure 1 As shown, the naked-eye 3D display method based on parallax compensation includes the following steps:

[0056] Step 1: Obtain original images and depth images of multiple viewing angles in the target scene and perform preprocessing operations.

[0057] In this embodiment, a capture system consisting of multiple fixed cameras is first constructed to synchronously capture image information of a target scene from different angles. The cameras are arranged in a ring around the target scene, with a total of nine capture positions corresponding to nine different viewing angles. The angle between adjacent viewing angles is 20°, ensuring coverage of the user's primary viewing area during naked-eye 3D display. Each camera is equipped with an RGB sensor and a depth sensor to simultaneously capture a color image (i.e., the original image) and a corresponding depth image. All cameras are triggered by a unified time synchronization signal to ensure temporal consistency of the captured images and avoid image misalignment caused by motion.

[0058] After acquisition, the original image and depth image from each view are stored as image pairs. Each pair consists of an RGB image (1920×1080 pixels) and a corresponding depth map (640×480 pixels, with depth values ​​in millimeters). The image data is then transferred to the image processing module for preprocessing.

[0059] 1. Resolution Unification and Image Registration: Due to the inconsistent resolutions of the RGB and depth images, we first upsampled the depth image to 1920×1080 using bilinear interpolation to match the spatial resolution of the RGB image. We then used a feature point-based affine transformation algorithm to align the two images, ensuring a one-to-one correspondence between each pixel in the RGB and depth images.

[0060] 2. Depth Image Denoising and Smoothing: Depth images often contain noise, especially at long distances. Therefore, a bilateral filtering algorithm is used to smooth depth images, effectively suppressing noise while preserving edge information and improving the reliability of depth data.

[0061] 3. Color Correction and White Balance Adjustment: Differences in lighting conditions and camera parameters can easily cause brightness and color deviations between images at different viewing angles. Therefore, an automatic white balance algorithm based on the grayscale world assumption is used to perform color correction on images from each viewing angle, ensuring consistent visual perception across all viewing angles.

[0062] 4. Image distortion correction: To address the barrel distortion that may be introduced by wide-angle lenses, the camera intrinsic parameter matrix and the distortion coefficient matrix are used to perform perspective transformation to eliminate the curvature of the image edges and ensure the correctness of the image geometry.

[0063] After completing the above preprocessing operations, a set of high-quality, spatially consistent, and temporally synchronized multi-view original images and their corresponding depth images are obtained, providing an accurate data basis for subsequent disparity offset modeling and image synthesis.

[0064] Step 2: Determine the disparity offset value of each pixel in the pre-processed original image and the depth image, and obtain a disparity offset matrix.

[0065] Step 21: Set the reference viewpoint to be on the central axis of the display, and define the angle between the i-th viewing angle and the reference viewpoint as θ i .

[0066] In this embodiment, the reference viewpoint of the naked-eye 3D display system is first set to be located on the central axis of the display, that is, the fifth viewing angle, which is considered to be the reference viewpoint of the user.

[0067] Step 22: Substitute the angle between the i-th viewing angle and the reference viewpoint into the parallax offset model to obtain the parallax deviation values ​​of the pixels in the original image of the i-th viewing angle at different depths.

[0068] First, the traditional parallax offset model is: (B × f) / z, where B is the baseline distance of the human eye, preferably 65mm. f is the display focal length, and z is the depth value of the pixel.

[0069] However, in practical applications, relying solely on a fixed parallax offset model cannot reflect the spatial differences between different perspectives. To this end, this application improves the traditional model by introducing the influence of the viewing angle θᵢ to form a new parallax offset model: i，j =(B×f×tan(θ i )) / z,X i，j is the jth pixel (x j ,y j ) is the parallax deviation value at depth z, θ i is the angle between the i-th viewing angle and the reference viewpoint.

[0070] The above B, f, angle and depth values ​​are obtained after standardization. Substitute the standardized B, f, angle and depth values ​​into X i，j Perform calculations.

[0071] Since image pixel coordinates must be integers, floating-point offset values ​​cannot be used directly for image synthesis. Therefore, the calculated disparity offset values ​​must be clipped and integerized. The specific steps include the following:

[0072] For each pixel point j at the i-th viewing angle, the disparity offset value X i,j , round down or round up to integer to get the final offset X i,j ′.

[0073] Set the maximum parallax offset threshold T1 (the first threshold below, which is 1% of the display width), when |X i,j When ′|>T1, it is truncated to ±T1 to prevent excessive offset from causing image edge loss or crosstalk. The disparity offset values ​​of all pixels are organized into a disparity offset matrix D according to the viewing angle number and pixel coordinates. The format is as follows:

[0074] , N is the total number of viewing angles, M is the total number of pixels for each viewing angle, each row represents the disparity offset value of all pixels under the i-th viewing angle, i=1, 2, 3, ..., N, and each column represents the disparity offset value of the j-th pixel under the i-th viewing angle, j=1, 2, 3, ..., M.

[0075] Traditional methods typically use a unified disparity offset model, ignoring the spatial differences between viewpoints. This results in uneven disparity distribution across different viewpoints, especially when dealing with scenes with significant depth variations, which can easily cause visual artifacts such as foreground object deformation or background blur. However, by introducing the angle between each viewing perspective and the reference perspective, a new disparity offset model is constructed, allowing disparity compensation to not only consider the depth information of the pixel but also the spatial position relationship of the pixel within the viewpoint. This method effectively reflects the perspective effect in the real physical world, ensuring a natural transition of disparity between different viewpoints and reducing visual discomfort caused by perspective changes. Furthermore, this angle-based disparity adjustment mechanism enhances the system's flexibility and adaptability, dynamically adjusting the displayed content based on the user's position to provide a more immersive naked-eye 3D experience. Therefore, compared to traditional fixed-ratio compression strategies, this method significantly improves viewing comfort and the consistency of the stereoscopic effect while maintaining image detail.

[0076] Step 3: Based on the disparity offset matrix, perform multi-view image synthesis on the pre-processed original image to obtain a multi-input view image sequence.

[0077] The core idea of ​​this embodiment is to resample and synthesize the pixels in the original image according to the parallax offset value to generate input viewing angle images adapted to different viewing angles.

[0078] Step 31 : Obtain the disparity deviation value of the current pixel in the pre-processed original image of the current viewing angle from the disparity offset matrix; and obtain the current input preset pixel according to the disparity deviation value of the current pixel and the current pixel.

[0079] Traverse each pixel (x, y) in the preprocessed original image of the current viewing angle and query its corresponding disparity deviation value Xi,j′ from the disparity offset matrix. Based on this deviation value, the position of the current input preset pixel is calculated as:

[0080] (x′,y′)=(x+X i,j ′,y) where x′ represents the offset coordinate in the horizontal direction and y remains unchanged. i,j ′>0, it means shifting to the right; if X i,j ′<0, it means shifting to the left.

[0081] Step 32: In each pre-processed original image of a viewing angle, select the current pixel as the center, and trigger the construction of a k×k neighborhood window from the center.

[0082] The position of the current input preset pixel point obtained through the above calculation may not exist in the original image. In the original image, a neighborhood window of size k×k is constructed with the current pixel point (x, y) as the center to search for a suitable replacement pixel.

[0083] Step 33: Search for a neighborhood pixel point in the neighborhood window as a target pixel point, and replace the current input preset pixel point with the target pixel point.

[0084] The pixel point with the same disparity deviation value direction between the neighborhood window and the current pixel point and the smallest grayscale gradient change is taken as the target pixel point; a positive disparity deviation value represents a direction to the right, and a negative disparity deviation value represents a direction to the left.

[0085] Within this neighborhood window, all pixels located in the direction of the disparity offset (i.e., positive or negative) are selected and their grayscale gradient values ​​are calculated using the Sobel operator. The pixel with the smallest grayscale gradient is selected as the target pixel, and the current input preset pixel is replaced with the target pixel. The current input preset pixel is replaced with the target pixel, preferably assigning the color value of the target pixel to the current input preset pixel (x′, y′).

[0086] Step 34: Repeat steps 31 to 33 to replace all current preset pixel points to obtain the current input perspective image.

[0087] Repeat the above operation for all pixels in the image, and replace all current input preset pixels under the current perspective in sequence, and finally obtain a complete input perspective image.

[0088] After completing step 34 to generate the input view image, a "pixel mapping table" is created to record the following information: input preset pixel coordinates, original viewing angle label, original pixel coordinates, depth value, and disparity offset value. This allows for quick access to the original depth or disparity value when performing disparity compression or depth correction on a pixel.

[0089] Step 35: Repeat step 34 to obtain a multi-view image sequence.

[0090] By performing the above steps on all N viewing angle images, a complete set of multi-input viewing angle image sequences can be generated for subsequent spatial light modulation and naked-eye 3D display.

[0091] Step 4: spatially modulate the multi-view image sequence through a cylindrical lens array, and project the multi-view images in the multi-view image sequence after spatial light modulation onto corresponding human eye areas for 3D display.

[0092] In this embodiment, the multi-view image sequence generated in step 3 is input to a glasses-free 3D display device, which includes a high-resolution display screen and a lenticular lens array positioned in front of it. The lenticular lens array consists of multiple parallel cylindrical microlenses, each corresponding to a number of pixel columns on the display screen, enabling directional projection of images from different perspectives. Specifically, the multi-view images are spliced ​​into a composite image in a preset order and displayed on the screen after being staggered in columns. The lenticular lens array performs spatial light modulation on the image on the screen, making each perspective image visible only in a specific direction, thereby forming multiple parallax images in space. These images are projected separately to the viewer's left and right eye regions, leveraging the human eye's parallax fusion mechanism to enable the viewer to perceive stereoscopic images without wearing glasses. Furthermore, the focal length, curvature, and arrangement period of the lenticular lenses are precisely matched to the display screen's pixel structure to ensure clear separation and crosstalk-free images from each perspective at the optimal viewing distance. In this way, the system achieves high-quality glasses-free 3D display, enhancing the spatial perception and immersive experience of the image. The above-described spatial light modulation process is a well-established conventional design and will not be described in detail in this embodiment.

[0093] Step 5: Dynamically adjust the disparity deviation value of the multi-input view images in the human eye area and update the multi-input view image sequence to optimize the display content.

[0094] Step 51: Collect feature point information of the user's face and perform three-dimensional reconstruction to obtain the spatial position coordinates of the user's face.

[0095] In this embodiment, a high-resolution camera integrated into the display device first captures a real-time image of the user's face. To ensure accurate identification and location of key feature points, advanced computer vision algorithms are employed, specifically convolutional neural networks (CNNs) from deep learning. These networks, trained with extensive data, can automatically extract the location information of key features such as the eyes, nose, and mouth from images. Specifically, the CNN analyzes the input facial image, identifies the pixel clusters that comprise these key features, and determines one or more feature points for each. For example, for the eyes, these might include locations such as the corners of the eyes and the center of the pupil; for the nose, these might include the tip of the nose and the sides of the nostrils. This creates a set of key feature points representing the user's facial structure.

[0096] Based on the spatial distribution of these feature points, the system then performs three-dimensional reconstruction. This process usually involves stereo matching technology or structured light scanning methods. Stereo matching technology relies on at least two images from different perspectives. By comparing the position differences of the same feature points in the two images, the parallax between them is calculated, and then the coordinates of each feature point in three-dimensional space are inferred. Structured light scanning, on the other hand, projects a series of light rays of known patterns onto the face and inversely calculates the surface shape based on the deformation caused by these rays on the facial surface. Regardless of which method is used, the ultimate goal is to generate an accurate three-dimensional model of the user's face.

[0097] Once the three-dimensional model of the face is obtained, the system can further calculate the specific position coordinates of the user's face in real space. This step usually involves placing the three-dimensional model in a virtual world coordinate system, and by comparing the positions of the feature points in the coordinate system, the precise position and orientation of the user's head relative to the device is obtained. This process not only requires a high degree of accuracy, but also needs to take into account the influence of environmental factors such as light changes and user movement. Through the above steps, the system can not only ensure the accuracy of facial feature extraction, but also provide the necessary input data for subsequent judgment of whether the user is in the effective viewing area, thereby achieving a more intelligent 3D display experience.

[0098] The above-mentioned key feature point identification, three-dimensional reconstruction and coordinate calculation processes are conventional technical means and will not be described in detail in this embodiment.

[0099] Step 52 , judging whether the user is in the effective viewing area according to the spatial position coordinates; if not, executing step 53 ; if yes, executing step 54 .

[0100] After obtaining the spatial coordinates of the user's face, they are compared and analyzed with the preset effective viewing area parameters. The effective viewing area is a spatial range established to ensure that the user's eyes can receive images from different perspectives, thereby enjoying the best naked-eye 3D experience. This area is primarily determined by the optical performance of the lenticular lens array, taking into account ergonomic factors such as viewing distance and angle. The ideal effective viewing zone includes a viewing distance of 0.5 meters to 2 meters from the screen, and an angle range of ±30° horizontal viewing angle and ±10° vertical viewing angle. When the user is within this range, the current 3D display mode is maintained and the process proceeds to step 54; if the user deviates from this area, the process proceeds to step 53.

[0101] In step 53 , the multi-view image sequence is switched to 2D display, and a user prompt is generated synchronously to guide the user to return to the valid area. When it is detected that the user has returned to the valid area, the 3D display mode is automatically restored, and step 54 is executed.

[0102] Step 54 , determining whether the current input view image in the multi-input view image sequence requires parallax compression. If so, compressing the parallax deviation value of the current target pixel in the current input view image to obtain an adjusted deviation value; if not, executing step 55 .

[0103] Step 541: Obtain the maximum disparity deviation value of all target pixels in the current input perspective image.

[0104] Step 542: Obtain the user's viewing comfort.

[0105] Viewing comfort is a comprehensive indicator used to evaluate whether the user's current viewing state is comfortable. In this embodiment, the user's viewing comfort is comprehensively judged based on the user's blinking frequency and pupil diameter.

[0106] Pupil dilation usually indicates fatigue or tension, while pupil contraction may mean relaxation. The average pupil diameter of an adult under normal lighting is 3.7mm, which is used as the reference pupil diameter in this embodiment, and then the user's real-time pupil diameter is obtained; under normal circumstances, humans blink about 15-20 times per minute. If the blinking frequency decreases significantly, it may indicate dry eyes or fatigue. In this embodiment, the human blink frequency of 18 times per minute is used as the reference blinking frequency, and the user's real-time blinking frequency is obtained. The above-mentioned real-time pupil diameter and blinking frequency, reference pupil diameter, and reference blinking frequency are standardized, and the viewing comfort is comprehensively calculated based on the standardized real-time pupil diameter and blinking frequency, reference pupil diameter, and reference blinking frequency.

[0107] C=w1×(1 / e -（dp-d0） )+w2×max(0,1-a×(fb-f0)),w1 and w2 represent weight coefficients,

[0108] w1=0.3, w2=0.7, e is the exponential function, dp is the real-time pupil diameter, d0 is the reference pupil diameter, max is the maximum value operation, fb is the real-time blink frequency, f0 is the reference blink frequency, a is the adjustment index, a=0.2.

[0109] Step 543: preset a first threshold and a second threshold.

[0110] The first threshold is 1% of the display width. The second threshold is used to determine when to activate the parallax adjustment mechanism. The second threshold is determined through statistical analysis of historical test data, and the viewing comfort level at which the parallax offset begins to be gradually reduced is used as the second threshold. The second threshold is preferably 0.7.

[0111] In step 544 , if the maximum disparity deviation value is greater than the first threshold, or the viewing comfort is greater than the second threshold, the disparity deviation value of the current target pixel in the current input perspective image is compressed to obtain an adjusted deviation value; if not, step 55 is executed.

[0112] The adjustment deviation value is obtained by the disparity deviation value of the current target pixel point in the current input perspective image, the first threshold, the second threshold and the maximum disparity deviation value.

[0113] X i，j ' =X i，j ×(1-r×(Xmax / T1+(T2-C) / T2)),X i，j ' is the jth pixel (x j ,y j ) is the adjustment deviation value at depth z, Xmax is the maximum parallax deviation value of the current input perspective image, T1 is the first threshold, T2 is the second threshold, C is the viewing comfort, r is the adjustment coefficient, and r<1.

[0114] The formula for calculating the adjustment deviation value is designed to optimize the naked-eye 3D display and enhance user viewing comfort. The theoretical basis of this formula lies in parallax control and human eye physiology. First, by introducing the maximum parallax deviation value Xmax and the first threshold T1, the formula dynamically adjusts the global parallax, ensuring that the parallax of the entire image does not exceed the physical limitations of the display, thereby avoiding visual fatigue and discomfort. Second, by combining the viewing comfort level C and the second threshold T2, the formula can make local adjustments based on the user's current physiological state (such as blink rate and pupil diameter) to adapt to individual differences and real-time changing viewing conditions.

[0115] Step 55: Obtain the depth value of the current target pixel in the current input perspective image, preset a reference depth value and a target depth range, calculate the absolute difference between the depth value and the reference depth value, and compare the absolute difference with the target depth range. If the absolute difference is greater than the target depth range, return to step 54 to compress the parallax deviation value of the current target pixel to obtain an adjusted deviation value; if not, correct the adjusted deviation value to obtain a corrected deviation value.

[0116] In this embodiment, steps 54 and 55 constitute a dual determination mechanism for achieving refined dynamic adjustment of the parallax deviation value. Specifically, in step 54, it is judged whether global or local parallax compression is required based on the maximum parallax deviation and viewing comfort; if it does not exceed the set threshold, the parallax is not adjusted temporarily. However, in the subsequent step 55, this embodiment further makes a secondary judgment by combining the depth information of the current target pixel: by calculating the absolute difference between the depth value of this pixel and the reference depth value, and comparing it with the preset target depth range. If the difference exceeds the target depth range, it indicates that this pixel is still in the visually sensitive area. Even if its parallax does not trigger the adjustment condition of step 54, it is necessary to return to step 54 to recompress the parallax deviation value of this pixel again to obtain a finer correction effect.

[0117] The advantages of this dual determination mechanism are as follows:

[0118] Enhance the robustness of parallax control: not only judge the adjustment requirement based on the parallax magnitude, but also introduce the depth perception factor to avoid stereo illusion distortion caused by depth mutation;

[0119] Improve visual consistency and comfort: actively optimize the parallax distribution in the shallow depth of field or depth edge area to reduce artifacts such as ghosting and jumping;

[0120] Achieve adaptive closed-loop adjustment: ensure that key pixel points are always within the optimal parallax range through the feedback mechanism, thereby improving the overall quality and user viewing experience of the naked-eye 3D display.

[0121] Select a prominent object at the exact center (i.e., on the axis) of the display as the reference point. This object is usually the position that the audience is most likely to focus on because it is at the visual focus of the screen. For example, in a 3D movie or game, design some key elements to appear in the center of the picture, and the depth values of these elements are used as the reference depth values.

[0122] Perform statistical analysis on all input perspective images, obtain the maximum depth value and the minimum depth value of all output perspective images and take the difference to get the maximum depth value zmax. The target depth range is k times the maximum depth value, 0 < k < 1. Preferably, k = 0.6. The target depth range is (0, zmax].

[0123] According to the target depth range, the adjustment deviation value, and the absolute difference, correct the adjustment deviation value to obtain the corrected deviation value.

[0124] X i，j '' =X i，j ' × (1 - b / zmax × |z - z0|); X i，j'' is the jth pixel (x j ,y j ) is the correction deviation value at depth z, zmax is the maximum depth value, z0 is the reference depth value, b is the sensitivity coefficient, 0 <b<1。

[0125] b is closely related to the distance between the actual depth z and the reference depth z0 of a pixel. When z is close to z0, i.e., when |z−z0| is small, a smaller b value should be chosen to maintain detail and clarity in the displayed area, resulting in a milder correction. Conversely, if z is farther from z0, a larger b value can be chosen to allow for more correction to accommodate different depth levels.

[0126] This correction method incorporates depth information on top of parallax adjustment, achieving refined control of pixel-level parallax distribution in naked-eye 3D displays. By dynamically adjusting the parallax deviation of each pixel to align it with the overall scene depth structure, it enhances stereoscopic realism and viewing comfort. After pre-compression, the adjusted deviation preserves the overall trend of the original parallax. Further local correction is performed based on the distance between the pixel's depth z and its reference depth z0, appropriately compressing the parallax of pixels further from the reference depth. This prevents visual discomfort caused by depth jumps or sudden parallax changes.

[0127] Step 56 : Based on the correction deviation value, return to step 3 and step 4 to obtain a multi-input viewing angle updated image sequence and display the optimized display content in 3D in the human eye area.

[0128] Based on the currently calculated correction deviation value, return to step 3 to step 4, and re-perform pixel-level synthesis and parallax adjustment on the multi-view input image sequence to generate optimized 3D display content. The specific implementation method is: call the parallax deviation value of the target pixel point in each input perspective image, and replace it with the correction deviation value as the new parallax offset; then, reposition the position of the target pixel point in the output image according to the deviation value, and re-copy the color value of the pixel to complete the image remapping and resynthesis. By executing this process pixel by pixel and perspective by perspective, a set of new image sequences optimized for depth perception is finally generated, so that the synthesized 3D picture can more naturally adapt to the depth distribution of the human eye viewing area, thereby improving the realism and comfort of naked-eye 3D display.

[0129] In order to implement the above embodiments, the present application also proposes a naked-eye 3D display system based on parallax compensation. Figure 2 This is a structural diagram of a naked-eye 3D display system based on parallax compensation provided in an embodiment of the present application. Figure 2As shown, the naked-eye 3D display system based on parallax compensation may include:

[0130] Image processing module: used to obtain original images and depth images of multiple viewing angles in the target scene and perform preprocessing operations;

[0131] A disparity offset value determination module: connected to the image processing module, used to determine the disparity offset value of each pixel in the pre-processed original image and the depth image, and obtain a disparity offset matrix;

[0132] A multi-input view image sequence determination module is connected to the parallax offset value determination module and is used to perform multi-view image synthesis on the pre-processed original image based on the parallax offset matrix to obtain a multi-input view image sequence;

[0133] 3D display module: connected to the multi-input view image sequence determination module, used to perform spatial light modulation on the multi-input view image sequence through a cylindrical lens array, and project the multi-input view images in the multi-input view image sequence after spatial light modulation onto the corresponding human eye areas for 3D display;

[0134] Display content optimization module: connected to the multi-input view image sequence determination module, used to dynamically adjust the disparity deviation value of the multi-input view image in the human eye area and update the multi-input view image sequence to optimize the display content.

[0135] In order to implement the above embodiment, the present application also proposes an electronic device. Figure 3 , Figure 3 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 3 As shown, the electronic device 500 includes: a processor 501, and a memory 502 communicatively connected to the processor 501; the memory 502 stores computer-executable instructions; the processor 501 executes the computer-executable instructions stored in the memory to implement the method provided in the aforementioned embodiment.

[0136] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0137] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0138] In the descriptions of the foregoing embodiments, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0139] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0140] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0141] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" is any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (not exhaustive) of computer-readable media include: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0142] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logical functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0143] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0144] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0145] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A naked-eye 3D display method based on parallax compensation, characterized in that: The following steps are involved: Step 1: Obtain original images and depth images of multiple viewing angles under the target scene and perform preprocessing operations; Step 2: determining the disparity offset value of each pixel in the pre-processed original image and the depth image, and obtaining a disparity offset matrix; Step 3: Based on the disparity offset matrix, perform multi-view image synthesis on the pre-processed original image to obtain a multi-input view image sequence; Step 31: Obtain the disparity deviation value of the current pixel in the pre-processed original image of the current viewing angle from the disparity offset matrix; and obtain the current input preset pixel according to the disparity deviation value of the current pixel and the current pixel. Step 32: In each pre-processed original image of each viewing angle, select the current pixel as the center and construct a k×k neighborhood window from the center; Step 33, searching for a neighborhood pixel point in the neighborhood window as a target pixel point, and replacing the current input preset pixel point with the target pixel point; Step 34, repeatedly executing steps 31 to 33, replacing all current preset pixel points to obtain the current input perspective image; Step 35, repeatedly executing step 34 to obtain a multi-view image sequence; Step 4: spatially modulating the multi-view image sequence through a cylindrical lens array, and projecting the multi-view images in the multi-view image sequence after spatial light modulation onto the human eye area for 3D display; Step 5: Dynamically adjust the disparity deviation value of the multi-input view images in the human eye area and update the multi-input view image sequence to optimize the display content; Step 54: determine whether the current input view image in the multi-view image sequence requires disparity compression. If so, compress the disparity deviation value of the current target pixel in the current input view image to obtain an adjusted deviation value. If not, proceed to step 55. Step 541, obtaining the maximum disparity deviation value of all target pixels in the current input perspective image; Step 542: Obtain the user's viewing comfort level; Step 543: Preset a first threshold and a second threshold; Step 544: If the maximum disparity deviation value is greater than the first threshold, or the viewing comfort level is greater than the second threshold, the disparity deviation value of the current target pixel in the current input viewing angle image is compressed to obtain an adjusted deviation value; if not, proceed to step 55; Step 55: Obtain the depth value of the current target pixel in the current input perspective image, preset a reference depth value and a target depth range, calculate the absolute difference between the depth value and the reference depth value, and compare the absolute difference with the target depth range. If the absolute difference is greater than the target depth range, return to step 54 to compress the parallax deviation value of the current target pixel to obtain an adjusted deviation value; if not, correct the adjusted deviation value to obtain a corrected deviation value. Step 56 : Based on the correction deviation value, return to step 3 and step 4 to obtain a multi-input viewing angle updated image sequence and display the optimized display content in 3D in the human eye area.

2. The method according to claim 1, characterized in that Determining the disparity offset value of each pixel in the preprocessed original image and the depth image includes the following steps: Step 21: Set the reference viewpoint to be on the central axis of the display, and define the angle between the i-th viewing angle and the reference viewpoint as θ i ; Step 22: Substitute the angle between the i-th viewing angle and the reference viewpoint into the parallax offset model to obtain the parallax deviation values ​​of the pixels in the original image of the i-th viewing angle at different depths.

3. The method according to claim 1, characterized in that The pixel whose disparity deviation value between the neighborhood window and the current pixel is in the same direction and whose grayscale gradient change is the smallest is taken as the target pixel; A positive parallax deviation value indicates a direction to the right, and a negative parallax deviation value indicates a direction to the left.

4. The method according to claim 1, wherein Dynamically adjust the disparity deviation value of multi-view images within the human eye area and update the multi-view image sequence to optimize the display content, including: Step 51: Collect feature point information of the user's face and perform three-dimensional reconstruction to obtain the spatial position coordinates of the user's face; Step 52: determine whether the user is in the valid viewing area based on the spatial position coordinates; if not, execute step 53; if yes, execute step 54; In step 53 , the multi-view image sequence is switched to 2D display, and a user prompt is generated synchronously to guide the user to return to the valid area. When it is detected that the user has returned to the valid area, the 3D display mode is automatically restored, and step 54 is executed.

5. The method according to claim 1, wherein The adjustment deviation value is obtained by the disparity deviation value of the current target pixel point in the current input perspective image, the first threshold, the second threshold and the maximum disparity deviation value.

6. The method according to claim 5, characterized in that The target depth range is k times the maximum depth value, <k<1。 7. The method according to claim 6, characterized in that The adjusted deviation value is corrected according to the target depth range, the adjusted deviation value and the absolute difference value to obtain a corrected deviation value.

8. The method according to claim 5, characterized in that The first threshold is 1% times the width of the display.

Images