Data processing method and apparatus for naked-eye 3D display

Abstract

The application discloses a data processing method and device for naked-eye 3D display. The method comprises the following steps: obtaining to-be-processed data, wherein the to-be-processed data comprises source data output by a received host computer; performing virtual viewpoint generation processing on the to-be-processed data based on depth information to obtain multi-viewpoint image data; performing viewpoint optimization processing on the multi-viewpoint image data based on a human eye viewing model to obtain optimized viewpoint set data; and performing raster mapping processing on the optimized viewpoint set data to obtain naked-eye 3D display data, wherein the naked-eye 3D display data is data for representing image frames after raster mapping. The source data is received at a display end side, naked-eye 3D display data is generated after raster mapping rendering of the source data, and complete link management of naked-eye 3D data display is realized.

Description

Technical Field

[0001] This application relates to the field of 3D image display, and more specifically, to a data processing method and apparatus for naked-eye 3D display. Background Technology

[0002] With the continuous development of technology, naked-eye 3D technology has emerged as an eye-catching technology, allowing users to view 3D images or videos without wearing any glasses or devices. The core principle is to project the images of the left and right eyes onto the corresponding eyeballs through optical elements such as gratings, lens arrays, or parallax barriers, and use the visual difference of the human brain to form a stereoscopic effect.

[0003] In existing technologies, glasses-free 3D display systems primarily rely on high-configuration PC hosts (typically equipped with RTX series discrete graphics cards). The PC host undertakes all the tasks in the entire chain: from building the 3D scene to generating multi-viewpoint images, and then to the most computationally intensive raster mapping. Finally, the PC sends a "composite image with completed raster arrangement" to the monitor via an HDMI / DP cable.

[0004] In existing naked-eye 3D display systems, the transmitted image is the result of raster mapping. The applicant discovered that the above-mentioned result image is composed of a large number of sub-pixel-level fragments, which requires extremely high image integrity. Such images cannot be compressed by conventional video. When compressed by encoding such as H.264 / H.265 (e.g., through network transmission or wireless projection), the compression algorithm will misjudge the fine raster arrangement as noise and smear it, causing the 3D effect at the receiving end to completely fail, resulting in serious crosstalk, ghosting and other problems.

[0005] Therefore, existing glasses-free 3D displays suffer from poor display quality. Summary of the Invention

[0006] The main objective of this application is to provide a data processing method and apparatus for naked-eye 3D display, so as to solve the technical problem that naked-eye 3D display has poor display effect in the prior art.

[0007] To achieve the above objectives, a first aspect of this application proposes a data processing method for glasses-free 3D display, applied to a glasses-free 3D display system, to generate 3D display data from received source data. The method includes: Acquire data to be processed, wherein the data to be processed includes source data received from the host computer output; The data to be processed is subjected to virtual viewpoint generation processing based on depth information to obtain multi-viewpoint image data; The multi-viewpoint image data is subjected to viewpoint optimization processing based on the human eye viewing model to obtain optimized viewpoint set data; The optimized viewpoint set data is subjected to raster mapping processing to obtain naked-eye 3D display data, wherein the naked-eye 3D display data is data used to represent image frames after raster mapping.

[0008] Furthermore, the data to be processed undergoes virtual viewpoint generation processing based on depth information to obtain multi-viewpoint image data, including: The data to be processed is subjected to recognition processing based on the source data to obtain 2D image data and depth map data; The 2D image data and the depth map data are subjected to depth-based image deformation processing to obtain image deformation data; The image deformation data is subjected to hole-filling-based image optimization processing to obtain the multi-viewpoint image data.

[0009] Furthermore, the image deformation data is subjected to hole-filling-based image optimization processing to obtain the multi-viewpoint image data, which includes: Hole detection processing is performed on the image deformation data to obtain hole region data, wherein the hole region data is data used to represent the hole region caused by image deformation; The data of the void region is subjected to a first void filling process to obtain a first filling result data, wherein the first void filling process is a void region scanning and filling. The first filling result data is subjected to a second hole filling process to obtain the second filling result data, wherein the second hole filling process is background filling of the hole area; The second filling result data is subjected to a third hole filling process to obtain the multi-view image data, wherein the third hole filling process is edge repair of the hole region.

[0010] Furthermore, the multi-viewpoint image data undergoes viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data, including: The data to be processed is subjected to identification processing based on viewing parameters to obtain viewing parameter data, wherein the viewing parameter data is parameter data used to represent the user's viewing distance and the user's viewing angle; The viewing parameter data is processed by viewpoint weight allocation based on the human eye viewing model to obtain viewpoint weight allocation data; The multi-viewpoint image data and the viewpoint weight allocation data are subjected to viewpoint optimization processing to obtain the optimized viewpoint set data, wherein the optimized viewpoint set data is a set of data used to represent the viewpoints after weight allocation.

[0011] Furthermore, the multi-viewpoint image data undergoes viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data, including: The viewing parameter data is identified and processed to obtain multi-user viewing parameter data, wherein the multi-user viewing parameter data is data used to represent the viewing parameters corresponding to multiple users respectively; The multi-user viewing parameter data is processed to obtain equivalent viewing parameter data; The equivalent viewing parameter data is processed by viewpoint weight allocation based on the human eye viewing model to obtain viewpoint weight allocation data; The multi-viewpoint image data and the viewpoint weight allocation data are subjected to viewpoint optimization processing to obtain the optimized viewpoint set data.

[0012] Furthermore, the optimized viewpoint set data is subjected to raster mapping processing to obtain naked-eye 3D display data, including: The data to be processed is subjected to grating parameter-based identification processing to obtain grating parameter data, wherein the grating parameter data is data used to represent the physical parameters of the grating lens; The grating parameter data is processed by geometric mapping model construction to obtain a grating mapping model, wherein the grating mapping model is a model used to represent the mapping relationship between screen pixels and virtual viewpoints; The optimized viewpoint set data is subjected to screen point inverse mapping processing based on the raster mapping model to obtain screen point pixel data, wherein the screen point pixel data is pixel data used to represent the virtual viewpoint corresponding to the screen point. The naked-eye 3D display data is obtained based on the screen pixel data.

[0013] Furthermore, the optimized viewpoint set data is subjected to inverse screen point mapping processing based on the raster mapping model to obtain screen point pixel data, including: Obtain display screen point data; The pixel space discretization processing is performed on the display screen point data to obtain sub-pixel data; Each sub-pixel data is subjected to viewpoint mapping processing based on the raster mapping model to obtain display viewpoint image data, wherein the display viewpoint image data is image data used to represent the viewpoint where the sub-pixel data should be displayed; Color extraction processing is performed on the displayed viewpoint image data to obtain the screen point pixel data.

[0014] According to a second aspect of this application, a data processing device for glasses-free 3D display is proposed, applied to a glasses-free 3D display system, to generate 3D display data from received source data, including: A data acquisition module is used to acquire data to be processed, wherein the data to be processed includes source data received from the host computer output. The parallax synthesis module is used to perform virtual viewpoint generation processing based on depth information on the data to be processed to obtain multi-viewpoint image data; The viewpoint optimization module is used to perform viewpoint optimization processing on the multi-viewpoint image data based on the human eye viewing model to obtain optimized viewpoint set data; The raster mapping module is used to perform raster mapping processing on the optimized viewpoint set data to obtain naked-eye 3D display data, wherein the naked-eye 3D display data is data used to represent image frames after raster mapping.

[0015] According to a third aspect of this application, a glasses-free 3D display system is proposed, characterized in that it comprises: a host computer, a raster mapping rendering module, and a glasses-free 3D display screen, wherein, The host computer generates source data and encodes the source data before transmitting it to the raster mapping rendering module. The raster mapping rendering module executes the data processing method described above for naked-eye 3D display; The naked-eye 3D display screen receives naked-eye 3D display data output by the raster mapping rendering module to achieve a naked-eye 3D effect.

[0016] According to a fourth aspect of this application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores computer instructions for causing the computer to perform the above-described data processing method for naked-eye 3D display.

[0017] The technical solutions provided by the embodiments of this application may include the following beneficial effects: In this application, data to be processed is acquired, including source data received from a host computer. The data to be processed undergoes virtual viewpoint generation processing based on depth information to obtain multi-viewpoint image data. The multi-viewpoint image data undergoes viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data. The optimized viewpoint set data undergoes raster mapping processing to obtain glasses-free 3D display data, wherein the glasses-free 3D display data represents image frames after raster mapping. By receiving source data at the display end and generating glasses-free 3D display data after raster mapping rendering of the source data, complete link management for glasses-free 3D data display is achieved. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A flowchart of a data processing method for glasses-free 3D display provided in this application; Figure 2 A flowchart of a data processing method for glasses-free 3D display provided in this application; Figure 3 A flowchart of a data processing method for glasses-free 3D display provided in this application; Figure 4 A flowchart of a data processing method for glasses-free 3D display provided in this application; Figure 5 This is a schematic diagram of a data processing device for glasses-free 3D display provided in this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0022] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0023] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0024] Explanation of related terms Source data refers to raw image data that conforms to human visual habits, such as 2D color images, depth maps, left and right eye views, and multi-viewpoint sequences. Its characteristics include high spatial correlation between pixels, making it suitable for video encoding and compression, and compression does not affect subsequent 3D processing.

[0025] FPGA (Field Programmable Gate Array) is an integrated circuit device whose internal logic functions can be configured through programming. Compared to fixed-function chips, FPGAs have advantages such as reconfigurability, shorter development cycles, and higher flexibility.

[0026] In existing technologies, PC-based 3D display systems suffer from technical problems such as image compression leading to 3D effect failure on the display side; they also rely on x86 architecture PCs, resulting in large size and high power consumption, which limits the commercial application of glasses-free 3D; and the rendering algorithms run on general-purpose GPUs, making it impossible to perform in-depth optimization for the specific needs of glasses-free 3D rasterization. Therefore, this application is proposed to address the aforementioned problems in existing technologies.

[0027] In an optional embodiment of this application, a data processing method for naked-eye 3D display is proposed and applied to a naked-eye 3D display system. The naked-eye 3D display system includes a host computer, raster mapping rendering hardware, and a raster lens-type naked-eye 3D display screen. The raster mapping rendering hardware is developed based on FPGA to decode and render the source data transmitted from the host computer, convert it into a raster mapping image, and display it on the 3D display screen. The raster mapping rendering hardware executes the data processing method for naked-eye 3D display proposed in this application.

[0028] In an optional embodiment of this application, a data processing method for glasses-free 3D display is proposed and applied to a glasses-free 3D display system to generate 3D display data from received source data. Figure 1 A flowchart of a data processing method for glasses-free 3D display provided in this application is shown below. Figure 1 As shown, the method includes the following steps: S101: Obtain the data to be processed; The data to be processed includes the source data received from the host computer, which can be a PC, server, cloud, etc., to generate and encode the source data for transmission. The source data format can be a spliced ​​image of "2D RGB video + Depth grayscale video", a "left-right eye (SBS)" stereoscopic format, or a multi-view sequence. In the optional embodiment of this application, since the transmitted data is source data, its pixel arrangement conforms to the visual laws of the human eye and has strong resistance to compression. Even when transmitted via ordinary WiFi, gigabit network, or public network, the image quality loss after encoding and decoding will not damage the physical structure of the 3D display and will not affect the local rendering and display of 3D content. The data to be processed also includes the optical parameters of the display screen, such as the lens period, tilt correction, and RGB sub-pixel offset of the current display screen; and the viewing requirements parameters of the user end, such as the number of users watching and the position of the users watching.

[0029] S102: Perform virtual viewpoint generation processing based on depth information on the data to be processed to obtain multi-viewpoint image data; In an optional embodiment of this application, a data processing method for glasses-free 3D display is proposed. Figure 2 A flowchart of a data processing method for glasses-free 3D display provided in this application is shown below. Figure 2 As shown, the method includes the following steps: S201: Perform recognition processing on the data to be processed based on the source data to obtain 2D image data and depth map data; S202: Perform depth-based image deformation processing on 2D image data and depth map data to obtain image deformation data; Real-time parallax calculation was performed on the 2D image data and depth map data. ,in, For parallax displacement, Z is the disparity coefficient, Z is the depth value, and Z0 is the reference depth. Based on the real-time disparity calculation, coordinate mapping transformation is performed through bilinear interpolation to realize the transfer of pixels to the corresponding positions in the virtual viewpoint image.

[0030] S203: Perform image optimization processing based on hole filling on the image deformation data to obtain multi-viewpoint image data.

[0031] In an optional embodiment of this application, during the above-mentioned image deformation process, due to pixel transfer, there may be empty or exposed areas in the original position, or cracks may be generated due to stretching. Hole detection is performed on the image deformation data and the empty areas are filled.

[0032] In an optional embodiment of this application, a data processing method for glasses-free 3D display is proposed, comprising: Hole detection processing is performed on the image deformation data to obtain hole region data, wherein the hole region data is used to represent the hole regions caused by image deformation; a first hole filling processing is performed on the hole region data to obtain a first filling result data, wherein the first hole filling processing is hole region scanning filling; a second hole filling processing is performed on the first filling result data to obtain a second filling result data, wherein the second hole filling processing is hole region background filling; a third hole filling processing is performed on the second filling result data to obtain multi-view image data, wherein the third hole filling processing is hole region edge repair.

[0033] In an optional embodiment of this application, the void filling process for the void region data includes: performing void type-based identification processing on the void region data to obtain first void region data, second void region data, and third void region data, wherein the first void region is a small, continuous crack void region, the second void region is a large-area void region, and the third void region is an edge-type void region; performing scan filling processing on the first void region data by scanning left and right to complete the first void; performing background filling processing on the second void region data by using pixels with large depth values ​​(background) to fill the void region; and performing edge repair filling on the third void region by using the Sobel operator to detect edges and smoothing the filled boundary.

[0034] In some optional embodiments of this application, the above-mentioned filling quality is evaluated.

[0035] Where a, b, and c are weighting coefficients; smoothness: the smoothness of the image region; edge_consistency: edge consistency; color_consistency: color consistency.

[0036] In some optional embodiments of this application, a data processing method for glasses-free 3D display is proposed, which performs virtual viewpoint parallel generation processing based on depth information on the data to be processed to obtain multi-viewpoint image data, including: The process involves: performing source image-based recognition on the data to be processed to obtain source image data; performing pixel-block processing on the source image data to obtain multiple blocks of data to be processed; performing depth-based image deformation processing on each block of data to obtain block image deformation data; performing hole-filling-based image optimization processing on the block image deformation data to obtain block viewpoint image data; and merging the block viewpoint image data to obtain multi-viewpoint image data.

[0037] S103: Perform viewpoint optimization processing on the multi-viewpoint image data based on the human eye viewing model to obtain optimized viewpoint set data; In an optional embodiment of this application, a data processing method for glasses-free 3D display is proposed. Figure 3 A flowchart of a data processing method for glasses-free 3D display provided in this application is shown below. Figure 3 As shown, the method includes the following steps: S301: Perform viewing parameter-based recognition processing on the data to be processed to obtain viewing parameter data; Viewing parameter data refers to parameter data that includes the user's viewing distance and viewing angle; S302: Perform viewpoint weight allocation processing on the viewing parameter data based on the human eye viewing model to obtain viewpoint weight allocation data; The viewing parameters mentioned above are processed by calculating viewpoint weights using a multidimensional comfort model. The viewing coefficient is then evaluated, and the multidimensional comfort model score represents the viewpoint weight corresponding to that viewing parameter, i.e., the weight of the viewpoint at that location. The multidimensional comfort model formula is as follows:

[0038] in, The distance is a Gaussian weight; The horizontal angle cosine weight; The weight is the cosine of the vertical angle. It is the user's viewing distance. It refers to the range of horizontal viewing angles for the user. It refers to the range of vertical viewing angles for the user.

[0039] S303: Perform viewpoint optimization processing on multi-viewpoint image data and viewpoint weight allocation data to obtain optimized viewpoint set data.

[0040] The optimized viewpoint set data is used to represent the set data of viewpoints after weight allocation.

[0041] In an optional embodiment of this application, the multi-viewpoint image data undergoes viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data, including: The viewing parameter data is identified and processed to obtain multi-user viewing parameter data, which represents the viewing parameters corresponding to multiple users respectively; the multi-user viewing parameter data is then processed to obtain equivalent viewing parameter data. The viewing parameter data of multiple users is processed to achieve equivalent viewing parameters through a multi-user weight superposition algorithm. The multi-user viewing parameter data includes first-user viewing parameter data, second-user viewing parameter data, and third-user viewing parameter data. The first-user viewing parameter data, second-user viewing parameter data, and third-user viewing parameter data are respectively processed by weight allocation based on the aforementioned multi-dimensional comfort model to obtain first-user viewpoint weight allocation data, second-user viewpoint weight allocation data, and third-user viewpoint weight allocation data. Optionally, by calculating viewpoint weight allocation for multiple viewing positions, multiple viewpoint weight allocation data corresponding to multiple viewing positions are obtained. A lookup table is constructed based on these multiple viewing positions and their corresponding multiple viewpoint weight allocation data. The lookup table is used to find the viewpoint weight allocation data corresponding to a viewing position, reducing the amount of real-time computational data in the system and improving data processing efficiency.

[0042] The equivalent viewing parameter data is processed by viewpoint weight allocation based on the human eye viewing model to obtain viewpoint weight allocation data; the multi-viewpoint image data and viewpoint weight allocation data are processed by viewpoint optimization to obtain optimized viewpoint set data.

[0043] S104: Perform raster mapping processing on the optimized viewpoint set data to obtain naked-eye 3D display data.

[0044] The data for naked-eye 3D display is used to represent the image frames after raster mapping.

[0045] In an optional embodiment of this application, a data processing method for glasses-free 3D display is proposed. Figure 4 A flowchart of a data processing method for glasses-free 3D display provided in this application is shown below. Figure 4 As shown, the method includes the following steps: S401: Perform grating parameter-based identification processing on the data to be processed to obtain grating parameter data; The grating parameter data is used to represent the physical parameters of the grating lens, which include: tilt angle Φ, lens period (Pitch), and sub-pixel arrangement.

[0046] In an optional embodiment of this application, by setting up FPGA-based raster mapping rendering hardware, the optical parameters of the local 3D display device are automatically adapted, eliminating the need for customized output from the host computer for a specific screen. This improves the versatility of naked-eye 3D display devices and achieves adaptability in various application scenarios by acquiring the optical parameters of different local terminal devices.

[0047] S402: Perform geometric mapping model construction on the grating parameter data to obtain the grating mapping model; The raster mapping model is a model used to represent the mapping relationship between screen pixels and virtual viewpoints; a geometric mapping model is established through the above-mentioned raster tilt angle Φ, lens period and sub-pixel arrangement structure.

[0048] In glasses-free 3D displays, the pitch of the grating lens is extremely small (typically covering only 4-8 sub-pixels). The tiny spatial distance (approximately 1 / 3 pixel width) between the red (R), green (G), and blue (B) sub-pixels leads to significant differences in the angle of light refraction. By treating these three sub-pixels (R, G, B) as independent geometric coordinate points (x, y), and setting sub-pixel offsets: R = x + 0, G = x + 0.33, B = x + 0.66; and by projecting the screen coordinates onto an axis perpendicular to the lens's orientation using the aforementioned grating tilt angle, the vertical distance of this pixel from the "zero point" can be calculated. Using the lens period described above, calculate the relative position of the distance N within the lens. .

[0049] In an optional embodiment of this application, a coordinate mapping lookup table (LUT) can also be generated based on the above geometric mapping model. Based on the physical parameters of the above grating, the phase value of each sub-pixel relative to the lens is calculated. This phase value is directly mapped to the virtual viewpoint space to determine which viewpoint (View ID) image data the sub-pixel should display. By pre-calculating the mapping relationship between screen pixel points and viewpoints, the real-time computing workload can be reduced during 3D display operation.

[0050] S403: Perform screen point inverse mapping processing based on the raster mapping model on the optimized viewpoint set data to obtain screen point pixel data; Screen point pixel data is the pixel data used to represent the virtual viewpoint corresponding to the screen point; After determining the viewpoint corresponding to the screen pixel through the above-mentioned screen pixel inverse mapping, the color value is extracted from the source image, and the texture map coordinates of the source image are calculated.

[0051] S404: Obtain naked-eye 3D display data based on screen pixel data.

[0052] Based on the calculated viewpoint and texture map coordinates of the screen position points, the data is mapped back to pixels. Bilinear interpolation is used, and a weighted average of the four source pixel values ​​surrounding the floating-point coordinates is performed to calculate the final output color. Simultaneously, adaptive filtering based on local contrast is used to smooth high-frequency noise. In an optional embodiment of this application, this is equivalent to passing a low-pass filter while reconstructing the image, effectively suppressing the generation of moiré patterns.

[0053] In an optional embodiment of this application, a data processing method for glasses-free 3D display is proposed, comprising: Acquire display screen point data; perform pixel space discretization on the display screen point data to obtain sub-pixel data; perform viewpoint mapping processing based on a raster mapping model on each sub-pixel data to obtain display viewpoint image data, wherein the display viewpoint image data is image data used to represent the viewpoint where the sub-pixel data should be displayed; perform color extraction processing on the display viewpoint image data to obtain screen point pixel data.

[0054] In another optional embodiment of this application, the source image is preprocessed with multi-level resolution maps; when the identified image has high-frequency texture, the preprocessed low-resolution small image is read; and the low-resolution small image is subjected to raster rendering mapping processing to reduce the generation of moiré patterns.

[0055] In an optional embodiment of this application, a data processing device for glasses-free 3D display is characterized in that it is applied to a glasses-free 3D display system to generate 3D display data from received source data. Figure 5 A schematic diagram of a data processing device for glasses-free 3D display provided in this application includes: The data acquisition module 51 is used to acquire data to be processed, wherein the data to be processed includes source data received from the host computer output. The parallax synthesis module 52 is used to perform virtual viewpoint generation processing based on depth information on the data to be processed to obtain multi-viewpoint image data. The viewpoint optimization module 53 is used to perform viewpoint optimization processing on the multi-viewpoint image data based on the human eye viewing model to obtain optimized viewpoint set data; The raster mapping module 54 is used to perform raster mapping processing on the optimized viewpoint set data to obtain naked-eye 3D display data, wherein the naked-eye 3D display data is data used to represent image frames after raster mapping.

[0056] In an optional embodiment of this application, a glasses-free 3D display system is proposed, comprising: a host computer, a raster mapping rendering module, and a glasses-free 3D display screen, wherein the host computer generates source data and encodes the source data and transmits it to the raster mapping rendering module; the raster mapping rendering module executes the aforementioned data processing method for glasses-free 3D display; and the glasses-free 3D display screen receives the glasses-free 3D display data output by the raster mapping rendering module to achieve a glasses-free 3D effect.

[0057] The specific methods of execution of each unit in the above embodiments have been described in detail in the embodiments of the method, and will not be elaborated here.

[0058] In this application, data to be processed is acquired, including source data received from a host computer. The data to be processed undergoes virtual viewpoint generation processing based on depth information to obtain multi-viewpoint image data. The multi-viewpoint image data undergoes viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data. The optimized viewpoint set data undergoes raster mapping processing to obtain glasses-free 3D display data, wherein the glasses-free 3D display data represents image frames after raster mapping. By receiving source data at the display end and generating glasses-free 3D display data after raster mapping rendering of the source data, complete link management for glasses-free 3D data display is achieved.

[0059] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0060] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0061] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A data processing method for glasses-free 3D display, characterized in that, A method for generating 3D display data from received source data, applicable to glasses-free 3D display systems, includes: Acquire data to be processed, wherein the data to be processed includes source data received from the host computer output; The data to be processed is subjected to virtual viewpoint generation processing based on depth information to obtain multi-viewpoint image data; The multi-viewpoint image data is subjected to viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data. This includes: performing viewpoint parameter recognition processing on the data to be processed to obtain viewpoint parameter data; performing viewpoint weight allocation processing on the viewpoint parameter data based on a human eye viewing model to obtain viewpoint weight allocation data; calculating viewpoint weights for the viewpoints using a human eye viewing model; and evaluating the viewpoint coefficients. The human eye viewing model score is used to represent the viewpoint weights corresponding to the viewpoints, i.e., the weights of the viewpoints corresponding to the viewpoints at the corresponding viewing positions. The human eye viewing model is... ,in, The distance is a Gaussian weight; The horizontal angle cosine weight; The weight is the cosine of the vertical angle. It is the user's viewing distance. It refers to the range of horizontal viewing angles for the user. It is the range of vertical viewing angles for users; viewpoint optimization processing is performed on multi-viewpoint image data and viewpoint weight allocation data to obtain optimized viewpoint set data, which is a set of data used to represent the viewpoints after weight allocation; The viewing parameter data is identified and processed to obtain multi-user viewing parameter data, which represents the viewing parameters corresponding to multiple users. The multi-user viewing parameter data is then subjected to viewing parameter equivalence processing to obtain equivalent viewing parameter data. A multi-user weight superposition algorithm is used to perform viewing parameter equivalence processing on the multi-user viewing parameter data. Viewpoint weight allocation processing based on a human eye viewing model is then applied to the equivalent viewing parameter data to obtain viewpoint weight allocation data. Finally, viewpoint optimization processing is performed on the multi-viewpoint image data and the viewpoint weight allocation data to obtain optimized viewpoint set data. The optimized viewpoint set data is subjected to raster mapping processing to obtain naked-eye 3D display data. This naked-eye 3D display data represents the image frames after raster mapping and includes: performing raster parameter-based recognition processing on the data to be processed to obtain raster parameter data; performing geometric mapping model construction processing on the raster parameter data to obtain a raster mapping model; treating the three sub-pixels (R), green (G), and blue (B) as independent geometric coordinate points (x, y), setting sub-pixel offsets: R = x + 0, G = x + 0.33, B = x + 0.66; and projecting the screen coordinates onto an axis perpendicular to the lens direction using the raster tilt angle Φ, calculating the vertical distance of the pixel from the "zero point". ; Calculate the relative position of distance N within the lens using the lens period. , The lens period is used; the optimized viewpoint set data is subjected to screen point inverse mapping based on the raster mapping model to obtain screen point pixel data.

2. The data processing method according to claim 1, characterized in that, The data to be processed is subjected to virtual viewpoint generation processing based on depth information to obtain multi-viewpoint image data, including: The data to be processed is subjected to recognition processing based on the source data to obtain 2D image data and depth map data; The 2D image data and the depth map data are subjected to depth-based image deformation processing to obtain image deformation data; The image deformation data is subjected to hole-filling-based image optimization processing to obtain the multi-viewpoint image data.

3. The data processing method according to claim 2, characterized in that, The image deformation data is subjected to hole-filling-based image optimization processing to obtain the multi-viewpoint image data, which includes: Hole detection processing is performed on the image deformation data to obtain hole region data, wherein the hole region data is data used to represent the hole region caused by image deformation; The data of the void region is subjected to a first void filling process to obtain a first filling result data, wherein the first void filling process is a void region scanning and filling. The first filling result data is subjected to a second hole filling process to obtain the second filling result data, wherein the second hole filling process is background filling of the hole area; The second filling result data is subjected to a third hole filling process to obtain the multi-view image data, wherein the third hole filling process is edge repair of the hole region.

4. The data processing method according to claim 1, characterized in that, The multi-viewpoint image data is subjected to viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data, including: The data to be processed is subjected to identification processing based on viewing parameters to obtain viewing parameter data, wherein the viewing parameter data is parameter data used to represent the user's viewing distance and the user's viewing angle; The viewing parameter data is processed by viewpoint weight allocation based on the human eye viewing model to obtain viewpoint weight allocation data; The multi-viewpoint image data and the viewpoint weight allocation data are subjected to viewpoint optimization processing to obtain the optimized viewpoint set data, wherein the optimized viewpoint set data is a set of data used to represent the viewpoints after weight allocation.

5. The data processing method according to claim 4, characterized in that, The multi-viewpoint image data is subjected to viewpoint optimization processing based on a human eye viewing model to obtain optimized viewpoint set data, including: The viewing parameter data is identified and processed to obtain multi-user viewing parameter data, wherein the multi-user viewing parameter data is data used to represent the viewing parameters corresponding to multiple users respectively; The multi-user viewing parameter data is processed to obtain equivalent viewing parameter data; The equivalent viewing parameter data is processed by viewpoint weight allocation based on the human eye viewing model to obtain viewpoint weight allocation data; The multi-viewpoint image data and the viewpoint weight allocation data are subjected to viewpoint optimization processing to obtain the optimized viewpoint set data.

6. The data processing method according to claim 1, characterized in that, The optimized viewpoint set data is subjected to raster mapping processing to obtain naked-eye 3D display data, including: The data to be processed is subjected to grating parameter-based identification processing to obtain grating parameter data, wherein the grating parameter data is data used to represent the physical parameters of the grating lens; The grating parameter data is processed by geometric mapping model construction to obtain a grating mapping model, wherein the grating mapping model is a model used to represent the mapping relationship between screen pixels and virtual viewpoints; The optimized viewpoint set data is subjected to screen point inverse mapping processing based on the raster mapping model to obtain screen point pixel data, wherein the screen point pixel data is pixel data used to represent the virtual viewpoint corresponding to the screen point. The naked-eye 3D display data is obtained based on the screen pixel data.

7. The data processing method according to claim 6, characterized in that, Performing inverse screen point mapping processing based on the raster mapping model on the optimized viewpoint set data yields screen point pixel data including: Obtain the display screen point data; The pixel space discretization processing is performed on the display screen point data to obtain sub-pixel data; Each sub-pixel data is subjected to viewpoint mapping processing based on the raster mapping model to obtain display viewpoint image data, wherein the display viewpoint image data is image data used to represent the viewpoint where the sub-pixel data should be displayed; Color extraction processing is performed on the displayed viewpoint image data to obtain the screen point pixel data.

8. A data processing device for glasses-free 3D display, characterized in that, Applied to glasses-free 3D display systems, it enables the generation of 3D display data from received source data, including: A data acquisition module is used to acquire data to be processed, wherein the data to be processed includes source data received from the host computer output. The parallax synthesis module is used to perform virtual viewpoint generation processing based on depth information on the data to be processed to obtain multi-viewpoint image data; The viewpoint optimization module is used to perform viewpoint optimization processing on the multi-viewpoint image data based on a human eye viewing model to obtain optimized viewpoint set data. This includes: performing viewing parameter recognition processing on the data to be processed to obtain viewing parameter data; performing viewpoint weight allocation processing on the viewing parameter data based on a human eye viewing model to obtain viewpoint weight allocation data; calculating viewpoint weights for the viewing parameters using the human eye viewing model; and evaluating the viewing coefficients. The human eye viewing model score is used to represent the viewpoint weights corresponding to the viewing parameters, i.e., the weights of the viewpoints corresponding to the viewing positions for the viewing parameters. The multidimensional comfort model is... ,in, The distance is a Gaussian weight; The horizontal angle cosine weight; The weight is the cosine of the vertical angle. It is the user's viewing distance. It refers to the range of horizontal viewing angles for the user. It is the range of vertical viewing angles for users; viewpoint optimization processing is performed on multi-viewpoint image data and viewpoint weight allocation data to obtain optimized viewpoint set data, which is a set of data used to represent the viewpoints after weight allocation; The viewing parameter data is identified and processed to obtain multi-user viewing parameter data, which represents the viewing parameters corresponding to multiple users. The multi-user viewing parameter data is then subjected to viewing parameter equivalence processing to obtain equivalent viewing parameter data. A multi-user weight superposition algorithm is used to perform viewing parameter equivalence processing on the multi-user viewing parameter data. Viewpoint weight allocation processing based on a human eye viewing model is then applied to the equivalent viewing parameter data to obtain viewpoint weight allocation data. Finally, viewpoint optimization processing is performed on the multi-viewpoint image data and the viewpoint weight allocation data to obtain optimized viewpoint set data. The raster mapping module is used to perform raster mapping processing on the optimized viewpoint set data to obtain naked-eye 3D display data. The naked-eye 3D display data represents the image frames after raster mapping and includes: performing raster parameter-based recognition processing on the data to be processed to obtain raster parameter data; performing geometric mapping model construction processing on the raster parameter data to obtain a raster mapping model; treating the three sub-pixels (R), green (G), and blue (B) as independent geometric coordinate points with coordinates (x, y), setting sub-pixel offsets: R = x + 0, G = x + 0.33, B = x + 0.66; and projecting the screen coordinates onto an axis perpendicular to the lens direction using the raster tilt angle Φ, calculating the vertical distance of the pixel from the "zero point". ; Calculate the relative position of distance N within the lens using the lens period. , The lens period is used; the optimized viewpoint set data is subjected to screen point inverse mapping based on the raster mapping model to obtain screen point pixel data.

9. A glasses-free 3D display system, characterized in that, include: The host computer, raster mapping rendering module, and naked-eye 3D display screen are included. The host computer generates source data and encodes the source data before transmitting it to the raster mapping rendering module. The raster mapping rendering module performs the data processing method for naked-eye 3D display as described in any one of claims 1-7; The naked-eye 3D display screen receives naked-eye 3D display data output by the raster mapping rendering module to achieve a naked-eye 3D effect.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the data processing method for naked-eye 3D display as described in any one of claims 1-7.

Images