Display driving method and system of three-dimensional image and computer equipment

Abstract

The invention discloses a display driving method and system for a three-dimensional image and computer equipment, and belongs to the technical field of display. The method comprises the following steps: acquiring a three-dimensional film source, wherein the three-dimensional film source comprises a plurality of two-dimensional images for displaying three-dimensional images; pixels of the plurality of two-dimensional images are rearranged by taking a row as a unit to obtain a plurality of sub-unit images, and any sub-unit image in the plurality of sub-unit images comprises the same row of pixels in the plurality of two-dimensional images; performing coding compression on the plurality of subunit images to obtain a plurality of row compression code streams; and sending the plurality of row compression code streams to the display device, wherein the plurality of row compression code streams are used for the display device to drive and display the three-dimensional image. According to the method, a plurality of two-dimensional images are rearranged by taking a row as a unit, so that the correlation of pixels in subunit images is relatively high, and the compression rate of compression coding is improved.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a method, system, and computer device for driving the display of three-dimensional images. Background Technology

[0002] In the field of 3D image display technology, the system needs to transmit 3D source material to the display end, which then drives the display of 3D images based on the source material. Because the 3D source material has a large data volume and the data transmission interface bandwidth is low, directly transmitting the 3D source material takes a long time, resulting in a low frame rate for displaying the 3D image.

[0003] Therefore, improving the transmission efficiency of 3D source material is an urgent problem to be solved in the process of driving the display of 3D images. Summary of the Invention

[0004] This application provides a method, system, and computer device for displaying three-dimensional images to improve the transmission efficiency of three-dimensional video sources. The technical solution is as follows:

[0005] On the one hand, a method for driving the display of a three-dimensional image is provided, the method comprising:

[0006] Acquire a three-dimensional source image, which includes multiple two-dimensional images for displaying a three-dimensional image;

[0007] The pixels of the multiple two-dimensional images are rearranged in rows to obtain multiple sub-unit images, wherein any sub-unit image in the multiple sub-unit images includes the same row of pixels in the multiple two-dimensional images;

[0008] The multiple sub-unit images are encoded and compressed to obtain multiple line compressed bitstreams;

[0009] The plurality of compressed line streams are sent to the display device, and the plurality of compressed line streams are used by the display device to drive the display of the three-dimensional image.

[0010] In one possible implementation, the step of encoding and compressing the plurality of sub-unit images to obtain a plurality of line compressed bitstreams includes: for any sub-unit image among the plurality of sub-unit images, dividing the any sub-unit image into N slice images, where N is the number of decoding sub-units that the display device performs in parallel decoding; for any slice image among the N slice images, obtaining a plurality of macroblock images based on the any slice image according to the macroblock size; and encoding and compressing the plurality of slice images at the granularity of macroblock images to obtain a line compressed bitstream corresponding to the any sub-unit image.

[0011] In one possible implementation, obtaining multiple macroblock images based on any slice image according to the macroblock size includes: obtaining a completed image corresponding to any slice image based on the macroblock size, wherein the number of row pixels in the completed image is an integer multiple of the number of rows in the macroblock size, and the number of column pixels in the completed image is an integer multiple of the number of columns in the macroblock size; and splitting the completed image into multiple macroblock images according to the macroblock size.

[0012] In one possible implementation, obtaining the completed image corresponding to any slice image based on the macroblock size includes: if the difference between the number of row pixels in any slice image and an integer multiple of the number of rows in the macroblock size is L, adding L rows of pixels to the any slice image, where L is a positive integer; or, if the difference between the number of column pixels in any slice image and an integer multiple of the number of columns in the macroblock size is C, adding C columns of pixels to the any slice image, where C is a positive integer; wherein any pixel in the L rows or C columns is a default pixel or a neighboring pixel, and the neighboring pixel is a pixel in the plurality of slice images that meets the distance requirement from any pixel.

[0013] In one possible implementation, splitting the completed image into multiple macroblock images according to the macroblock size includes: splitting the completed image into multiple macroblock images arranged in a single row order according to the macroblock size; wherein, when the number of row pixels of any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images in any group arranged in a column direction in the completed image are transformed into adjacent row direction arrangements.

[0014] In one possible implementation, any one of the multiple line compressed bitstreams includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image. The data volume of any one of the multiple sub-bitstreams is less than the decoding threshold of the display device, and the decoding threshold is less than X times the data volume of the sub-bitstream, where X is any number greater than 1. The decoding threshold is the maximum amount of data that the display device can decode within the screen refresh time.

[0015] On the other hand, a method for driving the display of a three-dimensional image is also provided, the method comprising:

[0016] Receive multiple lines of compressed bitstream sent by the system device;

[0017] The multiple line compressed bitstreams are decoded, and multiple sub-unit images are obtained based on the decoding results;

[0018] The three-dimensional image is displayed based on the image drive of the multiple sub-units.

[0019] In one possible implementation, decoding the plurality of line compressed bitstreams and obtaining a plurality of sub-unit images based on the decoding results includes: for any line compressed bitstream in the plurality of line compressed bitstreams, performing parallel decoding on any line compressed bitstream through a plurality of decoding sub-units, obtaining a plurality of slice images based on the decoding results; and obtaining a sub-unit image corresponding to any line compressed bitstream based on the plurality of slice images.

[0020] In one possible implementation, the any line of compressed bitstream includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image, and each slice image corresponding to a decoding subunit. The parallel decoding of the any line of compressed bitstream by the multiple decoding subunits, and the acquisition of multiple slice images based on the decoding results, includes: sending a first sub-bitstream corresponding to a first slice image in the any line of compressed bitstream to a first decoding subunit, where the first decoding subunit is any one of the multiple decoding subunits, and the first slice image is the slice image corresponding to the first decoding subunit; decoding the first sub-bitstream by the first decoding subunit, and acquiring the first slice image based on the decoding results, wherein the multiple decoding subunits perform parallel decoding.

[0021] In one possible implementation, the step of performing parallel decoding on any row of compressed bitstream using multiple decoding subunits and obtaining multiple slice images based on the decoding results includes: performing parallel decoding on any row of compressed bitstream using multiple decoding subunits and obtaining multiple completed images based on the decoding results, wherein the number of row pixels in the completed images is an integer multiple of the number of rows in the macroblock size, and the number of column pixels in the completed images is an integer multiple of the number of columns in the macroblock size; if the multiple completed images include L rows of pixels, deleting the L rows of pixels to obtain the multiple slice images, where L is a positive integer; if the multiple completed images include C columns of pixels, deleting the C columns of pixels to obtain the multiple slice images, where C is a positive integer; wherein any pixel in the L rows or the C columns is a default pixel or a neighboring pixel, and the neighboring pixel is a pixel whose distance from any given pixel meets a distance requirement.

[0022] In one possible implementation, the step of performing parallel decoding of any line of compressed bitstream by multiple decoding subunits and obtaining multiple completed images based on the decoding results includes: performing parallel decoding of any line of compressed bitstream by multiple decoding subunits to obtain multiple sets of macroblock images arranged in a single row; for any set of macroblock images in the multiple sets of macroblock images, if the number of row pixels in any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images arranged in adjacent rows in the set of macroblock images are transformed into column-arranged images to obtain the completed image corresponding to the set of macroblock images.

[0023] On the other hand, a display driving device for three-dimensional images is also provided, the device comprising:

[0024] An acquisition module is used to acquire a three-dimensional source image, wherein the three-dimensional source image includes multiple two-dimensional images for displaying a three-dimensional image;

[0025] The rearrangement module is used to rearrange the pixels of the plurality of two-dimensional images in rows to obtain a plurality of sub-unit images, wherein any sub-unit image in the plurality of sub-unit images includes the same row of pixels in the plurality of two-dimensional images;

[0026] The compression module is used to encode and compress the multiple sub-unit images to obtain multiple lines of compressed bitstream;

[0027] The sending module is used to send the plurality of line compressed bitstreams to the display device, the plurality of line compressed bitstreams being used by the display device to drive the display of the three-dimensional image.

[0028] In one possible implementation, the compression module is configured to: divide any one of the plurality of sub-unit images into N slice images, where N is the number of decoding sub-units that the display device performs parallel decoding; for any one of the N slice images, obtain a plurality of macroblock images based on the any one slice image according to the macroblock size; and encode and compress the plurality of slice images at the granularity of macroblock images to obtain the line compressed bitstream corresponding to the any one sub-unit image.

[0029] In one possible implementation, the compression module is configured to obtain a completed image corresponding to any slice image based on the macroblock size, wherein the number of row pixels in the completed image is an integer multiple of the number of rows in the macroblock size, and the number of column pixels in the completed image is an integer multiple of the number of columns in the macroblock size; and to split the completed image into multiple macroblock images according to the macroblock size.

[0030] In one possible implementation, the compression module is configured to add L rows of pixels to any slice image if the difference between the number of rows of any slice image and an integer multiple of the number of rows of the macroblock size is L, where L is a positive integer; or, if the difference between the number of columns of any slice image and an integer multiple of the number of columns of the macroblock size is C, add C columns of pixels to any slice image, where C is a positive integer; wherein any pixel in the L rows of pixels or the C columns of pixels is a default pixel or a neighboring pixel, and the neighboring pixel is a pixel in the plurality of slice images that is at a distance from any pixel that meets the distance requirement.

[0031] In one possible implementation, the compression module is used to split the completed image into multiple macroblock images arranged in a single row order according to the macroblock size; wherein, when the number of row pixels of any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images in any group arranged in a column direction in the completed image are transformed into an adjacent row direction arrangement.

[0032] In one possible implementation, any one of the multiple line compressed bitstreams includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image. The data volume of any one of the multiple sub-bitstreams is less than the decoding threshold of the display device, and the decoding threshold is less than X times the data volume of the sub-bitstream, where X is any number greater than 1. The decoding threshold is the maximum amount of data that the display device can decode within the screen refresh time.

[0033] On the other hand, a display driving device for three-dimensional images is also provided, the device comprising:

[0034] The receiving module is used to receive multiple line compressed bitstreams sent by the system device;

[0035] The decoding module is used to decode the multiple line compressed bitstreams and obtain multiple sub-unit images based on the decoding results;

[0036] The display module is used to display three-dimensional images based on the images of the multiple sub-units.

[0037] In one possible implementation, the decoding module is configured to perform parallel decoding of any one line of the compressed bitstream from the plurality of line compressed bitstreams through a plurality of decoding sub-units, obtain a plurality of slice images based on the decoding results, and obtain a sub-unit image corresponding to the any one line of compressed bitstream based on the plurality of slice images.

[0038] In one possible implementation, any line of compressed bitstream includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image, and each slice image corresponding to a decoding sub-unit. The decoding module is configured to send a first sub-bitstream corresponding to a first slice image in any line of compressed bitstream to a first decoding sub-unit. The first decoding sub-unit is any one of the multiple decoding sub-units, and the first slice image is the slice image corresponding to the first decoding sub-unit. The first sub-bitstream is decoded by the first decoding sub-unit, and the first slice image is obtained based on the decoding result. The multiple decoding sub-units decode in parallel.

[0039] In one possible implementation, the decoding module is configured to perform parallel decoding on any row of the compressed bitstream using multiple decoding subunits, and obtain multiple completed images based on the decoding results. The number of row pixels in each completed image is an integer multiple of the number of rows in the macroblock size, and the number of column pixels in each completed image is an integer multiple of the number of columns in the macroblock size. If the multiple completed images include L rows of pixels, the L rows of pixels are deleted to obtain the multiple slice images, where L is a positive integer. If the multiple completed images include C columns of pixels, the C columns of pixels are deleted to obtain the multiple slice images, where C is a positive integer. Any pixel in the L rows or the C columns is a default pixel or a neighboring pixel, where a neighboring pixel is a pixel whose distance from any given pixel meets a distance requirement.

[0040] In one possible implementation, the decoding module is configured to perform parallel decoding on any one line of compressed bitstream through multiple decoding subunits to obtain multiple sets of macroblock images arranged in a single row; for any one set of macroblock images in the multiple sets of macroblock images, if the number of row pixels in any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images arranged in adjacent rows in the set of macroblock images are transformed into column-arranged images to obtain the complete image corresponding to the set of macroblock images.

[0041] On the other hand, a display driving system for three-dimensional images is also provided, the system including a system device and a display device; the system device is used to execute the display driving method for three-dimensional images described in one aspect above, and the display device is used to execute the display driving method for three-dimensional images described in the other aspect above.

[0042] On the other hand, a computer device is also provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement the three-dimensional image display driving method described in both aspects above.

[0043] On the other hand, a computer-readable storage medium is also provided, wherein the non-transient computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable a computer to implement the three-dimensional image display driving method described in both aspects above.

[0044] On the other hand, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the three-dimensional image display driving method described in either or both of the preceding aspects.

[0045] The technical solution provided in this application has at least the following beneficial effects:

[0046] The technical solution provided in this application rearranges multiple two-dimensional images into multiple sub-unit images on a row-by-row basis, and encodes and compresses each sub-unit image separately. Since any sub-unit image includes pixels from the same row in multiple two-dimensional images, the pixel correlation within any sub-unit image is high, resulting in a higher compression ratio and consequently a higher transmission rate of the row-compressed bitstream, thus increasing the transmission rate of the 3D source image. Therefore, a higher compression ratio reduces image loss, and a higher transmission rate reduces the latency of the row-compressed bitstream, increasing the display frame rate of the 3D image and thus improving the display quality of the 3D image. Furthermore, when the display device refreshes the 3D image row-by-row, since one row-compressed bitstream corresponds to one row of pixels in the 3D image, the display device only needs to store one row-compressed bitstream to refresh one row of pixels in the 3D image once, saving storage space. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart of a three-dimensional image display driving method provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram illustrating the interaction process between a system device and a display device according to an embodiment of this application;

[0051] Figure 4 This is a schematic diagram of a multi-view image acquisition scenario provided in an embodiment of this application;

[0052] Figure 5This is a schematic diagram of a three-dimensional viewing scene provided in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram of a single-viewpoint image rendering process provided in an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of a rearrangement process provided in an embodiment of this application;

[0055] Figure 8 This is a schematic diagram of another rearrangement process provided in an embodiment of this application;

[0056] Figure 9 This is a schematic diagram of an image slicing process provided in an embodiment of this application;

[0057] Figure 10 This is a schematic diagram illustrating a macroblock arrangement transformation process provided in an embodiment of this application;

[0058] Figure 11 This is a schematic diagram of a macroblock combination method provided in an embodiment of this application;

[0059] Figure 12 This is a schematic diagram of a bitrate control process provided in an embodiment of this application;

[0060] Figure 13 This is a schematic diagram of a dither template structure provided in an embodiment of this application;

[0061] Figure 14 This is a schematic diagram of a dither processing flow provided in an embodiment of this application;

[0062] Figure 15 This is a schematic diagram of a bitstream processing procedure provided in an embodiment of this application;

[0063] Figure 16 This is a schematic diagram of the structure of a display screen provided in an embodiment of this application;

[0064] Figure 17 This is a scene diagram illustrating the determination of a viewpoint provided in an embodiment of this application;

[0065] Figure 18 This is a logic block diagram of a three-dimensional image display driving process provided in an embodiment of this application;

[0066] Figure 19 This is a schematic diagram of the structure of a three-dimensional image display driving device provided in an embodiment of this application;

[0067] Figure 20 This is a schematic diagram of another three-dimensional image display driving device provided in an embodiment of this application;

[0068] Figure 21 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0069] Figure 22 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0071] It should be noted that the terms "first," "second," etc. (if applicable) used in the specification of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0072] 3D image display technology is a display technology capable of producing stereoscopic visual effects. The system side of 3D image display technology processes and prepares 3D source material, i.e., image data containing stereoscopic visual information. The 3D source material contains enough information to generate images from two or more different viewpoints, thus simulating human stereoscopic vision; therefore, the data volume of the 3D source material is relatively large. The display side of 3D image display technology receives the 3D source material and drives its display as a 3D image.

[0073] The data transmission interface between the system and the display has limited bandwidth, making the direct transmission of large amounts of 3D source data time-consuming. This prolonged transmission time leads to a decrease in the display frame rate of the 3D images, impacting user experience and potentially causing errors and delays during data transmission, further affecting display quality. The display frame rate, or the number of image frames displayed per second, is used to measure the display effect of 3D images. A higher frame rate results in smoother images and a stronger sense of depth.

[0074] In related technologies, when compressing 3D video sources, the multi-viewpoint images in the 3D video source need to be rearranged according to the parameters of the optical lenses on the display screen. The correlation between pixels in the rearranged images is poor, resulting in a low compression ratio, easy image loss, and ultimately poor image quality. The optical lenses on the display screen are used to change the direction of light emitted from the screen, thereby achieving 3D display.

[0075] This application provides a method for driving the display of three-dimensional images, which can improve the transmission efficiency of three-dimensional video sources and thus improve image quality. Please refer to... Figure 1 This diagram illustrates an implementation environment for the three-dimensional image display driving method provided in this embodiment. The implementation environment may include a system device 101 and a display device 102, which are connected via wired or wireless means. The display device 102 includes a display screen.

[0076] System device 101 acquires a 3D source image, rearranges the pixels of multiple 2D images in the 3D source image line by line, and compresses them to obtain a line-compressed bitstream. System device 101 sends the line-compressed bitstream to display device 102, and display device 102 drives the display of the 3D image on the display screen based on the line-compressed bitstream.

[0077] This application does not limit the type of system device 101. For example, system device 101 can be a terminal or a server. Optionally, the terminal can be any electronic product that can interact with the user through one or more methods such as a keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device, such as PC (Personal Computer), mobile phone, smartphone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car system, smart TV, smart speaker, etc. The server can be a single server, a server cluster composed of multiple servers, or a cloud computing service center.

[0078] This application does not limit the type of display screen included in the display device. For example, the display screen may be an LED (Light Emitting Diode) screen or an LCD (Liquid Crystal Display) screen.

[0079] Those skilled in the art should understand that the above-described terminals, servers, and display screens are examples, and other existing or future terminals, servers, and display screens that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0080] This application provides a method for driving the display of a three-dimensional image, illustrated by an example of a system device and a display device executing the method. The display device includes a display screen. For example, the system device can be... Figure 1 The system device 101 shown may have a display device that can be Figure 1 The display device 102 shown is as follows. Figure 2 As shown, the display driving method for the three-dimensional image includes, but is not limited to, the following steps 201-207.

[0081] Step 201: The system device acquires a three-dimensional source image, which includes multiple two-dimensional images for displaying a three-dimensional image.

[0082] The system provides the environment necessary for running an operating system and performing calculations. The system's performance and compression methods can affect the processing speed and display quality of 3D images. High-performance systems process data faster and provide a smoother user experience. See also Figure 3 The diagram illustrates the interaction process between a system device and a display device, where the system device is the system end and the display device is the display end. The system device can be a PC (Personal Computer), Mobile (Mobile Device), NB (Narrow Band Internet of Things) device, or TPC (Touch Panel Controller) device, etc.

[0083] Optionally, the system device via Figure 3 The rendering module shown acquires 3D source material. The system's rendering module is used to render and generate the 3D source material. Rendering can be performed locally using hardware devices such as GPUs (Graphics Processing Units) or CPUs (Central Processing Units). Alternatively, a standalone rendering Internet Protocol (IP) can be used to meet specific needs or optimize performance. 3D source material can also be generated in the cloud, utilizing the computing power of cloud servers to render the 3D source material, and then transmitting the rendered image to the system via the network, thereby reducing the computational burden on local devices.

[0084] A 3D source material refers to a data set containing multiple 2D images used to display 3D images. In 3D image display technology, since the display screen itself is 2D, technical means are needed to simulate and present 3D effects. For example, multiple 2D images are used to synthesize and display 3D images. This application does not limit the type of 3D source material; for example, it may include multi-viewpoint source materials, light field display source materials, holographic display source materials, and integrated imaging source materials.

[0085] Among them, multi-viewpoint sources capture views of objects from multiple different angles, allowing users to experience a more three-dimensional effect. Each viewpoint corresponds to an image, and when the images from multiple viewpoints are correctly combined and displayed, a three-dimensional visual effect can be presented. Light field display sources record and reproduce the direction and intensity of light propagation in space to generate three-dimensional images that can present a realistic sense of depth and changes in perspective. Holographic display sources record and reproduce all the information of the object's wavefront, i.e., a hologram, thus presenting a three-dimensional image of the object. Integrated imaging sources capture and display objects based on microlens arrays, thereby achieving a three-dimensional effect for naked-eye viewing.

[0086] This application does not limit the rendering method of the 3D video source. Optionally, multiple cameras can be placed according to the human eye position for conventional rendering of the 3D video source. See also Figure 4 The diagram shows a scenario for acquiring multi-view images, in which multiple cameras 401 capture images of objects from different angles. The objects captured from different angles are different. The multiple images captured by the multiple cameras 401 are the multiple two-dimensional images in the embodiments of this application, i.e., three-dimensional source images.

[0087] The R (Red), G (Green), and B (Blue) values ​​of pixels corresponding to a 3D image in a 3D display can be obtained through light field rendering. See also... Figure 5 The schematic diagram of the 3D viewing scene shown in Part A illustrates that when a user views the 3D scene, the user's position is indicated by the center of their forehead 501. A 3D display panel 502 is superimposed in front of the 3D scene, and rendering is performed based on the optical characteristics of the 3D display panel 502. As shown in Part B, the 3D display panel 502 consists of the display panel itself and a 3D optical lens attached above the display panel. The display panel is responsible for displaying the image, and the 3D optical lens is responsible for changing the direction of light propagation to simulate a 3D effect. The light emitted from each sub-pixel (i.e., a pixel on the display panel) passes through multiple ( Figure 5 (Using 11 as an example) Three-dimensional optical lenses. Three-dimensional optical lenses are used to disperse the light emitted by sub-pixels in different directions, thereby simulating the effect of viewing an object from multiple different perspectives.

[0088] The system calculates and determines the ray closest to the center of the eyebrow 501 from the rays emitted from each sub-pixel. Based on the determined direction of this ray, the system renders the 3D scene in reverse, i.e., it inversely calculates the color and brightness that should be displayed on the display panel. The system assigns the RGB values ​​obtained from the reverse-ray rendering to that sub-pixel. As the light emitted from this sub-pixel travels from the display panel and passes through the 3D optical lens, a 3D image is formed in the user's eye.

[0089] In the light field image, within each 3D pixel cycle, the system records the distance from the rendered ray emitted by each pixel to the observer's brow. The distance at the brow is 0, to the left of the brow is negative, and to the right is positive. The system sorts these distance values ​​to extract rays from the same viewpoint. Based on this sorting, the system extracts pixels corresponding to rays with the same sorted position from each 3D pixel cycle. Pixels with the same sorted position form an image, which is the single-viewpoint image separated from the original light field image.

[0090] See Figure 6 The diagram illustrates a rendering process for a single-viewpoint image. In the 3D scene to be rendered, a 3D display panel 501 is established. The system calculates the positions of various views on the 3D display panel 501, including V_1_1 and V_1_2, etc. The view positions correspond to the angle from which the user views the 3D scene. Based on the view positions, the system divides the rendering viewport (i.e., the rendering window or area) into multiple small regions, each rendered as a single pixel. The rendering results of all small regions are combined to obtain a complete 2D image.

[0091] Furthermore, when the segmented region is located on the left and right sides of the image and its size exceeds half, the system extracts more than half of the region for rendering to obtain boundary pixels. These extracted boundary pixels are added to the 2D image to ensure that the edge portions of the 2D image are also rendered correctly.

[0092] In one possible implementation, the method for acquiring 3D video sources can also be to directly use pre-stored 3D video sources. These pre-stored video sources can be pre-made or obtained from other channels. Using existing stored video sources can save rendering time and computing resources.

[0093] Step 202: Rearrange the pixels of multiple two-dimensional images in rows to obtain multiple sub-unit images. Any sub-unit image in the multiple sub-unit images includes pixels in the same row of the multiple two-dimensional images.

[0094] In a two-dimensional image, a pixel is the most basic element. Taking a two-dimensional image with a resolution of m*n as an example, the image consists of m rows and n columns of pixels, meaning the number of pixels in the image is m*n, where m and n are arbitrary positive integers. Rearranging the pixels of multiple two-dimensional images row by row means rearranging the images based on the pixels of each row to generate new sub-unit images. For example, the number of sub-unit images is m, meaning that pixels in the same row of any two-dimensional image correspond to one sub-unit image.

[0095] The arrangement order of pixels in the same row of multiple two-dimensional images included in any sub-unit image is not limited in the embodiments of this application. Optionally, the multiple rows of pixels included in any sub-unit image are pixels in the same row of multiple two-dimensional images, or the multiple rows of pixels included in any sub-unit image are pixels in the same part of the multiple parts of pixels divided from pixels in the same row of multiple two-dimensional images.

[0096] For example, taking 16 two-dimensional images with a resolution of 2160*3840 as an example, see [link to example]. Figure 7 The diagram illustrates a rearrangement process. In 16 viewpoint images (i.e., 16 two-dimensional images), pixels in the same row, from View1 (1*3840) to View16 (1*3840), form a new image (i.e., a sub-unit image). Within the 16 rows of pixels in the new image (16*3840), View1 corresponds to the first row of pixels, View2 corresponds to the second row, and View16 corresponds to the 16th row, and so on.

[0097] Another example is seen in [reference 1]. Figure 8 The diagram illustrates another rearrangement process. For the same row of pixels View1 to View16 in the 16 viewpoint images, each row of pixels can be divided into multiple parts of pixels (e.g., different parts contain the same number of pixels). Then, the pixels of the same part in all the same row of pixels are taken and merged into a row of pixels in the new image, thus obtaining the new image (i.e., the sub-unit image).

[0098] This application does not limit the number of multipart pixels. Optionally, it can be determined based on the macroblock size. For example, the number of multipart pixels is an integer multiple of the macroblock size. Taking 16 multipart pixels as an example, as... Figure 8 As shown, View1 can be divided into 16 partial pixels, each of which includes 480 pixels. View2 to View16 are divided in the same way as View1. In the 16 rows of pixels in the new image, the first part of pixels from View1 to View16 is combined to form the first row of pixels in the 16 rows, the second part of pixels from View1 to View16 is combined to form the second row of pixels in the 16 rows, and so on. That is, the multiple rows of pixels are the same part of pixels in the multiple parts of pixels divided from the same row of pixels in multiple two-dimensional images.

[0099] in, Figure 7 The rearrangement method shown can be applied to situations where the differences between multiple two-dimensional images are small, that is, the correlation between multiple two-dimensional images is strong, and the row pixels of different two-dimensional images can be compressed together in the subsequent compression process. Figure 8The rearrangement method shown can be applied to situations where there are large differences between multiple two-dimensional images, that is, the correlation between multiple two-dimensional images is weak. In this case, a row of pixels of a two-dimensional image is split into multiple parts. The different parts of the same row of pixels are more correlated, and the different parts of the same row of pixels can be compressed together in the subsequent compression process.

[0100] Step 203: The system device encodes and compresses multiple sub-unit images to obtain multiple line compressed bitstreams.

[0101] A sub-unit image is an intermediate result obtained by rearranging the pixels of multiple two-dimensional images row by row by the system device. The sub-unit image contains the information needed to construct a three-dimensional image. The system device encodes and compresses the sub-unit image to remove redundant and irrelevant information, thereby reducing the data volume. The system device can employ different encoding techniques, such as lossless or lossy compression, selecting a suitable compression method based on actual needs and data characteristics.

[0102] After encoding and compression, the system device obtains multiple line-compressed bitstreams. Line-compressed bitstreams are the compressed data representation, occupying less storage space and offering faster transmission speeds. To ensure that the compressed data can be correctly decoded and displayed, corresponding decoding algorithms and processes need to be defined in the display device.

[0103] In one possible implementation, the process of encoding and compressing multiple sub-unit images to obtain multiple line compressed bitstreams includes: for any sub-unit image among the multiple sub-unit images, dividing the sub-unit image into N slice images, where N is the number of decoding sub-units that the display device performs in parallel decoding; for any slice image among the N slice images, obtaining multiple macroblock images based on the slice image according to the macroblock size; and encoding and compressing the multiple slice images at the macroblock image granularity to obtain the line compressed bitstream corresponding to any sub-unit image.

[0104] This application does not limit the method of segmenting any sub-unit image. For example, it can be uniform segmentation. Uniform segmentation results in more closely related slice images, facilitating subsequent compression operations. The system device uniformly segments any sub-unit image from multiple sub-unit images, thereby decomposing the sub-unit image into smaller image units for subsequent encoding and compression operations. The sub-unit image is uniformly segmented into N slice images, where N corresponds to the number of decoding sub-units performed in parallel by the display device, thus ensuring that each decoding sub-unit can independently process one slice image, improving the parallelism and efficiency of decoding.

[0105] See Figure 9 The diagram illustrates an image slicing process, taking 16 two-dimensional images as an example. Figure 7After obtaining the sub-unit image through the rearrangement method shown, the sub-unit image is divided into 8 slice images according to the number of IPs. The number of IPs is the number of decoding sub-units that the display device performs in parallel decoding, i.e., N is 8.

[0106] The system device performs macroblock splitting on any slice image from N slices. A macroblock is the basic processing unit in the encoding and compression process, and its size can be determined according to the encoding standard and actual needs. By splitting the slice image into multiple macroblock images, the system device can further refine the granularity of encoding and compression, thereby improving compression efficiency. After macroblock splitting, the system device performs encoding and compression on multiple slice images at the macroblock image granularity. Through encoding and compression, the line compressed bitstream corresponding to any sub-unit image is obtained.

[0107] In one possible implementation, the process of splitting any slice image into multiple macroblock images according to the macroblock size includes: obtaining a complete image corresponding to any slice image based on the macroblock size, wherein the number of row pixels of the complete image is an integer multiple of the number of rows of the macroblock size, and the number of column pixels of the complete image is an integer multiple of the number of columns of the macroblock size; and splitting the complete image into multiple macroblock images according to the macroblock size.

[0108] The completed image corresponding to any slice image acquired by the system device based on the macroblock size refers to an image whose row and column pixel counts both meet specific conditions. For example, the number of row pixels in the completed image is an integer multiple of the number of rows in the macroblock size, and the number of column pixels is also an integer multiple of the number of columns in the macroblock size. This ensures that the completed image can be completely split into multiple macroblock images without pixel mismatches or incomplete images. The system device obtains the completed image through preprocessing operations. For example, if the pixel count of the slice image does not meet the above conditions, the system device needs to pad or crop the image to obtain a completed image that meets the requirements. Preprocessing operations ensure the smooth progress of the subsequent splitting process.

[0109] After acquiring the completed image, the system divides it into multiple macroblock images according to the macroblock size. Each macroblock image contains a subset of pixels from the completed image, with pixel size matching the macroblock size. This allows for independent encoding and compression operations on each macroblock image. The choice of macroblock size significantly impacts the effectiveness and efficiency of encoding and compression. Excessively large macroblocks may result in poor compression, while excessively small macroblocks may increase the complexity and computational cost of encoding and compression. Therefore, a suitable macroblock size must be selected based on specific requirements and conditions.

[0110] In one possible implementation, the process of obtaining the completed image corresponding to any slice image based on the macroblock size includes: if the difference between the number of row pixels of any slice image and an integer multiple of the number of rows of the macroblock size is L, add L rows of pixels to any slice image, where L is a positive integer; or, if the difference between the number of column pixels of any slice image and an integer multiple of the number of columns of the macroblock size is C, add C columns of pixels to any slice image, where C is a positive integer; wherein any pixel in the L rows or C columns is a default pixel or a neighboring pixel, and a neighboring pixel is a pixel in multiple slice images that meets the distance requirement from any pixel.

[0111] The process of obtaining the complete image corresponding to any slice image based on the macroblock size ensures that each slice image can be completely split into multiple macroblock images without pixel mismatch or incomplete image.

[0112] The system checks if the number of rows of pixels in any slice image is an integer multiple of the number of rows in the macroblock size. If not, the difference is L, a positive integer representing the number of missing rows. To compensate for this difference, the system adds L rows of pixels to the bottom of the slice image. These added pixels can be default pixels or neighboring pixels. Default pixels are preset pixel values ​​that do not contain actual image information, such as black or white pixels. Neighboring pixels are pixels selected from multiple slice images that meet a certain distance requirement from the boundary of the current slice image. Using neighboring pixels as fillers allows the filled area to blend more seamlessly with the original image, reducing abruptness. The process of adding pixels should maintain image quality and continuity as much as possible. When selecting filler pixels, factors such as image characteristics, encoding standard requirements, and processing efficiency should be comprehensively considered to choose the most suitable filler method.

[0113] In addition, the system device checks whether the number of columns of pixels in any slice image is an integer multiple of the number of columns in the macroblock size. If not, the difference is C, where C is a positive integer representing the number of missing columns. To compensate for the difference, the system device adds C columns of pixels to the right of the slice image; these added pixels can be either default pixels or neighboring pixels.

[0114] Through the above process, the system ensures that the number of rows and columns of pixels in any slice image meets the macroblock size requirements, thus obtaining a complete slice image that can be fully divided into multiple macroblock images. This avoids problems such as pixel mismatch or incomplete images, improving the efficiency and accuracy of encoding and compression.

[0115] In one possible implementation, the process of splitting the completed image into multiple macroblock images according to the macroblock size includes: splitting the completed image into multiple macroblock images arranged in a single row order according to the macroblock size; wherein, if the number of row pixels of any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images in any set of column-oriented arrangements in the completed image are transformed into adjacent row-oriented arrangements.

[0116] The split macroblock images are arranged in a single-row order, meaning each macroblock image is located in the same row, forming a sequence of macroblock images row by row. When the number of pixels per row in any slice image is at least twice the number of rows in the macroblock size, at least two originally column-oriented macroblock images can be transformed into adjacent row-oriented images. This optimizes the arrangement of macroblock images and improves the efficiency of subsequent coding and compression. By transforming the originally column-oriented macroblock images into adjacent row-oriented images, the spatial distance between macroblock images can be reduced, making the correlation between adjacent macroblocks stronger, thereby improving the coding and compression effect.

[0117] During the adjustment process, the system identifies column-oriented macroblock image groups that meet the conditions in the completed image and transforms these column-oriented macroblock image groups into adjacent row-oriented groups. This transformation can be achieved by moving the positions of the macroblock images, adjusting their arrangement order, etc., to ensure that the transformed macroblock image sequence still maintains its integrity and continuity.

[0118] The transformation process yields an optimized macroblock image sequence. The arrangement of these optimized macroblock images facilitates subsequent coding and compression operations. The optimized macroblock image sequence retains information from the original image while improving the efficiency and quality of coding and compression. When adjusting the macroblock arrangement, the relative positional relationship between two macroblock images should be maintained to avoid introducing additional distortion or errors.

[0119] See Figure 10 The diagram illustrates a macroblock arrangement transformation process. Taking a macroblock size of 16*16 as an example, the number of rows (32 pixels) is twice the number of rows in the macroblock size of 16. Figure 10 As shown, the original column-oriented macroblock image is transformed into an adjacent row-oriented image. Optionally, multiple macroblocks can be combined and sent together based on the processing capability of the shallow compression coding subunit, see [link to documentation]. Figure 11 The diagram shows a macroblock combination method.

[0120] In one possible implementation, any one of the multiple line compressed bitstreams includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image. The data amount of any one of the multiple sub-bitstreams is less than the decoding threshold of the display device, and the decoding threshold is less than X times the data amount of the sub-bitstream, where X is any number greater than 1. The decoding threshold is the maximum amount of data that the display device can decode within the screen refresh time.

[0121] Each line of compressed bitstream consists of multiple sub-bitstreams, each representing the compressed data of a slice image. A slice image is an image portion obtained by uniformly dividing the original sub-unit image. Therefore, each sub-bitstream contains all the encoded compression information of that slice image.

[0122] The decoding threshold refers to the maximum amount of data a display device can decode within a single screen refresh time. Designing the data volume of any sub-stream to be less than the display device's decoding threshold ensures that the display device completes the decoding task promptly with each screen refresh, thus avoiding decoding delays or screen stuttering. Furthermore, the decoding threshold must be less than X times the sub-stream data volume. Not only must the data volume of each sub-stream be sufficient to be decoded within a single refresh time, but the decoding threshold itself cannot be too large, leading to wasted display device resources.

[0123] Regarding bitrate control, bitrate control techniques are used to ensure smooth bitrate transmission while maximizing channel bandwidth utilization. It's necessary to improve encoding performance as much as possible while meeting bandwidth constraints to achieve optimal image quality and transmission efficiency. For example, assuming the decoding capability of a 3D shallow compression decoding subunit is Tot_decode bit / us (microsecond), and the time to refresh a 3D screen slice image is Time_slice us (the time required for each screen refresh), the maximum amount of bitrate data that the 3D shallow compression decoding subunit can decode within Time_slice time can be calculated: Tot_slice = Tot_decode × Time_slice bit.

[0124] The rate control subunit controls the QP (Quantization Parameter) value of macroblock encoding during the encoding process, thereby controlling the number of bitstreams per slice and ensuring smooth decoding. The QP value affects the encoded image quality and bitstream size. By dynamically adjusting the QP value, the rate control subunit ensures that the bitstream data of each slice is less than Tot_slice, i.e., does not exceed the amount of bitstream data that the decoding subunit can process within the refresh time. The rate control strategy guarantees the real-time performance and stability of decoding and optimizes encoding performance. By controlling the number of bitstreams per slice, image quality can be maximized while meeting bandwidth limitations, providing users with a better visual experience.

[0125] See Figure 12 The diagram illustrates a bitrate control process. Taking X = 2 as an example, macroblocks within a slice image are encoded, and the size of the bitstream of all macroblocks within the slice image is counted. If the decoding threshold is greater than the bitstream size but less than twice the bitstream size, no adjustment is made; otherwise, the macroblock QP value is modified until the decoding threshold is greater than the bitstream size but less than twice the bitstream size.

[0126] Taking a slice image consisting of 8 macroblock images and a decoding IP count of 32 as an example, for Figure 7 The rearrangement method shown allows each decoded IP to correspond to a macroblock image that can be sequentially arranged as macroblock images 1, 2, 3, 4, 5, 6, 7, and 8. For Figure 8 The rearrangement method shown is in Figure 8 In the rearrangement method shown, the macroblock image and the viewpoint position on the display device correspond.

[0127] Optionally, steps 202-203 above can be performed by... Figure 3 The system device shown implements shallow compression. This shallow compression module is used to compress multi-view image data, i.e., stream processing, to reduce data size and transmission bandwidth. Stream processing can be performed through the encoding IP or a sub-IP included in the GPU. The encoding IP or the sub-IP included in the GPU may already have integrated data compression functionality, which can be directly utilized for shallow compression. Alternatively, a data compression solution tailored to the current image can be developed to adapt to specific application scenarios or optimize performance. Cloud encoding can also be performed in the cloud and then transmitted to the system.

[0128] Step 204: The system device sends multiple line compressed bitstreams to the display device, and the multiple line compressed bitstreams are used by the display device to drive the display of the three-dimensional image.

[0129] During transmission, the system can employ various technologies and transmission protocols to ensure data integrity and transmission efficiency. After the compressed bitstream arrives at the display device, the display device performs corresponding decoding and processing. Once decoding is complete, the display device can use the decoded information to drive the display panel or projection device, thereby presenting a three-dimensional image.

[0130] Display device (i.e., display terminal), see Figure 3 The diagram illustrates the interaction process between a system device and a display device. The decompression module in the display device decompresses the received compressed data back into raw data for subsequent processing. The decompression module includes a bitstream processing module and a decoding IP module.

[0131] The bitstream processing module processes the bitstream data according to the requirements of the decoding IP. These requirements may include the number of decoding IP modules and their decoding capabilities. By adapting the bitstream, the module ensures that it matches the decoding IP modules, thus achieving a more efficient decoding process. The bitstream processing module can be located inside the decompression module or within the shallow compression module; its location is determined by the system architecture and design requirements.

[0132] The decoding IP module is used to decode the line data stream. The decoding process restores the compressed stream data to the original image data. The decoded data is stored in the display device for use by subsequent data processing modules. The performance of the decoding IP module affects decompression efficiency and image quality; therefore, the appropriate decoding IP module must be selected based on user requirements. The data processing module further processes the decoded data. The data processing IP in the data processing module performs sub-pixel arrangement of multi-viewpoint images based on the optical characteristics of the 3D screen, ensuring the image presents the correct visual effect from different viewing angles. The display IP in the display module processes the data stream and drives the display of the 3D image. In addition to sub-pixel arrangement of multi-viewpoint images, the data processing module can also process image data, including gamma correction, dithering, and forward / reverse scanning. Data stream processing improves image display, increases contrast and brightness, and reduces noise and distortion.

[0133] Optionally, taking dithering as an example, the dithering template can be found here. Figure 13 As shown, the processing flow of dither is as follows: Figure 14 As shown, the input is N+2 bits of image data, where N+2 bits refers to the raw bit depth of the image data. The output is N bits of image data, where N bits is the bit depth that the target display device can process.

[0134] A processing unit is defined as 4*4 three-dimensional pixel periods. For each processing unit, there are two dithering templates (F0 and F1), used for different frames (frame 0 and frame 1). The templates can be pre-defined and are used to add appropriate noise or perturbation when reducing image depth. Templates can be, for example... Figure 13 The template shown.

[0135] Read the 10-bit R, G, B sub-pixel values ​​and take the last two bits (the lowest two bits) of each R, G, B sub-pixel. Based on the value of the last two bits (00, 01, 10, 11), find the corresponding template value (Value_M_R, Value_M_G, Value_M_B) in the corresponding dithering template (F0 or F1).

[0136] Obtain the high 8-bit values ​​(Value_R, Value_G, Value_B) of the R, G, and B sub-pixels respectively, and add the high 8-bit values ​​to the template values ​​obtained from the dithering template to obtain the final R, G, and B values ​​(Value_Result_R, Value_Result_G, Value_Result_B).

[0137] In frame 0, dithering is performed using the F0 template, and in frame 1, dithering is performed using the F1 template. For subsequent frames, the processing steps for F1 and F0 are similar and will not be repeated here. The dither-based image processing method reduces visual artifacts by applying dithering templates when reducing image depth, while preserving as much image detail as possible.

[0138] Step 205: The display device receives multiple line compressed bitstreams sent by the system device.

[0139] The display device is used to receive processed image data and convert it into visualized images or videos. The display device is equipped with a corresponding receiving interface to ensure stable and rapid reception of the bitstream sent by the system device. This application does not limit the type or specifications of the display screen installed on the display device; for example, the display screen type can be an LED screen or an LCD screen, and the display screen size can be 21 inches or 24 inches, etc.

[0140] For example, when the display device of this application receives a line compressed bitstream, since one line compressed bitstream corresponds to one row of pixels in a 3D image, and the display device refreshes the image line by line, the display device can refresh one row of pixels in the displayed 3D image once according to the line compressed bitstream each time it receives one. The refresh process may include a decoding process and a display driving process. The decoding process can be referred to in step 206 below, and the display driving process can be referred to in step 207 below. This allows the display device to store one line compressed bitstream at a time, which saves storage space compared to storing the compressed bitstream of the entire 3D image and then refreshing the image line by line according to the compressed bitstream of the entire 3D image.

[0141] Optionally, in addition to the line compression bitstream, the system device sends the verification information required by the display device to the display device. This verification information is used by the display device to adjust the parameters of the displayed image. In this embodiment, the verification information includes, but is not limited to, the user's eye coordinates or the correction parameters of the three-dimensional optical lens.

[0142] Step 206: The display device decodes the multiple line compressed bitstreams and obtains multiple sub-unit images based on the decoding results.

[0143] The decoding process can be performed by a decoder inside the display device. A decoder is a hardware or software module used to process compressed data. The decoder has a decoding algorithm corresponding to the compression algorithm, enabling it to reverse the compressed data and recover the original image data.

[0144] When the display device receives multiple lines of compressed bitstream, the decoder decodes each received bitstream individually. The decoding process includes decompressing and reconstructing the compressed data to restore the image data for each line. The decoder uses different decoding methods and parameters depending on the compression algorithm to ensure accuracy and efficiency.

[0145] During the decoding process, the display device needs to consider the synchronization and timing of the bitstreams. Since multiple line-compressed bitstreams may arrive simultaneously, the decoder needs to ensure that each bitstream can be decoded in the correct order and timing to avoid image data corruption or loss.

[0146] After decoding, the display device obtains multiple sub-unit images. These sub-unit images are arranged in row order according to the original image, and together they constitute the complete image data. The display device buffers these sub-unit images for subsequent image processing or display.

[0147] In one possible implementation, the process of decoding multiple line compressed bitstreams and obtaining multiple sub-unit images based on the decoding results includes: for any line compressed bitstream in the multiple line compressed bitstreams, performing parallel decoding on any line compressed bitstream using multiple decoding sub-units, and obtaining multiple slice images based on the decoding results; and obtaining the sub-unit image corresponding to any line compressed bitstream based on the multiple slice images. Wherein, the decoding sub-unit corresponds to a decoding IP.

[0148] Parallel decoding technology improves decoding efficiency by simultaneously decoding any line of the compressed bitstream using multiple decoding subunits. Optionally, multiple decoding subunits are used in the step of restoring the compressed bitstream data to the original image data; these multiple decoding subunits can be multiple decoders.

[0149] For each line of compressed bitstream, the compressed bitstream is fed into multiple decoding subunits for parallel processing. A decoding subunit is a hardware or software module used to process compressed data. During the decoding process, the decoding subunit generates multiple slice images, each a portion of the original image. Each decoding subunit generates a slice image, which is then combined in subsequent steps to reconstruct the complete subunit image. Sorting, stitching, and reconstructing the slice images ensure that they are combined in the correct order and position to form a complete image.

[0150] In one possible implementation, any line of compressed bitstream includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image, and each slice image corresponding to a decoding subunit. The process of parallel decoding of any line of compressed bitstream using multiple decoding subunits and obtaining multiple slice images based on the decoding results includes: sending a first sub-bitstream corresponding to a first slice image in any line of compressed bitstream to a first decoding subunit, where the first decoding subunit is any one of the multiple decoding subunits, and the first slice image is the slice image corresponding to the first decoding subunit; decoding the first sub-bitstream using the first decoding subunit; obtaining the first slice image based on the decoding results; and performing parallel decoding among the multiple decoding subunits.

[0151] Optionally, after receiving any line of compressed bitstream, the display device identifies multiple sub-bitstreams included in that line of compressed bitstream and determines the decoding sub-unit corresponding to each sub-bitstream. See also Figure 15 The diagram illustrates a bitstream processing procedure. The received compressed bitstream undergoes start code detection to determine any line of compressed bitstream; the line of compressed bitstream undergoes data header detection to distinguish different sub-bitstreams and saves these sub-bitstreams; the saved sub-bitstreams are rearranged according to different decoding sub-units, and the different sub-bitstreams are transmitted to their corresponding decoding sub-units.

[0152] For example, sub-streams corresponding to the same decoding subunit include the same data header. Sub-streams corresponding to the same decoding subunit are sub-streams corresponding to the same slice image; that is, the data headers of sub-streams corresponding to the same slice image are identical. The data header can refer to an SPS (Sequence Parameter Set) header, a PPS (Picture Parameter Set) header, an SEI header, or a slice header, etc. SEI is a technology in the H.264 standard. SEI does not contain image data information; it is a supplement to image data information or video streams. The slice header is the data header corresponding to the slice image. The image data of any slice image can be determined based on the slice header. The image data corresponding to one slice header is processed by one decoding IP; for example, slice1 is processed by decoding IP1, and slice2 is processed by decoding IP2.

[0153] Taking the first decoding subunit as an example, after receiving the first sub-bitstream, the first decoding subunit begins the decoding operation. It processes the first sub-bitstream using a decoding algorithm corresponding to the compression algorithm, restoring it to a sliced ​​image. The decoding process is independent and does not depend on the work of other decoding subunits. Other decoding subunits also perform decoding operations in parallel. Each receives its corresponding sub-bitstream and decodes it independently, generating the corresponding sliced ​​image. This improves decoding efficiency, allowing the entire decoding process to be completed in a shorter time.

[0154] In one possible implementation, the process of performing parallel decoding on any line of compressed bitstream using multiple decoding subunits and obtaining multiple slice images based on the decoding results includes: performing parallel decoding on any line of compressed bitstream using multiple decoding subunits, obtaining multiple completed images based on the decoding results, wherein the number of row pixels in the completed images is an integer multiple of the number of rows in the macroblock size, and the number of column pixels in the completed images is an integer multiple of the number of columns in the macroblock size; if the multiple completed images include L rows of pixels, deleting L rows of pixels to obtain multiple slice images, where L is a positive integer; if the multiple completed images include C columns of pixels, deleting C columns of pixels to obtain multiple slice images, where C is a positive integer; wherein any pixel in L rows of pixels or C columns of pixels is a default pixel or a neighboring pixel, and a neighboring pixel is a pixel whose distance from any pixel meets the distance requirement.

[0155] Multiple completed images are obtained by parallel decoding of any line of the compressed bitstream using multiple decoding subunits. These completed images, derived from the decoded compressed bitstream, contain all the data from the original image.

[0156] If multiple completed images include L rows of pixels added, delete those L rows. If multiple completed images include C columns of pixels added, delete those C columns. Deleting added pixels prevents display errors when displaying the image. By deleting added row or column pixels, multiple slice images are obtained. Slice images are a part of the original image. Because slice images are based on completed images and pixels that might affect image quality have been removed, slice images accurately reflect the information of the original image.

[0157] In one possible implementation, the process of performing parallel decoding on any line of compressed bitstream by multiple decoding subunits and obtaining multiple complete images based on the decoding results includes: performing parallel decoding on any line of compressed bitstream by multiple decoding subunits to obtain multiple sets of macroblock images arranged in a single row; for any macroblock image in the multiple sets of macroblock images, if the number of row pixels in any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images arranged in adjacent rows in any set of macroblock images are transformed into column-oriented arrangements to obtain the complete image corresponding to any set of macroblock images.

[0158] The process of obtaining multiple complete images by performing parallel decoding of any line of compressed bitstream by multiple decoding subunits involves the decoding and rearrangement of macroblock images.

[0159] Multiple decoding subunits perform parallel decoding on any line of the compressed bitstream, resulting in multiple sets of macroblock images arranged in a single-line sequence. Macroblock images are the basic components of a video frame, arranged in a specific order. If the number of pixels per row in any slice image is at least twice the number of rows in the macroblock size, at least two adjacent row-oriented macroblock images in any set are transformed into column-oriented macroblock images. This changes the arrangement of the macroblock images, transforming the original row-oriented macroblocks into column-oriented macroblocks, effectively restoring the original column-oriented macroblock images. After all macroblock images have undergone this process, multiple completed images are obtained.

[0160] Step 207: The display device displays a three-dimensional image based on the image drive of multiple sub-units.

[0161] The display device further processes and combines the sub-unit images. This may include operations such as stitching, scaling, and rotating the sub-unit images to ensure that the sub-unit images can be correctly combined into a three-dimensional image during display.

[0162] For example, see Figure 16 The diagram shows a display screen structure. Different two-dimensional images need to be displayed within corresponding areas, i.e., viewpoint intervals, to complete the display of three-dimensional images. The viewpoint interval for each pixel can be determined using the following formula:

[0163]

[0164] Where V is the viewpoint interval to which the sub-pixel belongs, x is the horizontal coordinate of the sub-pixel, y is the vertical coordinate of the sub-pixel, α is the tilt angle of the prism lens on the display screen, x is the line value, that is, the number of pixels in a row of pixels covered by a prism lens in the horizontal direction, Koff is the offset, (x-Koff-ytanα)modx is the modulus of (x-Koff-ytanα) with respect to x, Ntot is the total number of viewpoint intervals, P is the width of the prism lens, and P·cosα is x (i.e., the line value).

[0165] By determining the viewpoint interval corresponding to each pixel, the display parameters corresponding to each pixel are determined, and then the image is displayed. The prism lens corresponding to each viewpoint interval changes the direction of the light emitted from the image. Different viewpoint intervals correspond to different viewing angles, thereby generating a three-dimensional image.

[0166] In one possible implementation, see Figure 17 The diagram illustrates a scene for determining the viewpoint. A three-dimensional coordinate system OXYZ is established on a 3D screen with the screen center as the origin O. The tilt angle of the 3D optical lens is θ, and the magnitude of θ affects the viewing angle of the 3D image content. The coordinates of the center of the eyebrows E in the OXYZ coordinate system are (X_e, Y_e, Z_e). The projection of the center of the eyebrows E onto the 3D screen is point F, with coordinates (X_e, Y_e, 0). Point P (X_p, Y_p, 0) is the center of a 3D pixel located on the 3D screen, with a Z-coordinate of 0. The straight line PD passes through point P and makes an angle θ with the horizontal direction (i.e., the X-axis). Therefore, the slope of the straight line PD is k = tan(-θ). b is the Y-intercept of the straight line PD. From the point-slope form, b = Y_p - k*X_p, and the equation of the straight line PD is yk*xb = 0. Using the formula for the distance from a point to a line, we get FD = |Y_e - k * X_e - b| / √((-k)^θ + 1^θ). Using the tangent property of a right triangle, we calculate the angle ∠FED: tan(∠FED) = FD / Z_e. Finally, we obtain the angle value ∠FED = arctan(FD / Z_e) using the arctangent function.

[0167] A lookup table can be constructed to quickly find the corresponding viewpoint based on ∠FED. Each entry in the lookup table corresponds to a range of viewing angles and a viewpoint. Furthermore, if ∠FED exceeds the range of the lookup table (e.g., greater than 8α or less than -8α), it can be adjusted to fall within the effective range of the lookup table by adding or subtracting a larger angle (e.g., 16α). The lookup table is shown in Table 1 below.

[0168] Table 1

[0169]

[0170]

[0171] In this way, the display device can determine the viewpoint based on the position of the center of the eyebrows and the position of the three-dimensional pixels, thereby providing a correct three-dimensional visual effect.

[0172] Optionally, for the 3D subpixel layout subunit, a lookup table based on the center-of-the-eye viewpoint and the corresponding viewpoint within a pixel period can be created to complete the 3D subpixel layout process. Taking a total of 16 viewpoints on the screen as an example, there are 16 different viewpoints, each corresponding to a specific 3D perspective effect. In 3D display, a pixel period is used to display information from the same viewpoint. The center-of-the-eye viewpoint refers to the position of the viewer's eyes relative to the screen. By detecting or inputting the viewer's position, the system can determine the center-of-the-eye viewpoint and select appropriate viewpoint content accordingly. The lookup table maps the position of the center-of-the-eye viewpoint to the corresponding viewpoint within a pixel period. See Table 2 below for some of the correspondences between center-of-the-eye viewpoints and their corresponding viewpoints within a pixel period.

[0173] Table 2

[0174]

[0175] In summary, the method provided in this application rearranges multiple two-dimensional images row by row to obtain multiple sub-unit images, and encodes and compresses each sub-unit image. Since multiple rows of pixels in any sub-unit image correspond to the same row of pixels in multiple two-dimensional images, the pixel correlation within any sub-unit image is high, resulting in a higher compression ratio and thus a higher transmission rate of the row compressed bitstream, thereby increasing the transmission rate of the three-dimensional source image. Therefore, a higher compression ratio reduces image loss, and a higher transmission rate reduces the latency of the row compressed bitstream, increasing the display frame rate of the three-dimensional image and thus improving the display quality of the three-dimensional image. Furthermore, when the display device refreshes the three-dimensional image row by row, since one row compressed bitstream corresponds to one row of pixels in the three-dimensional image, the display device only needs to store one row compressed bitstream to refresh one row of pixels in the three-dimensional image once, saving storage space.

[0176] Below, in conjunction with Figure 18 The logic block diagram of the display driving process for the three-dimensional image shown illustrates the complete flow of the three-dimensional image display driving method provided in the embodiments of this application. Figure 18 As shown, the logic block diagram includes a 3D source generation subunit, a 3D source row arrangement subunit, a 3D shallow compression coding subunit, a bitrate control subunit, a bitstream splitting subunit, a 3D shallow compression decoding subunit, a 3D row subpixel arrangement subunit, a 3D display driving subunit, and a 3D display.

[0177] The system generates or receives external 3D source material. The 3D source material row arrangement subunit rearranges the 3D source material row by row to generate sub-unit images to be compressed. After compression by the 3D shallow compression encoding subunit, a sub-unit image bitstream is generated. The bitrate control subunit controls the compressed bitstream based on the performance of the 3D shallow compression decoding subunit. The bitstream splitting subunit splits the sub-unit image bitstream row by row and distributes it to one or more 3D shallow compression decoding subunits. The 3D shallow compression decoding subunit decodes the sub-unit image bitstream to generate the actual sub-unit image data. The 3D subpixel arrangement subunit arranges the subpixels within the sub-unit image according to the optical display requirements of 3D display to match the 3D display characteristics. The 3D display driving subunit drives the display of the subpixel-arranged image, ultimately achieving the 3D display effect.

[0178] Figure 18 The implementation methods of each module shown can be found in [reference needed]. Figure 2 The relevant implementation methods shown will not be described in detail here.

[0179] See Figure 19 , Figure 19 This is a schematic diagram of the structure of a three-dimensional image display driving device provided in an embodiment of this application. The device is applied to... Figure 2 System devices in, such as Figure 19 As shown, the device includes:

[0180] The acquisition module 1101 is used to acquire a three-dimensional source image, which includes multiple two-dimensional images for displaying a three-dimensional image.

[0181] The rearrangement module 1102 is used to rearrange the pixels of multiple two-dimensional images in rows to obtain multiple sub-unit images, wherein any sub-unit image in the multiple sub-unit images includes the same row of pixels in the multiple two-dimensional images.

[0182] Compression module 1103 is used to encode and compress multiple sub-unit images to obtain multiple line compressed bitstreams;

[0183] The sending module 1104 is used to send multiple line compressed bit streams to the display device, and the multiple line compressed bit streams are used by the display device to drive the display of three-dimensional images.

[0184] In one possible implementation, the compression module 1103 is used to divide any sub-unit image among multiple sub-unit images into N slice images, where N is the number of decoding sub-units that the display device performs parallel decoding; for any slice image among the N slice images, multiple macroblock images are obtained based on any slice image according to the macroblock size; and the multiple slice images are encoded and compressed at the granularity of macroblock images to obtain the line compressed bitstream corresponding to any sub-unit image.

[0185] In one possible implementation, the compression module 1103 is used to obtain the completed image corresponding to any slice image based on the macroblock size, wherein the number of row pixels of the completed image is an integer multiple of the number of rows of the macroblock size, and the number of column pixels of the completed image is an integer multiple of the number of columns of the macroblock size; and the completed image is split into multiple macroblock images according to the macroblock size.

[0186] In one possible implementation, the compression module 1103 is configured to add L rows of pixels to any slice image if the difference between the number of rows of any slice image and an integer multiple of the number of rows of the macroblock size is L, where L is a positive integer; or, if the difference between the number of columns of any slice image and an integer multiple of the number of columns of the macroblock size is C, add C columns of pixels to any slice image, where C is a positive integer; wherein any pixel in the L rows of pixels or the C columns of pixels is a default pixel or a neighboring pixel, and a neighboring pixel is a pixel in multiple slice images that meets the distance requirement from any pixel.

[0187] In one possible implementation, compression module 1103 is used to split the completed image into multiple macroblock images arranged in a single row order according to the macroblock size; wherein, when the number of row pixels of any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images in any set of column-oriented arrangement in the completed image are transformed into adjacent row-oriented arrangement.

[0188] In one possible implementation, any one of the multiple line compressed bitstreams includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image. The data amount of any one of the multiple sub-bitstreams is less than the decoding threshold of the display device, and the decoding threshold is less than X times the data amount of the sub-bitstream, where X is any number greater than 1. The decoding threshold is the maximum amount of data that the display device can decode within the screen refresh time.

[0189] See Figure 20 , Figure 20 This is a schematic diagram of another three-dimensional image display driving device provided in an embodiment of this application. The device is applied to... Figure 2 Display devices in, such as Figure 20 As shown, the device includes:

[0190] The receiving module 1201 is used to receive multiple line compressed bitstreams sent by the system device;

[0191] Decoding module 1202 is used to decode multiple line compressed bitstreams and obtain multiple sub-unit images based on the decoding results;

[0192] Display module 1203 is used to display three-dimensional images based on multiple sub-unit image drives.

[0193] In one possible implementation, the decoding module 1202 is used to perform parallel decoding of any one line of compressed bitstream in multiple line compressed bitstreams through multiple decoding sub-units, obtain multiple slice images based on the decoding results, and obtain the sub-unit image corresponding to any one line of compressed bitstream based on the multiple slice images.

[0194] In one possible implementation, any line of compressed bitstream includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image, and each slice image corresponding to a decoding subunit; the decoding module 1202 is used to send the first sub-bitstream corresponding to the first slice image in any line of compressed bitstream to the first decoding subunit, the first decoding subunit being any one of the multiple decoding subunits, and the first slice image being the slice image corresponding to the first decoding subunit; the first sub-bitstream is decoded by the first decoding subunit, and the first slice image is obtained based on the decoding result, with multiple decoding subunits performing parallel decoding.

[0195] In one possible implementation, the decoding module 1202 is used to perform parallel decoding on any line of compressed bitstream through multiple decoding subunits, and obtain multiple completed images based on the decoding results. The number of row pixels in the completed images is an integer multiple of the number of rows in the macroblock size, and the number of column pixels in the completed images is an integer multiple of the number of columns in the macroblock size. If the multiple completed images include L rows of pixels, L rows of pixels are deleted to obtain multiple slice images, where L is a positive integer. If the multiple completed images include C columns of pixels, C columns of pixels are deleted to obtain multiple slice images, where C is a positive integer. Any pixel in L rows or C columns is a default pixel or a neighboring pixel, and a neighboring pixel is a pixel that meets the distance requirement from any pixel.

[0196] In one possible implementation, the decoding module 1202 is used to perform parallel decoding on any line of compressed bitstream through multiple decoding subunits to obtain multiple sets of macroblock images arranged in a single line sequence; for any macroblock image in the multiple sets of macroblock images, if the number of row pixels in any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images arranged in adjacent rows in any set of macroblock images are transformed into column-arranged images to obtain the complete image corresponding to any set of macroblock images.

[0197] In summary, the 3D image display driving device provided in this application rearranges multiple 2D images into multiple sub-unit images on a row-by-row basis, and encodes and compresses each sub-unit image. Since multiple rows of pixels in any sub-unit image correspond to the same row of pixels in multiple 2D images, the pixel correlation within any sub-unit image is high, resulting in a higher compression ratio and thus a higher transmission rate of the row compression bitstream, thereby increasing the transmission rate of the 3D source image. Therefore, a higher compression ratio reduces image loss, and a higher transmission rate reduces the latency of the row compression bitstream, increasing the display frame rate of the 3D image and thus improving the display quality of the 3D image. Furthermore, when the display device refreshes the 3D image on a row-by-row basis, since one row compression bitstream corresponds to one row of pixels in the 3D image, the display device only needs to store one row compression bitstream to refresh one row of pixels in the 3D image once, saving storage space.

[0198] It should be noted that the above Figure 19 and Figure 20 The three-dimensional image display driving device provided in the embodiments is illustrated by the division of the above-described functional modules. In actual operation, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments.

[0199] This application provides a display driving system for three-dimensional images, which includes a system device and a display device; the system device is used to execute... Figure 2 The system device involved in the method for driving the display of 3D images includes the display device for execution. Figure 2 The display device involved in the display driving method of three-dimensional images.

[0200] Figure 21 This is a schematic diagram of a server structure provided in an embodiment of this application. The server can vary significantly due to differences in configuration or performance. It may include one or more processors 1301 and one or more memories 1302. The one or more memories 1302 store at least one computer program, which is loaded and executed by the one or more processors 1301 to enable the server to implement the three-dimensional image display driving method provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0201] Figure 22 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application, enabling the terminal to implement the three-dimensional image display driving method provided in the above-described method embodiments. The terminal may be, for example, a smartphone, tablet computer, media player, laptop computer, or desktop computer. The terminal may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0202] Typically, a terminal includes a processor 1401 and a memory 1402.

[0203] Processor 1401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0204] The memory 1402 may include one or more computer-readable storage media, which may be non-transitory. The memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1402 is used to store at least one instruction, which is executed by the processor 1401 to cause the terminal to implement the three-dimensional image display driving method provided in the method embodiments of this application.

[0205] In some embodiments, the terminal may also optionally include: a peripheral device interface 1403 and at least one peripheral device. The processor 1401, memory 1402, and peripheral device interface 1403 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1404, a display screen 1405, a camera assembly 1406, an audio circuit 1407, and a power supply 1408.

[0206] Peripheral device interface 1403 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1401 and memory 1402. In some embodiments, processor 1401, memory 1402 and peripheral device interface 1403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1401, memory 1402 and peripheral device interface 1403 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0207] The radio frequency (RF) circuit 1404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1404 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1404 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1404 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0208] Display screen 1405 is used to display a UI (User Interface). This UI may include graphics, text, icons, video, and any other combination thereof. When display screen 1405 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1401 for processing. In this case, display screen 1405 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1405 can be a single screen, located on the front panel of the terminal; in other embodiments, display screen 1405 can be at least two screens, respectively located on different surfaces of the terminal or in a folded design; in other embodiments, display screen 1405 can be a flexible display screen, located on a curved or folded surface of the terminal. Furthermore, display screen 1405 can be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 1405 can be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0209] The camera assembly 1406 is used to acquire images or videos. Optionally, the camera assembly 1406 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1406 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0210] The audio circuit 1407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1401 for processing, or input to the radio frequency circuit 1404 to achieve voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1401 or the radio frequency circuit 1404 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1407 may also include a headphone jack.

[0211] Power supply 1408 is used to power the various components in the terminal. Power supply 1408 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1408 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0212] In some embodiments, the terminal further includes one or more sensors 1409. The one or more sensors 1409 include, but are not limited to: an accelerometer 1410, a gyroscope 1411, a pressure sensor 1412, an optical sensor 1413, and a proximity sensor 1414.

[0213] Accelerometer 1410 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by the terminal. For example, accelerometer 1410 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1401 can control display screen 1405 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1410. Accelerometer 1410 can also be used for games or for acquiring user motion data.

[0214] The gyroscope sensor 1411 can detect the terminal's orientation and rotation angle. The gyroscope sensor 1411 can work in conjunction with the accelerometer sensor 1410 to collect the user's 3D movements on the terminal. Based on the data collected by the gyroscope sensor 1411, the processor 1401 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0215] The pressure sensor 1412 can be disposed on the side bezel of the terminal and / or the lower layer of the display screen 1405. When the pressure sensor 1412 is disposed on the side bezel of the terminal, it can detect the user's grip signal on the terminal, and the processor 1401 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1412. When the pressure sensor 1412 is disposed on the lower layer of the display screen 1405, the processor 1401 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1405. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0216] Optical sensor 1413 is used to collect ambient light intensity. In one embodiment, processor 1401 can control the display brightness of display screen 1405 based on the ambient light intensity collected by optical sensor 1413. Specifically, when the ambient light intensity is high, the display brightness of display screen 1405 is increased; when the ambient light intensity is low, the display brightness of display screen 1405 is decreased. In another embodiment, processor 1401 can also dynamically adjust the shooting parameters of camera assembly 1406 based on the ambient light intensity collected by optical sensor 1413.

[0217] The proximity sensor 1414, also known as a distance sensor, is typically installed on the front panel of the terminal. The proximity sensor 1414 is used to detect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1414 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1401 controls the display screen 1405 to switch from a screen-on state to a screen-off state; when the proximity sensor 1414 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1401 controls the display screen 1405 to switch from a screen-off state to a screen-on state.

[0218] Those skilled in the art will understand that Figure 22 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0219] In an exemplary embodiment, a computer device is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the above-described methods for displaying three-dimensional images.

[0220] In an exemplary embodiment, a non-transient computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-described three-dimensional image display driving methods.

[0221] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0222] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described three-dimensional image display driving methods.

[0223] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the 3D image sources involved in this application were all obtained under full authorization.

[0224] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0225] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for driving the display of a three-dimensional image, characterized in that, The method includes: Acquire a three-dimensional source image, which includes multiple two-dimensional images for displaying a three-dimensional image; The pixels of the multiple two-dimensional images are rearranged in rows to obtain multiple sub-unit images, wherein any sub-unit image in the multiple sub-unit images includes the same row of pixels in the multiple two-dimensional images; The multiple sub-unit images are encoded and compressed to obtain multiple line compressed bitstreams; The plurality of compressed line streams are sent to the display device, and the plurality of compressed line streams are used by the display device to drive the display of the three-dimensional image.

2. The method according to claim 1, characterized in that, The process of encoding and compressing the multiple sub-unit images to obtain multiple line compressed bitstreams includes: For any one of the plurality of sub-unit images, the any one sub-unit image is divided into N slice images, where N is the number of decoding sub-units that the display device performs in parallel decoding. For any slice image among the N slices, multiple macroblock images are obtained based on the any slice image according to the macroblock size; The multiple slice images are encoded and compressed at the macroblock image granularity to obtain the row compressed bitstream corresponding to any sub-unit image.

3. The method according to claim 2, characterized in that, The step of obtaining multiple macroblock images based on any slice image according to the macroblock size includes: The completed image corresponding to any slice image is obtained based on the macroblock size, wherein the number of rows of the completed image is an integer multiple of the number of rows of the macroblock size, and the number of columns of the completed image is an integer multiple of the number of columns of the macroblock size; The completed image is split into multiple macroblock images according to the macroblock size.

4. The method according to claim 3, characterized in that, The step of obtaining the completed image corresponding to any slice image based on the macroblock size includes: If the difference between the number of rows of pixels in any slice image and an integer multiple of the number of rows in the macroblock size is L, then add L rows of pixels to any slice image, where L is a positive integer; or... If the difference between the number of column pixels in any slice image and an integer multiple of the number of columns in the macroblock size is C, then add C column pixels to any slice image, where C is a positive integer; Wherein, any pixel in row L or column C is a default pixel or a neighboring pixel, and the neighboring pixel is a pixel in the plurality of slice images that is at a distance from any pixel that meets the distance requirement.

5. The method according to claim 3, characterized in that, The step of splitting the completed image into multiple macroblock images according to the macroblock size includes: The completed image is divided into multiple macroblock images arranged in a single row according to the macroblock size; Where the number of row pixels in any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images in any set of column-oriented images in the completed image are transformed into adjacent row-oriented images.

6. The method according to any one of claims 1-5, characterized in that, Any one of the multiple line compressed bitstreams includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image. The data volume of any one of the multiple sub-bitstreams is less than the decoding threshold of the display device, and the decoding threshold is less than X times the data volume of the sub-bitstream, where X is any number greater than 1. The decoding threshold is the maximum amount of data that the display device can decode within the screen refresh time.

7. A method for driving the display of a three-dimensional image, characterized in that, The method includes: Receive multiple lines of compressed bitstream sent by the system device; The multiple line compressed bitstreams are decoded, and multiple sub-unit images are obtained based on the decoding results; The three-dimensional image is displayed based on the image drive of the multiple sub-units.

8. The method according to claim 7, characterized in that, The step of decoding the multiple line compressed bitstreams and obtaining multiple sub-unit images based on the decoding results includes: For any one of the multiple compressed bitstreams, the compressed bitstream is decoded in parallel by multiple decoding subunits, and multiple slice images are obtained based on the decoding results. Based on the multiple slice images, obtain the sub-unit image corresponding to any line of the compressed bitstream.

9. The method according to claim 8, characterized in that, The compressed bitstream of any line includes multiple sub-bitstreams, each sub-bitstream corresponding to a slice image, and each slice image corresponding to a decoding subunit; the parallel decoding of the compressed bitstream of any line by multiple decoding subunits, and the acquisition of multiple slice images based on the decoding results, includes: Send a first sub-bit stream corresponding to the first slice image in any line of compressed bit stream to the first decoding sub-unit. The first decoding sub-unit is any one of the plurality of decoding sub-units, and the first slice image is the slice image corresponding to the first decoding sub-unit. The first sub-bitstream is decoded by the first decoding subunit, and the first slice image is obtained based on the decoding result. The multiple decoding subunits decode in parallel.

10. The method according to claim 8, characterized in that, The process of performing parallel decoding on any line of the compressed bitstream using multiple decoding subunits, and obtaining multiple slice images based on the decoding results, includes: The compressed bitstream of any line is decoded in parallel by multiple decoding subunits, and multiple complete images are obtained according to the decoding results. The number of row pixels of the complete image is an integer multiple of the number of rows of the macroblock size, and the number of column pixels of the complete image is an integer multiple of the number of columns of the macroblock size. If the plurality of completed images contain L rows of pixels, delete the L rows of pixels to obtain the plurality of slice images, where L is a positive integer; If the plurality of completed images include C columns of pixels, delete the C columns of pixels to obtain the plurality of slice images, where C is a positive integer; Wherein, any pixel in row L or column C is a default pixel or a neighboring pixel, and the neighboring pixel is a pixel that meets the distance requirement from any pixel.

11. The method according to claim 10, characterized in that, The process of performing parallel decoding on any line of the compressed bitstream using multiple decoding subunits, and obtaining multiple completed images based on the decoding results, includes: By performing parallel decoding on any line of the compressed bitstream using multiple decoding subunits, multiple sets of macroblock images are obtained in a single-line sequential arrangement. For any one of the multiple sets of macroblock images, if the number of row pixels in any slice image is at least twice the number of rows of the macroblock size, at least two macroblock images arranged in adjacent rows in any one set of macroblock images are transformed into columns to obtain the complete image corresponding to the one set of macroblock images.

12. A display driving system for three-dimensional images, characterized in that, The system includes system devices and display devices; The system device is used to execute the display driving method for a three-dimensional image as described in any one of claims 1 to 6, and the display device is used to execute the display driving method for a three-dimensional image as described in any one of claims 7 to 11.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement the display driving method for a three-dimensional image as described in any one of claims 1 to 6, or to enable the computer device to implement the display driving method for a three-dimensional image as described in any one of claims 7 to 11.

Images