Adaptive Resolution Adjustment 3D Display Architecture

Abstract

The present application discloses an adaptive resolution adjustment three-dimensional display architecture, which relates to the field of image display. The processor of the display architecture reads back the EDID information, parses the screen resolution parameters, and sends them to the logic reconfiguration control unit; the logic reconfiguration control unit matches the current screen resolution parameters with those of the previous read-back operation, determines the target viewpoint loading information of the current display device, and sends a read instruction to the startup device; the viewpoint calculation and sampling unit receives the target viewpoint sub-pixel mapping file in the startup device, calculates the viewpoint and samples the view according to the mapping relationship between the viewpoint sub-graph and the sub-pixel, and generates a raster encoded map; the video stream output unit receives the raster encoded map, generates an output video stream according to the instruction and timing, and displays it on the display device. This solution can calculate the viewpoint sampling order in real time by calling the hardware resources and the pre-stored mapping relationship between the viewpoint sub-graph and the sub-pixel, realizing adaptive resolution adjustment and bandwidth saving.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image display, and in particular, to an adaptive resolution adjustment three-dimensional display architecture. Background Art

[0002] The development of neural network technology, the continuous improvement of DIBR technology, and the rapid rise of current large models have all brought strong impetus to the field of three-dimensional display. The lenticular grating type naked-eye 3D display device is a major popular medium for realizing naked-eye 3D display at present. The dimming device of this display system is usually composed of a plurality of cylindrical lenses arranged horizontally, and uses the directional refraction ability of the lens to decode the images encoded with multiple viewpoints, refracting the light rays emitted from the pixels of each viewpoint to the corresponding positions in space, and completing the expression of the viewpoint content.

[0003] The current real-time processing of naked-eye 3D video streams is developing towards high resolution, multiple viewpoints, and large viewing angles. This also brings many problems to the real-time acceleration processing on the hardware side. If the viewpoint parameters are stored in the internal high-speed RAM / ROM, taking an 8K resolution 100-viewpoint screen as an example, the viewpoint parameters are encoded according to a 7-bit width, and about 80MB of storage space is required (7680 * 4320 * 3 * 7 / 8 ≈ 80MB). The capacity of the internal high-speed RAM / ROM of the chip usually cannot meet the requirements; if external storage is used and then transmitted to the hardware internally for processing in real time, taking a 30fps video stream as an example, the bandwidth required for reading only the viewpoint parameters is about 19.47Gbps (7680 * 4320 * 3 * 7 * 30), which will seriously squeeze the bandwidth requirements for the transmission of the remaining data and is also difficult to meet on the hardware.

[0004] Calculating the viewpoint parameters in real time on the hardware side can effectively avoid the above problems. However, there are huge differences in the hardware-side calculation and sampling processes of the viewpoint parameters for screens with different resolutions. If multiple resolutions are not compatible, it will result in a set of hardware that can only adapt to a lenticular grating screen with one resolution for three-dimensional display, lacking flexibility; if all common resolutions are compatible in a set of logic, it will not only greatly increase resource consumption but also easily lead to difficult timing convergence. Summary of the Invention

[0005] The embodiments of the present application provide an adaptive resolution adjustment three-dimensional display architecture to solve the problem of multi-resolution adaptation faced when calculating the viewpoint parameters in real time on the hardware side.

[0006] The display architecture includes a processor, a logic reconfiguration control unit, a viewpoint calculation and sampling unit, a video stream output unit, and a startup device;

[0007] The processor reads back the extended display identification data EDID information of the external display device, parses the screen resolution parameter information of the current display device, and sends it to the logical reconfiguration control unit;

[0008] The logical reconfiguration control unit matches the screen resolution parameter information obtained from the current read-back with that of the previous read-back operation, determines the target viewpoint loading information corresponding to the current display device, and sends a read instruction to the startup device; the read instruction is used to obtain the target sub-pixel mapping file corresponding to the viewpoint loading information;

[0009] The viewpoint calculation and sampling unit receives the target sub-pixel mapping file in the startup device, determines the mapping relationship between the sub-viewpoint map and the sub-pixels, and performs viewpoint calculation and view sampling according to the mapping relationship to generate a raster encoded map; among them, the horizontal and vertical sampling methods for the sub-viewpoint map are determined according to the resolution and size ratio of the input and output images, and upsampling magnification or downsampling reduction operations in the horizontal and vertical directions are performed;

[0010] The video stream output unit receives the raster encoded map, generates an output video stream according to the instruction and timing, and displays it on the display device.

[0011] Specifically, the historical viewpoint loading information of the previous read-back operation is stored in the logical reconfiguration control unit;

[0012] When the screen resolution parameters obtained from the two read-back operations are the same, it indicates that the resolution of the external display device remains unchanged; directly determine the stored historical viewpoint loading information as the target viewpoint loading information;

[0013] When the screen resolution parameters obtained from the two read-back operations are different, it indicates that the resolution of the external display device has changed; re-determine the matching target viewpoint loading information according to the actual resolution.

[0014] Specifically, when the resolution of the external display device remains unchanged, the viewpoint calculation and sampling unit uses the historical sub-pixel mapping file obtained from the previous read-back operation as the target sub-pixel mapping file for the current read-back operation;

[0015] When the resolution of the external display device changes, the logical reconfiguration control unit generates a read address and a read instruction according to the target viewpoint loading information, and sends them to the startup device; the viewpoint calculation and sampling unit reads the target sub-pixel mapping file from the startup device according to the address;

[0016] Among them, the startup device stores sub-pixel mapping files corresponding to different screen resolution parameters.

[0017] Specifically, the target view point sub-pixel mapping file records the view point sorting calculation formula for the sub-pixels corresponding to the coordinates of the cylindrical lens grating screen of the current display device under the corresponding screen resolution parameters, which is expressed as follows:

[0018]

[0019]

[0020] Wherein i represents the integer part of the operation result, f represents the fractional part of the operation result; N represents the corresponding view point of the current sub-pixel, (x, y) is the coordinate of the current sub-pixel in the pixel coordinate system, is the grating inclination angle, is the offset of the leftmost grating edge relative to the origin, is the arrangement period of the sub-pixels, is the total number of view points.

[0021] Specifically, the display architecture configures a static area and a dynamic area based on the DSP resources. The static area includes a logic reconfiguration control unit, a logic processing unit, and a video stream output unit. The dynamic area includes a view point calculation and sampling unit;

[0022] The processor converts the remainder and division of the view point sorting calculation formula into multiplication and subtraction to adapt to hardware processing, calls the DSP resources of the view point calculation and sampling unit for dynamic calculation, and encodes and samples the view point sub-map according to the N value to generate a grating encoded map.

[0023] Specifically, after the view point calculation and sampling unit obtains the target view point sub-pixel mapping file, the logic reconfiguration control unit sends a loading completion signal to the processor;

[0024] Based on the loading completion signal and the total number of view points the processor updates the quantization parameters of the parameter register in the view point calculation and sampling unit; the view point calculation and sampling unit performs calculations based on the updated quantization parameters.

[0025] Specifically, set the screen resolution of the output video stream to H*V, the vertical resolution sampling of the view point sub-map to V / v, and the horizontal resolution sampling to H / h;

[0026] Set , v = ; represents the floor symbol, is the total number of view points;

[0027] When the horizontal resolution of the original video stream is greater than H / h, horizontal downsampling is performed; otherwise, horizontal upsampling is performed. When the vertical resolution of the original video stream is greater than V / v, vertical downsampling is performed; otherwise, vertical upsampling is performed.

[0028] Specifically, after the encoding and sampling are completed by the viewpoint calculation and sampling unit, the raster encoded map is sent back to the video stream output unit in the static area. The video stream output unit integrates and outputs the video stream based on the timing logic and sends it to the display device.

[0029] Specifically, the static area further includes a video stream input unit. The display architecture receives the original video stream, and the video stream input unit samples based on the timing instructions to obtain a number of sub-viewpoint images of the same scene from different viewpoints.

[0030] Specifically, when the number of RGB images of the same scene from different viewpoints obtained from the original video stream is not less than the set value, the RGB images are spliced into one or more frames of images, and the target number of sub-viewpoint images are segmented by hardware and cached.

[0031] When the number of RGB images of the same scene from different viewpoints obtained from the original video stream is less than the set value, the RGB images and the corresponding depth images of the scene are obtained, and the target number of sub-viewpoint images are generated by the virtual viewpoint mapping method.

[0032] The beneficial effects brought by the technical solutions provided in the embodiments of the present application at least include:

[0033] 1. By performing real-time calculation of viewpoint parameters and upsampling (downsampling) calculation for reconfigurable compilation, multiple reconfigurable bitstream files with different resolutions can be stored in the startup device (such as flash), and selected according to the actual resolution parameters of the screen, so as to realize the dynamic adaptation of a set of programs to lenticular raster displays with different resolutions.

[0034] 2. Decouple the real-time calculation of viewpoint parameters, upsampling (downsampling) calculation from the overall architecture. The parameter calculation and sampling calculation corresponding to various resolutions reuse the same reconfigurable area, reducing the additional resource consumption brought by compatible different resolutions, and also facilitating timing convergence.

[0035] 3. Performing real-time calculation of viewpoint parameters at the hardware end reduces the storage requirements of the hardware and the bandwidth pressure of parameter transmission, enabling the system to meet the real-time processing requirements of high-resolution 3D video streams. Description of the Drawings

[0036] Figure 1 is a schematic diagram of the adaptive resolution adjustment 3D display architecture provided by the embodiments of the present application;

[0037] Figure 2It is a detailed schematic diagram of an adaptive resolution adjustment 3D display architecture;

[0038] Figure 3 It is a schematic diagram of the viewpoint parameter calculation simulation process provided by an embodiment of the present application;

[0039] Figure 4 Lists the lenticular grating parameters and the viewpoint diagram in a possible form;

[0040] Figure 5 It is a schematic diagram of the extraction of multi-view sub-pixels;

[0041] Figure 6 It is a schematic diagram of two schemes for obtaining multi-view sub-images provided by the present application;

[0042] Figure 7 Shows a schematic diagram of synthesizing a grating encoded image from an RGB image and a depth image;

[0043] Figure 8 Provides two simple sampling example diagrams shown in the embodiment solution of the present application. Specific embodiments

[0044] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0045] As used herein, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0046] See Figure 1 As shown, to solve the multi-resolution adaptation problem faced when calculating viewpoint parameters in real time on the hardware side, the overall design of the adaptive resolution adjustment 3D display architecture provided by the present application is a hardware-side device and an externally connected display device. The display device includes a lenticular grating screen for displaying a naked-eye 3D effect video (image). The hardware-side part is designed as structural units such as a core processor, a logic reconfiguration control unit, a viewpoint calculation and sampling unit, a video stream output unit, and a startup device.

[0047] The processor is the core part that controls the operation of the entire system architecture. During normal operation, it periodically reads back the Extended Display Identification Data (EDID) information of the external display device, parses the screen resolution parameter information of the current display device from it, and sends it to the logical reconfiguration control unit. Since the resolution parameters of the display can be adjusted by the user, or the size and model of the external display are changed, etc., the link state with the hardware side will be altered. Therefore, the processor needs to periodically detect the link state of the display device. After each re-link, the processor reads back the EDID information of the display and parses the resolution parameters of the display device.

[0048] The logical reconfiguration control unit is the key to the system architecture for achieving adaptive resolution adjustment. It is responsible for detecting the resolution change situation. Specifically, it compares the screen resolution parameter information obtained from the current read-back by the processor with that from the previous read-back operation to determine the target view point loading information corresponding to the current display device, and sends a read instruction to the Boot Device. The comparison of the screen resolution parameter information here is to detect whether the output image / video resolution needs to be changed. The target view point loading information is a kind of index information temporarily stored in the logical reconfiguration control unit (register or buffer) and used to represent the view point sub-pixel mapping file. For example, in the form of 2K resolution for size A, 8K resolution for size B, or 1080P resolution for size A, etc. Each size and resolution should have its own view point sub-pixel mapping file because display devices with different resolutions and sizes have corresponding physical parameters, such as grating tilt angle, number of view points, and offset. These information and sampling calculation rules need to be determined by the mapping file. The read instruction is issued when the logical reconfiguration control unit switches different resolution logics to obtain the target view point sub-pixel mapping file corresponding to the view point loading information.

[0049] The Boot Device is a device built into the hardware side or an external hardware device, and it can also be a storage device such as flash. It stores the view point sub-pixel mapping files for all possible sizes and resolutions, reads based on the instructions and addresses generated by the logical reconfiguration control unit, and reads back to the view point calculation and sampling unit.

[0050] The view point calculation and sampling unit receives the target view point sub-pixel mapping file read back by the Boot Device, determines the mapping relationship between the view point sub-graph and the sub-pixels therein, performs data configuration, view point calculation, and view sampling according to the mapping relationship, and generates a raster encoded graph. The raster encoded graph is a fused graph that combines multi-view point images and can present a stereoscopic effect under the action of a lenticular grating.

[0051] The video stream output unit receives the raster encoded image, generates an output video stream according to instructions and timing, and displays it on a display device. In some embodiments, the system architecture further includes a video stream input unit, which mainly receives an externally input video source, performs image sampling processing on the video source to obtain multi-view sub-images, and the view point calculation and sampling unit performs sampling operations based on the sampled multi-view sub-images.

[0052] This solution decouples the view point parameter calculation and sampling logic with different resolutions from the overall processing flow, compiles it into dynamically reconfigurable logic, selects and loads the corresponding reconfigurable logic according to the resolution requirements of the actual display device, and realizes the adaptation to lenticular raster display devices with different resolutions. For this reason, this application can be paired with hardware computing resources to perform real-time sampling calculations according to the mapping relationship, so that there is no need to store a large amount of view point parameters and the transmission bandwidth pressure can also be reduced.

[0053] For the logic reconfiguration control unit, since it needs to have reconfigurable logic and this logic is dynamically switched according to the resolution requirements, the logic reconfiguration control unit needs to store / cache the historical view point loading information of the previous read operation. When the screen resolution parameters obtained by two consecutive read operations are the same, it indicates that the resolution of the external display device remains unchanged; then the stored historical view point loading information can be directly determined as the target view point loading information. When the screen resolution parameters obtained by two consecutive read operations are different, it indicates that the resolution of the external display device has changed; at this time, the target view point loading information that matches the actual resolution is re-determined.

[0054] When the resolution of the external display device remains unchanged, that is, no switching is required, the view point calculation and sampling unit uses the historical view point sub-pixel mapping file obtained from the previous read operation as the target view point sub-pixel mapping file for the current read operation.

[0055] When the resolution of the external display device changes, that is, switching is required, the logic reconfiguration control unit generates a read address and a read instruction according to the target view point loading information and sends them to the startup device. The view point calculation and sampling unit reads the target view point sub-pixel mapping file from the startup device according to the address. After the switching is completed, a completion signal is fed back to the processor for subsequent operations. In this embodiment, a corresponding reconfigurable bit stream file can be provided for each resolution, stored at the corresponding address in the external startup device, and read back according to requirements.

[0056] The system architecture configures a static area and a dynamic area based on DSP resources or FPGA resources. The static area includes a logic reconfiguration control unit, a logic processing unit, and a video stream output unit, and the dynamic area includes a viewpoint calculation and sampling unit. The processor converts the remainder and division of the viewpoint sorting calculation formula into multiplication and subtraction to adapt to hardware processing, and calls the DSP resources or FPGA resources of the viewpoint calculation and sampling unit for dynamic calculation and processing.

[0057] For a determined target sub-pixel mapping file of viewpoints. It records the viewpoint sorting calculation formula of the corresponding coordinates of the sub-pixels on the cylindrical lens grating screen of the current display device under the corresponding screen resolution parameters, which is expressed as follows:

[0058]

[0059]

[0060] Among them, i represents the integer part of the operation result, f represents the fractional part of the operation result; N represents the corresponding viewpoint of the current sub-pixel, (x, y) is the coordinate of the current sub-pixel in the pixel coordinate system in the viewpoint sub-graph, is the grating tilt angle, is the offset of the leftmost grating edge relative to the origin, is the arrangement period of the sub-pixels, is the total number of viewpoints. Figure 4 Lists a possible form of the cylindrical lens grating parameters and the viewpoint graph. Xoff represents the horizontal displacement of the RGB sub-pixel (x, y) from the grating edge, and P represents the grating pitch, represents the sub-pixel width.

[0061] After the viewpoint calculation and sampling unit obtains the target sub-pixel mapping file of viewpoints, the logic reconfiguration control unit will send a loading completion signal to the processor. At this time, the processor can update the quantization parameters of the parameter register Para_Reg in the viewpoint calculation and sampling unit based on the loading completion signal and the total number of viewpoints The viewpoint calculation and sampling unit performs calculations based on the updated quantization parameters. The quantization here is the process of mapping continuous values (such as and in this application are all floating-point numbers) to finite discrete values. The quantization result is sent to the viewpoint parameter calculation and sampling unit through the bus, and then the N value is calculated and the viewpoint sub-graph is encoded and sampled to generate a grating encoded graph.

[0062] In this solution, the size of n depends on and The quantization precision required for these two numbers. If the number of viewpoints is relatively small, which is equivalent to a relatively large jump threshold between viewpoints, then n can be taken as a smaller value; if the total number of viewpoints is relatively large, the jump threshold between viewpoints will be relatively small, and in this case, the calculation precision needs to be improved, and n also needs to be taken as a larger value. As for this power-of-2 quantization, it is because implementing division on hardware is complex and resource-consuming. The difference between 1000 and 1024 is not significant, but dividing by 1000 requires using division for calculation; dividing by 1024 can be achieved on hardware by shifting a number to the right by 10 bits, which is fast in operation and less resource-consuming.

[0063] Because the hardware side uses hardware resources for calculation instead of reading the viewpoint parameter coding stream, and for the determined screen resolution parameters, in its mapping table 、 、 、 and are all fixed values, so they can be directly stored in registers. And according to the formula, its value can be split into an integer i and a decimal f two parts. It is equivalent to the integer i also being a fixed value. Through transformation, the following can be obtained:

[0064] In this way, the division and remainder operations in the original formula can be completely replaced by multiplication and subtraction. By calling DSP resources, the operation cycle can be shortened when the timing requirements are met, and de-quantization of the operation result can also be quickly achieved through shifting, which is beneficial to timing convergence at high clock frequencies. The simulation process of viewpoint parameter calculation is as shown in Figure 3 The two orange marked signals respectively represent the enable of calculation and the validity of the calculation result. The viewpoint parameters of a single sub-pixel can be obtained after a delay of about 20 clock cycles, and pipelined and parallelized calculations can be achieved.

[0065] After the encoding sampling is completed, the viewpoint calculation and sampling unit can send the raster encoded map back to the video stream output unit in the static area. The video stream output unit integrates and outputs the video stream based on the timing logic and sends it to the display device.

[0066] In the case where the encoding result (raster encoded map) has the same resolution as the viewpoint calculation sub-graph, the extraction schematic of multi-viewpoint sub-pixels is as shown in Figure 5As shown in the figure. Assume that the N values obtained by the viewpoint parameter calculation module for the viewpoint parameters of the RGB three channels of the current pixel are 3, 6, and 8 respectively, indicating that the R channel of the current coordinate encoding result needs to be extracted from the R channel of the coordinates corresponding to viewpoint 3, the G channel of the current coordinate encoding result needs to be extracted from the G channel of the coordinates corresponding to viewpoint 6, and the B channel of the current coordinate encoding result needs to be extracted from the B channel of the coordinates corresponding to viewpoint 8. The process of selecting these three channel points is the sub-viewpoint sampling process, and the three sampled points (i.e., the positions of 3, 6, and 8) are used as sub-pixels to jointly form the pixel at the i-th row and j-th column in the raster encoding map.

[0067] In some embodiments, the system may further include a process of generating a viewpoint count sub-map, that is Figure 2 the video stream input unit in, which mainly processes the input video source (original video stream) and samples to obtain multiple sub-viewpoint maps. This application provides two schemes for obtaining multiple sub-viewpoint maps. See Figure 6 As shown in the figure, for a determined input original video stream, when the number of RGB images of different viewpoints of the same scene obtained is less than the set value, the RGB images and corresponding depth images of the scene are obtained, and the target number of sub-viewpoint maps is generated by virtual viewpoint mapping, that is, all the sub-viewpoint maps of this scheme are obtained by virtual mapping of RGB and depth images. When the number of RGB images of different viewpoints of the same scene obtained is not less than the set value, the RGB images are spliced into one or more frames of images, and the target number of sub-viewpoint maps is segmented by hardware and cached, that is, all the sub-viewpoint maps of this scheme are obtained by splicing and segmenting RGB images.

[0068] Figure 7 shows a schematic diagram of synthesizing a raster encoding map from RGB images and depth images. Although the finally output video is 3D displayed, it is still a frame of raster encoding map. Each raster encoding map contains sub-viewpoint maps of different viewpoints of the same scene, and the method of obtaining multiple sub-viewpoint maps is arbitrary. It can come from the video stream exported and recombined by 3D modeling software, or can be generated in real time by deploying a viewpoint mapping algorithm at the hardware end. For the sampling process of sub-viewpoint maps, it is specifically divided into upsampling and downsampling.

[0069] Under standard conditions, the screen resolution size is designed according to international standards. That is, when the input and output resolutions are determined, if the output resolution is greater than the input resolution, the upsampling method is used; otherwise, the downsampling method is used. For example, if the input sub-image uses the 4k standard resolution (3840×2160) and the output is the 8k standard resolution (7680 ×4320), the images under this international standard size are all in the 16:9 size. The input and output images are enlarged or reduced proportionally (7680 / 3840 = 4320 / 2160 = 2). Therefore, only the input sub-image needs to be upsampled and enlarged simultaneously in the horizontal and vertical directions. If the output is the 2k standard resolution, the input sub-image needs to be downsampled and reduced simultaneously in the horizontal and vertical directions. As for the horizontal and vertical sampling frequencies, they are set according to international standards and can be quickly set by referring to the device manual or preset mode, which will not be elaborated here in detail.

[0070] However, in some special cases where the input or output is a non-standard resolution, since the input and output resolutions are not enlarged or reduced proportionally, it is necessary to consider according to the resolutions in the horizontal and vertical directions. Based on this, this application separately provides a method for obtaining sub-images of viewpoints with different perspectives in the case of non-standard resolutions, that is, first set the screen resolution of the output video stream to H*V, and the vertical resolution sampling of the viewpoint sub-image is V / v , and the horizontal resolution sampling is H / h . Set , ; where represents the floor function symbol. Here and are mainly considered for the universality of non-standard sizes, and the size ratios are close and the visual effects of the synthesized 3D images are more in line with the comfort of the human eye.

[0071] When the horizontal resolution of the original video stream is greater than H / h , horizontal downsampling is performed; otherwise, horizontal upsampling is performed. When the vertical resolution of the original video stream is greater than V / v , vertical downsampling is performed; otherwise, vertical upsampling is performed.

[0072] Example: Suppose there are 4 viewpoints and the 3D display device has an output resolution of 1920*1080. At this time, V = 1920, H = 1080, , v = 2, , so the vertical resolution sampling is V / v = 1080 / 2 = 540, and the horizontal resolution sampling is H / h = 1920 / 2 = 960. Suppose the original resolution of the sampled viewpoint sub-image is 1024*480, then it can be determined that horizontal downsampling and vertical upsampling should be performed for the 4 viewpoints.

[0073] For the convenience of presentation and demonstration, Figure 8 two simple sampling examples are provided in Figure 8 In (a) of , it means that the sub - figure and the viewpoint resolution of the lenticular grating screen are the same. In this case, the sub - pixels corresponding to the coordinates in the corresponding viewpoint can be directly extracted and filled. The sampling logic part is connected to the front and back modules in a direct - through manner. Figure 8 In (b) of , it means that the resolution of the sub - figure is not equal to the resolution of the lenticular grating (taking twice the length and width as an example). The four sub - pixels with coordinates (2i - 1, 2j - 1), (2i, 2j - 1), (2i - 1, 2j), and (2i, 2j) in the coding result are all extracted from the sub - pixel with coordinates (i, j) in the corresponding viewpoint. In this way, a simple nearest - neighbor interpolation (upsampling) of the sub - figure can be completed. The upsampling process here can be flexibly adjusted according to needs and actual hardware design. For example, when the difference between the sub - figure resolution and the lenticular grating resolution is large, it is difficult for nearest - neighbor interpolation to achieve a good display effect, and other methods can also be used to upsample the sub - figure.

[0074] In summary, compared with the traditional technical solution, the technical solution of this application has the following beneficial effects:

[0075] 1. By performing re - configurable compilation through real - time calculation of viewpoint parameters and upsampling (downsampling) calculation, multiple re - configurable bit - stream files with different resolutions can be stored in the startup device (such as flash), and selected according to the actual resolution parameters of the screen, realizing the dynamic adaptation of a set of programs to lenticular grating displays with different resolutions.

[0076] 2. Decouple the real - time calculation of viewpoint parameters, upsampling (downsampling) calculation from the overall architecture. The parameter calculations and sampling calculations corresponding to various resolutions share the same re - configurable area, reducing the additional resource consumption caused by compatibility with different resolutions and also facilitating timing convergence.

[0077] 3. Real - time calculation of viewpoint parameters on the hardware side reduces the storage requirements of the hardware and the bandwidth pressure of parameter transmission, enabling the system to meet the real - time processing requirements of high - resolution 3D video streams.

[0078] This specific embodiment is only an explanation of the present invention, and it is not a limitation of the present invention. Those skilled in the art can make modifications to this embodiment without creative contributions according to needs after reading this specification, but as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. An adaptive resolution adjustment three-dimensional display architecture, characterized in that, The display architecture includes a processor, a logic reconfiguration control unit, a viewpoint calculation and sampling unit, a video stream output unit, and a startup device; The processor reads back the extended display identification data (EDID) information of the external display device, parses the screen resolution parameter information of the current display device, and sends it to the logic reconfiguration control unit; The logic reconfiguration control unit matches the currently read screen resolution parameter information with that obtained in the previous read operation, determines the target viewpoint loading information corresponding to the current display device, and sends a read instruction to the startup device; the read instruction is used to obtain the target sub-pixel mapping file of the viewpoint corresponding to the viewpoint loading information; when the screen resolution parameters obtained in the two read operations are the same, it indicates that the resolution of the external display device remains unchanged; directly determines the stored historical viewpoint loading information as the target viewpoint loading information; When the screen resolution parameters obtained in the two read operations are different, it indicates that the resolution of the external display device has changed; Re-determines the matching target viewpoint loading information according to the actual resolution; The viewpoint calculation and sampling unit receives the target sub-pixel mapping file in the startup device, determines the mapping relationship between the viewpoint sub-map and the sub-pixels, and performs viewpoint calculation and view sampling according to the mapping relationship to generate a raster encoded map; among them, the horizontal and vertical sampling methods of the viewpoint sub-map are determined according to the resolution and size ratio of the input and output images, and perform upsampling magnification or downsampling reduction operations in the horizontal and vertical directions; The video stream output unit receives the raster encoded map, generates an output video stream according to the instructions and timing, and displays it on the display device.

2. The adaptive resolution adjustment three-dimensional display architecture according to claim 1, characterized in that When the resolution of the external display device remains unchanged, the viewpoint calculation and sampling unit uses the historical sub-pixel mapping file of the viewpoint obtained in the previous read operation as the target sub-pixel mapping file of the current read operation; When the resolution of the external display device changes, the logic reconfiguration control unit generates a read address and a read instruction according to the target viewpoint loading information and sends them to the startup device; the viewpoint calculation and sampling unit reads the target sub-pixel mapping file from the startup device according to the address; Among them, the startup device stores sub-pixel mapping files of viewpoints corresponding to different screen resolution parameters.

3. The adaptive resolution adjustment three-dimensional display architecture according to claim 1, wherein The target sub-pixel mapping file of the viewpoint records the viewpoint sorting calculation formula for the sub-pixels of the corresponding coordinates of the lenticular grating screen of the current display device under the corresponding screen resolution parameters, which is expressed as follows: Among them, i represents the integer part of the operation result, f represents the fractional part of the operation result; N represents the corresponding view point of the current sub-pixel, and (x, y) is the coordinate of the current sub-pixel in the pixel coordinate system. is the grating tilt angle, is the offset of the leftmost grating edge relative to the origin, is the arrangement period of the sub-pixels, is the total number of view points.

4. The adaptive resolution adjustment three-dimensional display architecture according to claim 3, wherein The display architecture configures a static area and a dynamic area based on the DSP resources. The static area includes a logic reconfiguration control unit, a logic processing unit, and a video stream output unit, and the dynamic area includes a viewpoint calculation and sampling unit; The processor converts the remainder and division of the viewpoint sorting calculation formula into multiplication and subtraction to adapt to hardware processing, calls the DSP resources of the viewpoint calculation and sampling unit for dynamic calculation, and encodes and samples the viewpoint sub-map according to the N value to generate a raster encoded map.

5. The adaptive resolution adjustment three-dimensional display architecture according to claim 4, characterized in that, After the viewpoint calculation and sampling unit obtains the target sub-pixel mapping file of the viewpoint, the logic reconfiguration control unit sends a loading completion signal to the processor; The processor updates the quantization parameters of the parameter register in the view point calculation and sampling unit based on the load completion signal and the total number of view points. The view point calculation and sampling unit performs calculations based on the updated quantization parameters.

6. The adaptive resolution adjustment three-dimensional display architecture according to any one of claims 1-5, characterized in that, When the input and output image sizes are not in proportion, set the screen resolution of the output video stream to H*V, and sample the vertical resolution of the viewpoint sub-image as V / v , and sample the horizontal resolution as H / h ; Set , ; represents the floor symbol, is the total number of viewpoints; When the horizontal resolution of the original video stream is greater than H / h perform horizontal downsampling; otherwise, perform horizontal upsampling. When the vertical resolution of the original video stream is greater than V / v perform vertical downsampling; otherwise, perform vertical upsampling.

7. The adaptive resolution adjustment three-dimensional display architecture according to claim 5, wherein After the viewpoint calculation and sampling unit completes encoding and sampling, it returns the raster encoded map to the video stream output unit in the static area. The video stream output unit integrates and outputs the video stream based on the timing logic and sends it to the display device.

8. The adaptive resolution adjustment three-dimensional display architecture according to claim 1, characterized in that The static area also includes a video stream input unit. The display architecture receives the original video stream, and the video stream input unit samples based on the timing instructions to obtain a number of sub-viewpoint images of the same scene from different viewpoints.

9. The adaptive resolution adjustment three-dimensional display architecture according to claim 8, characterized in that When the number of RGB images of the same scene from different viewpoints obtained from the original video stream is not less than the set value, splice the RGB images into one or more frames of images, and use hardware to segment the target number of sub-viewpoint images and cache them; When the number of RGB images of the same scene from different viewpoints obtained from the original video stream is less than the set value, obtain the RGB images and the corresponding depth images of the scene, and generate the target number of sub-viewpoint images through virtual viewpoint mapping.

Images