3D display system and 3D display method

Abstract

The present invention provides a 3D display system and a 3D display method. The 3D display system includes a 3D display, a memory, and one or more processors. The memory records multiple modules, and the processor accesses and executes the modules recorded in the memory. These modules include a bridge module and a 3D display service module. When the processor executes an application, the bridge module establishes a virtual extended screen and moves the application to the virtual extended screen. The bridge module obtains the two-dimensional content frame of the application from the virtual extended screen through a screen acquisition function. The 3D display service module converts the two-dimensional content frame into a 3D format frame by communicating with a third-party software development kit, and provides the 3D format frame to the 3D display for display.

Description

Technical Field

[0001] The present invention relates to a display system, and in particular to a 3D display system and a 3D display method. Background Art

[0002] With the rapid development of 3D stereoscopic display technology, people can be provided with an immersive experience. In terms of 3D stereoscopic display technology, it can be divided into glasses-type 3D image display technology and naked-eye 3D image display technology. Although glasses-type 3D image display technology is relatively mature, 3D glasses are inconvenient to use. Therefore, naked-eye 3D image display technology has gradually become the focus of various industries.

[0003] Naked 3D image display technology also uses the principle of binocular parallax, and uses the optical technology of the screen itself to allow both eyes to receive different images. In naked-eye 3D image display technology, lens-type 3D image display technology has the advantage of better light transmittance compared to shielded 3D image display technology, so it can provide brighter images to viewers in image presentation. However, no matter which naked-eye 3D image display technology is used, it must be combined with the specifications, hardware design and user-related parameters of the naked-eye 3D display to produce naked-eye 3D image content suitable for display by the naked-eye 3D display. However, there are not many naked-eye 3D image contents on the market, so even if users have a naked-eye 3D display, they still cannot fully and arbitrarily enjoy the display effect brought by the naked-eye 3D display. Summary of the invention

[0004] The present disclosure relates to a 3D display system and a 3D display method, which can allow users to fully experience the display effect brought by a 3D display.

[0005] An embodiment of the present invention provides a 3D display system, which includes a 3D display, a memory, and one or more processors. The memory records multiple modules, and the processor is connected to the 3D display and the memory to access and execute the modules recorded in the memory. These modules include a bridge module and a 3D display service module. When the processor executes an application, the bridge module establishes a virtual extended screen and moves the application to the virtual extended screen. The 3D display service module obtains the two-dimensional content frame of the application from the virtual extended screen through a screen acquisition function, converts the two-dimensional content frame into a 3D format frame by communicating with a third-party software development kit, and provides the 3D format frame to the 3D display for display.

[0006] An embodiment of the present invention provides a 3D display method, which is applicable to a 3D display system. The method comprises the following steps. When a processor executes an application, a virtual extended screen is established, and the application is moved to the virtual extended screen. A two-dimensional content frame of the application is acquired from the virtual extended screen through a screen acquisition function. The two-dimensional content frame is converted into a 3D format frame by communicating with a third-party software development kit. The 3D format frame is provided to a 3D display for display.

[0007] An embodiment of the present invention provides a 3D display system, which includes a 3D display, a memory, and one or more processors. The memory records a plurality of modules, and the processor is connected to the 3D display and the memory, accesses and executes the modules recorded in the memory to perform the following steps. When the processor executes an application, a virtual extended screen is established, and an application executed by the processor is moved to the virtual extended screen. The two-dimensional content frame of the application is obtained from the virtual extended screen through the screen acquisition function. The 3D display parameters associated with the 3D display are obtained. The two-dimensional content frame is converted into a 3D format frame according to the 3D display parameters and the image weaving function, and the 3D format frame is provided to the 3D display for display.

[0008] Based on the above, in an embodiment of the present invention, 2D content frames of various applications can be converted into 3D format frames conforming to a specific 3D format without making any changes to the applications, so as to expand the 3D content that can be displayed by the 3D display. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of a 3D display system according to an embodiment of the present invention;

[0010] Figure 2 is a schematic diagram of a 3D display according to an embodiment of the present invention;

[0011] Figure 3 is a schematic diagram of a 3D display according to an embodiment of the present invention;

[0012] Figure 4 is a flow chart of a 3D display method according to an embodiment of the present invention;

[0013] Figure 5 is a schematic diagram of a software architecture of a 3D display system according to an embodiment of the present invention;

[0014] Figure 6 FIG. 4 is a schematic diagram of a frame buffer space in a memory according to an embodiment of the present invention.

[0015] Description of Reference Numerals

[0016] 10: 3D display system;

[0017] 110: 3D display;

[0018] 120: memory;

[0019] 130: processor;

[0020] 121: application;

[0021] 122: bridge module;

[0022] 123: 3D display service module;

[0023] 124: side-by-side image processing module;

[0024] 125: Third-party software development kit;

[0025] 111: display panel;

[0026] 112: lens layer;

[0027] 511: Desktop application;

[0028] 512: Standardized applications;

[0029] 513: Special development applications;

[0030] 125_1: lens control module;

[0031] 125_2: eye tracking module;

[0032] 125_3: image weaving module;

[0033] L1: lens;

[0034] 140: Camera;

[0035] 53: main display screen;

[0036] 55: interface tool;

[0037] 61, 62: image frame cache space;

[0038] F1: Operation screen;

[0039] F2: 2D content frame;

[0040] F3, F_3d: 3D format image frame;

[0041] S201~S204: steps. DETAILED DESCRIPTION

[0042] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0043] Figure 1 is a schematic diagram of a 3D display system according to an embodiment of the present invention. Figure 1 , the 3D display system 10 may include a 3D display 110, a memory 120, and one or more processors (the processor 130 will be used as an example below). The processor 130 couples the 3D display 110 and the memory 120. The 3D display system 10 may be a single integrated system or a separate system. Specifically, the 3D display 110, the memory 120, and the processor 130 in the 3D display system 10 may be implemented as an all-in-one (AIO) electronic device, such as a laptop or a tablet computer. Alternatively, the 3D display 110 may be connected to the processor 130 of the computer system via a wired transmission interface or a wireless transmission interface.

[0044] The memory 120 is used to store data and program codes (such as operating systems, applications, and drivers) for access by the processor 130, and can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or a combination thereof. In one embodiment, the memory 120 also includes a memory of the image processing unit.

[0045] The processor 130 is coupled to the memory 120, and is, for example, a central processing unit (CPU), an application processor (AP), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image signal processor (ISP), graphics processing unit (GPU), or other similar devices, integrated circuits, and combinations thereof. The processor 130 can access and execute program codes and software modules recorded in the memory 120 to implement the 3D display method in the embodiment of the present invention.

[0046] Figure 2 is a schematic diagram of multiple modules according to an embodiment of the present invention. Figure 2In one embodiment, the memory 120 may store an application 121, a bridge module 122, a 3D display service module 123, a side-by-side image processing module 124, and a third-party software development kit 125. In one embodiment, the bridge module 122, the 3D display service module 123, and the side-by-side image processing module 124 are, for example, computer programs, programs, or instructions, which can be loaded into the processing unit 130, so that the 3D display system 10 can realize the function of converting the two-dimensional content frame of the application 121 into a 3D format frame. The bridge module 122, the 3D display service module 123, and the side-by-side image processing module 124 can, for example, be run as a background program in an operating system to implement the method proposed in the embodiment of the present invention, and the details are described in detail below.

[0047] Figure 3 is a schematic diagram of a 3D display according to an embodiment of the present invention. In one embodiment, the 3D display 110 is a naked-eye 3D display, which can provide different images with aberrations to the left eye and the right eye through the lens refraction principle, so that the viewer can experience a stereoscopic display effect. The 3D display 110 may include a display panel 111 and a lens layer 112. The lens layer 112 is disposed above the display panel 111, and the user sees the image content provided by the display panel 111 through the lens layer 112. The present invention does not limit the type and structure of the display panel 111, and the display panel 111 may also be a self-luminous display panel. The lens layer 112 refracts light to refract different display contents to different places in space, so that the human eye receives two images with parallax. The structure and implementation method of the 3D display 110 are not the focus of the present invention and will not be repeated here. It should be noted that, in one embodiment, the developer or supplier of the 3D display 110 will provide a third-party software development kit that meets the hardware design specifications of the 3D display 110, and the third-party software development kit can be recorded in the memory 120, such as the third-party software development kit 125 stored in the memory 120. In this way, the 3D display system 10 assembled with the 3D display 110 can realize the auto-stereoscopic 3D display function according to the third-party software development kit 125.

[0048] Figure 4 is a flow chart of a 3D display method according to an embodiment of the present invention. Figure 1 , Figure 2 and Figure 4 The method of this embodiment is applicable to the 3D display system 10 in the above embodiment. The following will describe the detailed steps of this embodiment with the components in the 3D display system 10.

[0049] In step S201, when the processor 130 executes an application 121, the bridge module 122 creates a virtual extended screen and moves the application 121 to the virtual extended screen. Here, the application 121 may include a desktop application or a browser application. In one embodiment, the bridge module 122 may use a driver / API of the Windows operating system to create a virtual extended screen. This virtual extended screen will not be displayed on the 3D display 110, but the image processing unit (not shown) of the 3D display system 10 will generate the image data of the virtual extended screen and configure the frame buffer space of the virtual extended screen in the memory. The bridge module 122 may move the window of the application 121 to the virtual extended screen in response to a user operation. The above-mentioned user operation is, for example, dragging the mouse or pressing a shortcut key.

[0050] Next, in step S202, the bridge module 122 may acquire the two-dimensional content frame of the application 121 from the virtual extended screen through a screen acquisition function. For example, the bridge module 122 may acquire the two-dimensional content frame of the application from the virtual extended screen through a screen acquisition technology such as the "DesktopDuplication API" of the Windows operating system. The two-dimensional content frame of the application 121 is, for example, image content provided by the application 121 operating in full-screen mode, but the present invention is not limited to this. The two-dimensional content frame may or may not include a user interface of the application 121. For example, assuming that the application 121 is a photo playback program, the two-dimensional content frame of the application 121 may be a photo played in full-screen mode.

[0051] In step S203, the 3D display service module 123 converts the two-dimensional content frame into a 3D format frame by communicating with the third-party software development kit 125. In detail, the 3D display service module 123 can obtain the two-dimensional content frame of the application 121 from the bridge module 122. In one embodiment, the bridge module 122 can be implemented as an application programming interface (API), so that the 3D display service module 123 can obtain the two-dimensional content frame of the application 121 through the bridge module 122. Then, the 3D display service module 123 can convert the two-dimensional content frame into a 3D format frame according to the 3D display parameters provided by the third-party software development kit 125 and the image weaving function provided by the third-party software development kit 125. In step S204, the 3D display service module 123 provides the 3D format frame to the 3D display 110 for display. It can be seen that in one embodiment, the bridge module 122 can move the application executed by the processor 130 to the virtual extended screen in response to the user operation. Furthermore, the 3D display 110 responds to the user operation and switches from the operation screen associated with the application 121 to displaying the 3D format image frame.

[0052] It is worth mentioning that, in one embodiment, the side-by-side image processing module 124 can receive the two-dimensional content frame of the application 121 from the 3D display service module 123, and determine whether the two-dimensional content frame conforms to the side-by-side image format. If not, the side-by-side image processing module 124 can convert the two-dimensional content frame into a side-by-side image format. Thus, the 3D display service module 123 can convert the two-dimensional content frame that conforms to the side-by-side image format into a 3D format frame. Here, the image that conforms to the side-by-side image format will include a left-eye image and a right-eye image that are side by side. In one embodiment, the side-by-side image processing module 124 can convert the two-dimensional content frame into a side-by-side image format through a neural network model.

[0053] However, it should be noted that the above embodiment is described by taking the 3D display parameters and image weaving functions provided by the third-party software development kit as an example. However, in other embodiments, the functions implemented by the bridge module 122, the 3D display service module 123, and the side-by-side image processing module 124 can also be integrated with the third-party software development kit 125. In this case, in one embodiment, the processor 130 can perform the following steps by executing multiple modules in the memory 120. When the processor 130 executes an application 121, the processor 130 establishes a virtual extended screen and moves the application 121 to the virtual extended screen. The processor 130 obtains the two-dimensional content frame of the application from the virtual extended screen through the screen acquisition function. The processor 130 obtains 3D display parameters associated with the 3D display, such as line of sight information, etc. The processor 130 converts the two-dimensional content frame into a 3D format frame according to the 3D display parameters and the image weaving function, and provides the 3D format frame to the 3D display 110 for display.

[0054] Figure 5 is a schematic diagram of the software architecture of a 3D display system according to an embodiment of the present invention. Figure 5 The applications executable by the 3D display system 10 include desktop applications 511, standardized applications 512 that comply with the Openxr standard or the Openvr standard, and special development applications 513.

[0055] In this embodiment, in order to generate a 3D format frame F_3d suitable for display by the 3D display 110 based on the two-dimensional content frame of the application, the bridge module 122 can obtain the two-dimensional content frame of the desktop application 511, the standardized application 512, or the special development application 513. Depending on the different types of applications, the bridge module 122 can use different methods to obtain the corresponding two-dimensional content frame, which will be described in detail later. Therefore, the 3D display service module 123 can convert the two-dimensional content frame into a 3D format frame F_3d based on the 3D display parameters provided by the third-party software development kit 125 and the image weaving function provided by the third-party software development kit 125. The image weaving function is suitable for converting image content that conforms to the side-by-side image format into a 3D format frame F_3d determined based on the hardware architecture of the 3D display 110 (such as the lens architecture or the panel pixel design, etc.).

[0056] In this embodiment, the third-party software development kit 125 may include a lens control module 125_1, an eye tracking module 125_2, and an image knitting module 125_3. The lens control module 125_1 is used to control the opening and closing of the lens L1 so that the 3D display 110 can perform 2D display or 3D display. In one embodiment, in response to the 3D display service module 123 detecting the user operation of enabling the 3D display function, the 3D display service module 123 can control the lens L1 to be turned on through the lens control module 125_1, and the eye tracking module 125_2 detects the user's line of sight information through the camera 140, so that the 3D format frame F_3d can be generated according to the user's line of sight information. In one embodiment, the eye tracking module 125_2 can report the 3D display parameters related to the line of sight information to the 3D display service module 123.

[0057] In addition, the side-by-side image processing module 124 can detect whether the two-dimensional content frame obtained by the bridge module 122 conforms to the side-by-side image format. If not, the side-by-side image processing module 124 can generate a two-dimensional content frame conforming to the side-by-side image format, and report the two-dimensional content frame conforming to the side-by-side image format to the 3D display service module 123. Afterwards, the 3D display service module 123 can generate a 3D format frame F_3d by using the library (Library) and 3D display parameters provided by the image knitting module 125_3.

[0058] In this embodiment, the 3D display service module 123 may transmit the 3D format frame F_3d to the main display screen 53, so that the 3D display 110 corresponding to the main display screen 53 displays the 3D format frame F_3d. That is, the 3D format frame F_3d is written into the frame buffer space corresponding to the main display screen 53. However, in other embodiments, the 3D display service module 123 may transmit the 3D format frame to the 3D display 110 for display through other wired or wireless transmission technologies.

[0059] It should be noted that, in the case where the processor 130 executes the desktop application 511, such as when the user operates a drawing software program to draw or uses a multimedia player to play images, the bridge module 122 can respond to the user operation and move the desktop application 511 to the virtual extended screen. Then the bridge module 122 can perform a screen capture operation on the virtual extended screen to obtain the two-dimensional content frame of the desktop application 511. In one embodiment, before the bridge module 122 moves the desktop application 511 to the virtual extended screen, the desktop application 511 can be switched to operate in full-screen mode, so that the virtual extended screen only includes the two-dimensional content frame of the desktop application 511 and does not include other application operation interfaces. In this way, the bridge module 122 can provide the two-dimensional content frame of the desktop application 511 to the 3D display service module 123 for subsequent operations.

[0060] Figure 6 FIG. 1 is a schematic diagram of a frame buffer space in a memory according to an embodiment of the present invention. Figure 6 After the virtual extended screen is established, the memory in the 3D display system may include a frame cache space 62 corresponding to the virtual extended screen and a frame cache space 61 corresponding to the main display screen. Before the application executed by the processor is moved to the virtual extended screen, the frame cache space 61 of the main display screen stores the operation screen F1 of the application, and the frame cache space 62 of the virtual extended screen has not yet stored any frame data. Then, after the application executed by the processor is moved to the virtual extended screen, the two-dimensional content frame F2 of the application will be recorded in the frame cache space 62 of the virtual extended screen, and the bridge module 122 can obtain the two-dimensional content frame F2 of the application from the frame cache space 62. Then, the frame cache space record 61 corresponding to the main display screen will store the 3D format frame F3 provided by the 3D display service module 123, so that the 3D display 110 can perform 3D display.

[0061] based on Figure 6 It can be seen from the description that by moving the application to the virtual extended screen, the bridge module 122 can continuously obtain multiple two-dimensional content frames of the application from the virtual extended screen, so as to achieve the purpose of continuously generating corresponding 3D format frames. On the contrary, if the virtual extended screen and the operation of moving the application to the virtual extended screen are not recommended (that is, only the main display screen is configured), the bridge module 122 will obtain a 3D format frame after the main display screen displays the 3D format frame F3, and will not be able to continue to obtain the two-dimensional content frame of the application. Therefore, the 3D display system will not be able to continue to perform the expected 3D display based on the two-dimensional content frame of the application.

[0062] On the other hand, in one embodiment, for a standardized application 512 that complies with the Openxr standard or the Openvr standard, the bridge module 122 can realize an execution environment (runtime) that can execute the standardized application 512 through an interface tool 55. Based on this, the bridge module 122 can obtain the two-dimensional content frame of the standardized application 512 through the interface tool 55, and provide the two-dimensional content frame of the standardized application 512 to the 3D display service module 123 for subsequent operations. In addition, in one embodiment, for the special development application 513, it is developed based on the bridge module 122 as an application programming interface. Therefore, the special development application 513 can provide the two-dimensional content frame to the 3D display service module 123 through the bridge module 122 for subsequent operations.

[0063] In summary, in the embodiments of the present invention, the two-dimensional content frames of various applications can be converted into 3D format frames that conform to a specific 3D format without making any changes to the application, so as to enrich the 3D content that can be displayed by the auto-stereoscopic 3D display. In this way, users can convert the two-dimensional content frames of various applications into 3D format for auto-stereoscopic 3D display, which is not limited by the image format and has a wide range of applications.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D display system, characterized in that: include: 3D display, providing different images with disparity to the left and right eyes; Memory,records have multiple modules; as well as A processor is connected to the 3D display and the memory to access and execute the module recorded in the memory, wherein the module includes: a bridge module, which, when the processor executes the application, creates a virtual extended screen that will not be displayed on the 3D display, moves the window of the application to the virtual extended screen, and acquires the two-dimensional content frame of the application from the virtual extended screen through a screen acquisition function provided by an application program interface of the operating system; and a 3D display service module, which obtains the two-dimensional content frame of the application from the bridge module, converts the two-dimensional content frame into a 3D format frame by communicating with a third-party software development kit, and provides the 3D format frame to the 3D display for display, The memory includes a frame cache space corresponding to the virtual extended screen and a frame cache space corresponding to the main display screen. After the window of the application executed by the processor is moved to the virtual extended screen, the frame cache space corresponding to the main display screen records the 3D format frame, so that the 3D display serving as the main display screen displays the 3D format frame.

2. The 3D display system according to claim 1, characterized in that: The module also includes a side-by-side image processing module, which receives the two-dimensional content frame of the application from the 3D display service module and determines whether the two-dimensional content frame conforms to the side-by-side image format; if not, converts the two-dimensional content frame to conform to the side-by-side image format.

3. The 3D display system according to claim 1, characterized in that: The 3D display service module converts the two-dimensional content frame into the 3D format frame according to the 3D display parameters provided by the third-party software development toolkit and using the image weaving function provided by the third-party software development toolkit.

4. The 3D display system according to claim 1, characterized in that: The 3D display service module transmits the 3D format image frame to the main display screen, so that the 3D display displays the 3D format image frame.

5. The 3D display system according to claim 1, characterized in that: The bridge module moves the application executed by the processor to the virtual extended screen in response to a user operation, and the 3D display switches from an operation screen associated with the application to displaying the 3D format frame in response to the user operation.

6. A 3D display method, applicable to a 3D display system, characterized in that: include: When the processor executes the application, a virtual extended screen is created that will not be displayed on the 3D display, and a window of the application is moved to the virtual extended screen; Acquire the two-dimensional content frame of the application from the virtual extended screen through a screen acquisition function provided by an application program interface of the operating system; Converting the two-dimensional content image frame into a 3D format image frame by communicating with a third-party software development kit; as well as The 3D format image frame is provided to the 3D display for display, and the 3D display provides different images with disparity to the left eye and the right eye, The memory of the 3D display system includes a frame buffer space corresponding to the virtual extended screen and a frame buffer space corresponding to the main display screen. The method further includes: after moving the application executed by the processor to the virtual extended screen, recording the 3D format frame to a frame buffer space corresponding to the main display screen, so that the 3D display serving as the main display screen displays the 3D format frame.

7. The 3D display method according to claim 6, characterized in that: The method further comprises: Determining whether the two-dimensional content frame conforms to a side-by-side image format; and If not, the two-dimensional content frame is converted to conform to the side-by-side image format.

8. The 3D display method according to claim 6, characterized in that: The step of converting the two-dimensional content frame into the 3D format frame by communicating with the third-party software development kit comprises: The two-dimensional content image frame is converted into the 3D format image frame according to the 3D display parameters provided by the third-party software development toolkit and by utilizing the image weaving function provided by the third-party software development toolkit.

9. The 3D display method according to claim 6, characterized in that: The step of providing the 3D format image frame to the 3D display for display comprises: The 3D format image frame is transmitted to the main display screen, so that the 3D display displays the 3D format image frame.

10. The 3D display method according to claim 9, characterized in that: The application is moved to the virtual extended screen in response to the user operation, and the 3D display is switched from an operation screen associated with the application to display the 3D format frame in response to the user operation.

11. A 3D display system, characterized in that: include: 3D display, providing different images with disparity to the left and right eyes; Memory,records have multiple modules; as well as A processor is connected to the 3D display and the memory, accesses and executes the module recorded in the memory to: When the processor executes the application, a virtual extended screen is created that will not be displayed on the 3D display, and a window of the application is moved to the virtual extended screen; Acquire the two-dimensional content frame of the application from the virtual extended screen through a screen acquisition function provided by an application program interface of the operating system; Acquiring 3D display parameters associated with the 3D display; as well as converting the two-dimensional content image frame into a 3D format image frame according to the 3D display parameter and the image knitting function, and providing the 3D format image frame to the 3D display for display, The memory includes a frame cache space corresponding to the virtual extended screen and a frame cache space corresponding to the main display screen. After the application executed by the processor is moved to the virtual extended screen, the frame cache space corresponding to the main display screen records the 3D format frame, so that the 3D display serving as the main display screen displays the 3D format frame.

12. The 3D display system according to claim 11, characterized in that: The processor executes the module to: determine whether the two-dimensional content frame conforms to a side-by-side image format; if not, convert the two-dimensional content frame into conforming to the side-by-side image format.

13. The 3D display system according to claim 11, characterized in that: The processor executes the modules to: transmit the 3D format image frame to the main display screen, so that the 3D display displays the 3D format image frame.

14. The 3D display system according to claim 13, characterized in that: The processor executes the module to: move the application executed by the processor to the virtual extended screen in response to a user operation, wherein the 3D display switches from an operation screen associated with the application to display the 3D format frame in response to the user operation.

Images