A holographic display method and device based on holographic sand table

By combining sand tray and head-mounted display devices in the holographic sand tray display system, using a motion capture camera to match the user's perspective, the problem of insufficient three-dimensionality of the existing 3D display carrier is solved, and a holographic with adjustable brightness and small size is achieved Image display improves user experience.

CN113376858BActive Publication Date: 2024-11-12SHENZHEN REALIS MULTIMEDIA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110606420.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-27
Publication Date
2024-11-12
Estimated Expiration
2041-05-27

AI Technical Summary

Technical Problem

Most of the existing 3D display carriers are vertical screens, with a weak sense of three-dimensionality and a weak sense of interaction. The existing VR glasses require complex data processing, resulting in large size and weight of the device and poor user physical feeling.

Method used

Using a holographic display method based on holographic sand table, the left and right eye display signals are generated by combining the sand table display device and the head-mounted display device, and the motion capture camera is used to match the user's perspective to realize alternating image display, and in darker scenes Adjust the brightness to improve the display effect.

Benefits of technology

It realizes holographic image display with strong three-dimensional sense and high picture quality, reduces the volume and weight of the head-mounted display device, and improves the user's somatosensory comfort and interactivity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113376858B_ABST
    Figure CN113376858B_ABST
Patent Text Reader

Abstract

The present application discloses a holographic display method and device based on a holographic sandbox, which is based on a holographic display system. The holographic display system includes: a display processing device, a sandbox display device and a head-mounted display device. The display processing device is electrically connected to the sandbox display device and the head-mounted display device respectively. Among them, the display processing device generates a first display signal for the left eye and a first display signal for the right eye, processes the first display signal for the left eye and the first display signal for the right eye, generates a first display signal, and outputs it to the sandbox display device, so that the sandbox display device outputs an image according to the first display signal. In addition, the display processing device generates a synchronization signal and sends it to the head-mounted display device, so that the head-mounted display device collects the image output by the sandbox display device according to the synchronization signal, so as to alternately display the collected image to the user wearing the head-mounted display device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of simulation display technology, and in particular to a holographic display method and device based on a holographic sandbox. Background Art

[0002] Front-Projected Holographic Display, also known as virtual imaging technology, is a technology that uses the principles of interference and diffraction to record and reproduce the real three-dimensional image of an object. It has the advantage of being able to meet all the perceptions of human vision, and viewers can even watch three-dimensional images without the help of auxiliary devices such as helmets and glasses. With the continuous development of display technology, holographic display technology has received more and more attention.

[0003] In application scenarios such as cinemas, science and technology museums, museums, and even virtual reality (VR) helmets, arc-shaped or ring-shaped holographic images can enable viewers to obtain a surround-view effect, thereby greatly improving the viewer's viewing experience and making the viewer feel immersive.

[0004] Most existing 3D display carriers are vertical screens. For example, when watching 3D movies in a movie theater, the stereoscopic effect is not strong, the viewing experience of the 3D picture is not good, and the sense of interaction is not strong. It is very rare to lay the screen flat to watch the holographic 3D picture. Summary of the invention

[0005] The embodiments of the present application provide a holographic display method and device based on a holographic sandbox to at least partially solve the above technical problems.

[0006] The present application embodiment adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a holographic display method based on a holographic sandbox, wherein the method is based on a holographic display system, wherein the holographic display system comprises: a display processing device, a sandbox display device, and a head-mounted display device; the sandbox display device is a holographic sandbox;

[0008] The display processing device is electrically connected to the sandbox display device and the head-mounted display device respectively; the holographic display method based on the holographic sandbox is executed by the display processing device, and the method includes:

[0009] Generate a left-eye first display signal and a right-eye first display signal;

[0010] Processing the left-eye first display signal and the right-eye first display signal to generate a first display signal, and outputting the first display signal to the sandbox display device, so that the sandbox display device outputs an image according to the first display signal;

[0011] Generate a synchronization signal and send it to the head-mounted display device, so that the head-mounted display device collects the images output by the sandbox display device according to the synchronization signal, so as to alternately display the collected images to the user wearing the head-mounted display device.

[0012] In an optional embodiment of the present specification, the holographic display system further includes a receiving card;

[0013] Generating a first display signal and outputting it to the sandbox display device includes:

[0014] A first display signal is generated and sent to the receiving card, so that the receiving card outputs the first display signal to the sandbox display device.

[0015] In an optional embodiment of the present specification, generating a left-eye display signal and a right-eye display signal includes:

[0016] Based on the position signal, a left-eye display signal and a right-eye display signal are generated so that the left-eye display signal and the right-eye display signal match the user's viewing angle and the image displayed to the user by the sandbox display device, and the position signal shows the position of the user's two eyes.

[0017] In an optional embodiment of the present specification, the holographic display system further comprises: a motion capture camera;

[0018] The position signal is obtained based on the data collected by the motion capture camera.

[0019] In an optional embodiment of the present specification, the motion capture camera is arranged on the sandbox display device.

[0020] In an optional embodiment of the present specification, the display processing device includes: a display server, a display processor and a synchronization transmitter;

[0021] The left-eye display signal and the right-eye display signal are generated by the display server, the first display signal is generated by the display processor, and the synchronization signal is sent to the head-mounted display device by the synchronization transmitter.

[0022] In an optional embodiment of this specification, the method further includes:

[0023] When an interaction instruction is received, a second display signal is generated and output to the sandbox display device, so that the sandbox display device outputs an image for displaying the interaction handle according to the second display signal, so that the user can operate the interaction handle to achieve interaction; the interaction includes at least one of the following: enlarging, reducing, moving, rotating, selecting, and replacing the image output by the sandbox display device.

[0024] In a second aspect, the embodiments of the present application further provide a holographic display device for realizing any one of the aforementioned holographic display devices.

[0025] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0026] Processor; and

[0027] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of the first aspect.

[0028] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method in the first aspect.

[0029] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the holographic display method and device based on the holographic sandbox in this specification adopts a sandbox display device and a head-mounted display device to display holographic images to users. The sandbox display device has strong controllability, and its display brightness can be adjusted according to the brightness of the scene in which it is located. Even in darker scenes, a good holographic image display effect can be obtained. Since the holographic image in this specification is assisted by the sandbox display device, the head-mounted display device in this specification does not need to perform more complex data processing compared to existing VR glasses, which is conducive to reducing the volume and weight of the head-mounted display device, making the user feel more comfortable. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 A schematic diagram of a holographic display scene provided in an embodiment of this specification;

[0032] Figure 2 A schematic diagram of a holographic display process based on a holographic sandbox provided in an embodiment of this specification;

[0033] Figure 3 A schematic diagram of the structure of a holographic display device provided in an embodiment of this specification;

[0034] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, and this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0036] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0037] The serial numbers of the components in this document, such as "second", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0038] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0039] In order to realize holographic display, this specification provides a holographic display method based on a holographic sandbox. The holographic display method based on a holographic sandbox is based on a holographic display system, and the holographic display system includes: a display processing device, a sandbox display device and a head-mounted display device. The display processing device is electrically connected to the sandbox display device and the head-mounted display device respectively. The scene of using the holographic display system to display a holographic image to a user is schematically shown as follows: Figure 1 As shown, in Figure 1 In the scene shown, the sandbox display device is fixedly arranged in the holographic display venue, and the user wears the head-mounted display device on his body. In an optional embodiment of the present specification, the display processing device can be integrated with the sandbox display device. The head-mounted display device can be glasses with display function.

[0040] This specification does not impose any specific restrictions on the arrangement of the sandbox display device in the holographic display venue. For example, the sandbox display device can be hung on a wall or arranged on the ground.

[0041] This specification does not specifically limit the number of sandbox display devices and head-mounted display devices included in the holographic display system. In the case of multiple sandbox display devices, the placement of different sandbox display devices in the venue can be the same or different. In the case of multiple head-mounted display devices, different head-mounted display devices can be worn by different users.

[0042] The holographic display process based on the holographic sandbox in this specification can be executed by a display processing device in a holographic display system, and the holographic display process based on the holographic sandbox can include at least part of the following steps.

[0043] S200: The display processing device generates a left-eye first display signal and a right-eye first display signal.

[0044] The process in this specification is intended to output a holographic image to the user. In order to enable the user to experience the stereoscopic sense of the holographic image, the display processing device outputs display signals according to the perspectives of the user's two eyes, namely, the left eye first display signal and the right eye first display signal. There is a certain difference between the left eye first display signal and the right eye first display signal to simulate the difference in the perspectives of the user's two eyes, thereby realizing a display with a stereoscopic sense.

[0045] In an optional embodiment of the present specification, the image specifications corresponding to the left-eye first display signal and the right-eye first display signal are both 1920 pixels*1080 pixels, and the output frequencies of the left-eye first display signal and the right-eye first display signal are both 60 Hz.

[0046] S202: The display processing device processes the left-eye first display signal and the right-eye first display signal to generate a first display signal, and outputs the first display signal to the sandbox display device.

[0047] The display processing device processes the left-eye first display signal and the right-eye first display signal so that the obtained first display signal can be displayed through the sandbox display device.

[0048] In an optional embodiment of the present invention, the sandbox display device is an LED (Light Emitting Diode) display device. The image specification corresponding to the first display signal is 1920 pixels * 1080 pixels, and the output frequency of the first display signal is 120 Hz, so as to realize the output of the picture to the user's binocular alternately at a frequency of 60 Hz.

[0049] After receiving the first display signal, the sandbox display device outputs an image according to the first display signal for viewing by a user wearing the head mounted display device.

[0050] S204: The display processing device generates a synchronization signal and sends it to the head mounted display device.

[0051] In order to make the sandbox display device and the head-mounted display device cooperate, the display processing device generates a synchronization signal according to the left eye first display signal and the right eye first display signal, so that the head-mounted display device collects the image output by the sandbox display device according to the synchronization signal, so as to alternately display the collected image to the user wearing the head-mounted display device. It can be seen that the synchronization signal is used to realize the output screen of the user's binoculars alternately.

[0052] In an optional embodiment of the present specification, the head-mounted display device is shutter-type 3D glasses.

[0053] It should be noted that the execution order of step S202 and step S204 is not particular. In an optional embodiment, step S202 and step S204 are executed simultaneously.

[0054] The holographic display method based on the holographic sandbox in this specification uses a sandbox display device and a head-mounted display device to display holographic images to users. The sandbox display device has strong controllability, and its display brightness can be adjusted according to the brightness of the scene in which it is located. Even in darker scenes, a good holographic image display effect can be obtained. Since the holographic image in this specification is assisted by the sandbox display device, the head-mounted display device in this specification does not need to perform more complex data processing compared to existing VR glasses, which is conducive to reducing the volume and weight of the head-mounted display device, making the user feel more comfortable.

[0055] In addition, existing 3D display products usually have only a very narrow viewing angle or can only see the ideal effect at a fixed position, that is, existing 3D display products only support a single viewing angle, but the 3D stereoscopic interactive display system with a single viewing angle is increasingly unable to meet customer needs. The holographic display method based on the holographic sandbox in this specification adopts a sandbox display device and a head-mounted display device to cooperate, and even if the user's viewing angle changes (which may be caused by the user's movement), it can also display a holographic image with a three-dimensional sense to the user, which is conducive to improving the user experience.

[0056] To achieve data transmission between the display processing device and the sandbox display device, the holographic display system may further include a receiving card. The display processing device generates a first display signal and sends it to the receiving card, so that the receiving card outputs the first display signal to the sandbox display device.

[0057] It can be seen that the receiving card in this specification has a certain interface function to cooperate with multiple sandbox display devices to display images simultaneously, and / or to cooperate with sandbox display devices of different models to display images. In addition, the receiving card in this specification also has a certain driving function, and the receiving card drives the sandbox display device to generate images corresponding to the user's eyes respectively according to the first display signal.

[0058] Through the holographic display method based on the holographic sandbox in this specification, a holographic image with strong stereoscopic sense and high image quality can be displayed to the user in a variety of scenes with different brightness. In actual scenes, the user may move in the venue where the sandbox display device is set according to his own needs, and there may be a phenomenon that the left-eye first display signal and the right-eye first display signal do not match the user's current viewing angle due to the user's movement.

[0059] In order to provide a holographic image with excellent display effect during the movement of the user, in an optional embodiment of the present specification, the holographic display system also includes: a motion capture camera. The motion capture camera is arranged on the sandbox display device. The number of motion capture cameras in this specification, the setting positions on the sandbox display device, and the shooting angles are determined according to the setting placement, shape, size, etc. of the sandbox display device. Specifically, the number of motion capture cameras is 4, which are respectively arranged at the corners of the sandbox display device. The motion capture camera is used to capture images of the environment around the sandbox display device to capture the user's position in the scene, and then obtain the position of the head-mounted display device relative to the sandbox display device to generate a position signal.

[0060] In other implementations of this specification, the motion capture camera can be set above the sandbox display device to capture images of the environment from a bird's-eye view.

[0061] After obtaining the position signal, the display processing device generates a left-eye display signal and a right-eye display signal according to the position signal, so that the left-eye display signal and the right-eye display signal match the user's viewing angle and the image displayed to the user by the sandbox display device.

[0062] In an optional embodiment of the present specification, the motion capture camera first obtains the 3D coordinate information of the head-mounted display device in space, and then combines the obtained human eye view coordinate information of the user wearing the head-mounted display device in the motion capture space to complete the tracking and positioning of the user's viewing angle, and then transmits the positioning information to the display processing device. The position signal of the display processing device in the motion capture space is synchronized to the holographic image, and the three-dimensional position of the head-mounted display device in the motion capture space is used as the position of the virtual camera in the holographic image (i.e., the position of the head-mounted display device) to calculate the view angle of the virtual camera.

[0063] Furthermore, the head-mounted display device includes shutter-type active 3D glasses and a rigid body composed of at least three reflective marking points. The reflective marking points may be 4. The at least three reflective marking points are embedded in the shell structure of the shutter-type active 3D glasses. The shutter-type active 3D glasses are used to receive the viewing angle picture (3D signal) transmitted by the sandbox display device, so as to obtain the correct left and right eye pictures. Specifically, the shutter-type active 3D glasses receive the 3D signal emitted by the 3D signal transmitter in the sandbox display device.

[0064] The rigid body composed of at least three reflective marking points is used to mark the position of the shutter-type active 3D glasses in the motion capture space, and can capture the position information of the head-mounted display device in the motion capture space, as well as the perspective position information of the user wearing the head-mounted display device in the motion capture space. By capturing the position of the shutter-type active 3D glasses in the motion capture space, the user can see the three-dimensional picture of the display content floating in the air by wearing the shutter-type active 3D glasses, and can watch the three-dimensional picture (i.e., holographic image) from various angles.

[0065] It should be noted that, in an optional embodiment of the present specification, the display processing device may be composed of a plurality of apparatuses, and the functions of the display processing device may be implemented by the cooperation of the apparatuses constituting the display processing device.

[0066] Specifically, the display processing device may include: a display server, a display processor, and a synchronization transmitter. The left-eye display signal and the right-eye display signal are generated by the display server, and after the display server generates the left-eye display signal and the right-eye display signal, it sends them to the display processor. The display processor generates a first display signal according to the left-eye display signal and the right-eye display signal, and sends it to the sandbox display device. The synchronization signal is sent to the head-mounted display device by the synchronization transmitter.

[0067] Optionally, the display server is electrically connected to the display processor via a DP or HDMI cable; the display processor is electrically connected to the receiving card via a network port; the display processor is electrically connected to the synchronous transmitter via a cable; and the receiving card is electrically connected to the sandbox display device via a flat cable.

[0068] In an optional embodiment of the present specification, a sandbox display device may be composed of a small-pitch LED display screen, and the sandbox display device includes an LED display screen body, a three-dimensional video amalgamator, a three-dimensional signal transmitter, and the like.

[0069] The 3D video fusion device is connected to the sending card of the LED display screen. The 3D signal transmitter is connected to the 3D video fusion device, which receives the visual image information of the virtual camera (i.e., the head-mounted display device) processed by the display processor in the display processing device, adjusts the display frequency and displays the image on the LED display screen through the sending card to display the left and right eye images. The display processor is equipped with a holographic 3D display algorithm. The 3D signal transmitter synchronously sends the display frequency to the holographic 3D glasses to realize real-time display of the image adjusted according to the different viewing angles of the user wearing the holographic 3D glasses.

[0070] In this description, in a virtual three-dimensional scene, the three-dimensional position of the head-mounted display device in the motion capture space is used as the position of the virtual camera in the virtual space. In this way, the position of a virtual camera can be simulated to capture the image at the location of the head-mounted display device. The virtual three-dimensional scene is set according to the position of the virtual camera in the virtual space.

[0071] The image captured by the virtual camera (i.e., the image from the perspective of the head-mounted display device) is the three-dimensional virtual scene image (i.e., holographic image) seen from the perspective of the user wearing the head-mounted display device, that is, the virtual three-dimensional scene captured by the virtual camera is the 3D perspective image viewed by the user. The display processing device can process the image to be displayed in real time according to the perspective position information of the head-mounted display device in the motion capture space, and display the perspective image using a sandbox display device, so as to achieve an excellent holographic three-dimensional display effect, and obtain an excellent screen-out feeling and holographic visual feeling.

[0072] Compared with other products, the holographic display system provided by this application has greatly reduced costs, and the picture display is more delicate and rich, and the colors are richer. The display effect is stable, the user is more convenient to use, and the operation and control are simpler. Among them, the sandbox display device has a delicate display effect, high brightness, and low cost. It uses LED active 3D stereoscopic display technology combined with the spatial position information of the observation angle to achieve a 3D stereoscopic effect of virtual objects suspended on the LED display screen, which can improve the user's viewing experience and improve the effect of holographic display.

[0073] In addition, in an optional embodiment of the present specification, the process in the present specification can not only be used to display holographic images to users, but also to achieve interaction with users. Specifically, when receiving an interaction instruction, the display processing device generates a second display signal and outputs it to the sandbox display device, so that the sandbox display device outputs an image for displaying an interactive handle according to the second display signal, so that the user can operate the interactive handle to achieve interaction; the interaction includes at least one of the following: zooming in, reducing, moving, rotating, selecting, and replacing the image output by the sandbox display device.

[0074] Optionally, the interactive instruction is triggered by a specified action of the user, and the specified action may be captured by a motion capture camera. The image of the interactive handle may be a stick-shaped image, and one end of the interactive handle cooperates with the hand movement of the user and moves in the image generated by the sandbox display device following the hand movement of the user.

[0075] In an optional embodiment of the present specification, the display processing device adopts the UE4 processing system. The holographic display system and the holographic display method based on the holographic sandbox in the present specification are applicable to a wide range of scenarios, specifically, they can be applied to military situation deduction, industrial equipment simulation operation, simulation design, education and teaching training room, medical simulation training and other scenarios.

[0076] In addition, in other optional embodiments of the present specification, when the sandbox display device displays a holographic image according to a first display signal, if an interaction instruction is received, the hand position of the user wearing the head-mounted display device is identified; the viewpoint information of the user is obtained, and the viewpoint information shows the position of the user's eyes and the viewing angle of the user's eyes; according to the hand position, a first position of a handheld end for generating an interactive handle is determined; and according to the hand position and the viewpoint information, a second position of an indicating end for generating an interactive handle is determined; the handheld end is one end of the interactive handle, and the interactive end is the other end of the interactive handle; according to the first position and the second position, a second display signal is generated and sent to the sandbox display device, so that the sandbox display device displays a holographic image according to the first display signal and the second display signal.

[0077] In an optional embodiment of the present specification, the method further includes: determining whether the sandbox display device is displaying a holographic image according to a first display signal; if so, detecting an interactive instruction; if not, executing the determination again.

[0078] In an optional embodiment of the present specification, a second position of an indicating end for generating an interactive handle is determined based on the hand position and the viewpoint information, including: extending a specified length from the hand position as a starting point so that the end point obtained by the extension intersects with the line of sight of the user, and determining that the end point is the second position; the line of sight of the user is obtained based on the viewpoint information.

[0079] In an optional embodiment of the present specification, the interaction achieved through the interactive handle includes at least one of the following: zooming in, zooming out, moving, rotating, selecting, and replacing the image output by the sandbox display device.

[0080] In an optional embodiment of the present specification, the method further includes: determining whether the outline of the holographic image displayed by the sandbox display device intersects with the indicating end of the interactive handle; if not, enlarging the holographic image displayed by the sandbox display device until the outline of the holographic image intersects with the indicating end of the interactive handle.

[0081] In an optional embodiment of the present specification, the method further includes: generating a left-eye first display signal and a right-eye first display signal according to the viewpoint information of the user; processing the left-eye first display signal and the right-eye first display signal to generate a first display signal, and outputting it to the sandbox display device, so that the sandbox display device outputs an image according to the first display signal; generating a synchronization signal, and sending it to the head-mounted display device, so that the head-mounted display device collects the image output by the sandbox display device according to the synchronization signal, so as to alternately display the collected images to the user wearing the head-mounted display device.

[0082] In an optional embodiment of the present specification, the holographic display system further includes: a virtual camera; the position of the virtual camera is the same as the position of the user's binoculars, and the viewing angle of the virtual camera is the same as the viewing angle of the user's binoculars; the virtual camera is used to capture images of the environment in which the head-mounted display device is located according to its position and viewing angle. The method further includes: acquiring the image captured by the virtual camera; generating a first display signal based on the image captured by the virtual camera, so that at least part of the content in the environment is shown in the holographic image displayed by the sandbox display device.

[0083] In an optional embodiment of the present specification, the holographic display system further includes: a motion capture camera; and the user's hand position is obtained based on data collected by the motion capture camera.

[0084] When the holographic display system in this specification is used to display for multiple users at the same time, the head-mounted display device can be a customized radio frequency stereo glasses supporting dual viewpoints. The display processing device controls the opening and closing of the radio frequency stereo glasses through the synchronous transmitter for two customized radio frequency stereo glasses, so that the radio frequency stereo glasses supporting dual viewpoints A and the radio frequency stereo glasses supporting dual viewpoints B can display their respective tracked viewpoints in time sharing; the viewpoints of the radio frequency stereo glasses supporting dual viewpoints A and the radio frequency stereo glasses supporting dual viewpoints B corresponding to the viewpoints are output in time sharing to the sandbox display device, so that the sandbox display device can display the radio frequency stereo glasses supporting dual viewpoints A and the radio frequency stereo glasses supporting dual viewpoints B in time sharing.

[0085] The coordination process of customized dual-viewpoint RF stereo glasses and sandbox display device is, for example: the display processing device controls the switching of the RF stereo glasses through the synchronous transmitter for two pairs of customized RF stereo glasses, so that the dual-viewpoint RF stereo glasses A and the dual-viewpoint RF stereo glasses B can display their respective tracked viewpoints in a time-sharing manner; the viewpoints of the dual-viewpoint RF stereo glasses A and the dual-viewpoint RF stereo glasses B corresponding to the viewpoints are output to the sandbox display device in a time-sharing manner, so that the sandbox display device can display the dual-viewpoint RF stereo glasses A and the dual-viewpoint RF stereo glasses B in a time-sharing manner.

[0086] In addition, the synchronization signal is output to the 3D synchronization transmitter at the same time, and the 3D synchronization transmitter transmits the radio frequency signal to control the opening and closing of the two pairs of dual-viewpoint radio frequency stereo glasses, so that the two pairs of dual-viewpoint radio frequency stereo glasses can independently observe their corresponding stereo videos, that is, dual-viewpoint radio frequency stereo glasses A can only see stereo video signal A, and dual-viewpoint radio frequency stereo glasses B can only see stereo video signal B. By using dual-viewpoint technology, two independent tracked viewpoints can be displayed in time-sharing on the same display screen in the same VR environment, so that the dual-viewpoint radio frequency stereo glasses A and the dual-viewpoint radio frequency stereo glasses B have corresponding viewpoints, thereby improving the immersive feeling of the system.

[0087] When tracking a user, for each dynamic auxiliary camera, an image captured by the dynamic auxiliary camera is obtained; a user included in the image captured by the dynamic auxiliary camera is identified; if the image captured by the dynamic auxiliary camera does not include a user, the number of users photographed by other dynamic auxiliary cameras other than the dynamic auxiliary camera is identified; if the number of users photographed by other dynamic auxiliary cameras is greater than a preset first threshold number, the other dynamic auxiliary camera is determined to be a reference camera; and the shooting angle of the dynamic auxiliary camera is adjusted so that the difference between the adjusted shooting angle of the dynamic auxiliary camera and the shooting angle of the reference camera is not greater than a preset threshold angle.

[0088] In an optional embodiment of the present specification, after adjusting the shooting angle of the dynamic auxiliary camera, it also includes: obtaining the shooting angle of each dynamic auxiliary camera, and determining whether there is a shooting blind spot within a specified range around the sand table display device; if the judgment result is yes, then among each dynamic auxiliary camera, determining the dynamic auxiliary camera whose shooting angle is the smallest different from the shooting blind spot, as the first target camera; adjusting the shooting angle of the first target camera so that the shooting blind spot is within the adjusted shooting angle of the first target camera.

[0089] In an optional embodiment of the present specification, the method also includes: when it is detected that the number of users in the images captured by each dynamic auxiliary camera is less than a second threshold number, adjusting the shooting angle of each dynamic auxiliary camera so that the shooting angle of each dynamic auxiliary camera is evenly distributed around the sand table display device; the second threshold number is less than the first threshold number.

[0090] In an optional embodiment of the present specification, the method further includes: for each dynamic auxiliary camera, determining the movement speed of the user in the image captured by the dynamic auxiliary camera within a specified time period from the current moment; if the movement speed of the user is greater than a threshold speed, determining that the dynamic auxiliary camera is the second target camera; among the other dynamic auxiliary cameras except the second target camera, determining the dynamic auxiliary camera with the smallest difference in shooting angle with the second target camera as the third target camera; adjusting the shooting angle of the third target camera so that the image captured by the third target camera includes the user whose movement speed is greater than the threshold speed. In an optional embodiment of the present specification, the dynamic auxiliary cameras are arranged around the sandbox display device to capture images around the sandbox display device; or, the dynamic auxiliary cameras are suspended above the sandbox display device to capture images around the sandbox display device.

[0091] In an optional embodiment of the present specification, the method further includes: generating a left-eye first display signal and a right-eye first display signal according to the viewpoint information of the user; processing the left-eye first display signal and the right-eye first display signal to generate a first display signal, and outputting it to the sandbox display device, so that the sandbox display device outputs an image according to the first display signal; generating a synchronization signal, and sending it to the customized dual-viewpoint supporting radio frequency stereo glasses, so that the customized dual-viewpoint supporting radio frequency stereo glasses collect the image output by the sandbox display device according to the synchronization signal, so as to alternately display the collected images to the user wearing the customized dual-viewpoint supporting radio frequency stereo glasses.

[0092] In an optional embodiment of the present specification, the holographic display system further includes: a virtual camera; the position of the virtual camera is the same as the position of the user's binoculars, and the viewing angle of the virtual camera is the same as the viewing angle of the user's binoculars; the virtual camera is used to capture images of the environment in which the customized dual-viewpoint radio frequency stereo glasses are located according to its position and viewing angle;

[0093] The method further includes: acquiring an image captured by the virtual camera; and generating a first display signal according to the image captured by the virtual camera, so that at least part of the content in the environment is shown in the holographic image displayed by the sandbox display device.

[0094] In an optional embodiment of the present specification, there are multiple customized radio frequency stereo glasses supporting dual viewpoints, and the customized radio frequency stereo glasses supporting dual viewpoints correspond to users one by one.

[0095] In addition, images captured by each dynamic-assisted camera can be obtained; users in each image can be identified; for each identified user, it is determined whether there is an image in which the user is located in a valid area of ​​the image; the valid area is an area in the image where the distance from each edge of the image is greater than a specified distance; the specified distance is obtained based on the size of the image, and the specified distance is not greater than one-fourth of the size of the shorter side of the image; if the judgment result is no, the dynamic-assisted camera whose captured images contain the least number of users is determined among the dynamic-assisted cameras as the designated dynamic-assisted camera; and the user is determined to be the designated user; the shooting angle of the designated dynamic-assisted camera is adjusted so that the designated user is located in the valid area of ​​the image captured by the designated dynamic-assisted camera.

[0096] In an optional embodiment of the present specification, after identifying the users in each image, the method further includes: determining the total number of users in the images captured by each dynamic-assisted camera; judging whether the total number is greater than a preset number threshold; if so, reducing the specified distance; if not, increasing the specified size, and making the specified distance no greater than one-fourth of the size of the shorter side of the image.

[0097] In an optional embodiment of the present specification, after identifying the users in each image, the method further includes: for each identified user, determining whether the user's moving speed is greater than a preset speed threshold; if so, increasing the specified distance; if not, reducing the specified distance and making the specified distance no greater than one quarter of the size of the shorter side of the image.

[0098] In an optional embodiment of the present specification, the holographic display system is also used to interact with a user; the method further includes: for a user interacting with the holographic display system, determining that a specified distance corresponding to an image containing the user is no greater than one fifth of the size of a shorter side of the image.

[0099] Based on the same idea, the present specification also provides a corresponding embodiment Figure 2 A holographic display device for the partial process shown. The holographic display device is applied to a display processing device of a holographic display system. In addition, the holographic display system also includes a sandbox display device and a head-mounted display device. The display processing device is electrically connected to the sandbox display device and the head-mounted display device respectively. Figure 3 As shown, the holographic display device may include one or more of the following modules:

[0100] The first generating module 300 is configured to generate a left-eye first display signal and a right-eye first display signal.

[0101] The second generating module 302 is configured to: process the left-eye first display signal and the right-eye first display signal to generate a first display signal, and output the first display signal to the sandbox display device, so that the sandbox display device outputs an image according to the first display signal.

[0102] The third generating module 304 is configured to: generate a synchronization signal and send it to the head-mounted display device, so that the head-mounted display device collects the images output by the sandbox display device according to the synchronization signal, so as to alternately display the collected images to the user wearing the head-mounted display device.

[0103] In an optional embodiment of the present specification, the holographic display system further includes a receiving card. The second generating module 302 is specifically configured to: generate a first display signal and send it to the receiving card, so that the receiving card outputs the first display signal to the sandbox display device.

[0104] In an optional embodiment of the present specification, the first generating module 300 is specifically configured to generate a left-eye display signal and a right-eye display signal according to a position signal, so that the left-eye display signal and the right-eye display signal match the user's viewing angle and the image displayed to the user by the sandbox display device, and the position signal shows the position of the user's eyes.

[0105] In an optional embodiment of the present specification, the holographic display system further includes: a motion capture camera; and the position signal is obtained based on data collected by the motion capture camera.

[0106] In an optional embodiment of the present specification, the motion capture camera is arranged on the sandbox display device.

[0107] In an optional embodiment of the present specification, the display processing device includes: a display server, a display processor and a synchronization transmitter; the left-eye display signal and the right-eye display signal are generated by the display server, the first display signal is generated by the display processor, and the synchronization signal is sent by the synchronization transmitter to the head-mounted display device.

[0108] In an optional embodiment of the present specification, the holographic display device further includes an interaction module 306. The interaction module 306 is configured to: upon receiving an interaction instruction, generate a second display signal, and output it to the sandbox display device, so that the sandbox display device outputs an image for displaying an interaction handle according to the second display signal, so that the user can operate the interaction handle to achieve interaction; the interaction includes at least one of the following: zooming in, zooming out, moving, rotating, selecting, and replacing the image output by the sandbox display device.

[0109] It can be understood that the above-mentioned holographic display device can implement various steps of the holographic display process based on the holographic sandbox performed by the holographic display device provided in the aforementioned embodiments, and the relevant explanations on the holographic display method based on the holographic sandbox are applicable to the holographic display device and will not be repeated here.

[0110] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0111] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0112] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0113] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a holographic display device and / or a second holographic display device at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the aforementioned holographic display processes based on the holographic sandbox.

[0114] The above application Figure 2 The method performed by a holographic display device and / or a second holographic display device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0115] The electronic device may also perform Figure 2At least part of the method steps performed by the holographic display device in the embodiment of the present invention, and realizing a holographic display device and / or a second holographic display device in Figure 2 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0116] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 2 The method executed by the first holographic display device and / or the second holographic display device in the illustrated embodiment is specifically used to execute any of the aforementioned holographic display methods based on the holographic sandbox.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0123] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0126] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A holographic display method based on a holographic sandbox, the method is based on a holographic display system, and the holographic display system comprises: Display processing equipment, sandbox display devices and head-mounted display devices; The sandbox display device is a holographic sandbox; The display processing device is electrically connected to the sandbox display device and the head mounted display device respectively; The holographic display method based on the holographic sandbox is performed by the display processing device, and the method includes: Generate a left-eye first display signal and a right-eye first display signal; Processing the left-eye first display signal and the right-eye first display signal to generate a first display signal, and outputting the first display signal to the sandbox display device, so that the sandbox display device outputs an image according to the first display signal; Generate a synchronization signal and send it to the head-mounted display device, so that the head-mounted display device collects the images output by the sandbox display device according to the synchronization signal, so as to alternately display the collected images to the user wearing the head-mounted display device; The holographic display system further includes a receiving card; the receiving card has a driving function, and the receiving card drives the sandbox display device to generate images corresponding to the user's eyes respectively according to the first display signal; generating a first display signal and outputting it to the sandbox display device, including: generating a first display signal and sending it to the receiving card, so that the receiving card outputs the first display signal to the sandbox display device; The holographic display system further includes: a motion capture camera; generating a left-eye display signal and a right-eye display signal, including: generating the left-eye display signal and the right-eye display signal according to the position signal, so that the left-eye display signal and the right-eye display signal match the user's perspective and the image displayed to the user by the sandbox display device, and the position signal shows the position of the user's eyes; the position signal is obtained according to the data collected by the motion capture camera; the motion capture camera is arranged above the sandbox display device, and collects images of the environment from a bird's eye perspective; The method further includes: when the sandbox display device displays a holographic image according to the first display signal, if an interaction instruction is received, identifying the hand position of the user wearing the head-mounted display device; obtaining the viewpoint information of the user, wherein the viewpoint information shows the position of the user's eyes and the viewing angle of the user's eyes; determining a first position of a handheld end for generating an interactive handle according to the hand position; and determining a second position of an indicating end for generating an interactive handle according to the hand position and the viewpoint information; the handheld end is one end of the interactive handle and the interactive end is the other end of the interactive handle; generating a second display signal according to the first position and the second position, and sending the second display signal to the sandbox display device, so that the sandbox display device displays a holographic image according to the first display signal and the second display signal; The head-mounted display device comprises shutter-type active 3D glasses and a rigid body composed of at least three reflective marking points, wherein the at least three reflective marking points are embedded in the outer shell structure of the shutter-type active 3D glasses. The shutter-type active 3D glasses are used to receive the viewing angle picture and the 3D signal transmitted by the sandbox display device, so as to obtain the correct left and right eye pictures. Specifically, the shutter-type active 3D glasses receive the 3D signal transmitted by the 3D signal transmitter in the sandbox display device. The rigid body composed of at least three reflective marking points is used to mark the position of the shutter-type active 3D glasses in the motion capture space, and can capture the position information of the head-mounted display device in the motion capture space, as well as the viewing angle position information of the user wearing the head-mounted display device in the motion capture space. By capturing the position of the shutter-type active 3D glasses in the motion capture space, the user can see a three-dimensional picture of the display content floating in the air by wearing the shutter-type active 3D glasses, and can view the three-dimensional picture from various angles, that is, a holographic image.

2. The method of claim 1, wherein: The display processing device includes: a display server, a display processor and a synchronization transmitter; The left-eye display signal and the right-eye display signal are generated by the display server, the first display signal is generated by the display processor, and the synchronization signal is sent to the head-mounted display device by the synchronization transmitter.

3. The method of claim 1, wherein: The method further comprises: The interaction includes at least one of the following: zooming in, zooming out, moving, rotating, selecting, and replacing the image output by the sandbox display device.

4. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by one or more processors, the one or more processors can execute any one of the methods described in claims 1 to 3.

5. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform any of the methods of claims 1 to 3.