Three-dimensional video display method, head-mounted display device, readable medium and product

Abstract

The embodiment of the invention discloses a three-dimensional video display method, head-mounted display equipment, a readable medium and a product. According to one specific embodiment, the method comprises the steps that a three-dimensional video used for visual training is obtained, at least one motion element is displayed in the three-dimensional video, a head-mounted display device comprises a first display module and a second display module, the first display module comprises a first light machine and a first display screen, and the second display module comprises a second light machine and a second display screen; the second display module comprises a second ray machine and a second display screen; the first light machine is controlled to drive the first display screen to display first video content of the corresponding three-dimensional video, the second light machine is controlled to drive the second display screen to display second video content of the three-dimensional video, and parallax exists between the first video content and the second video content. According to the embodiment, the experience of the user in the visual training process is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a three-dimensional video display method, a head-mounted display device, a readable medium, and a product. Background Art

[0002] With the development of computer display technology, vision training functions can be integrated into display devices. At present, when displaying content for vision training, the commonly used methods are: displaying the content on a two-dimensional display screen for users to directly watch the two-dimensional display content; or allowing users to directly watch naked-eye three-dimensional display content.

[0003] However, the inventors have found that when the above method is used, the following technical problems often occur: the two-dimensional display method cannot provide a stereoscopic effect because there is no information to distinguish between the left and right eyes, resulting in a poor experience for users during vision training; the naked-eye three-dimensional display method requires users to watch at a close distance, which easily causes eye fatigue, resulting in a poor experience for users during vision training.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a three-dimensional video display method, a head-mounted display device, a computer-readable medium, and a computer program product to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a three-dimensional video display method, which is applied to a head-mounted display device, the method comprising: obtaining a three-dimensional video for vision training, wherein at least one motion element is displayed in the three-dimensional video, the head-mounted display device comprising a first display module and a second display module, the first display module comprising a first optical machine and a first display screen, and the second display module comprising a second optical machine and a second display screen; controlling the first optical machine to drive the first display screen to display a first video content corresponding to the three-dimensional video, and controlling the second optical machine to drive the second display screen to display a second video content of the three-dimensional video, wherein there is parallax between the first video content and the second video content.

[0008] Optionally, the first display module corresponds to the user's left eye, and the first display module also includes a first beam splitter and a first reflecting device. The reflecting surfaces of the first beam splitter and the first reflecting device are placed opposite to each other. After the light corresponding to the first video content displayed on the first display screen is irradiated on the first beam splitter, it is reflected by the first beam splitter to the first reflecting device. After receiving the light, the first reflecting device returns the light toward the first beam splitter. The light returning to the first beam splitter enters the left eye of the target user, thereby forming virtual content corresponding to the first video content. The target user wears the head-mounted display device.

[0009] Optionally, the second display module corresponds to the user's right eye, and the second display module also includes a second beam splitter and a second reflecting device. The reflecting surfaces of the second beam splitter and the second reflecting device are placed opposite to each other. After the light corresponding to the second video content displayed on the second display screen is irradiated on the second beam splitter, it is reflected by the second beam splitter to the second reflecting device. After receiving the light, the second reflecting device returns the light toward the second beam splitter. The light returning to the second beam splitter enters the right eye of the target user, forming virtual content corresponding to the second video content, so that the virtual content corresponding to the first video content and the second video content presents a stereoscopic visual effect in the binoculars of the target user.

[0010] Optionally, the above-mentioned acquisition of the three-dimensional video for vision training includes: in response to detecting a selection operation of the user on the vision training control, determining the three-dimensional video identifier corresponding to the vision training control; and acquiring the three-dimensional video corresponding to the three-dimensional video identifier.

[0011] Optionally, the above-mentioned controlling the first optical machine to drive the first display screen to display the first video content corresponding to the above-mentioned three-dimensional video, and controlling the above-mentioned second optical machine to drive the above-mentioned second display screen to display the second video content of the above-mentioned three-dimensional video, includes: for each video frame included in the above-mentioned three-dimensional video, performing the following steps: controlling the above-mentioned first optical machine to drive the above-mentioned first display screen to display the first video frame in the above-mentioned first video content corresponding to the above-mentioned video frame; controlling the above-mentioned second optical machine to drive the above-mentioned second display screen to display the second video frame in the above-mentioned second video content corresponding to the above-mentioned video frame, wherein the display time of the above-mentioned second video frame is the same as the display time of the above-mentioned first video frame.

[0012] Optionally, the three-dimensional video is generated by the following steps: determining at least one motion element, wherein each motion element corresponds to motion configuration information; controlling the motion of each motion element in the at least one motion element in the three-dimensional space according to the motion configuration information corresponding to each motion element in the at least one motion element, and controlling the first virtual camera and the second virtual camera to record the three-dimensional space to obtain a first video content and a second video content, wherein the distance between the first virtual camera and the second virtual camera is a preset binocular distance, the first virtual camera corresponds to the first display module, and the second virtual camera corresponds to the second display module; determining the first video content and the second video content as three-dimensional videos.

[0013] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising: a first display module, comprising a first optical engine and a first display screen; a second display module, comprising a second optical engine and a second display screen; one or more processors; a storage device, on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect above.

[0014] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0015] In a fourth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner of the above-mentioned first aspect when executed by a processor.

[0016] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the three-dimensional video display method of some embodiments of the present disclosure, the user's experience in the process of vision training is improved. Specifically, the reason for the poor experience of the user in the process of vision training is that: the two-dimensional display mode cannot provide a stereoscopic effect because there is no information to distinguish the left and right eyes, resulting in a poor experience of the user in the process of vision training; the naked eye three-dimensional display mode requires the user to watch at a close distance, and the eyes are easily fatigued, resulting in a poor experience of the user in the process of vision training. Based on this, the three-dimensional video display method of some embodiments of the present disclosure, first, obtains a three-dimensional video for vision training, wherein at least one motion element is displayed in the three-dimensional video, the head-mounted display device includes a first display module and a second display module, the first display module includes a first optical machine and a first display screen, and the second display module includes a second optical machine and a second display screen. Thus, the obtained three-dimensional video can be displayed in the head-mounted display device for viewing by the wearing user to perform vision training. Then, the first optical machine is controlled to drive the first display screen to display the first video content corresponding to the three-dimensional video, and the second optical machine is controlled to drive the second display screen to display the second video content of the three-dimensional video, wherein there is parallax between the first video content and the second video content. Thus, the three-dimensional video can be displayed on the two display screens of the head-mounted display device in a manner with parallax, so that the left and right eyes of the user wearing the device can watch the training video content with parallax, and then a three-dimensional effect can be imaged in front of the eyes of the user wearing the device. Thus, the user experience during the vision training process is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0018] Figure 1 is a flow chart of some embodiments of the three-dimensional video display method according to the present disclosure; Figure 2 are flow charts of other embodiments of the three-dimensional video display method according to the present disclosure; Figure 3 It is a schematic diagram of the structure of a head-mounted display device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] It should be noted that the three-dimensional video display method disclosed in the present invention is used for non-therapeutic purposes. Specifically, it can be used to improve the user's experience when performing vision training.

[0025] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0026] Figure 1 The process 100 of some embodiments of the 3D video display method according to the present disclosure is shown. The 3D video display method comprises the following steps: Step 101, obtaining a three-dimensional video for vision training.

[0027] In some embodiments, the execution subject of the three-dimensional video display method (such as a head-mounted display device) can obtain a three-dimensional video for vision training from a server or local storage. The three-dimensional video can be a video for users to watch in a stereoscopic form for vision training. The three-dimensional video can correspond to a first video content and a second video content, and there is a parallax between the first video content and the second video content, which are respectively used to be displayed in two display screens to present a stereoscopic effect. The first video content can be a video after parallax mapping of each frame image of the original video for the first display screen. The second video content can be a video after parallax mapping of each frame image of the original video for the second display screen. The first video content and the second video content can also be videos shot by dual cameras. The dual cameras can be used to simulate the user's eyes, one camera is used to capture the left eye perspective, and the other camera is used to capture the right eye perspective. The three-dimensional video can directly include the first video content and the second video content, or it can include the original video without parallax mapping and the first video content and the second video content. At least one motion element can be displayed in the three-dimensional video. The motion element can be a pre-set visualization element used to cause a change in the user's visual focus when watching in a moving manner. The moving element may move along a preset moving path, and may also have at least one of the following changes during the movement: color change, size change, shape change.

[0028] Optionally, the three-dimensional video is generated by the following steps: The first step is to determine at least one moving element. Each moving element has corresponding motion configuration information. The at least one moving element may be pre-configured by a technician. The motion configuration information is state-related information pre-configured by a technician for the moving element. The motion configuration information may include but is not limited to at least one of the following: moving path, moving speed, color change information, size change information, and shape change information. The moving path may include various three-dimensional coordinates. The moving speed may include a uniform speed and may also include acceleration corresponding to each time period. The color change information may include display colors corresponding to each time period. The size change information may include an element scaling ratio corresponding to each time period. The shape change information may include an element shape identifier corresponding to each time period.

[0029] In the second step, according to the motion configuration information corresponding to each motion element in the at least one motion element, the motion of each motion element is controlled in the three-dimensional space, and the first virtual camera and the second virtual camera are controlled to record the three-dimensional space to obtain the first video content and the second video content. The distance between the first virtual camera and the second virtual camera is the preset binocular distance. The first virtual camera corresponds to the first display module. The second virtual camera corresponds to the second display module. The preset binocular distance can be the distance between two eyes (such as pupils) that is preset. The three-dimensional space can be a pre-constructed three-dimensional coordinate system. The first virtual camera and the second virtual camera can be observation points for simulating binocular positions in the three-dimensional space. The content captured by the first virtual camera and the second virtual camera at a distance of the preset binocular distance has parallax.

[0030] The third step is to determine the first video content and the second video content as three-dimensional video. Thus, the three-dimensional video with parallax can be directly shot by the camera simulating binoculars.

[0031] The head-mounted display device may include a first display module and a second display module. The first display module may include a first optical machine and a first display screen. The second display module includes a second optical machine and a second display screen. The first display module may correspond to the left eye, and the second display module may correspond to the right eye. Alternatively, the first display module may correspond to the right eye, and the second display module may correspond to the left eye, which is not specifically limited here. The first display screen and the second display screen may be micro display screens. The head-mounted display device may be, but is not limited to, one of the following: AR glasses, VR glasses, MR glasses. The first optical machine and the second optical machine may be used to superimpose digital information onto the user's actual field of view, and the type of the optical machine corresponds to the type of the head-mounted display device.

[0032] In some optional implementations of some embodiments, the above-mentioned execution subject may obtain the three-dimensional video for vision training through the following steps: In the first step, in response to detecting a selection operation of a user acting on a vision training control, a 3D video identifier corresponding to the vision training control is determined. The vision training control may be a control for determining to start vision training. Each vision training control may correspond to a 3D video. The 3D video identifier may be used to uniquely identify the 3D video. The selection operation may include but is not limited to at least one of the following: a touch screen click operation, a head control confirmation operation, a voice confirmation operation, and a gesture confirmation operation.

[0033] The second step is to obtain the 3D video corresponding to the 3D video identifier. In practice, the execution subject can determine whether there is a 3D video corresponding to the 3D video identifier in the local storage. Then, in response to the determination that there is, the 3D video corresponding to the 3D video identifier can be obtained from the local storage. Secondly, in response to the determination that there is no 3D video, the 3D video corresponding to the 3D video identifier can be obtained from the server. In this way, the wearer can select the 3D video for vision training as needed.

[0034] Step 102: Control the first optical engine to drive the first display screen to display first video content corresponding to the three-dimensional video, and control the second optical engine to drive the second display screen to display second video content of the three-dimensional video.

[0035] In some embodiments, the execution subject may control the first optical machine to drive the first display screen to display the first video content corresponding to the three-dimensional video, and control the second optical machine to drive the second display screen to display the second video content of the three-dimensional video. In practice, the execution subject may control the first optical machine to drive the first display screen to display the first video content corresponding to the three-dimensional video, and synchronously control the second optical machine to drive the second display screen to display the second video content of the three-dimensional video. Optionally, the execution subject may also synchronously control the sound playback device included in the head-mounted display device to play the audio corresponding to the three-dimensional video.

[0036] Optionally, the first display module corresponds to the left eye of the user. The first display module also includes a first beam splitter and a first reflecting device. The first beam splitter may be a polarizing beam splitter. The first reflecting device may be a semi-transparent and semi-reflective curved mirror. The reflecting surfaces of the first beam splitter and the first reflecting device are placed opposite to each other. The reflecting surface of the first reflecting device is a concave surface. After the light corresponding to the first video content displayed in the first display screen is irradiated by the first beam splitter, it is reflected by the first beam splitter to the first reflecting device. After receiving the light, the first reflecting device returns the light toward the first beam splitter. After the light returned to the first beam splitter, it enters the left eye of the target user, forming virtual content corresponding to the first video content. At the same time, external ambient light may also enter the left eye of the target user through the first reflecting device and the first beam splitter in sequence. The target user wears the head-mounted display device. Thus, virtual content can be presented in the left eye of the user.

[0037] Optionally, the second display module corresponds to the right eye of the user. The second display module also includes a second beam splitter and a second reflecting device. The second beam splitter may be a polarizing beam splitter. The second reflecting device may be a semi-transparent and semi-reflective curved mirror. The reflecting surfaces of the second beam splitter and the second reflecting device are placed opposite to each other. The reflecting surface of the second reflecting device is a concave surface. After the light corresponding to the second video content displayed on the second display screen is irradiated by the second beam splitter, it is reflected by the second beam splitter to the second reflecting device. After receiving the light, the second reflecting device returns the light toward the second beam splitter. The light returned to the second beam splitter enters the right eye of the target user, forming virtual content corresponding to the second video content, so that the virtual content corresponding to the first video content and the second video content presents a stereoscopic visual effect in the binocular eyes of the target user.

[0038] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the three-dimensional video display method of some embodiments of the present disclosure, the user's experience in the process of vision training is improved. Specifically, the reason for the poor experience of the user in the process of vision training is that: the two-dimensional display mode cannot provide a stereoscopic effect because there is no information to distinguish the left and right eyes, resulting in a poor experience of the user in the process of vision training; the naked eye three-dimensional display mode requires the user to watch at a close distance, and the eyes are easily fatigued, resulting in a poor experience of the user in the process of vision training. Based on this, the three-dimensional video display method of some embodiments of the present disclosure, first, obtains a three-dimensional video for vision training, wherein at least one motion element is displayed in the three-dimensional video, the head-mounted display device includes a first display module and a second display module, the first display module includes a first optical machine and a first display screen, and the second display module includes a second optical machine and a second display screen. Thus, the obtained three-dimensional video can be displayed in the head-mounted display device for viewing by the wearing user to perform vision training. Then, the first optical machine is controlled to drive the first display screen to display the first video content corresponding to the three-dimensional video, and the second optical machine is controlled to drive the second display screen to display the second video content of the three-dimensional video, wherein there is parallax between the first video content and the second video content. Thus, the three-dimensional video can be displayed on the two display screens of the head-mounted display device in a manner with parallax, so that the left and right eyes of the user wearing the device can watch the training video content with parallax, and then a three-dimensional effect can be imaged in front of the eyes of the user wearing the device. Thus, the user experience during the vision training process is improved.

[0039] Further references Figure 2 , which shows a process 200 of another embodiment of a 3D video display method. The process 200 of the 3D video display method includes the following steps: Step 201, obtaining a three-dimensional video for vision training.

[0040] In some embodiments, the specific implementation of step 201 and the technical effects brought about can be referred to Figure 1 The corresponding step 101 in the embodiments will not be described in detail here.

[0041] Step 202: for each video frame included in the 3D video, perform the following steps: Step 2021: Control the first optical engine to drive the first display screen to display a first video frame corresponding to a video frame in the first video content.

[0042] In some embodiments, the execution subject of the three-dimensional video display method (e.g., a head-mounted display device) may control the first optical machine to drive the first display screen to display the first video frame corresponding to the video frame in the first video content. The three-dimensional video may include the original video. The video frame may be a video frame in the original video. The first video frame corresponding to the video frame may be a video frame obtained by performing parallax mapping on the video frame. The first video frame corresponding to the video frame may also be a video frame having the same number of frames in the first video content as the number of frames corresponding to the video frame.

[0043] Optionally, before step 2021, the execution entity may further perform the following steps: The first step is to generate a depth image corresponding to the video frame according to the video frame. In practice, the execution subject may input the video frame into a pre-trained depth image generation model to obtain a depth image corresponding to the video frame. The depth image generation model may be a pre-trained neural network model with an image as input data and a depth image corresponding to the image as output data. The depth image may include depth values ​​of each pixel. For example, the depth image generation model may be a CNN. For another example, the depth image generation model may include an encoder and a decoder. The encoder may be used to extract image features. The decoder may be used to restore depth information from the features extracted by the encoder. The encoder may include an input layer, a convolution layer, a residual layer, and an average pooling layer. The decoder may include an upsampling layer, a convolution layer, a skip connection, and a prediction layer. The input layer may be used to accept an RGB image as input. The convolution layer in the encoder may be used to preliminarily extract features. The residual layer may include 4 residual block groups, each residual block group may include two residual blocks. Each residual block may include 2 3×3 convolution layers. The average pooling layer may be a global average pooling layer, which may be used to reduce the dimension of the feature map. The upsampling layer can use deconvolution or nearest neighbor upsampling to increase the resolution of the feature map. The decoder includes convolutional layers that can be used to further process the feature map. Skip connections can be used to bring in feature maps from different stages of the encoder to preserve detail information. The prediction layer can be used to generate the final depth image.

[0044] In the second step, the product of the preset binocular distance and the pixel resolution corresponding to the above video frame is determined as a first value. The preset binocular distance can be a preset average horizontal distance between the two eyes. For example, the preset binocular distance can be 6.5 centimeters.

[0045] Step 3: For each pixel in the above video frame, perform the following steps: The first sub-step is to determine the depth information corresponding to the pixel point according to the depth image. In practice, the execution subject may determine the depth value of the pixel point in the depth image as the depth information.

[0046] The second sub-step is to determine the ratio of the first value to the depth information as the pixel offset corresponding to the pixel point, wherein the pixel offset can be used as the horizontal offset of the pixel.

[0047] The fourth step is to perform a left-shift processing on the video frame according to the pixel offset corresponding to each pixel point in the video frame to obtain a left-shifted image. In practice, for each pixel point in the video frame, the execution subject may shift the pixel point to the left by the pixel offset amount. Afterwards, the execution subject may determine the video frame after each pixel point is shifted to the left as a left-shifted image.

[0048] The fifth step is to perform right-shift processing on the video frame according to the pixel offset corresponding to each pixel point in the video frame to obtain a right-shifted image. In practice, for each pixel point in the video frame, the execution subject may shift the pixel point to the right by the pixel offset amount. Afterwards, the execution subject may determine the video frame after each pixel point is shifted to the right as the right-shifted image.

[0049] Step 6: Generate a first video frame corresponding to the above video frame based on the above left-shifted image.

[0050] Step 7: Generate a second video frame corresponding to the video frame according to the right-shifted image. Thus, when the acquired 3D video only includes 2D video, the first video content and the second video content with parallax can be generated according to the 2D video.

[0051] In some optional implementations of some embodiments, the execution subject may generate a first video frame corresponding to the video frame according to the left-offset image through the following steps: In the first step, each pixel point in the above-mentioned left-offset image that meets the preset super-boundary condition is determined as a super-boundary pixel point set. The preset super-boundary condition may be that the pixel value of the pixel point is empty, and the corresponding original pixel point is offset beyond the boundary of the image. Here, the image may refer to a left-offset image. The boundary of the left-offset image may be based on the pixel point at the right vertex in the corresponding video frame after being left-offset, and the image boundary of the left-offset image may be constructed according to the width and height of the video frame image. For the left-offset image, the pixel point exceeds the boundary of the image after being offset, which may refer to the pixel point exceeding the left boundary of the left-offset image after being offset.

[0052] In the second step, for each super-boundary pixel point in the above super-boundary pixel point set, perform the following steps: In the first sub-step, each pixel point in the left-shifted image that satisfies the preset neighbor condition corresponding to the super-boundary pixel point is determined as a neighbor pixel point set, wherein the preset neighbor condition may be that the distance between the pixel point and the super-boundary pixel point is less than a preset distance and the pixel value is not empty.

[0053] The second sub-step is to generate a filling pixel value corresponding to the above-mentioned super-boundary pixel point according to the pixel value of each neighboring pixel point in the above-mentioned neighboring pixel point set. In practice, the above-mentioned execution subject can interpolate the pixel value of the above-mentioned super-boundary pixel point according to the pixel value of each neighboring pixel point in the above-mentioned neighboring pixel point set to obtain the filling pixel value corresponding to the above-mentioned super-boundary pixel point. For example, the interpolation process can be a mean interpolation process or a bilinear interpolation process.

[0054] In the third step, pixel filling processing is performed on each super-boundary pixel point included in the super-boundary pixel point set in the left offset image according to each generated filling pixel value, and the left offset image after pixel filling processing is obtained as the left offset filling image. In this way, the pixel points beyond the boundary of the offset image after offset can be filled to improve the integrity of the offset image.

[0055] In the fourth step, each pixel point in the left-offset filled image that meets a preset missing condition is determined as a missing pixel point set, wherein the preset missing condition may be that the pixel value corresponding to the pixel point is a null value.

[0056] Step 5: For each missing pixel in the above missing pixel set, perform the following steps: In the first sub-step, each pixel point in the above-mentioned left-offset filled image that satisfies the preset adjacent condition corresponding to the above-mentioned missing pixel point is determined as a set of adjacent pixel points. Among them, the number of each adjacent pixel point included in the above-mentioned set of adjacent pixel points is a preset number. The preset adjacent condition can be that the pixel point is adjacent to the missing pixel point. Here, when the preset number is 4, adjacent can include: left adjacent, right adjacent, upper adjacent, lower adjacent. When the preset number is 8, adjacent can include: left adjacent, right adjacent, upper adjacent, lower adjacent, upper left corner adjacent, upper right corner adjacent, lower left corner adjacent, and lower right corner adjacent.

[0057] The second sub-step is to generate a complementary pixel value corresponding to the missing pixel point according to the pixel values ​​of each adjacent pixel point in the adjacent pixel point set. In practice, the execution subject may determine the average pixel value of each adjacent pixel point as the complementary pixel value corresponding to the missing pixel point.

[0058] In the sixth step, according to the generated complementary pixel values, pixel complementary processing is performed on each missing pixel point included in the above missing pixel point set in the above left offset complementary image, and the left offset complementary image after pixel complementary processing is obtained as the first video frame corresponding to the above video frame. Thus, the pixel points with missing pixel values ​​can be further checked and complementary to further improve the integrity of the first video frame.

[0059] It should be noted that the specific implementation method of generating the second video frame corresponding to the above video frame can refer to the method of generating the first video frame corresponding to the above video frame in the first to sixth steps, which will not be repeated here.

[0060] Step 2022: Control the second optical engine to drive the second display screen to display a second video frame corresponding to a video frame in the second video content.

[0061] In some embodiments, the execution subject may control the second optical machine to drive the second display screen to display the second video frame corresponding to the video frame in the second video content. The second video frame corresponding to the video frame may be a video frame obtained after parallax mapping of the video frame. The second video frame corresponding to the video frame may also be a video frame whose corresponding frame number in the second video content is the same as the corresponding frame number of the video frame. The display time of the second video frame is the same as the display time of the first video frame. Thus, the synchronous display of the first video frame and the corresponding second video frame may be achieved.

[0062] from Figure 2 It can be seen that Figure 1 Compared with the description of some corresponding embodiments, Figure 2The process 200 of the 3D video display method in some corresponding embodiments embodies the steps extended to display the first video frame and the corresponding second video frame. Therefore, the schemes described in these embodiments can control the synchronous display of the first video content and the second video content from the dimension of the video frame.

[0063] Reference below Figure 3 , which shows a schematic structural diagram of a head-mounted display device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The head mounted display device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0064] like Figure 3 As shown, the head mounted display device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processor, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the head mounted display device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0065] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; and communication devices 309. The communication devices 309 may allow the head mounted display device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The head mounted display device 300 having various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0066] The input device includes a first display module and a second display module, wherein the first display module includes a first optical engine and a first display screen, and the second display module includes a second optical engine and a second display screen.

[0067] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0068] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0069] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0070] The computer-readable medium may be included in the head-mounted display device; or it may exist independently without being assembled into the head-mounted display device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device: obtains a three-dimensional video for vision training, wherein at least one motion element is displayed in the three-dimensional video, and the head-mounted display device includes a first display module and a second display module, wherein the first display module includes a first optical machine and a first display screen, and the second display module includes a second optical machine and a second display screen; controls the first optical machine to drive the first display screen to display the first video content corresponding to the three-dimensional video, and controls the second optical machine to drive the second display screen to display the second video content of the three-dimensional video, wherein there is parallax between the first video content and the second video content.

[0071] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0074] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which implements any of the above-mentioned three-dimensional video display methods when executed by a processor.

[0075] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A three-dimensional video display method, applied to a head-mounted display device, comprising: Acquire a three-dimensional video for vision training, wherein at least one motion element is displayed in the three-dimensional video, and the head-mounted display device includes a first display module and a second display module, the first display module includes a first optical machine and a first display screen, and the second display module includes a second optical machine and a second display screen; The first optical engine is controlled to drive the first display screen to display first video content corresponding to the three-dimensional video, and the second optical engine is controlled to drive the second display screen to display second video content of the three-dimensional video, wherein there is parallax between the first video content and the second video content.

2. The method according to claim 1, wherein: The first display module corresponds to the user's left eye. The first display module also includes a first beam splitter and a first reflecting device. The reflecting surfaces of the first beam splitter and the first reflecting device are placed opposite to each other. After the light corresponding to the first video content displayed on the first display screen is irradiated on the first beam splitter, it is reflected by the first beam splitter to the first reflecting device. After receiving the light, the first reflecting device returns the light toward the first beam splitter. The light returning to the first beam splitter enters the left eye of the target user, thereby forming virtual content corresponding to the first video content. The target user wears the head-mounted display device.

3. The method according to claim 2, wherein: The second display module corresponds to the user's right eye. The second display module also includes a second beam splitter and a second reflecting device. The reflecting surfaces of the second beam splitter and the second reflecting device are placed opposite to each other. After the light corresponding to the second video content displayed on the second display screen is irradiated on the second beam splitter, it is reflected by the second beam splitter to the second reflecting device. After receiving the light, the second reflecting device returns the light toward the second beam splitter. The light returning to the second beam splitter enters the right eye of the target user to form virtual content corresponding to the second video content, so that the virtual content corresponding to the first video content and the second video content presents a stereoscopic visual effect in the binoculars of the target user.

4. The method according to claim 1, wherein: The method of obtaining a three-dimensional video for vision training comprises: In response to detecting a selection operation performed by a user on a vision training control, determining a three-dimensional video identifier corresponding to the vision training control; A three-dimensional video corresponding to the three-dimensional video identifier is obtained.

5. The method according to claim 1, wherein: The controlling the first optical engine to drive the first display screen to display first video content corresponding to the three-dimensional video, and controlling the second optical engine to drive the second display screen to display second video content of the three-dimensional video, comprises: For each video frame included in the three-dimensional video, the following steps are performed: Controlling the first optical engine to drive the first display screen to display a first video frame corresponding to the video frame in the first video content; The second optical engine is controlled to drive the second display screen to display a second video frame corresponding to the video frame in the second video content, wherein a display time of the second video frame is the same as a display time of the first video frame.

6. The method according to claim 1, wherein: The three-dimensional video is generated by the following steps: Determine at least one motion element, wherein each motion element corresponds to motion configuration information; According to the motion configuration information corresponding to each motion element in the at least one motion element, control each motion element to move in a three-dimensional space, and control a first virtual camera and a second virtual camera to record the three-dimensional space, so as to obtain a first video content and a second video content, wherein the distance between the first virtual camera and the second virtual camera is a preset binocular distance, the first virtual camera corresponds to the first display module, and the second virtual camera corresponds to the second display module; The first video content and the second video content are determined to be three-dimensional videos.

7. A head mounted display device, comprising: A first display module, comprising a first optical engine and a first display screen; The second display module includes a second optical engine and a second display screen; one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images