Image content interaction method, head-mounted display device and computer readable storage medium

Abstract

The invention discloses an image content interaction method, head-mounted display equipment and a computer readable storage medium, the head-mounted display equipment comprises a first display screen, a second display screen and a controller, the first display screen is arranged on the inner side of the head-mounted display equipment, and a display interface of the first display screen faces the inner side of the head-mounted display equipment; the second display screen is arranged on the outer side of the head-mounted display device, a display interface of the second display screen faces the outer side of the head-mounted display device, the outer side is the side, away from the face of the user, of the head-mounted display device in the wearing state, and the controller is electrically connected with the first display screen and the second display screen. According to the invention, the convenience and efficiency of image content interaction of the head-mounted display device can be improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent interaction technology, and in particular to an image content interaction method, a head-mounted display device and a computer-readable storage medium. Background Art

[0002] With the increasing diversification of smart wearable devices, head-mounted display devices have gradually entered people's lives. However, the images displayed by the relevant head-mounted display devices can only be viewed by the wearer, that is, extended reality experiences such as VR (Virtual Reality) are basically the user's own image content experience. If more people want to join the interactive experience of the image content, the user needs to project the image content to a PC (personal computer) through wireless projection. That is, the solution needs to configure an additional electronic display device for projection to the head-mounted display device as an interactive platform for users and people around them, so as to realize image content interaction such as sharing a video with a third party, discussing a piece of clothing or a game, etc. The convenience and efficiency of image content interaction are low. Summary of the invention

[0003] The main purpose of the present invention is to provide an image content interaction method, a head-mounted display device and a computer-readable storage medium, aiming to solve the technical problems of the convenience and low efficiency of image content interaction in the current head-mounted display device.

[0004] To achieve the above object, the present invention provides a head mounted display device, comprising:

[0005] A first display screen, wherein the first display screen is disposed on the inner side of the head mounted display device, and a display interface of the first display screen faces the inner side of the head mounted display device, wherein the inner side is a side of the head mounted display device close to the user's face when the head mounted display device is worn;

[0006] A second display screen, wherein the second display screen is disposed on the outside of the head-mounted display device, and a display interface of the second display screen faces the outside of the head-mounted display device, wherein the outside is a side of the head-mounted display device that is away from the user's face when the head-mounted display device is worn;

[0007] A controller is electrically connected to the first display screen and the second display screen.

[0008] Optionally, the head mounted display device further includes a proximity sensor, and the proximity sensor is electrically connected to the controller.

[0009] Optionally, the head mounted display device further includes a camera, and the camera is electrically connected to the controller.

[0010] Optionally, the second display screen is arranged on the front shell of the head-mounted display device.

[0011] In addition, to achieve the above object, the present invention further provides an image content interaction method, the image content interaction method is applied to the head mounted display device as described above, the head mounted display device includes a first display screen and a second display screen, the method includes:

[0012] Detect whether the screen sharing command is entered;

[0013] If the input of the screen sharing instruction is detected, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display.

[0014] Optionally, the head mounted display device includes a proximity sensor, and before the step of detecting whether a screen sharing instruction is input, the method further includes:

[0015] Detecting the distance between the head mounted display device and other people in a surrounding preset area through the proximity sensor;

[0016] After detecting that the distance is less than a first preset distance, generating and outputting a preset mode selection window;

[0017] The step of detecting whether a screen sharing instruction is input comprises:

[0018] It is detected whether a screen sharing instruction is input in response to the mode selection window.

[0019] Optionally, the head mounted display device includes a camera, and after the step of synchronously transmitting the image content currently displayed on the first display screen to the second display screen for display, the method further includes:

[0020] detecting, by means of the camera, actual gestures made by other persons;

[0021] If the matching rate between the actual gesture and the pre-stored target interaction gesture is greater than a preset ratio value, generating a target interaction instruction corresponding to the target interaction gesture;

[0022] The image contents of the first display screen and the second display screen are synchronously updated based on the target interaction instruction to complete human-computer interaction.

[0023] Optionally, after the step of synchronously transmitting the image content displayed on the first display screen to the second display screen for display, the method further includes:

[0024] After detecting that the time duration for which the distance is greater than a second preset distance is greater than a preset time threshold, controlling the second display screen to turn off, wherein the first preset distance is less than or equal to the second preset distance;

[0025] Return to execution: the step of detecting the distance between the head mounted display device and other people in a surrounding preset area by using the proximity sensor.

[0026] Optionally, after the step of generating and outputting a preset mode selection window, the method further includes:

[0027] If it is detected that an immersive mode instruction is input in response to the mode selection window, the preset do not disturb prompt information is transmitted to the second display screen for display.

[0028] In addition, the present application also provides a computer-readable storage medium, on which an image content interaction program is stored. When the image content interaction program is executed by a processor, the steps of the image content interaction method described above are implemented.

[0029] The present application sets the head-mounted display device to include a first display screen, a second display screen and a controller. The first display screen is set on the inner side of the head-mounted display device, and the display interface of the first display screen faces the inner side of the head-mounted display device, wherein the inner side is the side of the head-mounted display device close to the user's face when the head-mounted display device is worn. The second display screen is then set on the outer side of the head-mounted display device, and the display interface of the second display screen faces the outer side of the head-mounted display device, wherein the outer side is the side of the head-mounted display device away from the user's face when the head-mounted display device is worn. The controller is then electrically connected to the first display screen and the second display screen, so that the head-mounted display device of the present application builds a dual-screen display, that is, the first display screen is electrically connected to the second display screen, so that the head-mounted display device of the present application builds a dual-screen display, that is, the first display screen is electrically connected to the second display screen, and ... and the first display screen is electrically connected to the second display screen, and the first display screen is electrically connected to the second display screen, so that the head-mounted display device of the present application builds a dual-screen display, that A display screen (user screen) and a second display screen (interactive screen). The second display screen serves as an interactive platform for the user and the people around him. Other users can communicate and discuss through the second display screen. For example, when the user (the wearer of the head-mounted display device) wants to share the screen with the people around him, he can share the content through the second display screen integrated in the head-mounted display device. In this way, the image content seen by the user from the first display screen is consistent with the image content seen by the people around him from the second display screen. In this way, without adding wireless projection, it is possible to share a video, discuss a piece of clothing or a game with a third party, and other image content interactions, thereby improving the convenience and efficiency of image content interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0031] Figure 1 A schematic diagram of the structure of a head mounted display device according to an embodiment of the present invention;

[0032] Figure 2 It is a flowchart of the first embodiment of the image content interaction method of the present invention;

[0033] Figure 3 A schematic diagram of an interactive process of a head mounted display device controlling dual-screen display in an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of a display scene of a head mounted display device in response to a screen sharing instruction in an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of a display scene of a head mounted display device in response to an immersive mode instruction in an embodiment of the present invention;

[0036] Figure 6 It is a flowchart of a second embodiment of the image content interaction method of the present invention;

[0037] Figure 7 The figure is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0038] Description of Figure Numbers:

[0039] Label name Label name 100 Head-mounted display devices 3 Controller 1 First display screen 4 Front shell 2 Second display 5 Optical lens

[0040] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0043] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0044] In this embodiment, the head-mounted display device of the present application can be, for example, a mixed reality (Mixed Reality)-MR device (such as MR glasses or MR helmet), an augmented reality (Augmented Reality)-AR device (such as AR glasses or AR helmet), a virtual reality-(Virtual Reality)-VR device (such as VR glasses or VR helmet), an extended reality (Extended Reality)-XR device (such as XR glasses or XR helmet) or some combination thereof.

[0045] At present, the images displayed by the relevant head-mounted display devices can only be viewed by the wearer, that is, extended reality experiences such as VR (Virtual Reality) are basically the user's own image content experience. If more people are wanted to join the interactive experience of the image content, the user needs to project the image content to a PC (personal computer) through wireless projection. That is, the solution requires the head-mounted display device to be additionally equipped with an electronic display device for projection as an interactive platform for the user and the people around him, so as to realize image content interaction such as sharing a video with a third party, discussing a piece of clothing or a game, etc. The convenience and efficiency of image content interaction are relatively low.

[0046] Based on this, the present application embodiment provides a head mounted display device 100, please refer to Figure 1 , the head mounted display device 100 comprises:

[0047] A first display screen 1, which is disposed on the inner side of the head mounted display device 100, and a display interface of the first display screen 1 faces the inner side of the head mounted display device 100, wherein the inner side is a side of the head mounted display device 100 close to the user's face when the head mounted display device 100 is worn;

[0048] A second display screen 2, which is disposed on the outside of the head mounted display device 100, and a display interface of the second display screen 2 faces the outside of the head mounted display device 100, wherein the outside is a side of the head mounted display device 100 away from the user's face when the head mounted display device 100 is worn;

[0049] The controller 3 is electrically connected to the first display screen 1 and the second display screen 2 .

[0050] The controller 3 may be electrically connected to the first display screen 1 and the second display screen 2 via a PCB (Printed circuit board).

[0051] In this embodiment, the first display screen 1 may be referred to as a user display screen, and is used to display extended reality content such as VR (Virtual Reality) to a user wearing the head mounted display device 100. The second display screen 2 may be referred to as an interactive screen, and is used to realize sharing of image content with a third party (the third party is other persons who are different from those wearing the head mounted display device 100). For example, the image content currently displayed on the first display screen 1 may be synchronously displayed on the second display screen 2 through the controller 3 (based on the connection relationship between the controller 3 and the first display screen 1 and the second display screen 2 that are electrically connected), so that when the user wants to share the screen with people around, a content can be shared through the second display screen 2. The user can also move the cursor on the second display screen 2 for discussion by manipulating the handle, eye control or hand control, etc., so as to realize the interaction of sharing a video, discussing a piece of clothing or a game, etc.

[0052] Exemplarily, the second display screen 2 is disposed on the front shell 4 of the head mounted display device 100 .

[0053] In one embodiment, the outer side of the front shell 4 has a mounting groove, and the second screen can be arranged at the mounting groove. In another embodiment, the front shell 4 has a transparent shell area, the second display screen 2 is arranged on the inner side of the front shell 4, and the display interface of the second screen faces the transparent shell area.

[0054] In this embodiment, the head-mounted display device 100 is provided with a first display screen 1, a second display screen 2 and a controller 3, wherein the first display screen 1 is provided on the inner side of the head-mounted display device 100, and the display interface of the first display screen 1 faces the inner side of the head-mounted display device 100, wherein the inner side is the side of the head-mounted display device 100 close to the user's face when the head-mounted display device 100 is worn, and then the second display screen 2 is provided on the outer side of the head-mounted display device 100, and the display interface of the second display screen 2 faces the outer side of the head-mounted display device 100, wherein the outer side is the side of the head-mounted display device 100 away from the user's face when the head-mounted display device 100 is worn, and then the controller 3 is electrically connected to the first display screen 1 and the second display screen 2, so that the head-mounted display device of the embodiment of the present application is 100 has built a dual-screen display, namely a first display screen 1 (user screen) and a second display screen 2 (interactive screen). The second display screen 2 serves as an interactive platform for the user and the people around him. Other users can communicate and discuss through the second display screen 2. For example, when the user (the wearer of the head-mounted display device 100) wants to share the screen with the people around him, he can share the content through the second display screen 2 integrated in the head-mounted display device 100. In this way, the image content seen by the user from the first display screen 1 is consistent with the image content seen by the people around him from the second display screen 2, thereby achieving image content interaction such as sharing a video, discussing a piece of clothing or a game with a third party without adding wireless projection, thereby improving the convenience and efficiency of image content interaction.

[0055] In one practicable manner, the first display screen 1 and the second display screen 2 are stacked.

[0056] In this embodiment, the first display screen 1 and the second display screen 2 can be stacked to reduce the volume requirement of the installation space, which is more conducive to the miniaturization and integration of the structure of the head-mounted display device 100.

[0057] Of course, in another practicable manner, the first display screen 1 and the second display screen 2 may be an integrated double-sided display screen (this embodiment is not shown in the figure), the first surface of the double-sided display screen is the first display screen 1, and the second surface of the double-sided display screen is the second display screen 2.

[0058] In this embodiment, the first display screen 1 and the second display screen 2 can be integrated to make the structure of the head-mounted display device 100 more compact, thereby further reducing the installation space requirement in the front shell 4 of the head-mounted display device 100, and facilitating the miniaturization and lightweight of the structure of the head-mounted display device 100.

[0059] In addition, the head mounted display device 100 includes an optical lens 5 . In an example, the optical lens 5 is disposed on the inner side of the first display screen 1 , and the optical lens 5 is disposed parallel to the first display screen 1 .

[0060] In a possible implementation, the head mounted display device 100 further includes a proximity sensor (not shown), and the proximity sensor is electrically connected to the controller 3 .

[0061] In this embodiment, the type of the proximity sensor may include a photoelectric proximity sensor. Those skilled in the art have made in-depth research on the type and principle of the proximity sensor, which will not be elaborated here.

[0062] In this embodiment, the head mounted display device 100 is provided with a proximity sensor, and the proximity sensor is electrically connected to the controller 3, so that it is convenient to detect the distance between the head mounted display device 100 and other people in a preset surrounding area through the proximity sensor. After detecting that the distance is less than a certain value, a proximity signal is sent to the controller 3, so that the controller 3 controls the image content currently displayed on the first display screen 1 to be synchronously transmitted to the second display screen 2 for display, thereby achieving the sharing of image content with a third party without adding wireless screen projection, and further improving the convenience and efficiency of image content interaction.

[0063] In addition, it is worth mentioning that the present embodiment can also detect the distance between the head mounted display device 100 and other people in the surrounding preset area through the proximity sensor. After detecting that the distance is greater than the preset value for a period of time greater than the preset time, a distance signal is sent to the controller 3, so that the controller 3 controls the second display screen 2 to turn off, thereby realizing timely extinguishing of the second display screen 2 when the surrounding people are away to reduce power consumption.

[0064] Exemplarily, the head mounted display device 100 further includes a camera (not shown), which is electrically connected to the controller 3 .

[0065] In this embodiment, the type of the camera includes but is not limited to a depth camera, an infrared camera, etc., which is not specifically limited in this embodiment. It is easy to understand that the shooting direction of the camera should be consistent or almost consistent with the direction of the display interface of the second display screen 2.

[0066] In this embodiment, the head mounted display device 100 is provided with a camera, and the camera is electrically connected to the controller 3, so that the camera can detect gestures made by people in the surrounding preset area facing the second display screen 2. If a gesture is detected in the surrounding preset area, the gesture information is sent to the controller 3. After receiving the gesture information, the controller 3 polls and matches a target interaction gesture whose matching rate with the gesture information is greater than a preset ratio value from a pre-stored gesture template database, and generates a target interaction instruction mapped by the target interaction gesture. Based on the target interaction instruction, the image content of the first display screen 1 and the second display screen 2 are synchronously updated to complete the human-computer interaction between the third party and the head mounted display device 100.

[0067] In addition, the embodiment of the present application also provides an image content interaction method, which is applied to the head-mounted display device as described above, wherein the head-mounted display device includes a first display screen and a second display screen, please refer to Figure 2 In a first embodiment of the image content interaction method, the image content interaction method includes the following steps:

[0068] Step S10, detecting whether a screen sharing instruction is input;

[0069] In this embodiment, the user (wearer of the head mounted display device) can input the screen sharing instruction by manipulating the handle, eye control or hand control, etc. For the hand control operation, the head mounted display device can generate and obtain the screen sharing instruction in response to the touch operation on the physical button, and the head mounted display device can also generate and obtain the screen sharing instruction in response to the touch operation on the virtual button.

[0070] Step S20: If it is detected that the screen sharing instruction is input, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display.

[0071] In this embodiment, specifically, the image content interaction method is applied to the controller of the head mounted display device as described above.

[0072] In this embodiment, the head-mounted display device is configured to include a first display screen, a second display screen and a controller, and detects whether a screen sharing instruction is input. After detecting that the user has input the screen sharing instruction, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display, so that the head-mounted display device of the embodiment of the present application builds a dual-screen display, namely the first display screen (user screen) and the second display screen (interactive screen). The second display screen serves as an interactive platform for the user and the people around him. The user can communicate and discuss through the second display screen. For example, when the user (the wearer of the head-mounted display device) wants to share the screen with the people around him, the content can be shared through the second display screen integrated in the head-mounted display device. In this way, the image content seen by the user from the first display screen is consistent with the image content seen by the people around him from the second display screen, thereby achieving image content interaction such as sharing a video, discussing a piece of clothing or a game with a third party without adding wireless projection, thereby improving the convenience and efficiency of image content interaction.

[0073] To help understand, here is an example scenario: Figure 4 As shown, Figure 4 is a schematic diagram of a display scene of a head mounted display device in response to a screen sharing instruction in an embodiment of the present invention, Figure 4 The smiley face pattern on the first display screen (i.e., the user screen) is the image content currently being displayed. Figure 4 The smiley face pattern currently displayed on the second display screen (ie, the interactive screen) is the same image content as the first display screen. It should be noted that the smiley face image displayed in this example scene is only an example of image content and does not constitute a limitation on the image content played on the display screen.

[0074] In one possible implementation, please refer to Figure 6 , the head mounted display device includes a proximity sensor, and before the step of detecting whether a screen sharing instruction is input, the method further includes:

[0075] Step S30, detecting the distance between the head mounted display device and other people in a surrounding preset area through the proximity sensor;

[0076] Step S40, after detecting that the distance is less than a first preset distance, generating and outputting a preset mode selection window;

[0077] In this embodiment, it should be noted that “after detecting that the distance is less than the first preset distance, generating and outputting a preset mode selection window” can be interpreted as “when it is detected that the distance is less than the first preset distance for a time period greater than the preset time period, generating and outputting a preset mode selection window”, and can also be interpreted as “once it is detected that the distance is less than the first preset distance, generating and outputting a preset mode selection window”, both implementation modes should be within the scope of protection of the present application.

[0078] The step S10 of detecting whether a screen sharing instruction is input includes:

[0079] Step S11 , detecting whether a screen sharing instruction is input in response to the mode selection window.

[0080] In this embodiment, if it is detected that a screen sharing instruction is input in response to the mode selection window, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display. The first preset distance is not specifically limited in this embodiment, and those skilled in the art can set it according to actual conditions to better predict whether there are people around approaching the second display screen to interact with the image content.

[0081] In this embodiment, a head mounted display device is provided with a proximity sensor, and the distance between the head mounted display device and other people in a preset surrounding area is detected by the proximity sensor. After detecting that the distance is less than a first preset distance, a preset mode selection window is generated and outputted. If it is detected that a screen sharing instruction is input in response to the mode selection window, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display, thereby achieving image content interaction such as sharing a video, discussing a piece of clothing or a game with a third party without adding wireless screen projection, thereby further improving the convenience and efficiency of image content interaction.

[0082] In addition, it is worth mentioning that, when the distance to other people in the surrounding preset area is detected to be less than the first preset distance, the present embodiment does not immediately synchronously transmit the image content currently displayed on the first display screen to the second display screen for display, but first displays a preset mode selection window to the user, asking the user to confirm whether to enter the screen sharing mode, that is, confirms with the user to share the image content currently played on the first display screen with other people, and only enters the screen sharing mode after obtaining the user's confirmation instruction, and synchronously transmits the image content currently displayed on the first display screen to the second display screen for display, thereby achieving effective protection of the user's privacy information while satisfying the function of image content interaction, thereby improving the robustness of the image content interaction method of the embodiment of the present application.

[0083] Further, after the step of generating and outputting a preset mode selection window, the method further includes:

[0084] Step A10: If it is detected that an immersive mode instruction is input in response to the mode selection window, the preset do not disturb prompt information is transmitted to the second display screen for display.

[0085] In this embodiment, the mode selection window is a guide selection window for guiding the user to select a screen sharing mode or an immersive mode.

[0086] The preset do not disturb prompt information may be text information and / or pattern information that prompts other people in the surrounding preset area not to disturb, and this embodiment does not make specific limitations. Figure 5 As shown, Figure 5 is a schematic diagram of a display scene of a head mounted display device in response to an immersive mode instruction in an embodiment of the present invention, Figure 5 The smiley face pattern on the first display screen (i.e., the user screen) is the image content currently being displayed. Figure 5The "Do Not Disturb" text on the second display screen (i.e., the interactive screen) is the Do Not Disturb prompt message currently being displayed. Those skilled in the art will appreciate that the "Do Not Disturb" text message displayed on the second display screen is only an example of a "Do Not Disturb prompt message", and may also be similar text messages such as "Please create a quiet environment, a customer is having an immersive experience, thank you", or may also be pattern information used to express the Do Not Disturb prompt message, or may also be graphic information that combines text and pattern information used to express the Do Not Disturb prompt message. This embodiment does not make specific limitations on this, and simple transformations in more forms based on this technical concept should all be within the scope of protection of this application.

[0087] This embodiment transmits the preset Do Not Disturb prompt information to the second display screen for display after detecting that an immersive mode command is input in response to the mode selection window. This allows the user to remind people around them through the Do Not Disturb prompt information on the front screen (i.e., the second display screen) when they want to have an immersive experience such as VR content, thereby avoiding being disturbed by people around them as much as possible.

[0088] In order to further help understand the technical concept of the present application, a specific embodiment is listed, please refer to Figure 3 , Figure 3 The following is a schematic diagram of the interactive process of the head mounted display device controlling the dual-screen display in an embodiment of the present invention:

[0089] In this specific embodiment, the head-mounted display device has a dual-screen display, namely a first display screen (i.e., a user screen) and a second display screen (i.e., an interactive screen). The second display screen serves as an interactive platform between the user and the people around him / her, and the user can communicate and discuss through the second display screen. In addition, the user can input a preset first interactive instruction (i.e., an immersive mode instruction) in the APP (Application) of the head-mounted display device by means of a handle, eye control, or hand control, and then the APP informs the controller that the user needs to have an immersive experience at this time, and the controller transmits the preset Do Not Disturb prompt information to the second display screen for display, while the first display screen plays the extended reality content that the current user is immersed in normally. In addition, the user can input a preset second interactive instruction (i.e., a screen sharing instruction) to the APP of the head-mounted display device by means of a handle, eye control, or hand control, and then the APP informs the controller that the user needs to share the image content with the people around him / her at this time, and the controller synchronously transmits the image content currently displayed on the first display screen to the second display screen for display.

[0090] It should be noted that the specific embodiments shown above are only used to assist in understanding the technical concept of the present application and do not constitute a limitation of the present application. More simple transformations based on the technical concept of the present application should all be within the scope of protection of the present application.

[0091] In an practicable manner, the head mounted display device includes a camera, and after the step of synchronously transmitting the image content currently displayed on the first display screen to the second display screen for display, the method further includes:

[0092] Step B10, detecting actual gestures made by other persons through the camera;

[0093] Step B20, if the matching rate between the actual gesture and the pre-stored target interaction gesture is greater than a preset ratio value, generating a target interaction instruction corresponding to the target interaction gesture;

[0094] Step B30: synchronously updating the image contents of the first display screen and the second display screen based on the target interaction instruction to complete human-computer interaction.

[0095] In this embodiment, a gesture template database may be pre-stored in the system of the head-mounted display device, and the gesture template database includes multiple template gestures, each template gesture maps an interaction instruction, wherein the template gesture whose matching rate with the actual gesture is greater than a preset ratio value is the target interaction gesture, and the interaction instruction mapped by the target interaction gesture is the target interaction instruction.

[0096] In this embodiment, a head-mounted display device is provided with a camera, and actual gestures made by a person in a preset area facing the second display screen are detected through the camera. If it is detected that the matching rate of the actual gesture from the pre-stored target interaction gestures is greater than a preset ratio value, a target interaction instruction corresponding to the target interaction gesture is generated, and the image contents of the first display screen and the second display screen are synchronously updated based on the target interaction instruction to complete the human-computer interaction between the third party and the head-mounted display device, thereby effectively realizing image content interaction and further improving the convenience and efficiency of image content interaction.

[0097] The present application also provides a possible implementation manner. In this embodiment, after the step of synchronously transmitting the image content displayed on the first display screen to the second display screen for display, the method further includes:

[0098] Step C10, after detecting that the time duration of the distance being greater than the second preset distance is greater than a preset time threshold, controlling the second display screen to turn off, wherein the first preset distance is less than or equal to the second preset distance;

[0099] Step C20, returning to the step of detecting the distance between the head mounted display device and other persons in a surrounding preset area by using the proximity sensor.

[0100] In this embodiment, the preset time threshold is not specifically limited in this embodiment, and those skilled in the art may set it according to actual conditions to better predict that the surrounding people have left the second display screen and will not interact with the image content for a short period of time.

[0101] It can be understood that after the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display, the proximity sensor is still detecting the distance to other people in the surrounding preset area in real time, for example, by detecting whether the time when the distance to other people is greater than the second preset distance is greater than the preset time threshold, thereby predicting whether the surrounding people have left the second display screen and whether they will no longer interact with the image content in a short time.

[0102] This embodiment uses a proximity sensor to detect the distance between the head-mounted display device and other people in a surrounding preset area. After detecting that the distance is greater than a second preset distance for a period longer than a preset time threshold, the second display screen is controlled to turn off, thereby achieving timely extinguishing of the second display screen when people around move away to reduce power consumption.

[0103] In order to further help understand the technical concept of the present application, the following specific examples of interactive scenes of image content are listed:

[0104] In this scenario, when the user wearing the display device wants to share the screen with people around him, the user can actively input the screen sharing command to make the interactive screen (i.e., the second display screen) share the image content. The user can also move the cursor on the second display screen for discussion by manipulating the handle, eye control, or hand control. When the user is in immersive mode, the user can actively input the immersive mode command to inform the people around him of the immersive experience in text form, so as to avoid being disturbed by the people around him as much as possible. When the user detects that the people around him (i.e., other people in the surrounding preset area) are away through the distance sensor (i.e., proximity sensor), the interactive screen can be turned off to save power consumption.

[0105] It should be noted that the interaction scenarios shown above are only used to assist in understanding the technical concept of the present application and do not constitute a limitation of the present application. More simple transformations based on the technical concept of the present application should all be within the scope of protection of the present application.

[0106] like Figure 7 As shown, Figure 7 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0107] The terminal of the embodiment of the present invention is a head mounted display device. The head mounted display device may be, for example, a mixed reality (MixedReality)-MR device (e.g., MR glasses or MR helmet), an augmented reality (AugmentedReality)-AR device (e.g., AR glasses or AR helmet), a virtual reality-VR device (e.g., VR glasses or VR helmet), an extended reality (ExtendedReality)-XR device (e.g., XR glasses or XR helmet), or a combination thereof.

[0108] like Figure 7 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0109] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, the sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the terminal device is moved to the ear. Of course, the terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0110] Those skilled in the art will understand that Figure 7 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0111] like Figure 7 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image content interaction program.

[0112] exist Figure 7In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and perform data communication with the client; and the processor 1001 can be used to call the image content interaction program stored in the memory 1005 and perform the following operations:

[0113] Detect whether the screen sharing command is entered;

[0114] If the input of the screen sharing instruction is detected, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display.

[0115] The present invention further provides a computer-readable storage medium, on which an image content interaction program is stored, and when the image content interaction program is executed by a processor, the steps of the image content interaction method described above are implemented.

[0116] The specific implementation of the computer-readable storage medium of the present invention is basically the same as the above-mentioned embodiments of the image content interaction method, and will not be repeated here.

[0117] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0118] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0120] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A head mounted display device, characterized in that: include: A first display screen, wherein the first display screen is disposed on the inner side of the head mounted display device, and a display interface of the first display screen faces the inner side of the head mounted display device, wherein the inner side is a side of the head mounted display device close to the user's face when the head mounted display device is worn; A second display screen, wherein the second display screen is disposed on the outside of the head-mounted display device, and a display interface of the second display screen faces the outside of the head-mounted display device, wherein the outside is a side of the head-mounted display device that is away from the user's face when the head-mounted display device is worn; A controller is electrically connected to the first display screen and the second display screen.

2. The head mounted display device according to claim 1, wherein: The head mounted display device further includes a proximity sensor, which is electrically connected to the controller.

3. The head mounted display device according to claim 1, wherein: The head mounted display device also includes a camera, which is electrically connected to the controller.

4. The head mounted display device according to any one of claims 1 to 3, characterized in that: The second display screen is arranged on the front shell of the head mounted display device.

5. An image content interaction method, characterized in that: The image content interaction method is applied to a head mounted display device according to any one of claims 1 to 4, wherein the head mounted display device comprises a first display screen and a second display screen, and the method comprises: Detect whether the screen sharing command is entered; If the input of the screen sharing instruction is detected, the image content currently displayed on the first display screen is synchronously transmitted to the second display screen for display.

6. The image content interaction method according to claim 5, characterized in that: The head mounted display device includes a proximity sensor, and before the step of detecting whether a screen sharing instruction is input, the method further includes: Detecting the distance between the head mounted display device and other people in a surrounding preset area through the proximity sensor; After detecting that the distance is less than a first preset distance, generating and outputting a preset mode selection window; The step of detecting whether a screen sharing instruction is input comprises: It is detected whether a screen sharing instruction is input in response to the mode selection window.

7. The image content interaction method according to claim 6, characterized in that: The head mounted display device includes a camera, and after the step of synchronously transmitting the image content currently displayed on the first display screen to the second display screen for display, the method further includes: detecting, by means of the camera, actual gestures made by other persons; If the matching rate between the actual gesture and the pre-stored target interaction gesture is greater than a preset ratio value, generating a target interaction instruction corresponding to the target interaction gesture; The image contents of the first display screen and the second display screen are synchronously updated based on the target interaction instruction to complete human-computer interaction.

8. The image content interaction method according to claim 6, characterized in that: After the step of synchronously transmitting the image content displayed on the first display screen to the second display screen for display, the method further includes: After detecting that the time duration for which the distance is greater than a second preset distance is greater than a preset time threshold, controlling the second display screen to turn off, wherein the first preset distance is less than or equal to the second preset distance; Return to execution: the step of detecting the distance between the head mounted display device and other people in a surrounding preset area by using the proximity sensor.

9. The image content interaction method according to claim 6, characterized in that: After the step of generating and outputting a preset mode selection window, the method further includes: If it is detected that an immersive mode instruction is input in response to the mode selection window, the preset do not disturb prompt information is transmitted to the second display screen for display.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an image content interaction program, and when the image content interaction program is executed by a processor, the steps of the image content interaction method according to any one of claims 5 to 9 are implemented.

Images