Image content interaction method, head-mounted display device, and computer-readable storage medium

Abstract

The application discloses an image content interaction method, a head-mounted display device and a computer readable storage medium. The head-mounted display device comprises a first display screen, a second display screen and a controller. The first display screen is arranged on the inner side of the head-mounted display device, and the display interface of the first display screen faces the inner side of the head-mounted display device. The second display screen is arranged on the outer side of the head-mounted display device, and the display interface of the second display screen faces the outer side of the head-mounted display device. The outer side is the side of the head-mounted display device which is away from the face of a user in the wearing state. The controller is electrically connected with the first display screen and the second display screen. The application can improve the convenience and efficiency of the head-mounted display device in image content interaction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent interaction, and in particular to an image content interaction method, a head-mounted display device and a computer readable storage medium. BACKGROUND

[0002] With the increasing diversification of smart wearable devices, head-mounted display devices have gradually entered people's lives. However, the images displayed by the related head-mounted display devices can only be viewed by the wearer, that is, the extended reality experience such as VR (Virtual Reality) is basically the image content experience of the user himself. If more people want to join the image content interaction experience, the user needs to project the image content to a PC (personal computer) through wireless projection, that is, the scheme needs to additionally configure an electronic display device for the head-mounted display device to provide projection as an interaction platform for the user and the surrounding people, so as to realize the image content interaction of sharing a video, discussing a piece of clothing or a game with a third party, and the convenience and efficiency of image content interaction are low. SUMMARY

[0003] The main purpose of the present application is to provide an image content interaction method, a head-mounted display device and a computer readable storage medium, which aims to solve the technical problem of low convenience and efficiency of image content interaction of the current head-mounted display device.

[0004] To achieve the above-mentioned purpose, the present application provides a head-mounted display device, comprising:

[0005] A first display screen is arranged on the inner side of the head-mounted display device, and the display interface of the first display screen faces the inner side of the head-mounted display device, wherein the inner side is the side close to the user's face in the wearing state of the head-mounted display device;

[0006] A second display screen is arranged on the outer side of the head-mounted display device, and the display interface of the second display screen faces the outer side of the head-mounted display device, wherein the outer side is the side away from the user's face in the wearing state of the head-mounted display device;

[0007] A controller is electrically connected with the first display screen and the second display screen.

[0008] Optionally, the head-mounted display device further comprises a proximity sensor, and the proximity sensor is electrically connected with the controller.

[0009] Optionally, the head-mounted display device further comprises a camera, and the camera is electrically connected with the controller.

[0010] Optionally, the second display screen is arranged on a front shell of the head-mounted display device.

[0011] Further, to achieve the above object, the present application provides an image content interaction method, applied to the head-mounted display device as above, comprising a first display screen and a second display screen, the method comprising:

[0012] detecting whether a screen sharing instruction is inputted;

[0013] if the screen sharing instruction is detected, synchronously transmitting image content currently displayed by the first display screen to the second display screen for display.

[0014] Optionally, the head-mounted display device comprises a proximity sensor, and before the step of detecting whether the screen sharing instruction is inputted, the method further comprises:

[0015] detecting, by the proximity sensor, a distance between the head-mounted display device and other people in a preset area around the head-mounted display device;

[0016] after detecting that the distance is less than a first preset distance, generating and outputting a preset mode selection window;

[0017] the step of detecting whether the screen sharing instruction is inputted comprises:

[0018] detecting whether the screen sharing instruction is inputted in response to the mode selection window.

[0019] Optionally, the head-mounted display device comprises a camera, and after the step of synchronously transmitting the image content currently displayed by the first display screen to the second display screen for display, the method further comprises:

[0020] detecting, by the camera, an actual gesture made by the other people;

[0021] if a matching rate of the actual gesture and a target interaction gesture stored in advance is greater than a preset rate value, generating a target interaction instruction corresponding to the target interaction gesture;

[0022] synchronously updating image content of the first display screen and the second display screen based on the target interaction instruction, so as to complete human-computer interaction.

[0023] Optionally, after the step of synchronously transmitting the image content currently displayed by the first display screen to the second display screen for display, the method further comprises:

[0024] after detecting that the distance is greater than a second preset distance for a time greater than a preset time threshold, controlling the second display screen to be turned off, wherein the first preset distance is less than or equal to the second preset distance.

[0025] returning to execute: the step of detecting, by the proximity sensor, the distance between the head-mounted display device and other people in the surrounding preset area.

[0026] Optionally, after the step of generating and outputting the preset mode selection window, the method further comprises:

[0027] If it is detected that an immersive mode instruction is input in response to the mode selection window, preset do-not-disturb prompt information is transmitted to the second display screen for display.

[0028] In addition, the present application also provides a computer readable storage medium, the computer readable storage medium has an image content interaction program stored thereon, the computer readable storage medium has an image content interaction program stored thereon, and the image content interaction program is executed by a processor to realize the steps of the image content interaction method described above.

[0029] The present application sets the head-mounted display device to include a first display screen, a second display screen and a controller, the first display screen is arranged on the inner side of the head-mounted display device, and the display interface of the first display screen faces the inner side of the head-mounted display device, wherein the inner side is the side close to the user's face in the wearing state of the head-mounted display device, then the second display screen is arranged on the outer side of the head-mounted display device, and the display interface of the second display screen faces the outer side of the head-mounted display device, wherein the outer side is the side away from the user's face in the wearing state of the head-mounted display device, and the controller is electrically connected with the first display screen and the second display screen, so that the head-mounted display device of the present application builds a double-screen display, that is, the first display screen (user screen) and the second display screen (interaction screen), the second display screen serves as an interactive platform for the user and the surrounding people, and other users can communicate and discuss through the second display screen, for example, when the user (wearer of the head-mounted display device) wants to share the screen with the surrounding people, the user can share the content through the second display screen integrated in the head-mounted display device, so that the image content seen by the user from the first display screen is consistent with the image content seen by the surrounding people from the second display screen, thereby realizing the image content interaction of sharing a video, discussing a piece of clothing or a game, etc. with a third party without increasing wireless projection, and improving the convenience and efficiency of image content interaction. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to the structures shown in these drawings without creative labor for those skilled in the art.

[0031] Figure 1 Structure diagram of a head-mounted display device according to an embodiment of the present application;

[0032] Figure 2 Flow diagram of a method for image content interaction according to a first embodiment of the present application;

[0033] Figure 3 Interaction flow diagram of a head-mounted display device controlling dual-screen display according to an embodiment of the present application;

[0034] Figure 4 Display scenario diagram of a head-mounted display device responding to a screen sharing instruction according to an embodiment of the present application;

[0035] Figure 5 Display scenario diagram of a head-mounted display device responding to an immersion mode instruction according to an embodiment of the present application;

[0036] Figure 6 Flow diagram of a method for image content interaction according to a second embodiment of the present application;

[0037] Figure 7 Terminal structure diagram of a hardware running environment involved in an embodiment of the present application.

[0038] Explanation of reference numerals:

[0039] Reference Name Reference Name 100 Head-mounted display device 3 Controller 1 First display screen 4 Front housing 2 Second display screen 5 Optical lens

[0040] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. The above-described drawings have shown the specific embodiments of the present application, and the following will have more detailed description. These drawings and textual descriptions are not intended to limit the scope of the present application concept by any means, but to illustrate the present application concept to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0042] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional indications will also change accordingly.

[0043] In addition, if the description of "first", "second" and the like is involved in the embodiments of the present application, the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of the person skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the present application.

[0044] In the present embodiment, the head-mounted display device of the present application can be, for example, a Mixed Reality (MR) device (such as MR glasses or an MR helmet), an Augmented Reality (AR) device (such as AR glasses or an AR helmet), a Virtual Reality (VR) device (such as VR glasses or a VR helmet), an Extended Reality (XR) device (such as XR glasses or an XR helmet), or some combination thereof.

[0045] At present, the image displayed by the related head-mounted display device can only be viewed by the wearer, that is, the extended reality experience such as VR (Virtual Reality) is basically the image content experience of the user himself. If more people want to join the interactive experience of the image content, the user needs to project the image content to a PC (personal computer) through wireless projection, that is, the scheme needs to additionally configure an electronic display device for the head-mounted display device to provide projection, which serves as an interactive platform for the user and the surrounding people, so as to realize the image content interaction of sharing a video, discussing a piece of clothing or a game with a third party, and the convenience and efficiency of the image content interaction are low.

[0046] Based on this, the present embodiment provides a head-mounted display device 100, please refer to Figure 1 The head-mounted display device 100 comprises:

[0047] A first display screen 1, the first display screen 1 is arranged on the inner side of the head-mounted display device 100, and the display interface of the first display screen 1 faces the inner side of the head-mounted display device 100, wherein the inner side is the side close to the user's face in the wearing state of the head-mounted display device 100;

[0048] The second display screen 2 is arranged on the outer side of the head-mounted display device 100, and the display interface of the second display screen 2 faces the outer side of the head-mounted display device 100, wherein the outer side is the side of the head-mounted display device 100 away from the face of the user in the wearing state.

[0049] The controller 3 is electrically connected with the first display screen 1 and the second display screen 2.

[0050] The controller 3 can be electrically connected with the first display screen 1 and the second display screen 2 through a PCB (printed circuit board).

[0051] In this embodiment, the first display screen 1 can be referred to as a user display screen, which is used to display extended reality content such as VR (Virtual Reality) to the user wearing the head-mounted display device 100. The second display screen 2 can be referred to as an interactive screen, which is used to realize sharing of image content with a third party (the third party is different from other people wearing the head-mounted display device 100). For example, the image content currently displayed on the first display screen 1 can be synchronously displayed on the second display screen 2 through the controller 3 based on the electrical connection between the controller 3 and the first display screen 1 and the second display screen 2, so that when the user wants to share the screen with people around, the user can share the content through the second display screen 2. The user can also move the cursor on the second display screen 2 through the handle manipulation, eye control or hand control to discuss, so as to realize sharing of a video, discussion of a piece of clothing or a game image content, and the like.

[0052] For example, the second display screen 2 is arranged on the front shell 4 of the head-mounted display device 100.

[0053] In one embodiment, the outer side of the front shell 4 has a mounting groove, and the second screen is arranged in the mounting groove. In another embodiment, the front shell 4 has a transparent shell area, the second display screen 2 is arranged on the inner side of the front shell 4, and the display interface of the second screen faces the transparent shell area.

[0054] The head-mounted display device 100 of the embodiment is provided with a first display screen 1, a second display screen 2 and a controller 3. The first display screen 1 is arranged on the inner side of the head-mounted display device 100, and the display interface of the first display screen 1 faces the inner side of the head-mounted display device 100. The inner side is the side close to the face of the user in the wearing state of the head-mounted display device 100. The second display screen 2 is arranged on the outer side of the head-mounted display device 100, and the display interface of the second display screen 2 faces the outer side of the head-mounted display device 100. The outer side is the side away from the face of the user in the wearing state of the head-mounted display device 100. The controller 3 is electrically connected with the first display screen 1 and the second display screen 2. Thus, the head-mounted display device 100 of the embodiment builds a double-screen display, that is, the first display screen 1 (user screen) and the second display screen 2 (interaction screen). The second display screen 2 serves as an interaction platform for the user and the surrounding people. Other users can communicate and discuss through the second display screen 2. For example, when the user (the wearer of the head-mounted display device 100) wants to share the screen with the surrounding people, the user can share the content through the second display screen 2 integrated in the head-mounted display device 100. In this way, the image content seen by the user from the first display screen 1 is consistent with the image content seen by the surrounding people from the second display screen 2. Thus, without increasing wireless projection, the image content interaction of sharing a video, discussing a piece of clothing or a game, etc. with a third party is realized, thereby improving the convenience and efficiency of the image content interaction.

[0055] In an implementable manner, the first display screen 1 and the second display screen 2 are arranged in layers.

[0056] The first display screen 1 and the second display screen 2 are arranged in layers, thereby reducing the size requirement of the installation space, and facilitating the miniaturization and integration of the structure of the head-mounted display device 100.

[0057] Of course, in another implementable manner, the first display screen 1 and the second display screen 2 can be a double-sided display screen (not shown in the embodiment) arranged in one body. The first surface of the double-sided display screen is the first display screen 1, and the second surface of the double-sided display screen is the second display screen 2.

[0058] The first display screen 1 and the second display screen 2 are arranged in one body, thereby making the structure of the head-mounted display device 100 more compact, and further reducing the requirement of the installation space in the front shell 4 of the head-mounted display device 100, and facilitating the miniaturization and light weight of the structure of the head-mounted display device 100.

[0059] In addition, the head-mounted display device 100 includes an optical lens 5. In an example, the optical lens 5 is arranged on the inner side of the first display screen 1, and the optical lens 5 is arranged in parallel with the first display screen 1.

[0060] In a possible implementation, the head-mounted display device 100 further comprises a proximity sensor (not shown) which is electrically connected to the controller 3.

[0061] In the embodiment, the proximity sensor can be an optical proximity sensor, and the type and principle of the proximity sensor are well known to those skilled in the art and will not be described here.

[0062] In the embodiment, the head-mounted display device 100 is provided with the proximity sensor which is electrically connected to the controller 3, so that the distance between the head-mounted display device 100 and other people in the surrounding preset area can be detected by the proximity sensor, and when the distance is less than a certain value, a proximity signal is sent to the controller 3, so that the controller 3 controls the image content currently displayed on the first display screen 1 to be synchronously transmitted to the second display screen 2 for display, thereby achieving sharing of the image content with the third party without increasing wireless projection, and further improving the convenience and efficiency of image content interaction.

[0063] In addition, it is worth mentioning that the distance between the head-mounted display device 100 and other people in the surrounding preset area can also be detected by the proximity sensor, and when the distance is greater than a preset value for a time greater than a preset time, a distancing signal is sent to the controller 3, so that the controller 3 controls the second display screen 2 to be turned off, thereby extinguishing the second display screen 2 in time to reduce power consumption when the surrounding people are far away.

[0064] Exemplarily, the head-mounted display device 100 further comprises a camera (not shown) which is electrically connected to the controller 3.

[0065] In the embodiment, the camera can be, but is not limited to, a depth camera, an infrared camera, etc., and the embodiment is not limited in this regard. It is easy to understand that the shooting direction of the camera should be consistent or almost consistent with the orientation of the display interface of the second display screen 2.

[0066] In the embodiment, the head-mounted display device 100 is provided with the camera which is electrically connected to the controller 3, so that the gestures made by people in the surrounding preset area facing the second display screen 2 can be detected by the camera, and if a gesture is made in the surrounding preset area, the gesture information of the gesture is sent to the controller 3, and then the controller 3 receives the gesture information, polls a target interactive gesture from a pre-stored gesture template database, the matching rate of the target interactive gesture and the gesture information being greater than a preset ratio value, and generates a target interactive instruction mapped by the target interactive gesture, and synchronously updates the image content of the first display screen 1 and the second display screen 2 based on the target interactive instruction, so as to complete the human-computer interaction between the third party and the head-mounted display device 100.

[0067] Further, the image content interaction method is applied to the head-mounted display device as described above, the head-mounted display device comprises a first display screen and a second display screen, please refer to Figure 2 In the first embodiment of the image content interaction method, the image content interaction method comprises the following steps:

[0068] Step S10, detecting whether a screen sharing instruction is inputted;

[0069] In the embodiment, the user (wearer of the head-mounted display device) can input the screen sharing instruction through human-computer interaction modes such as handle manipulation, eye control or hand control. For the human-computer interaction mode of hand control operation, the head-mounted display device can generate and acquire the screen sharing instruction in response to the touch operation on the physical button, and the head-mounted display device can also generate and acquire the screen sharing instruction in response to the touch operation on the virtual button.

[0070] Step S20, if it is detected that the screen sharing instruction is inputted, synchronously transmitting the image content currently displayed by the first display screen to the second display screen for display.

[0071] In the embodiment, specifically, the image content interaction method is applied to the controller of the head-mounted display device as described above.

[0072] The embodiment synchronously transmits the image content currently displayed by the first display screen to the second display screen for display after detecting that the user inputs the screen sharing instruction, so that the head-mounted display device of the embodiment builds a double-screen display, i.e. the first display screen (user screen) and the second display screen (interaction screen), the second display screen serves as an interaction platform for the user and the surrounding people, the user can communicate and discuss through the second display screen, for example, when the user (wearer of the head-mounted display device) wants to share the screen with the surrounding people, the user can share the content through the second display screen integrated in the head-mounted display device, so that the image content seen by the user from the first display screen is consistent with the image content seen by the surrounding people from the second display screen, thereby achieving image content interaction such as sharing a video, discussing a piece of clothing or a game with a third party without increasing wireless projection, and improving the convenience and efficiency of image content interaction.

[0073] To facilitate understanding, an example scenario is listed as follows: Figure 4 As shown in Figure 4 is a display scene diagram of the head-mounted display device in the embodiment in response to the screen sharing instruction, Figure 4 the smiling face pattern of the first display screen (i.e. user screen) in is the image content currently being displayed,Figure 4 The smiley face pattern currently displayed by the second display screen (i.e., the interactive screen) in the first display screen is consistent with the image content. It should be noted that the smiley face image shown in this example scenario is only an example of image content and does not limit the image content played by the display screen.

[0074] In one possible implementation, referring to Figure 6 The head-mounted display device includes a proximity sensor, and before the step of detecting whether a screen sharing instruction is input, the method further includes:

[0075] Step S30, detecting the distance between the head-mounted display device and other people in the surrounding preset area through the proximity sensor;

[0076] Step S40, after detecting that the distance is less than the first preset distance, generating and outputting a preset mode selection window;

[0077] In this embodiment, it should be noted that "after detecting that the distance is less than the first preset distance, generating and outputting a preset mode selection window" can be interpreted as "after detecting that the distance is less than the first preset distance for a duration greater than a preset duration, generating and outputting a preset mode selection window", or as "as soon as the distance is detected to be less than the first preset distance, generating and outputting a preset mode selection window". Both of these implementations should be within the scope of the present application.

[0078] The step S10 of detecting whether a screen sharing instruction is input includes:

[0079] Step S11, detecting whether a screen sharing instruction is input in response to the mode selection window.

[0080] In this embodiment, if it is detected that a screen sharing instruction is input in response to the mode selection window, the image content currently displayed by the first display screen is synchronously transmitted to the second display screen for display. The first preset distance is not specifically limited in this embodiment, and can be set by a person skilled in the art according to actual conditions, so as to better predict whether there are people around the second display screen for image content interaction.

[0081] The embodiment sets the head-mounted display device to include a proximity sensor, detects the distance between the head-mounted display device and other people in the surrounding preset area through the proximity sensor, generates and outputs a preset mode selection window after detecting that the distance is less than a first preset distance, and if it is detected that a screen sharing instruction is input in response to the mode selection window, synchronously transmits the image content currently displayed by the first display screen to the second display screen for display, so that the image content interaction of sharing a video, discussing a piece of clothing or a game, etc. with a third party is realized without increasing wireless projection, and the convenience and efficiency of image content interaction are further improved.

[0082] In addition, it is worth mentioning that, in the embodiment, after detecting that the distance between the head-mounted display device and other people in the surrounding preset area is less than a first preset distance, the image content currently displayed by the first display screen is not synchronously transmitted to the second display screen for display immediately, but a preset mode selection window is displayed to the user first, so that the user confirms whether to enter a screen sharing mode, i.e., the user confirms whether to share the image content currently played by the first display screen with other people, and after obtaining a confirmation instruction from the user, the screen sharing mode is entered, and the image content currently displayed by the first display screen is synchronously transmitted to the second display screen for display, so that the privacy information of the user is effectively protected while the function of image content interaction is met, and the robustness of the image content interaction method of the embodiment is improved.

[0083] Further, after the step of generating and outputting the preset mode selection window, the method further includes:

[0084] Step A10, if it is detected that an immersive mode instruction is input in response to the mode selection window, preset do-not-disturb prompt information is transmitted to the second display screen for display.

[0085] In the embodiment, the mode selection window is a guided selection window that guides the user to select a screen sharing mode or an immersive mode.

[0086] The preset do-not-disturb prompt information can be text information and / or pattern information that prompts other people in the surrounding preset area not to disturb, and the embodiment is not limited in this regard. As an example, as shown in Figure 5 Figure 5 is a display scene diagram of the head-mounted display device in response to the immersive mode instruction in the embodiment, Figure 5 the smiley face pattern of the first display screen (i.e., the user screen) in Figure 5 ​The "please do not disturb" text displayed on the second display screen (i.e., the interactive screen) in the first display screen (i.e., the user screen) is the current displayed do-not-disturb prompt information. Those skilled in the art can understand that the "please do not disturb" text information displayed on the second display screen is only an example of the "do-not-disturb prompt information", and can also be "please create a quiet environment, a customer is experiencing immersive experience, thank you" and the like, or can also be pattern information for expressing the do-not-disturb prompt information, or can also be graphic information for expressing the do-not-disturb prompt information by combining text and pattern. The present embodiment does not make specific limitations thereon, and more forms of simple changes based on the technical concept should be within the protection scope of the present application.

[0087] The present embodiment transmits the preset do-not-disturb prompt information to the second display screen for display after detecting that the immersive mode instruction is input in response to the mode selection window, so that the user can remind the surrounding people through the do-not-disturb prompt information of the front screen (i.e., the second display screen) when he wants to experience the VR content and the like, thereby avoiding being disturbed by the surrounding people as much as possible.

[0088] In order to further assist in understanding the technical concept of the present application, a specific embodiment is listed, please refer to Figure 3 , Figure 3 The interactive flowchart of the head-mounted display device for controlling the dual-screen display in the embodiment of the present application is as follows:

[0089] In the specific embodiment, the head-mounted display device sets up dual-screen display, i.e., the first display screen (i.e., the user screen) and the second display screen (i.e., the interactive screen). The second display screen serves as an interactive platform for the user and the surrounding people, and the user can communicate and discuss through the second display screen. In addition, the user can input the preset first interactive instruction (i.e., the immersive mode instruction) in the APP (Application) of the head-mounted display device through the handle, eye control or hand control, and then the APP informs the controller that the user needs to experience the immersive experience at this time, and the controller transmits the preset do-not-disturb prompt information to the second display screen for display, while the first display screen normally plays the extended reality content that the user is immersed in. In addition, the user can input the preset second interactive instruction (i.e., the screen sharing instruction) in the APP of the head-mounted display device through the handle, eye control or hand control, and then the APP informs the controller that the user needs to share the image content to the surrounding people at this time, and the controller synchronously transmits the image content currently displayed on the first display screen to the second display screen for display.

[0090] It should be noted that the specific embodiments shown above are only used to assist in understanding the technical concept of the present application and do not constitute a limitation of the present application. More forms of simple changes based on the technical concept of the present application should be within the protection scope of the present application.

[0091] In an implementable manner, the head-mounted display device comprises a camera, and after the step of synchronously transmitting the image content currently displayed by the first display screen to the second display screen for display, the method further comprises:

[0092] Step B10, detecting an actual gesture made by the other person through the camera;

[0093] Step B20, if a matching rate of the actual gesture and the pre-stored target interaction gesture is greater than a preset ratio value, generating a target interaction instruction corresponding to the target interaction gesture;

[0094] Step B30, synchronously updating image content of the first display screen and the second display screen based on the target interaction instruction to complete human-computer interaction.

[0095] In this embodiment, a gesture template database can be pre-stored in the system of the head-mounted display device, the gesture template database comprising a plurality of template gestures, each of which maps an interaction instruction, wherein a template gesture with a matching rate greater than a preset ratio value is a target interaction gesture, and an interaction instruction mapped by the target interaction gesture is a target interaction instruction.

[0096] In this embodiment, the head-mounted display device is provided with a camera, and an actual gesture made by the other person facing the second display screen in a preset area around the head-mounted display device is detected through the camera. If a matching rate of the actual gesture and a pre-stored target interaction gesture is greater than a preset ratio value, a target interaction instruction corresponding to the target interaction gesture is generated, and image content of the first display screen and the second display screen is synchronously updated based on the target interaction instruction to complete human-computer interaction of the third party with the head-mounted display device, thereby effectively realizing image content interaction and improving the convenience and efficiency of image content interaction.

[0097] The present application also provides a possible implementable manner. In this embodiment, after the step of synchronously transmitting the image content displayed by the first display screen to the second display screen for display, the method further comprises:

[0098] Step C10, after detecting that the distance greater than the second preset distance is greater than a preset time threshold, controlling the second display screen to be turned off, wherein the first preset distance is less than or equal to the second preset distance;

[0099] Step C20, returning to execute the step of detecting the distance between the head-mounted display device and the other person in the preset area around the head-mounted display device through the proximity sensor.

[0100] In the embodiment, the preset time threshold is not specifically limited, and a person skilled in the art can set it according to actual conditions to better predict that the people around the second display have left and will not interact with the image content in a short time.

[0101] It can be understood that after the image content currently displayed by the first display is synchronously transmitted to the second display for display, the proximity sensor still detects the distance between the people in the surrounding preset area and other people in real time, for example, by detecting whether the time during which the distance between the people in the surrounding preset area and other people is greater than the second preset distance is greater than the preset time threshold, to predict whether the people around the second display have left and whether they will not interact with the image content in a short time.

[0102] The embodiment detects the distance between the head-mounted display device and other people in the surrounding preset area by using the proximity sensor, and controls the second display to be turned off after detecting that the time during which the distance is greater than the second preset distance is greater than the preset time threshold, so that the second display is turned off in time when the people around the second display are far away to reduce power consumption.

[0103] In order to further assist in understanding the technical concept of the present application, an interaction scene of image content in a specific example is listed as follows:

[0104] In the scene, when the user wearing the display device wants to share the screen with the people around, the user can make the interactive screen (i.e., the second display) share the image content by actively inputting a screen sharing instruction, and the user can also move the cursor on the second display by using a handle, eye control, or hand control to discuss, etc. When the user is in an immersive mode, the user can inform the people around in the form of text by actively inputting an immersive mode instruction to inform the people around in the form of text, so as to avoid being disturbed by the people around as much as possible. When the user detects that the people around (i.e., other people in the surrounding preset area) are far away by using the distance sensor (i.e., the proximity sensor), the interactive screen can be turned off to save power consumption.

[0105] It should be noted that the above-mentioned interaction scene is only used to assist in understanding the technical concept of the present application, and does not constitute a limitation on the present application. Based on the technical concept of the present application, more forms of simple transformation are within the protection scope of the present application.

[0106] As shown in Figure 7 , the terminal structure diagram of the hardware running environment involved in the embodiment of the present application is shown in Figure 7

[0107] ​Embodiments of the present application are terminals for head-mounted display devices. The head-mounted display devices can be, for example, Mixed Reality (MR) devices (e.g., MR glasses or MR headsets), Augmented Reality (AR) devices (e.g., AR glasses or AR headsets), Virtual Reality (VR) devices (e.g., VR glasses or VR headsets), Extended Reality (XR) devices (e.g., XR glasses or XR headsets), or some combination thereof.

[0108] As shown in Figure 7 The terminal can include a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen, an input unit such as a keyboard, and can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0109] Optionally, the terminal can also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. The sensor can include an ambient light sensor, a motion sensor, and other sensors. Specifically, the ambient light sensor can adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor can turn off the display screen and / or backlight when the terminal device is moved to the ear. Of course, the terminal device can also be configured with a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and other sensors, which will not be described here.

[0110] Those skilled in the art can understand that Figure 7 The terminal structure shown in the above description does not constitute a limitation on the terminal, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0111] As shown in Figure 7 The memory 1005, as a computer storage medium, can include an operating system, a network communication module, a user interface module, and an image content interaction program.

[0112] In Figure 7In the terminal shown, the network interface 1004 is mainly used for connecting a background server and communicating data with the background server; the user interface 1003 is mainly used for connecting a client (user end) and communicating data with the client; and the processor 1001 can be used for calling an image content interaction program stored in the memory 1005 and performing the following operations:

[0113] detecting whether a screen sharing instruction is input;

[0114] If the screen sharing instruction is detected, the image content currently displayed by the first display screen is synchronously transmitted to the second display screen for display.

[0115] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores an image content interaction program, and the image content interaction program is executed by a processor to implement the steps of the image content interaction method.

[0116] The computer readable storage medium of the application has basically the same implementation as the above-mentioned image content interaction method, and thus will not be described here.

[0117] It should be noted that, in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0118] The above-mentioned application example numbers are only for description, and do not represent the advantages and disadvantages of the examples.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the application.

[0120] The above merely describes the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application. Any equivalent structural changes made according to the content of the present application specification and drawings, or direct / indirect application in other related technical fields, are included in the patent protection scope of the present application.

Claims

1. An image content interaction method, characterized by, The image content interaction method is applied to a head-mounted display device, the head-mounted display device comprising a controller, and a first display screen, a second display screen and a proximity sensor electrically connected to the controller, and a display interface of the first display screen faces an inner side, and a display interface of the second display screen faces an outer side, wherein the inner side is a side of the head-mounted display device close to a face of a user in a wearing state, and the outer side is a side of the head-mounted display device away from the face of the user in the wearing state, and the method comprises: detecting, by the proximity sensor, a distance between the head-mounted display device and another person in a preset area around the head-mounted display device; generating and outputting a preset mode selection window when a time length during which the distance is less than a first preset distance is greater than a preset time length; detecting whether a screen sharing instruction is input in response to the mode selection window; if the screen sharing instruction is detected to be input, synchronously transmitting image content currently displayed on the first display screen to the second display screen for display; wherein the head-mounted display device comprises a camera, and a gesture template database is pre-stored in a system of the head-mounted display device, the gesture template database comprising a plurality of template gestures, and each template gesture is mapped to an interaction instruction, and after the step of synchronously transmitting the image content currently displayed on the first display screen to the second display screen for display, the method further comprises: detecting, by the camera, an actual gesture made by the other person; if a matching rate of the actual gesture and a target interaction gesture pre-stored is greater than a preset rate value, generating a target interaction instruction corresponding to the target interaction gesture, wherein the target interaction gesture is a template gesture in the gesture template database having a matching rate greater than the preset rate value with the actual gesture, and the target interaction instruction is an interaction instruction mapped by the target interaction gesture; synchronously updating image content of the first display screen and the second display screen based on the target interaction instruction to complete human-computer interaction.

2. The image content interaction method of claim 1, wherein, after the step of synchronously transmitting the image content displayed on the first display screen to the second display screen for display, the method further comprises: controlling the second display screen to be turned off after detecting that a time length during which the distance is greater than a second preset distance is greater than a preset time length threshold, wherein the first preset distance is less than or equal to the second preset distance; returning to execute the step of detecting, by the proximity sensor, the distance between the head-mounted display device and the other person in the preset area around the head-mounted display device.

3. The image content interaction method of claim 1, wherein, after the step of generating and outputting the preset mode selection window, the method further comprises: if an immersive mode instruction is detected to be input in response to the mode selection window, transmitting preset do-not-disturb prompt information to the second display screen for display.

4. A computer-readable storage medium, characterized in that, The computer readable storage medium stores an image content interaction program, and the image content interaction program, when executed by a processor, implements steps of the image content interaction method according to any one of claims 1 to 3.

Images