Display method in motion mode, head-mounted display device, medium and product

Abstract

The embodiment of the invention discloses a display method in a motion mode, head-mounted display equipment, a medium and a product. According to one specific embodiment, the method comprises the steps that in response to the detected head rotation operation of a wearing user, rotation angle information corresponding to the head rotation operation is determined, and the rotation angle information comprises a rotation angle; in response to determining that the rotation angle meets a rotation condition corresponding to a preset angle threshold value, rotating display content in a three-dimensional display space of the head-mounted display equipment according to the rotation angle information; and in response to determining that the rotation angle does not meet the rotation condition, keeping the display content in the three-dimensional display space at an original position. According to the embodiment, the jitter of the display content in the head-mounted display device is weakened, the dizziness of a user during watching is weakened, and the user experience is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a display method, a head-mounted display device, a medium, and a product in a motion mode. Background Art

[0002] Head-mounted display devices (such as AR glasses) provide users with an immersive viewing experience. Currently, when users watch in head-mounted display devices, the method usually adopted is: the display window in the head-mounted display device moves as the user's head turns.

[0003] However, when the above method is adopted, the following technical problems often occur: when the user's head shakes slightly, the display window also moves, causing the display window to shake frequently, and the user feels dizzy when watching, resulting in a poor user experience.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a display method in motion mode, a head-mounted display device, a computer-readable medium, and a computer program product to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a display method in a motion mode, the method comprising: in response to detecting a head rotation operation of a wearing user, determining rotation angle information corresponding to the head rotation operation, wherein the rotation angle information includes a rotation angle; in response to determining that the rotation angle satisfies a rotation condition corresponding to a preset angle threshold, rotating display content in a three-dimensional display space of the head-mounted display device according to the rotation angle information; in response to determining that the rotation angle does not satisfy the rotation condition, maintaining the display content in the three-dimensional display space in the original position.

[0008] Optionally, the method also includes: in response to detecting a movement operation of the wearing user, determining the movement distance information corresponding to the above movement operation, wherein the above movement distance information includes the movement distance; in response to determining that the above movement distance satisfies the movement condition corresponding to a preset distance threshold, moving the display content in the three-dimensional display space of the above head-mounted display device according to the above movement distance information.

[0009] Optionally, the method further includes: in response to determining that the moving distance does not satisfy the moving condition, maintaining the display content in the three-dimensional display space at an original position.

[0010] Optionally, before determining the rotation angle information corresponding to the head rotation operation in response to detecting the head rotation operation of the wearing user, the method also includes: determining the current scene type; and determining the anti-shake threshold information corresponding to the above scene type based on the above scene type, wherein the above anti-shake threshold information includes the above preset angle threshold.

[0011] Optionally, the anti-shake threshold information includes preset angle thresholds corresponding to each rotation direction, and the rotation angle information also includes a rotation direction; and in response to determining that the rotation angle satisfies the rotation condition corresponding to the preset angle threshold, rotating the display content in the three-dimensional display space of the head-mounted display device according to the rotation angle information, including: determining the preset angle threshold corresponding to the rotation direction among the preset angle thresholds; determining whether the rotation angle satisfies the rotation condition corresponding to the determined preset angle threshold; in response to determining that the rotation condition is satisfied, rotating the display content in the three-dimensional display space of the head-mounted display device according to the rotation direction and rotation angle included in the rotation angle information.

[0012] Optionally, the anti-shake threshold information also includes preset distance thresholds corresponding to each moving direction, and the moving distance information also includes the moving direction; and in response to determining that the moving distance satisfies the moving condition corresponding to the preset distance threshold, the display content in the three-dimensional display space of the head-mounted display device is moved according to the moving distance information, including: determining the preset distance threshold corresponding to the moving direction among the preset distance thresholds; determining whether the moving distance satisfies the moving condition corresponding to the determined preset distance threshold; in response to determining that the moving condition is satisfied, the display content in the three-dimensional display space of the head-mounted display device is moved according to the moving direction and moving distance included in the moving distance information.

[0013] Optionally, the above determination of the current scene type includes: determining a pre-set scene type as the current scene type; acquiring at least one scene image corresponding to the real scene; identifying the real scene type based on the at least one scene image; in response to determining that the real scene type belongs to a preset scene range, updating the scene type to the real scene type.

[0014] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising: a display module for imaging in front of the eyes of the wearer; one or more processors; and a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0015] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0016] In a fourth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner of the above-mentioned first aspect when executed by a processor.

[0017] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the display method in the motion mode of some embodiments of the present disclosure, the jitter of the displayed content in the head-mounted display device is reduced, the dizziness of the user when watching is reduced, and the user experience is improved. Specifically, the reason for the poor user experience is that when the user's head shakes slightly, the display window also moves with it, resulting in frequent jitter of the display window, and the user's dizziness is strong when watching, resulting in poor user experience. Based on this, the display method in the motion mode of some embodiments of the present disclosure, first, in response to detecting the head rotation operation of the wearing user, determines the rotation angle information corresponding to the head rotation operation, wherein the rotation angle information includes the rotation angle. Thus, when the user turns his head, the rotation angle related information can be recorded. Then, in response to determining that the rotation angle meets the rotation condition corresponding to the preset angle threshold, according to the rotation angle information, the display content in the three-dimensional display space of the head-mounted display device is rotated. Thus, the display content in the three-dimensional display space can be rotated only when the rotation angle meets a certain threshold condition. Finally, in response to determining that the rotation angle does not meet the rotation condition, the display content in the three-dimensional display space is kept in the original position. Therefore, when the rotation angle does not meet the threshold condition, the position of the picture in the three-dimensional display space can be controlled to remain unchanged. Also, because it is first determined whether the rotation angle meets the threshold condition before moving the display content in the three-dimensional display space, the display content can be rotated accordingly after it is determined that the rotation angle meets the threshold condition, and there is no need to follow the rotation of the display content at all rotation angles, thereby reducing the jitter of the display content in the head-mounted display device, reducing the dizziness of the user when watching, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0019] Figure 1 is a flow chart of some embodiments of the display method in the sports mode according to the present disclosure;

[0020] Figure 2 are flow charts of other embodiments of the display method in the sports mode according to the present disclosure;

[0021] Figure 3-4 is a schematic diagram of an application scenario of a display method in a sports mode according to some embodiments of the present disclosure;

[0022] Figure 5It is a schematic diagram of the structure of a head-mounted display device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0025] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0026] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0027] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0028] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0029] Figure 1 The process 100 of some embodiments of the display method in the sports mode according to the present disclosure is shown. The display method in the sports mode includes the following steps:

[0030] Step 101: in response to detecting a head rotation operation of a user wearing the device, determining rotation angle information corresponding to the head rotation operation.

[0031] In some embodiments, the execution subject (e.g., head-mounted display device) of the display method in the motion mode may determine the rotation angle information corresponding to the head rotation operation in response to detecting the head rotation operation of the wearing user. Among them, the head-mounted display device may be a display device for the wearing user to view virtual display content. For example, the head-mounted display device includes but is not limited to: AR glasses, MR glasses, VR glasses. The wearing user may be a user wearing the head-mounted display device. The head rotation operation may be an operation of rotating the head along the rotation direction. The rotation direction may include but is not limited to: left, right, up, and down. It should be noted that a gyroscope may be provided in the head-mounted display device. The gyroscope may be used to measure the rotation angle of the device. The rotation angle information may be information related to the head rotation angle of the wearing user. The rotation angle information may include but is not limited to the rotation angle. The rotation angle may be the angle at which the head of the wearing user rotates along the rotation direction. In practice, the execution subject may measure the rotation angle information of the head rotation operation by a gyroscope provided on the head-mounted display device.

[0032] Step 102: In response to determining that the rotation angle satisfies a rotation condition corresponding to a preset angle threshold, display content in a three-dimensional display space of the head mounted display device is rotated according to the rotation angle information.

[0033] In some embodiments, the execution subject may rotate the display content in the three-dimensional display space of the head-mounted display device according to the rotation angle information in response to determining that the rotation angle satisfies the rotation condition corresponding to the preset angle threshold. The preset angle threshold may be a preset angle threshold for determining whether the rotation angle satisfies the rotation condition. The rotation condition may be that the rotation angle is greater than the preset angle threshold. The rotation condition may also be that the rotation angle is greater than or equal to the preset angle threshold. The three-dimensional display space may be a three-dimensional space for display in the head-mounted display device. The display content may be the content displayed in the head-mounted display device. The display content may include but is not limited to at least one of the following: a display window, a picture, a video, and an icon. It is understood that when rotating or moving, the contents displayed in the head-mounted display device may rotate or move together as a whole. In practice, the execution subject may rotate the display content by an angle of the rotation angle in the three-dimensional display space with the observation point as the center and along the rotation direction. The observation point may be a corner point of the field of view. It can be understood that when the rotation condition is met, the user's head rotates, and the display content in the three-dimensional display space of the head-mounted display device rotates along with the rotation of the field of view angle.

[0034] Optionally, the rotation angle information may also include rotation acceleration information. The rotation acceleration information may include rotation acceleration. The rotation acceleration may be measured by a gyroscope.

[0035] In some optional implementations of some embodiments, the execution subject may rotate the display content in the three-dimensional display space according to the rotation angle information. The rotation acceleration of the display content corresponds to the rotation acceleration information. The angle between the center point of the display content after rotation and the center point of the display content before rotation corresponds to the rotation angle. In practice, the execution subject may rotate the display content in the three-dimensional display space along the rotation direction according to the rotation acceleration included in the rotation acceleration information. Thus, the display content may be rotated according to the acceleration when the user turns his head, thereby improving the user's head control experience.

[0036] Step 103: In response to determining that the rotation angle does not satisfy the rotation condition, the display content in the three-dimensional display space is maintained at the original position.

[0037] In some embodiments, the execution subject may maintain the display content in the three-dimensional display space at the original position in response to determining that the rotation angle does not satisfy the rotation condition. At this time, the viewing angle of the wearer rotates with the rotation of the head, and the display content that the wearer can see is part of the display content within the current viewing angle range.

[0038] Optionally, the execution subject may also determine the moving distance information corresponding to the moving operation in response to detecting the moving operation of the wearing user. The moving distance information includes the moving distance. The moving operation may be an operation in which the wearing user moves the head along the moving direction. The moving direction may include but is not limited to: forward, backward, left, right, upward, and downward. It should be noted that an accelerometer may be provided in the head-mounted display device. The accelerometer may be used to measure the acceleration and position of the device. The rotation angle information may be information related to the rotation angle of the head of the wearing user. The moving distance information may include but is not limited to the moving distance. The moving distance may be the distance that the head of the wearing user moves along the moving direction. In practice, the execution subject may measure the moving distance information of the moving operation by an accelerometer provided on the head-mounted display device. Thus, the moving distance information of the wearing user may be recorded.

[0039] Then, in response to determining that the moving distance satisfies the moving condition corresponding to the preset distance threshold, the display content in the three-dimensional display space of the head-mounted display device can be moved according to the moving distance information. The preset distance threshold may be a pre-set distance threshold for determining whether the moving distance satisfies the moving condition. The moving condition may be that the moving distance is greater than the preset distance threshold. The moving condition may also be that the moving distance is greater than or equal to the preset distance threshold. In practice, the execution subject may move the display content in the three-dimensional display space along the moving direction by a distance equal to the moving distance. It can be understood that when the moving condition is met, the user's head moves, and the display content in the three-dimensional display space of the head-mounted display device moves along with the movement of the field of view angle.

[0040] Secondly, in response to determining that the moving distance does not meet the moving condition, the display content in the three-dimensional display space can be kept in the original position. At this time, the field of view of the user wearing the headset moves with the movement of the head, and the display content that the user wearing the headset can see is part of the display content within the current field of view. As a result, there is no need to follow the moving display content under all moving events, thereby reducing the jitter of the display content in the head-mounted display device, reducing the user's dizziness when watching, and improving the user experience.

[0041] Optionally, the execution subject may further display anti-shake prompt information in the three-dimensional display space in response to determining that the rotation angle does not meet the rotation condition. The anti-shake prompt information may be information for prompting the user that the anti-shake function is taking effect. The anti-shake prompt information may include but is not limited to at least one of the following: text, icon, animated image. For example, the text included in the anti-shake prompt information may be "the anti-shake function is taking effect". Here, the specific setting of the anti-shake prompt information is not limited. The anti-shake prompt information may be displayed in the form of a pop-up window, or may be fixed at a preset position in the three-dimensional display space. For example, the preset position may be the upper left corner. Secondly, in response to detecting a closing operation corresponding to the anti-shake prompt information or the rotation angle meeting the rotation condition, the anti-shake prompt information may be hidden in the three-dimensional display space. The closing operation may be an operation for closing the displayed anti-shake prompt information. The closing operation may include but is not limited to at least one of the following interactive operations: touch screen operation, head control operation, voice control, gesture control. Thus, when the position of the displayed content remains unchanged, the user is prompted that the anti-shake function is taking effect, thereby improving the user's cognition of the anti-shake effect process.

[0042] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the display method in the motion mode of some embodiments of the present disclosure, the jitter of the displayed content in the head-mounted display device is reduced, the dizziness of the user when watching is reduced, and the user experience is improved. Specifically, the reason for the poor user experience is that when the user's head shakes slightly, the display window also moves with it, resulting in frequent jitter of the display window, and the user's dizziness is strong when watching, resulting in poor user experience. Based on this, the display method in the motion mode of some embodiments of the present disclosure, first, in response to detecting the head rotation operation of the wearing user, determines the rotation angle information corresponding to the head rotation operation, wherein the rotation angle information includes the rotation angle. Thus, when the user turns his head, the rotation angle related information can be recorded. Then, in response to determining that the rotation angle meets the rotation condition corresponding to the preset angle threshold, according to the rotation angle information, the display content in the three-dimensional display space of the head-mounted display device is rotated. Thus, the display content in the three-dimensional display space can be rotated only when the rotation angle meets a certain threshold condition. Finally, in response to determining that the rotation angle does not meet the rotation condition, the display content in the three-dimensional display space is kept in the original position. Therefore, when the rotation angle does not meet the threshold condition, the position of the picture in the three-dimensional display space can be controlled to remain unchanged. Also, because it is first determined whether the rotation angle meets the threshold condition before moving the display content in the three-dimensional display space, the display content can be rotated accordingly after it is determined that the rotation angle meets the threshold condition, and there is no need to follow the rotation of the display content at all rotation angles, thereby reducing the jitter of the display content in the head-mounted display device, reducing the dizziness of the user when watching, and improving the user experience.

[0043] Further references Figure 2 , which shows a process 200 of another embodiment of the display method in the motion mode. The process 200 of the display method in the motion mode includes the following steps:

[0044] Step 201, determine the current scene type.

[0045] In some embodiments, the execution subject (e.g., head mounted display device) on which the display method in the sports mode runs can determine the current scene type. The scene type can be a user-preconfigured or default scene type. For example, the scene type can include, but is not limited to: driving scene, riding scene, sports scene, reading scene.

[0046] In some optional implementations of some embodiments, the execution subject may determine the current scene type through the following steps:

[0047] The first step is to determine the preset scene type as the current scene type.

[0048] The second step is to obtain at least one scene image corresponding to the real scene. In practice, the execution subject may collect at least one scene image corresponding to the real scene through a camera device disposed on the front side of the head mounted display device.

[0049] The third step is to identify the type of real scene based on the at least one scene image. In practice, the execution entity may input the at least one scene image into a pre-trained real scene recognition model to obtain the real scene type. The real scene recognition model may be a neural network model that takes at least one scene image as input and the recognized real scene type as output. For example, the real scene recognition model may be a convolutional neural network model.

[0050] In the fourth step, in response to determining that the above-mentioned real scene type belongs to the preset scene range, the above-mentioned scene type is updated to the above-mentioned real scene type. Among them, the above-mentioned preset scene range may include various preset real scene types. Each real scene type in the above-mentioned preset scene range corresponds to preset anti-shake threshold information. Therefore, the preset anti-shake threshold information can be adaptively matched according to the type of real scene identified in real time.

[0051] Optionally, the execution subject may further identify the type of the real scene according to the at least one scene image through the following steps:

[0052] The first step is to obtain the scene audio of the above-mentioned real scene. In practice, the above-mentioned execution subject can collect the scene audio of the above-mentioned real scene through a microphone set on the head-mounted display device. The duration of the scene audio can be a preset duration.

[0053] The second step is to obtain the inertial measurement data of the head-mounted display device. In practice, the execution entity may provide an inertial measurement unit on the head-mounted display device to collect inertial measurement data. The inertial measurement data includes but is not limited to: acceleration, angular velocity. The start acquisition time of the scene audio, the inertial measurement data and the at least one scene image may be the same. For example, the sampling rate of the scene audio may be 16kHz, and the sampling rate of the inertial measurement data may be 100 samples / second. Each sample may include accelerations in 3 axes and angular velocities in 3 axes. The inertial measurement data may include a sample sequence. For example, the inertial measurement data may include 100 samples with a time sequence.

[0054] The third step is to input the at least one scene image into the visual feature extraction network included in the pre-trained real scene recognition model to obtain visual feature information. Among them, the real scene recognition model may include a visual feature extraction network, a sound feature extraction network, a motion feature extraction network, a fusion layer, a classification layer and an output layer. The visual feature extraction network may be a neural network for extracting image features. The visual feature extraction network may include a convolution layer and a global average pooling layer. For example, the convolution layer may use ResNet-50 as a basic feature extractor, which may be used to extract high-level features of the image. The global average pooling layer may be used to compress the feature map output by the convolution layer into a vector of a fixed size. The output of the global average pooling layer may be used as visual feature information. For example, the fixed size may be 2048 dimensions.

[0055] The fourth step is to input the scene audio into the sound feature extraction network included in the real scene recognition model to obtain sound feature information. The sound feature extraction network can be a neural network for extracting audio features. The sound feature extraction network can include a convolutional layer. For example, the convolutional layer included in the sound feature extraction network can use the VGGish model as a basic feature extractor to extract high-level features of the audio. The output of the convolutional layer included in the sound feature extraction network can be a vector of a preset dimension size, that is, the sound feature information. For example, the preset dimension size can be 128 dimensions.

[0056] In the fifth step, the inertial measurement data is input into the motion feature extraction network included in the real scene recognition model to obtain motion feature information. The motion feature extraction network may be a neural network for extracting motion features. The motion feature extraction network may be a long short-term memory network. The long short-term memory network may be used to extract time series data features from inertial measurement data. The output of the long short-term memory network may be a vector of a preset dimension size, i.e., motion feature information. For example, the preset dimension size may be 128 dimensions.

[0057] In a sixth step, the visual feature information, the sound feature information, and the motion feature information are input into a fusion layer included in the real scene recognition model to obtain fused feature information. The fusion layer may include a fully connected layer. The fully connected layer may be used for feature fusion. The output of the fully connected layer may be a vector of a preset dimension size. For example, the preset dimension size may be 1024 dimensions.

[0058] The seventh step is to input the above-mentioned fused feature information into the classification layer included in the above-mentioned real scene recognition model to obtain probability distribution information. Among them, the above-mentioned classification layer can be used to output the probability distribution of each real scene type. The above-mentioned classification layer may include a fully connected layer and a softmax layer. The fully connected layer included in the above-mentioned classification layer can be used to map the fused feature vector to the category space. The softmax layer can be used to convert the output into a probability distribution. The probability distribution information may include each probability distribution corresponding to each real scene type.

[0059] In the eighth step, the probability distribution information is input into the output layer of the real scene recognition model to obtain the real scene type. The output layer can be used to take the real scene type with the highest corresponding probability distribution as the final recognition result, that is, the output real scene type. In this way, the visual, sound and motion data can be combined to improve the robustness and recognition accuracy of the real scene recognition model.

[0060] Step 202: Determine anti-shake threshold information corresponding to the scene type according to the scene type.

[0061] In some embodiments, the execution subject may determine the anti-shake threshold information corresponding to the scene type according to the scene type. Each scene type may correspond to pre-configured anti-shake threshold information. The anti-shake threshold information may be various threshold information configured for anti-shake. The anti-shake threshold information may include but is not limited to a preset angle threshold. Thus, the corresponding threshold may be adapted according to the scene type to achieve different degrees of anti-shake effects.

[0062] As an example, refer to Figure 3 , the preset angle threshold in the riding scenario can be 6°. When the user turns his head upward at an angle of 5°, the open window or content remains in the original position and does not rotate with the field of view. Figure 4 , the preset angle threshold in the motion scene can be 11°. When the user turns his head upward at an angle of 10°, the open window or content remains in the original position and does not rotate with the field of view.

[0063] Step 203: in response to detecting a head rotation operation of the wearer, determining rotation angle information corresponding to the head rotation operation.

[0064] Step 204 , in response to determining that the rotation angle satisfies the rotation condition corresponding to the preset angle threshold, the display content in the three-dimensional display space of the head mounted display device is rotated according to the rotation angle information.

[0065] Optionally, the anti-shake threshold information may include preset angle thresholds corresponding to respective rotation directions. The rotation angle information may also include the rotation direction. Thus, a corresponding angle threshold may be configured in each rotation direction.

[0066] In some optional implementations of some embodiments, the execution subject may rotate the display content in the three-dimensional display space of the head mounted display device according to the rotation angle information in response to determining that the rotation angle satisfies the rotation condition corresponding to the preset angle threshold through the following steps:

[0067] The first step is to determine the preset angle threshold corresponding to the above rotation direction from among the above preset angle thresholds. Thus, the preset angle threshold corresponding to the current rotation direction can be determined from among the preset angle thresholds.

[0068] The second step is to determine whether the above rotation angle satisfies the rotation condition corresponding to the determined preset angle threshold.

[0069] In the third step, in response to determining that the above rotation condition is met, the display content in the three-dimensional display space of the above head mounted display device is rotated according to the rotation direction and rotation angle included in the above rotation angle information. Here, the implementation method of the rotation can refer to Figure 1 The corresponding step 102 in the embodiments will not be described in detail here.

[0070] Step 205: In response to determining that the rotation angle does not satisfy the rotation condition, the display content in the three-dimensional display space is maintained at the original position.

[0071] In some embodiments, the specific implementation of steps 203-205 and the technical effects brought about can be referred to Figure 1 The corresponding steps 101-103 in the embodiments are not described in detail here.

[0072] Optionally, the anti-shake threshold information may further include preset distance thresholds corresponding to respective moving directions. The moving distance information may also include moving directions. Thus, corresponding distance thresholds may be configured in respective moving directions.

[0073] In some optional implementations of some embodiments, the execution subject may, in response to determining that the moving distance satisfies the moving condition corresponding to the preset distance threshold, move the display content in the three-dimensional display space of the head mounted display device according to the moving distance information through the following steps:

[0074] The first step is to determine the preset distance threshold corresponding to the above moving direction from among the above preset distance thresholds. Thus, the preset distance threshold corresponding to the current moving direction can be determined from among the preset distance thresholds.

[0075] The second step is to determine whether the above moving distance meets the moving condition corresponding to the determined preset distance threshold.

[0076] In the third step, in response to determining that the above movement condition is met, the display content in the three-dimensional display space of the above head mounted display device is moved according to the movement direction and movement distance included in the above movement distance information. Here, the implementation method of the movement can refer to Figure 1 The corresponding optional embodiments will not be described in detail here.

[0077] from Figure 2 It can be seen that Figure 1 Compared with the description of some corresponding embodiments, Figure 2 The process 200 of the display method in the motion mode in some corresponding embodiments embodies the steps of expanding the applicability of the scene. Therefore, the solutions described in these embodiments can adapt the corresponding thresholds according to the scene type to achieve different degrees of anti-shake effects.

[0078] Reference below Figure 5 , which shows a schematic structural diagram of a head-mounted display device 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The head mounted display device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0079] like Figure 5 As shown, the head mounted display device 500 may include a processing device 501 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the head mounted display device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0080] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a micro display, a speaker, a vibrator, etc.; and a communication device 509. The communication device 509 may allow the head mounted display device 500 to communicate with other devices wirelessly or by wire to exchange data. The display module included in the head mounted display device 500 may include a micro display and an optical element. Although Figure 5The head mounted display device 500 having various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0081] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0082] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0083] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0084] The computer-readable medium may be included in the head-mounted display device; or it may exist independently without being assembled into the head-mounted display device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device: in response to detecting the head rotation operation of the wearing user, determines the rotation angle information corresponding to the head rotation operation, wherein the rotation angle information includes the rotation angle; in response to determining that the rotation angle satisfies the rotation condition corresponding to the preset angle threshold, rotates the display content in the three-dimensional display space of the head-mounted display device according to the rotation angle information; in response to determining that the rotation angle does not satisfy the rotation condition, keeps the display content in the three-dimensional display space at the original position.

[0085] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0086] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0087] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0088] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which implements the display method in any of the above-mentioned motion modes when executed by a processor.

[0089] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A display method in a sports mode, applied to a head-mounted display device, comprising: In response to detecting a head rotation operation of the wearing user, determining rotation angle information corresponding to the head rotation operation, wherein the rotation angle information includes a rotation angle; In response to determining that the rotation angle satisfies a rotation condition corresponding to a preset angle threshold, rotating display content in a three-dimensional display space of the head mounted display device according to the rotation angle information; In response to determining that the rotation angle does not satisfy the rotation condition, the display content in the three-dimensional display space is maintained at an original position.

2. The method according to claim 1, wherein: The method further comprises: In response to detecting a movement operation of the wearing user, determining movement distance information corresponding to the movement operation, wherein the movement distance information includes a movement distance; In response to determining that the moving distance satisfies a moving condition corresponding to a preset distance threshold, the display content in the three-dimensional display space of the head mounted display device is moved according to the moving distance information.

3. The method according to claim 2, wherein: The method further comprises: In response to determining that the movement distance does not satisfy the movement condition, the display content in the three-dimensional display space is maintained at an original position.

4. The method according to claim 3, wherein: Before determining the rotation angle information corresponding to the head rotation operation in response to detecting the head rotation operation of the wearing user, the method further includes: Determine the current scene type; Determine, according to the scene type, anti-shake threshold information corresponding to the scene type, wherein the anti-shake threshold information includes the preset angle threshold.

5. The method according to claim 4, wherein: The anti-shake threshold information includes each preset angle threshold corresponding to each rotation direction, and the rotation angle information also includes the rotation direction; and in response to determining that the rotation angle satisfies the rotation condition corresponding to the preset angle threshold, rotating the display content in the three-dimensional display space of the head mounted display device according to the rotation angle information, comprising: Determine a preset angle threshold corresponding to the rotation direction among the preset angle thresholds; Determining whether the rotation angle satisfies a rotation condition corresponding to a determined preset angle threshold; In response to determining that the rotation condition is satisfied, the display content in the three-dimensional display space of the head mounted display device is rotated according to the rotation direction and the rotation angle included in the rotation angle information.

6. The method according to claim 4, wherein: The anti-shake threshold information further includes respective preset distance thresholds corresponding to respective moving directions, the moving distance information further includes the moving direction; and in response to determining that the moving distance satisfies the moving condition corresponding to the preset distance threshold, moving the display content in the three-dimensional display space of the head mounted display device according to the moving distance information, comprising: Determine a preset distance threshold corresponding to the moving direction among the preset distance thresholds; Determining whether the moving distance satisfies a moving condition corresponding to a determined preset distance threshold; In response to determining that the movement condition is satisfied, the display content in the three-dimensional display space of the head mounted display device is moved according to the movement direction and the movement distance included in the movement distance information.

7. The method according to claim 4, wherein: Determining the current scene type includes: Determine a preset scene type as the current scene type; Acquire at least one scene image corresponding to a real scene; According to the at least one scene image, identifying a real scene type; In response to determining that the real scene type belongs to a preset scene range, the scene type is updated to the real scene type.

8. The method according to claim 7, wherein: The step of identifying the type of the real scene according to the at least one scene image comprises: Acquiring scene audio of the real scene; Acquiring inertial measurement data of the head mounted display device; Inputting the at least one scene image into a visual feature extraction network included in a pre-trained real scene recognition model to obtain visual feature information, wherein the real scene recognition model includes a visual feature extraction network, a sound feature extraction network, a motion feature extraction network, a fusion layer, a classification layer, and an output layer; Inputting the scene audio into the sound feature extraction network included in the real scene recognition model to obtain sound feature information; Inputting the inertial measurement data into a motion feature extraction network included in the real scene recognition model to obtain motion feature information; Inputting the visual feature information, the sound feature information and the motion feature information into a fusion layer included in the real scene recognition model to obtain fusion feature information; Inputting the fused feature information into the classification layer included in the real scene recognition model to obtain probability distribution information; The probability distribution information is input into the output layer of the real scene recognition model to obtain the real scene type.

9. The method according to claim 1, wherein: The rotation angle information also includes rotation acceleration information; and rotating the display content in the three-dimensional display space of the head mounted display device according to the rotation angle information includes: The display content is rotated in the three-dimensional display space according to the rotation angle information, wherein the rotation acceleration of the display content corresponds to the rotation acceleration information, and the angle between the center point of the display content after rotation and the center point of the display content before rotation corresponds to the rotation angle.

10. The method according to claim 1, wherein: The method further comprises: In response to determining that the rotation angle does not satisfy the rotation condition, displaying anti-shake prompt information in the three-dimensional display space; In response to detecting a closing operation corresponding to the anti-shake prompt information or the rotation angle satisfying the rotation condition, hiding the anti-shake prompt information in the three-dimensional display space.

11. A head mounted display device, comprising: A display module, used for forming images in front of the eyes of the wearer; one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.

12. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images