Display object control method of XR equipment and electronic equipment

Abstract

The invention discloses a display object control method of XR equipment and electronic equipment, and belongs to the field of augmented reality. The method can comprise the steps that target parameters are obtained, and the target parameters comprise position information of the head of a first user; determining whether the direction of the display object needs to be adjusted or not according to the target parameter and the position information of the display object; and when it is determined that the direction of the display object needs to be adjusted, rotating the front side of the display object to face the first user.

Description

Technical Field

[0001] The present application relates to the field of extended reality, and specifically to a display object control method and electronic device for an XR device. Background Art

[0002] When a user uses an extended reality (XR) device, if the user's position in the virtual space moves, the user's viewing angle of the displayed object will change, and the user may even see the back of the displayed object, which seriously affects the viewing experience. Figure 1 As shown, when user 2 is in position 1, display window 1 of the XR device is open and facing user 2, providing a good viewing experience. However, when user 2 moves from position 1 to position 2, they see the back of display window 1, resulting in a poor viewing experience. To maintain the viewing experience after the user's spatial displacement, the display object needs to be repositioned.

[0003] Currently, triggering repositioning requires manual activation via physical buttons or system menus, resulting in redundant operations and disrupting the viewing experience. Furthermore, the current display object repositioning algorithm moves the display object directly from its original position to directly in front of the user at the changed position. This can cause users to feel disoriented, mistaking their position for the same position. Furthermore, the repositioning can disrupt the viewing distance that users have painstakingly adjusted by moving their position, further degrading the viewing experience.

[0004] The above problems together lead to a degradation of user experience during the relocation process, which urgently needs to be improved. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a display object control method of an XR device and an electronic device to enhance the user's immersive experience.

[0006] In a first aspect, an embodiment of the present application provides a method for controlling a display object of an XR device, the method comprising: Acquiring target parameters, wherein the target parameters include position information of the first user's head; determining whether a direction of the display object needs to be adjusted according to the target parameter and the position information of the display object; If it is determined that the direction of the display object needs to be adjusted, the front side of the display object is rotated to face the first user.

[0007] In a second aspect, an embodiment of the present application provides a display object control device for an XR device, the device comprising: A parameter acquisition module, configured to acquire target parameters, where the target parameters include the position information of the head of a first user; A determination module, configured to determine whether to adjust the direction of the display object according to the target parameters and the position information of the display object; An adjustment module, configured to rotate the front of the display object to face the first user when it is determined that the direction of the display object needs to be adjusted.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In the embodiments of the present application, on the one hand, by acquiring target parameters and according to the target parameters and the position information of the display object, it can be automatically determined whether to adjust the direction of the display object, without the need for the user to manually trigger the repositioning of the display object, there is no redundant operation, and the continuous viewing experience of the first user will not be interrupted; on the other hand, when it is determined that the direction of the display object needs to be adjusted, the display object is rotated to face the head of the first user after the position is changed, rather than directly moving the display object to the front of the first user after the position is changed, so that the first user can view the display object at the adjusted distance and position, which better meets the viewing needs of the first user, and thus can significantly improve the user's immersive experience ultimately. Description of the Drawings

[0013] Figure 1 It is a schematic diagram for comparing the viewing experiences obtained before and after the user changes the position when the display object repositioning function is not enabled.

[0014] Figure 2 It is a schematic diagram of the model of the display object given in an embodiment of the present application.

[0015] Figure 3 It is a top - down schematic view showing the relationship between a display object and a user when the user's position remains unchanged in an embodiment of the present application.

[0016] Figure 4 It is a schematic flowchart of a method for controlling a display object of an XR device proposed in an embodiment of the present application.

[0017] Figure 5 It is a schematic diagram of the principle of an implementation manner of a method for controlling a display object of an XR device proposed in an embodiment of the present application.

[0018] Figure 6 It is a schematic diagram of the principle of another implementation manner of a method for controlling a display object of an XR device proposed in an embodiment of the present application.

[0019] Figure 7 It is a schematic diagram of the principle of yet another implementation manner of a method for controlling a display object of an XR device proposed in an embodiment of the present application.

[0020] Figure 8 It is a schematic diagram of the principle of yet another implementation manner of a method for controlling a display object of an XR device proposed in an embodiment of the present application.

[0021] Figure 9 It is a schematic structural diagram of a device for controlling a display object of an XR device proposed in an embodiment of the present application.

[0022] Figure 10 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

[0023] Figure 11 It is a schematic hardware structural diagram of an electronic device provided in an embodiment of the present application. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0025] The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / ", generally represents an "or" relationship between the associated objects before and after.

[0026] To enhance the user's immersive experience, embodiments of this application propose a method, device, and electronic device for controlling display objects of an XR device. A method for controlling display objects of an XR device proposed in this application can be applied to application scenarios such as extended reality (XR) and three-dimensional human-computer interaction. Among them, XR can include, but is not limited to, representative forms such as augmented reality (AR), mixed reality (MR), and virtual reality (VR), as well as the cross fields between them. Specifically, a method for controlling display objects of an XR device provided by embodiments of this application can be executed by an electronic device with the ability of human-computer interaction.

[0027] First, the basic concepts involved in the embodiments of this application will be described below.

[0028] The display object can include any virtual object displayed in the XR device. For example, the display object can include one or more of virtual objects such as a display window, a 2D display screen, a doll that can be viewed, and an ornament that can be viewed. Below, the display object will be mainly described by taking the display window as an example. It can be understood that the control of other display objects is the same as that of the display window.

[0029] Among them, the display window can be an invisible cube wrapping frame that carries virtual content. The size of the display window varies according to different virtual contents, but generally speaking, the display window will not exceed the comfortable visual range of the human eye due to the limitation of the comfortable visual range of the human eye.

[0030] As Figure 2 shown, the center point of display window 1 is the center point 11 of the front side of the invisible cube.

[0031] As Figure 3As shown, to ensure a friendly user experience, the newly opened display window 1 always appears in front of user 2 and faces user 2; if user 2 does not move at the same physical location, then the newly opened display window 1 or the relocated display window 1 after triggering relocation will be on a concentric circle.

[0032] The following will describe in detail a method for controlling a display object of an XR device proposed in this application with reference to the accompanying drawings.

[0033] As Figure 4 shown, a method for controlling a display object of an XR device proposed in an embodiment of this application may include: Step 401, obtain target parameters, where the target parameters include the position information of the head of a first user.

[0034] In some embodiments, the first user may be the user wearing the XR device. As an example, the XR device may be a head-mounted display device (abbreviation: head-mounted display).

[0035] In other embodiments, in a multi-user scenario, that is, when at least two users jointly view the display object through the XR devices worn by each of them, the first user is one of the at least two users.

[0036] In some embodiments, the at least two users include a host user and at least one guest user. Before step 401, Figure 4 the method shown may further include: in response to the host user's control user setting operation on the display object, setting the host user as the first user; or, in response to a guest user's control user setting operation on the display object, setting the guest user as the first user. That is to say, in a multi-user scenario including a host-guest mode, it can be determined according to the user's setting whether the display object follows the position adjustment of the host user or the guest user.

[0037] In some embodiments, the obtaining of the target parameters may include: obtaining the target parameters through the XR device worn by the first user. For example, the position information of the XR device may be determined in real time through the XR device worn by the first user and Simultaneous Localization and Mapping (SLAM), and the position information of the XR device may be used as the position information of the head of the first user. The SLAM technology is the core technology for realizing the autonomous perception and positioning of a device in an unknown environment, and its principle is: constructing an environmental map in real time through sensor data and simultaneously determining the position of the device in the map. It should be noted that using the SLAM technology for XR device positioning belongs to the prior art, so it will not be described in detail in this article.

[0038] In some embodiments, the position information of the head of the first user may be the coordinate information of the head of the first user.

[0039] In some embodiments, the obtaining of the target parameter may include: in response to the first user enabling the display object repositioning function, obtaining the target parameter. That is, when the XR device detects that the first user turns on the display object repositioning function, the target parameter will be obtained in real time. Specifically, when the XR device detects that the first user turns on the display object repositioning function, the target parameter can be obtained at a preset time interval.

[0040] Step 402, determine whether to adjust the direction of the display object according to the target parameter and the position information of the display object.

[0041] In some embodiments, the position information of the display object may include the coordinates of the center point of the display object. The coordinates of the center point of the display object are positioned with the coordinates of the head of the first user as the origin of the virtual world coordinate system. Among them, the center point of the display object may be the central symmetry point of the display object, or the center point of the front of the display object, so as to more accurately calibrate the movement trajectory of the user relative to the display object.

[0042] In some embodiments, if it is determined that the direction of the display object needs to be adjusted, then proceed to step 403.

[0043] In other embodiments, if it is determined that the direction of the display object does not need to be adjusted, then return to step 401, or end this process.

[0044] In some embodiments, the display object may include but is not limited to at least one of the following: Display window; 2D display screen; Ornamental doll; Ornamental ornament.

[0045] Step 403, when it is determined that the direction of the display object needs to be adjusted, rotate the front of the display object to face the first user.

[0046] Among them, the front of the display object generally refers to the functional side, visible side, or operable side of the display object, etc. Or rather, the front of the display object is the most core visual surface of the display object in terms of functionality, design intention, or interaction scenario, and is the visual surface that can most attract the user's attention. For example, in the case where the display object is a display window, the front of the display object is usually the visual surface for displaying image content and for the user to view; in the case where the display object is a 2D display screen, the front of the display object is the visual surface for displaying image content and for the user to view; in the case where the display object is an ornamental doll, the front of the display object usually refers to the facial orientation surface of the doll; in the case where the display object is an ornamental ornament, the front of the display object is usually the main visual surface of the artistic design.

[0047] In the above step 402, there can be many implementation manners for determining whether to adjust the direction of the display object according to the target parameter and the position information of the display object. Three of them are listed below.

[0048] In the first implementation manner, the target parameter includes the position information of the head of the first user. Step 402 may specifically include: According to the position information of the head of the first user and the position information of the display object, determine the angle between the first connection line and the normal line of the display object; in the case where the angle is greater than or equal to a preset threshold, determine that it is necessary to adjust the direction of the display object; on the contrary, in the case where the angle is less than or equal to the preset threshold, determine that it is not necessary to adjust the direction of the display object.

[0049] The first connection line is the connection line between the position where the first user is located and the position where the display object is located.

[0050] In the embodiments of the present application, optionally, the position where the first user is located is the center position of the head of the first user, and the position where the display object is located is the center point of the display object. That is, the first connection line is the connection line between the center of the head of the first user and the center point of the display object.

[0051] Among them, the preset threshold can be set flexibly. For example, the preset threshold can be 10°.

[0052] Correspondingly, the above step 403 may specifically include: in the case where it is determined that it is necessary to adjust the direction of the display object, rotate the display object by the angle towards the first user so that the front of the display object faces the first user.

[0053] Such as Figure 5As shown in the figure, from a top-down perspective, assume that when the first user 3 is at position 1 and opens the display window 1 of the XR device, the display window 1 is facing the first user 3 directly. The normal line 12 of the display window 1 overlaps with the first connection line 4 (the line connecting the center of the first user 3's head and the center point of the display window 1 when the first user 3 is at position 1). That is, when the first user 3 is at position 1, the angle between the first connection line 4 and the normal line 12 of the display window 1 is 0°.

[0054] Further, as Figure 5 shown, if the first user 3 moves from position 1 to position 2 and the display window 1 still faces position 1, at this time, the angle between the normal line 12 of the display window 1 and the first connection line 4 is θ1. Assume that θ1 is greater than or equal to the preset threshold. Then, it is necessary to rotate the display window 1 around its own center point by θ1 towards position 2 so that the display window 1 faces position 2. It is not difficult to see that at this time, the normal line of the display window 1 overlaps with the first connection line corresponding to position 2, enabling the first user to obtain a good viewing experience when at position 2.

[0055] Further, as Figure 5 shown, if the first user 3 moves from position 2 to position 3 and the display window 1 still faces position 2, at this time, the angle between the normal line of the display window 1 and the first connection line is θ2. Assume that θ2 is less than the preset threshold. Then, it is determined that there is no need to adjust the direction of the display window 1, and the display window 1 still faces position 2.

[0056] Further, as Figure 5 shown, if the first user 3 continues to move from position 3 to position n and the display window 1 still faces position 2, at this time, the angle between the normal line of the display window 1 and the first connection line 4 is θ3. Assume that θ3 is greater than or equal to the preset threshold. Then, it is necessary to rotate the display window 1 around its own center point by θ1 towards position n so that the display window 1 faces position n. It is not difficult to see that at this time, the normal line of the display window 1 overlaps with the first connection line corresponding to position n, enabling the first user to obtain a good viewing experience when at position n.

[0057] Through the above first implementation method, the position information of the first user obtained by the XR device can be used to automatically and dynamically adjust the direction of the display object, improving the viewing and interaction experience. This implementation method responds quickly and can rotate the display screen required by the first user to the desired angle in advance.

[0058] In the second implementation method, the target parameter includes not only the position information of the head of the first user but also the position information of the eye fixation point of the first user. Among them, the position information of the eye fixation point of the first user can also be detected by the XR device worn by the first user. The detection process belongs to the prior art and will not be described in detail in this article. At this time, step 402 can specifically include: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the center of the head of the first user and the center point of the display object; When the angle is greater than or equal to a preset threshold, further determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; conversely, when the angle is less than the preset threshold, determine that there is no need to adjust the direction of the display object; When the eye fixation point of the first user is located on the display object, determine that the direction of the display object needs to be adjusted; conversely, when the eye fixation point of the first user is not located on the display object, determine that there is no need to adjust the direction of the display object.

[0059] Correspondingly, step 403 above may specifically include: when it is determined that the direction of the display object needs to be adjusted, rotate the display object towards the first user by the angle so that the front of the display object faces the first user.

[0060] Optionally, when it is determined that the direction of the display object needs to be adjusted, rotate the display object towards the first user by the angle so that the front of the display object faces the head of the first user, for the convenience of the user to view the display object. As Figure 6 shown, in the top-down view, assume that the first user 3 opens the display window 1 of the XR device at position 1. At this time, the display window 1 is facing the first user 3 directly, and the normal line 12 of the display window 1 overlaps with the first connection line 4 (the connection line between the center of the head of the first user 3 at position 1 and the center point of the display window 1), that is, when the first user 3 is at position 1, the angle between the first connection line 4 and the normal line 12 of the display window 1 is 0°.

[0061] Furthermore, as Figure 6 shown, if the first user 3 moves from position 1 to position 2, the display window 1 still faces position 1. At this time, the angle between the normal line 12 of the display window 1 and the first connection line 4 is θ4. Assume that θ4 is greater than or equal to the preset threshold, and the eye fixation point of the first user 3 is located on the display window 1, then the display window 1 needs to be rotated by θ4 around its own center point towards position 2 so that the display window 1 faces position 2. It is not difficult to see that at this time, the normal line of the display window 1 overlaps with the first connection line corresponding to position 2, so that the first user can obtain a good viewing experience when at position 2.

[0062] Through the above second implementation manner, the position information of the first user and the position information of the eye fixation point obtained through the XR device can be used to automatically and dynamically adjust the direction of the display object, improving the viewing and interaction experience. This implementation manner has a relatively fast response and can also rotate the display screen required by the first user to the desired angle in advance.

[0063] In the third implementation manner, the target parameters include not only the position information of the head of the first user, but also the position information of the eye fixation point of the first user and the fixation time information. Among them, the position information of the eye fixation point of the first user and the fixation time information can also be detected by the XR device worn by the first user. The detection process belongs to the prior art and will not be described in detail in this article. At this time, step 402 may specifically include: According to the position information of the head of the first user and the position information of the display object, determine the angle between the first connection line and the normal line of the display object, where the first connection line is the connection line between the center of the head of the first user and the center point of the display object; When the angle is greater than or equal to the preset threshold, further determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; conversely, when the angle is less than the preset threshold, determine that there is no need to adjust the direction of the display object; When the eye fixation point of the first user is located on the display object, determine whether the fixation duration of the eye of the first user on the display object exceeds the preset duration according to the fixation time information; conversely, when the eye fixation point of the first user is not located on the display object, determine that there is no need to adjust the direction of the display object; When the fixation duration of the eye of the first user on the display object exceeds the preset duration, determine that it is necessary to adjust the direction of the display object; conversely, when the fixation duration of the eye of the first user on the display object does not exceed the preset duration, determine that there is no need to adjust the direction of the display object.

[0064] Correspondingly, step 403 above may specifically include: when it is determined that it is necessary to adjust the direction of the display object, rotate the display object towards the first user by the angle so that the front of the display object faces the first user.

[0065] Such as Figure 7As shown in the figure, in the top-down view, assume that when the first user 3 is at position 1 and opens the display window 1 of the XR device, at this time, the display window 1 is facing the first user 3, and the normal line 12 of the display window 1 overlaps with the first connection line 4 (the connection line between the center of the head of the first user 3 at position 1 and the center point of the display window 1), that is, when the first user 3 is at position 1, the angle between the first connection line 4 and the normal line 12 of the display window 1 is 0°.

[0066] Furthermore, as Figure 8 shown, if the first user 3 moves from position 1 to position 2 and the display window 1 still faces position 1, at this time, the angle between the normal line 12 of the display window 1 and the first connection line 4 is θ5. Assume that θ5 is greater than or equal to the preset threshold. The eye fixation point of the first user 3 at position 2 is located on the display window 1, and the fixation duration of the first user 3's eye on the display window 1 exceeds the preset duration. Then, it is necessary to rotate the display window 1 around its own center point by θ5 towards position 2 so that the display window 1 faces position 2. It is not difficult to see that at this time, the normal line of the display window 1 overlaps with the first connection line corresponding to position 2, and the display window 1 is exactly facing the face of the first user 3, which enables the first user to obtain a very good viewing experience when at position 2.

[0067] Through the above third implementation method, on the one hand, the position information of the first user, the position information of the eye fixation point, and the fixation time information obtained by the XR device can be used to automatically and dynamically adjust the direction of the display object. This can not only improve the viewing and interaction experience, but also clarify the intention of the first user and display the picture required by the user to the first user, which better meets the needs of the user; on the other hand, first judge whether the angle between the first connection line and the normal line of the display object is greater than or equal to the preset threshold. If not, directly determine that there is no need to adjust the direction of the display object; if so, further judge whether the eye fixation point of the first user is located on the display object. If the result is no, directly determine that there is no need to adjust the direction of the display object. If yes, further judge whether the fixation duration of the first user's eye on the display object exceeds the preset duration. This progressive judgment method can determine the situation where there is no need to adjust the direction of the display object as early as possible without having to make subsequent judgments, which can save computing power and reduce power consumption.

[0068] Figure 4A method for controlling a display object of an XR device proposed by the illustrated embodiment, on the one hand, can automatically determine whether to adjust the direction of the display object by obtaining target parameters and according to the target parameters and the position information of the display object, without the need for the user to manually trigger the repositioning of the display object, there is no redundant operation, and it will not interrupt the continuity of the first user's viewing experience; on the other hand, in the case of determining that the direction of the display object needs to be adjusted, the display object is rotated to face the head of the first user after the position change, rather than directly moving the display object to the front of the first user after the position change, so that the first user can view the display object at the adjusted distance and position, which better meets the viewing needs of the first user, and thus can significantly enhance the user's immersive experience in the end.

[0069] A method for controlling a display object of an XR device provided by an embodiment of the present application, the execution subject may be a display object control device of the XR device. In the embodiment of the present application, taking the display object control device of the XR device to execute the human-computer interaction method as an example, the human-computer interaction device provided by the embodiment of the present application is described.

[0070] As Figure 9 As shown, an embodiment of the present application proposes a display object control device for an XR device, and the device may include: a parameter acquisition module 901, a determination module 902, and an adjustment module 903.

[0071] The parameter acquisition module 901 is configured to acquire target parameters, where the target parameters include the position information of the head of the first user.

[0072] In some embodiments, the first user may be a user wearing the XR device. As an example, the XR device may be a head-mounted display device (abbreviated as a head-mounted display).

[0073] In other embodiments, in a multi-user scenario, that is, in the case where at least two users jointly view the display object through their respective XR devices worn, the first user is one of the at least two users.

[0074] In some embodiments, the at least two users include a host user and at least one guest user, Figure 9 The illustrated device may further include: a setting response module, configured to, before acquiring the target parameters, in response to a control user setting operation of the host user on the display object, set the host user as the first user; or, in response to a control user setting operation of a guest user on the display object, set the guest user as the first user. That is to say, in a multi-user scenario including a host-guest mode, it can be determined according to the user's setting whether the display object follows the position adjustment of the host user or the guest user.

[0075] In some embodiments, the parameter acquisition module 901 may specifically be configured to: obtain the target parameter through the XR device worn by the first user. For example, the position information of the XR device may be determined in real time through the XR device worn by the first user and Simultaneous Localization and Mapping (SLAM), and the position information of the XR device may be used as the position information of the head of the first user. The SLAM technology is the core technology for realizing the autonomous perception and positioning of a device in an unknown environment. Its principle is: constructing an environmental map in real time through sensor data and simultaneously determining the position of the device in the map. It should be noted that using the SLAM technology for XR device positioning belongs to the prior art, so it will not be described in detail in this article.

[0076] In some embodiments, the position information of the head of the first user may be the coordinate information of the head of the first user.

[0077] In some embodiments, the parameter acquisition module 901 may specifically be configured to: in response to the first user enabling the display object repositioning function, obtain the target parameter. That is to say, when the XR device detects that the first user has turned on the display object repositioning function, the target parameter will be obtained in real time. Specifically, when the XR device detects that the first user has turned on the display object repositioning function, the target parameter may be obtained at a preset time interval.

[0078] The determination module 902 is configured to determine whether it is necessary to adjust the direction of the display object according to the target parameter and the position information of the display object.

[0079] In some embodiments, if it is determined that the direction of the display object needs to be adjusted, the adjustment module 903 is triggered.

[0080] In some other embodiments, if it is determined that the direction of the display object does not need to be adjusted, the parameter acquisition module 901 is returned to be triggered.

[0081] In some embodiments, the display object may include but is not limited to at least one of the following: A display window; A 2D display screen; An ornamental doll; An ornamental ornament.

[0082] The adjustment module 903 is configured to, when it is determined that the direction of the display object needs to be adjusted, rotate the front of the display object to face the first user.

[0083] Among them, the front of the display object generally refers to the functional side, visible side, or operable side of the display object, etc. Or rather, the front of the display object is the most core visual surface of the display object in terms of functionality, design intention, or interaction scenario, and is the visual surface that can most attract the user's attention. For example, in the case where the display object is a display window, the front of the display object is usually the visual surface that displays image content for the user to view; in the case where the display object is a 2D display screen, the front of the display object is the visual surface that displays image content for the user to view; in the case where the display object is an ornamental doll, the front of the display object usually refers to the facial orientation surface of the doll; in the case where the display object is an ornamental ornament, the front of the display object is usually the main visual surface of the artistic design.

[0084] There can be many implementation manners for the above determination module 902 to determine whether to adjust the direction of the display object according to the target parameter and the position information of the display object. Three of them are listed below.

[0085] In the first implementation manner, the target parameter includes the position information of the head of the first user. Specifically, the determination module 902 can be used to: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object; in the case where the angle is greater than or equal to a preset threshold, determine that it is necessary to adjust the direction of the display object; conversely, in the case where the angle is less than or equal to the preset threshold, determine that it is not necessary to adjust the direction of the display object. Among them, the first connection line is the connection line between the center of the head of the first user and the center point of the display object.

[0086] Among them, the preset threshold can be set flexibly. For example, the preset threshold can be 10°.

[0087] Correspondingly, the above adjustment module 903 can be specifically used to: in the case where it is determined that it is necessary to adjust the direction of the display object, rotate the display object towards the first user by the angle so that the front of the display object faces the first user.

[0088] In the second implementation manner, the target parameter includes not only the position information of the head of the first user but also the position information of the eye fixation point of the first user. Among them, the position information of the eye fixation point of the first user can also be detected by the XR device worn by the first user. The detection process belongs to the prior art and will not be described in detail in this article. At this time, the determination module 902 can be specifically used to: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the center of the head of the first user and the center point of the display object; When the angle is greater than or equal to a preset threshold, further determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; conversely, when the angle is less than the preset threshold, determine that the direction of the display object does not need to be adjusted; When the eye fixation point of the first user is located on the display object, determine that the direction of the display object needs to be adjusted; conversely, when the eye fixation point of the first user is not located on the display object, determine that the direction of the display object does not need to be adjusted.

[0089] Correspondingly, the above adjustment module 903 can specifically be used to: when it is determined that the direction of the display object needs to be adjusted, rotate the display object towards the first user by the angle so that the front of the display object faces the first user.

[0090] In the third implementation manner, the target parameter further includes the position information and fixation time information of the eye fixation point of the first user, where determining whether the direction of the display object needs to be adjusted according to the target parameter and the position information of the display object includes: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; Determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; Determine whether the fixation duration of the eyes of the first user on the display object exceeds a preset duration according to the fixation time information; When the angle is greater than or equal to the preset threshold, the eye fixation point of the first user is located on the display object, and the fixation duration of the eyes of the first user on the display object exceeds the preset duration, determine that the direction of the display object needs to be adjusted.

[0091] The target parameter includes not only the position information of the head of the first user, but also the position information and fixation time information of the eye fixation point of the first user. The position information and fixation time information of the eye fixation point of the first user can also be detected by the XR device worn by the first user, and the detection process belongs to the prior art and will not be described in detail in this article.

[0092] Optionally, the determination module 902 can be specifically used for: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the center of the head of the first user and the center point of the display object; When the angle is greater than or equal to a preset threshold, further determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; conversely, when the angle is less than the preset threshold, determine that there is no need to adjust the direction of the display object; When the eye fixation point of the first user is located on the display object, determine whether the fixation duration of the eye of the first user on the display object exceeds a preset duration according to the fixation time information; conversely, when the eye fixation point of the first user is not located on the display object, determine that there is no need to adjust the direction of the display object; When the fixation duration of the eye of the first user on the display object exceeds the preset duration, determine that it is necessary to adjust the direction of the display object; conversely, when the fixation duration of the eye of the first user on the display object does not exceed the preset duration, determine that there is no need to adjust the direction of the display object.

[0093] Correspondingly, the above adjustment module 903 can be specifically used for: when it is determined that it is necessary to adjust the direction of the display object, rotate the display object by the angle towards the first user, so that the front of the display object faces the first user.

[0094] Figure 9 A display object control device for an XR device provided by the illustrated embodiment, on the one hand, can automatically determine whether it is necessary to adjust the direction of the display object by obtaining target parameters and according to the target parameters and the position information of the display object, without the user manually triggering the repositioning of the display object, there is no redundant operation, and it will not interrupt the continuity of the viewing experience of the first user; on the other hand, when it is determined that it is necessary to adjust the direction of the display object, rotate the display object to face the head of the first user after the position change, rather than directly moving the display object to the front of the first user after the position change, so that the first user can view the display object at the adjusted distance and position, which better meets the viewing needs of the first user, and thus can ultimately significantly improve the immersive experience of the user.

[0095] The display object control device of an XR device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0096] The display object control device of an XR device in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0097] The display object control device of an XR device provided in the embodiments of the present application can implement Figure 4 each process implemented by the method embodiments. To avoid repetition, it will not be elaborated here.

[0098] Optionally, as Figure 10 shown, the embodiments of the present application further provide an electronic device 1000, including a processor 1001 and a memory 1002. A program or instruction that can run on the processor 1001 is stored on the memory 1002. When the program or instruction is executed by the processor 1001, it implements each step of the above-mentioned method embodiment for controlling the display object of the XR device and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0099] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0100] Figure 11 It is a schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application. The electronic device 1100 includes, but is not limited to: a radio frequency unit 1108, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, a processor 1110, and other components.

[0101] Those skilled in the art can understand that the electronic device 1100 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1110 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 11 The structure of the electronic device shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here. The processor 1110 is configured to obtain a target parameter, where the target parameter includes the position information of the head of a first user; determine whether to adjust the direction of the display object according to the target parameter and the position information of the display object; and in the case of determining that the direction of the display object needs to be adjusted, rotate the front of the display object to face the first user.

[0102] In an embodiment of the present application, on the one hand, by obtaining the target parameter and according to the target parameter and the position information of the display object, it can be automatically determined whether to adjust the direction of the display object, without the need for the user to manually trigger the repositioning of the display object, there is no redundant operation, and the continuity of the viewing experience of the first user will not be interrupted. On the other hand, in the case of determining that the direction of the display object needs to be adjusted, the display object is rotated to face the head of the first user after the position change, rather than directly moving the display object to the front of the first user after the position change, so that the first user can view the display object at the adjusted distance and position, which better meets the viewing needs of the first user, and thus can significantly improve the immersive experience of the user ultimately.

[0103] It should be understood that in the embodiments of the present application, the input unit 1104 may include a Graphics Processing Unit (GPU) 11041 and a microphone 11042. The GPU 11041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of, for example, a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also referred to as a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. The other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated herein.

[0104] The memory 1109 can be used to store software programs and various data. The memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1109 can include volatile memory or non-volatile memory, or the memory 1109 can include both volatile and non-volatile memory. Among them, the non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 1109 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

[0105] The processor 1110 may include one or more processing units; optionally, the processor 1110 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1110.

[0106] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the embodiment of the display object control method for the above XR device and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0107] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc.

[0108] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above-mentioned embodiment of the display object control method for XR devices, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0109] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0110] The embodiments of the present application provide a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above-mentioned embodiment of the display object control method for XR devices, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0111] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0112] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0113] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. A method for controlling a display object of an XR device, characterized in that The method includes: Obtaining a target parameter, where the target parameter includes the position information of the head of a first user; Determining whether to adjust the direction of the display window according to the target parameter and the position information of the display object; When it is determined that the direction of the display object needs to be adjusted, rotating the front of the display object to face the first user.

2. The method according to claim 1, characterized in that The obtaining of the target parameter includes: In response to the first user enabling the display object repositioning function, obtaining the target parameter.

3. The method according to claim 1, wherein The determining whether to adjust the direction of the display object according to the target parameter and the position information of the display object includes: Determining the angle between a first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; When the angle is greater than or equal to a preset threshold, determining that the direction of the display object needs to be adjusted.

4. The method according to claim 1, wherein The target parameter further includes the position information of the eye fixation point of the first user. The determining whether to adjust the direction of the display object according to the target parameter and the position information of the display object includes: Determining the angle between a first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; When the angle is greater than or equal to the preset threshold, determining whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; When the eye fixation point of the first user is located on the display object, determining that the direction of the display object needs to be adjusted.

5. The method according to claim 1, characterized in that The target parameter further includes the position information of the eye fixation point of the first user and the fixation time information. The determining whether to adjust the direction of the display object according to the target parameter and the position information of the display object includes: Determining the angle between a first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; Determining whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; Determining whether the fixation duration of the eyes of the first user on the display object exceeds a preset duration according to the fixation time information; When the angle is greater than or equal to the preset threshold, the eye fixation point of the first user is located on the display object, and the fixation duration of the eyes of the first user on the display object exceeds the preset duration, determining that the direction of the display object needs to be adjusted.

6. The method according to any one of claims 3-5, characterized in that, The rotating the front of the display object to face the first user includes: Rotate the display object by the angle towards the first user so that the front of the display object faces the first user.

7. The method according to any one of claims 3-5, characterized in that The position where the first user is located is the center position of the first user's head, the position where the display object is located is the center point position of the display object, and the first connection line is the connection line between the center of the first user's head and the center point of the display object.

8. The method according to any one of claims 1-5, characterized in that, The display object includes at least one of the following: A display window; A 2D display screen; An ornamental doll; An ornamental ornament.

9. The method according to any one of claims 1-5, characterized in that, When at least two users jointly view the display object through XR devices worn by each of them, the first user is one of the at least two users; The at least two users include a host user and at least one guest user. Before obtaining the target parameter, the method further includes: In response to the host user's control user setting operation on the display object, set the host user as the first user; Or, In response to a guest user's control user setting operation on the display object, set the guest user as the first user.

10. A display object control device for an XR device, characterized in that, The device includes: A parameter acquisition module for acquiring a target parameter, where the target parameter includes the position information of the head of the first user; A determination module for determining whether to adjust the direction of the display object according to the target parameter and the position information of the display object; An adjustment module for rotating the front of the display object to face the first user when it is determined that the direction of the display object needs to be adjusted.

11. The device according to claim 10, characterized in that, The parameter acquisition module is specifically used for: In response to the first user enabling the display object repositioning function, acquire the target parameter.

12. The device according to claim 10, characterized in that, The determination module is specifically used for: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; When the angle is greater than or equal to a preset threshold, determine that the direction of the display object needs to be adjusted.

13. The device according to claim 10, characterized in that, The target parameter further includes the position information of the eye fixation point of the first user, where the determination module is specifically used for: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; When the angle is greater than or equal to a preset threshold, determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; When the eye fixation point of the first user is located on the display object, determine that the direction of the display object needs to be adjusted.

14. The device according to claim 10, characterized in that, The target parameter further includes the position information of the eye fixation point of the first user and the fixation time information, where the determination module is specifically used for: Determine the angle between the first connection line and the normal line of the display object according to the position information of the head of the first user and the position information of the display object, where the first connection line is the connection line between the position where the first user is located and the position where the display object is located; Determine whether the eye fixation point of the first user is located on the display object according to the position information of the eye fixation point of the first user; Determine whether the fixation duration of the eyes of the first user on the display object exceeds a preset duration according to the fixation time information; When the angle is greater than or equal to the preset threshold, the eye fixation point of the first user is located on the display object, and the fixation duration of the eyes of the first user on the display object exceeds the preset duration, determine that the direction of the display object needs to be adjusted.

15. The device according to any one of claims 12-14, characterized in that, The adjustment module is specifically configured to: Rotate the display object by the angle towards the head of the first user, so that the front of the display object faces the head of the first user.

16. The method according to any one of claims 12-14, characterized in that, The position where the first user is located is the center position of the head of the first user, the position where the display object is located is the center point position of the display object, and the first connection line is the connection line between the center of the head of the first user and the center point of the display object.

17. The device according to any one of claims 10-14, characterized in that The display object includes at least one of the following: Display window; 2D display screen; Ornamental doll; Ornamental display.

18. The device according to any one of claims 10 - 14, characterized in that, When at least two users jointly view the display object through the XR devices worn by them respectively, the first user is one of the at least two users; The at least two users include a host user and at least one guest user, and the device further includes: a user setting module; Wherein, the user setting module is used for: Before obtaining the target parameter, in response to the control user setting operation of the host user on the display object, set the host user as the first user; Or, Before obtaining the target parameter, in response to the control user setting operation of a guest user on the display object, set the guest user as the first user.

19. An electronic device, characterized in that, It includes a processor and a memory, and the memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method for controlling the display object of the XR device according to any one of claims 1-9 are implemented.

20. A readable storage medium, characterized in that, The program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, the steps of the method for controlling the display object of the XR device according to any one of claims 1-9 are implemented.

Images