Methods, apparatuses, devices, and products for display of a head-mounted display device

Abstract

Embodiments of the present disclosure relate to a method, apparatus, electronic device and product for display of a head-mounted display device. The method comprises obtaining information of a physical object in response to a first area of an interactor of the head-mounted display device intersecting a second area of the physical object in a physical environment. The method further comprises determining a see-through area for see-through display in the head-mounted display device based on the information of the physical object. In addition, the method further comprises see-through displaying at least a portion of the physical object in the see-through area of the head-mounted display device. According to embodiments of the present disclosure, a user is dynamically allowed to see through a partial area selectively, instead of setting a fixed safety area, so that the user can interact freely without being bound by the fixed safety area, which not only gives the user a better immersive experience, but also ensures higher safety.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of computers, and more specifically, to a method, apparatus, electronic device and product for display of a head-mounted display device. BACKGROUND

[0002] Extended Reality (XR) is a comprehensive integration of Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) technologies. In an XR environment, users can immerse themselves in a completely virtual world or interact with virtual elements in a real environment, experiencing the interaction between virtual and real. These virtual elements can be any imagined object, scene or character, through high-definition display devices, precise position tracking and realistic virtual elements, users feel as if they are in a completely new world.

[0003] XR devices are the key to realizing this virtual reality experience. These devices usually include head-mounted displays (HMDs), interaction controllers, and sensors. The head-mounted display is responsible for presenting images of the virtual world, the interaction controller is used for user interaction with virtual elements, and the sensor is used to track the user's position and actions to ensure real-time feedback from the virtual world. SUMMARY

[0004] Embodiments of the present disclosure provide a method, apparatus, electronic device and product for display of a head-mounted display device.

[0005] According to a first aspect of the present disclosure, a method for display of a head-mounted display device is provided. The method comprises obtaining information of a physical object in response to a first area of an interactor of the head-mounted display device intersecting a second area of the physical object in a physical environment. The method further comprises determining a see-through area for see-through display in the head-mounted display device based on the information of the physical object. In addition, the method further comprises see-through displaying at least a portion of the physical object in the see-through area of the head-mounted display device.

[0006] In a second aspect of the present disclosure, an apparatus for display of a head-mounted display device is provided. The apparatus comprises an information obtaining module configured to obtain information of a physical object in response to a first area of an interactor of the head-mounted display device intersecting a second area of the physical object in a physical environment. The apparatus further comprises a see-through area determining module configured to determine a see-through area for see-through display in the head-mounted display device based on the information of the physical object. In addition, the apparatus further comprises a display module configured to see-through display at least a portion of the physical object in the see-through area of the head-mounted display device.

[0007] In a third aspect of the disclosure, an electronic device is provided. The electronic device includes a processor and a memory coupled with the processor, the memory having stored therein instructions which, when executed by the processor, cause the electronic device to perform the method according to the first aspect.

[0008] In a fourth aspect of the disclosure, a computer program product having stored thereon instructions including computer executable instructions, wherein the computer executable instructions are executed by a processor to implement the method of the first aspect.

[0009] The summary is provided to introduce some aspects of the concepts in a simplified form that are further described below in the DETAILED DESCRIPTION. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, aspects and advantages of various embodiments of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. In the drawings:

[0011] Figure 1 A schematic diagram illustrating an example environment in which some embodiments of the present disclosure can be implemented is shown;

[0012] Figure 2 A flow diagram illustrating a method for display of a head-mounted display device of some embodiments of the present disclosure is shown;

[0013] Figure 3A A schematic diagram illustrating a display flow for a head-mounted display device of some embodiments of the present disclosure is shown;

[0014] Figure 3B A schematic diagram illustrating a collision volume of an interactor of some embodiments of the present disclosure is shown;

[0015] Figure 3C A schematic diagram illustrating a distance of an interactor from a grid point on a physical object of some embodiments of the present disclosure is shown;

[0016] Figure 3D A schematic diagram illustrating a normalized mapping curve of a distance of an interactor from a grid point on a physical object of some embodiments of the present disclosure is shown;

[0017] Figure 3E A schematic diagram illustrating a weight of a distance of an interactor from a grid point on a physical object of some embodiments of the present disclosure is shown;

[0018] Figure 3F A schematic diagram illustrating a collision center of an interactor of some embodiments of the present disclosure is shown;

[0019] Figure 3G A diagram showing the determination of the see-through region according to the collision center of the interactor of some embodiments of the present disclosure;

[0020] Figure 3H A diagram showing the see-through display of physical objects of some embodiments of the present disclosure;

[0021] Figure 4A A diagram showing another display flow for a head-mounted display device of some embodiments of the present disclosure;

[0022] Figure 4B A diagram showing the recording of the interactor center point coordinates upon collision of some embodiments of the present disclosure;

[0023] Figure 4C A diagram showing the recording of the interactor center point coordinates based on a time interval upon collision of some embodiments of the present disclosure;

[0024] Figure 4D A diagram showing the recording of the interactor center point coordinates based on the movement distance of the interactor upon collision of some embodiments of the present disclosure;

[0025] Figure 4E A diagram showing the recorded set of interactor center point coordinates of some embodiments of the present disclosure;

[0026] Figure 4F A diagram showing the see-through display of physical objects according to the trust level of some embodiments of the present disclosure;

[0027] Figure 5A A diagram showing yet another display flow for a head-mounted display device of some embodiments of the present disclosure;

[0028] Figure 5B A diagram showing the triggering of the see-through region based on the movement speed of the head interactor of some embodiments of the present disclosure;

[0029] Figure 5C A diagram showing the triggering of the see-through region based on the distance between the head interactor and the mesh point of the physical object reaching a threshold of some embodiments of the present disclosure;

[0030] Figure 5D A diagram showing the triggering of the see-through region based on the speed of the hand interactor reaching a threshold speed of some embodiments of the present disclosure;

[0031] Figure 6 A block diagram of an apparatus for display of a head-mounted display device of some embodiments of the present disclosure; and

[0032] Figure 7 A block diagram of an electronic device showing some embodiments of the present disclosure is shown.

[0033] In all the drawings, like or similar reference numerals refer to like or similar elements. DETAILED DESCRIPTION

[0034] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws, regulations and provisions.

[0035] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0036] In the description of embodiments of the present disclosure, the term "comprising" and similar terms are to be understood as open-ended, i.e., "including but not limited to". The term "based on" is to be understood as "based at least in part on". The term "one embodiment" or "the embodiment" is to be understood as "at least one embodiment". The terms "first", "second" and the like can refer to different or identical objects unless otherwise explicitly stated. Further explicit and implicit definitions can be included below.

[0037] In the MR experience, the related art sets a safety area for the safety of the experimenter, which aims to prevent the experimenter from being injured due to too intense action or accidental collision in the immersive experience. At the same time, it can also prevent other non-experimenters from entering the experience area by mistake, so as to avoid them from being injured due to the behavior of the experimenter. However, since the range of the safety area is usually fixed, it can limit the range of activities of the user, thereby affecting the immersive experience of the user.

[0038] According to embodiments of the present disclosure, based on the intersection of the interactor and the physical object in the real physical environment, a part of the area is selectively penetrated for the user, and a part of the real physical object is penetrated in the head-mounted display device. This method of not setting a fixed safety area breaks the limitation of having to set a fixed safety area in traditional interaction, so that the user can freely interact with the surrounding environment without the constraint of a fixed safety area. This dynamic interaction method not only improves the immersive experience of the user, but also ensures the safety during the interaction, bringing the user an unprecedented new interactive experience.

[0039] Figure 1A schematic diagram of an example environment 100 in which some embodiments of this disclosure may be implemented is shown. For example... Figure 1 As shown, 140 in example environment 100 is the virtual environment 140 originally presented by the head-mounted display device. In an XR experience, the user can view the virtual environment 140 through a specific head-mounted display device, such as XR glasses, and immerse themselves in a completely virtual environment 140 constructed by a computer system, or interact with virtual elements in the real environment, experiencing the interaction between virtual and reality.

[0040] Continue to refer to Figure 1 When the interactive element 110 of the head-mounted display device approaches a real object 130 in the real environment to a certain distance, the head-mounted display device will dynamically display an area of ​​a certain size on the user's screen 140, such as... Figure 1 The transparent area 120 is shown. Within this transparent area 120, a real object 130 that the user is about to encounter will be dynamically displayed. For example, when a sphere representing the interactor, with a radius R from the center point of the interactor 110, intersects with the real object 130, a camera configured on the head-mounted display device will acquire real-time mesh data of the real object 130 in the physical environment and dynamically determine the collision center of the intersecting real object 130. This collision center will then be used to determine the extent of the transparent area 120, which will be dynamically displayed on the user's display device. The size, shape, and effects of the transparent area 120 can be set by the user or automatically planned by the computer system. These interactors can be interactive elements such as controllers, hands, or heads.

[0041] The following will combine Figures 2 to 7 The process according to embodiments of this disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0042] Figure 2A flowchart of a method 200 for displaying a head-mounted display device according to some embodiments of the present disclosure is shown. At block 202, in response to an intersection between a first area of ​​the interactive element of the head-mounted display device and a second area of ​​a physical object in the physical environment, information about the physical object is acquired. The first area of ​​the interactive element of the head-mounted display device refers to an area centered on the center point of the interactive element; for example, it could be a spherical area with a radius of R centered on the center point of the interactive element 110. When this spherical area intersects with a physical object 130, information about the physical object 130 can be acquired using a camera on the head-mounted display device to obtain mesh data (polygonal grid data) of the physical object 130.

[0043] In box 204, the transparent area that will be displayed in the head-mounted display device is determined based on information about the physical object. Once the mesh data of the physical object 130 is acquired, the transparent area can be determined based on this data. For example, the center point of the transparent area 120 can be determined using the grid points of the physical object 130, and the size of the transparent area 120 can be determined using this center point.

[0044] In frame 206, at least a portion of a physical object is projected through the transparent area of ​​the head-mounted display device. Once the extent of the transparent area 120 is determined, at least a portion 130 of the physical object can be displayed in the virtual environment 140 displayed in the head-mounted display device.

[0045] In this embodiment, when the outer area of ​​the interactor intersects with a real object, information about that object is acquired. Then, based on this information, a transparent area can be determined, within which at least a portion of the real object is displayed transparently on the head-mounted display device. In this way, the user can see both virtual content and the real object integrated with the virtual content, thereby enhancing the visual experience without diminishing the user's immersive experience and ensuring the safety of the experience.

[0046] Figure 3A A schematic diagram of a display process 300A for a head-mounted display device according to some embodiments of the present disclosure is shown. (See reference...) Figure 3A In section 310A, collision events are detected. A collision event occurs when the interactor's collider intersects with a physical object in the physics environment. Then, through a series of calculations and transformations, the potential collision location and the relative distance between the interactor and the collision area are predicted. In section 322A, all vertices of the Mesh are selected within a spherical region of radius R centered on the interactor. (Reference) Figure 3BIn 300B, a collision volume 320B representing the interactor can be obtained, which can be a sphere with the interactor 310B as the center and a radius R. The radius can be arbitrarily set or can vary according to the speed of the interactor. Alternatively, the collision volume can also be other arbitrary shapes with the center of the interactor as the center.

[0047] In some embodiments, the interactor can be an interactive element such as a hand, a hand portion, a head, a ring, a bracelet, and the like. In some embodiments, the radius R of the collision volume can vary according to the speed of the interactor. In some embodiments, the radius of the interactor can also vary according to the danger level of the real object. Alternatively, the shape of the collision volume 320B can also be other shapes. All the vertices of the Mesh can be selected within the spherical region 320B, as shown in 330C, which are the selected vertices of the Mesh. Figure 3C

[0048] Continuing to refer to Figure 3A In 324A, the distance of each vertex of the Mesh to the center of the sphere is calculated, and the weight of each vertex is generated. In combination with Figure 3C In 300C, the collision volume is a sphere with the center of the interactor 320C as the center and a radius R. The collision volume intersects with the real object 310C, and the vertices 330C are all the selected vertices of the Mesh within the collision volume. Then, the distance D 340C of each selected vertex of the Mesh to the center of the interactor 320C can be calculated. Assuming that the Mesh has 5 vertices with coordinates V1(1, 2, 3), V2(4, 5, 6), V3(7, 8, 9), V4(10, 11, 12), and V5(13, 14, 15), and the center of the sphere has coordinates C(0, 0, 0), the distance of each vertex to the center of the sphere can be calculated as follows: D1 = 3.74, D2 = 8.66, D3 = 13.60, D4 = 18.52, and D5 = 23.44.

[0049] Referring to Figure 3A For example, when 16 vertices are selected (not shown in the figure), the distance D 340C of the 16 vertices of the Mesh to the center of the interactor 320C can be calculated as follows: [D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11, D12, D13, D14, D15, D16]. Then, the weight of each vertex can be calculated according to the 16 distances D. Referring to Figure 3D ​At 300D, the calculated distance values can be normalized, and then mapped through a sigmoid curve to generate the weight W for each vertex. In some embodiments, the normalized distances of the 16 points can be statistically analyzed using a normal distribution, and mapped using a sigmoid curve of the normal distribution. In some embodiments, points with normalized distance values close to 1 are selected for subsequent operations. For example, assume there are 20 points after normalization, and 10 of them are in the range of [0.8, 1], then these 10 points can be selected for subsequent operations. Each normalized distance value is mapped to the sigmoid curve to generate the corresponding weight W. The closer the distance value is to 1 (i.e., the closer to the sphere center), the higher the mapped weight W value, indicating a higher probability of user touching these vertices. Conversely, the closer the distance value is to 0, the lower the weight W value, indicating a lower probability of user touching. The weights of the 10 selected points are combined to determine the collision center 310F. Figure 3E At 300E, the points are selected again within the sphere with the center at the center point of the interactor and a radius of R to calculate the collision center. As shown in 310E, the weights of each selected vertex are W1, W2, W3, W4, W5, W6, W7, W8, W9, and W10.

[0050] Continuing to refer to Figure 3A At 326A, the collision center is determined. The collision center is the center point of the 10 selected Mesh vertices. In some embodiments, the total weight of the 10 selected Mesh vertices is first calculated, and the total weight can be obtained by adding the weights W of all vertices. Then the weighted coordinates are calculated, and for each vertex, the X, Y, and Z coordinates are multiplied by the corresponding weight W to obtain the weighted coordinates. Then the weighted coordinates are divided by the total weight to obtain the coordinates of the center point O. For example, the total weight can be calculated as W1+W2+W3+W4+W5+W6+W7+W8+W9+…+Wn. Then the weighted coordinates are calculated by multiplying the coordinates of each vertex by the respective weight to obtain the weighted coordinates. The weighted x coordinate = W1x1+W2x2+W3x3+…+Wnxn; the weighted y coordinate = W1y1+W2y2+W3y3+…+Wnyn; and the weighted z coordinate = W1z1+W2z2+W3z3+…+Wnzn. Then the center point O of the collision probability is determined by dividing the weighted coordinates by the total weight to obtain the coordinates of the center point O. The x coordinate of O = (weighted x coordinate) / total weight; the y coordinate of O = (weighted y coordinate) / total weight; and the z coordinate of O = (weighted z coordinate) / total weight. In this way, the coordinates of the collision center 310F in FIG. 3 can be determined. Figure 3F

[0051] As Figure 3F ​As shown, in some embodiments, the distance 320F can be determined using the determined collision center 310F and the center point of the interactor, as well as the radius R of the collision volume of the interactor. In this way, the size of the range of the pass-through region can be conveniently calculated.

[0052] With continued reference to Figure 3A At 328A, the pass-through region is calculated. In some embodiments, when the collision center O is determined, the distance between the collision center O and the center of the interactor can be calculated. As shown, Figure 3F As shown, at 300F, the distance L between the collision center 310F and the center of the interactor is 320F. In some embodiments, the range of the pass-through region can be determined based on the relationship between the distance L between the collision center and the center of the interactor and the radius R of the collision volume of the interactor. For example, the distance L can be divided by R to obtain a metric I (normalized) that approximates the range of the collision, and the value of I can be statistically normalized and mapped by a sigmoid curve as a numerical reference for optimizing the range of the pass-through portion. The curve mapping of the metric I is related to the visual effect of the output. The higher the probability of touch, the greater the intensity / range of the pass-through portion. In some embodiments, the higher the probability of being touched by the user, the greater the intensity and range of the pass-through portion. In some embodiments, a sphere can be drawn at the collision center, and the real-world picture can be passed through within the range of the sphere. As shown, Figure 3G As shown, at 300G, the pass-through region 320G is a spherical region with a radius of 330G and the collision center 310G as the center of the sphere.

[0053] With continued reference to Figure 3A At 330A, the visual output is outputted. When the pass-through region is determined, the real-world picture can be passed through within the pass-through region. As shown, Figure 3H As shown, at 300H, the real picture 320H is a spherical region with the collision center 310H as the center of the sphere and a radius of I. In some embodiments, the shape of the pass-through region is not fixed and can be other shapes with the collision center as the center point. In some embodiments, the blur radius, radius, and transparency of the pass-through region can be set. It can be understood that the visual effects are not unique.

[0054] Based on the collision event triggering the distance detection between the environmental space mesh vertex and the interactor, and using the mesh vertex position to determine the center of the pass-through region and the pass-through intensity everywhere, this method significantly improves the accuracy of the interactor collision warning function and ensures the safety and quality of experience of the user during the interaction.

[0055] Figure 4A Another schematic diagram of a display flow 400A for a head-mounted display device is shown, which illustrates some embodiments of the present disclosure. As shown, Figure 4A As shown, at 410A, the range of a specific region centered on the interactor is set. With reference toFigure 3B This particular region is a sphere 320B with the center of the flower pot 310B as the center of the sphere and R as the radius. At 420A, the coordinates of the center of the interactive object in space are recorded. In some embodiments, the coordinates of the center of the interactive object in space are recorded each time the particular region centered on the interactive object intersects the object. For example, at 422A, the coordinates of the center of the interactive object in space are recorded each time the particular region centered on the interactive object intersects the object. As shown at 400A, the interactive object moves toward the object 420A. Each time the particular region 410A centered on the interactive object intersects the object 420A, the coordinates of the center of the interactive object in space are recorded. For example, the coordinates of the center of the interactive object in space are recorded when the particular region 412A centered on the interactive object intersects the object 420A. As another example, the coordinates of the center of the interactive object in space are recorded when the particular region 414A centered on the interactive object intersects the object 420A. As yet another example, the coordinates of the center of the interactive object in space are recorded when the particular region 416A centered on the interactive object intersects the object 420A. Figure 4B As shown at 400B, the interactive object moves toward the object 420B. Each time the particular region 410B centered on the interactive object intersects the object 420B, the coordinates of the center of the interactive object in space are recorded. For example, the coordinates of the center of the interactive object in space are recorded when the particular region 412B centered on the interactive object intersects the object 420B. As another example, the coordinates of the center of the interactive object in space are recorded when the particular region 414B centered on the interactive object intersects the object 420B. As yet another example, the coordinates of the center of the interactive object in space are recorded when the particular region 416B centered on the interactive object intersects the object 420B.

[0056] As shown at 400B, the interactive object moves toward the object 420B. Each time the particular region 410B centered on the interactive object intersects the object 420B, the coordinates of the center of the interactive object in space are recorded. For example, the coordinates of the center of the interactive object in space are recorded when the particular region 412B centered on the interactive object intersects the object 420B. As another example, the coordinates of the center of the interactive object in space are recorded when the particular region 414B centered on the interactive object intersects the object 420B. As yet another example, the coordinates of the center of the interactive object in space are recorded when the particular region 416B centered on the interactive object intersects the object 420B. Figure 4C As shown at 400C, the coordinates of the center of the interactive object in space are recorded each time the particular region 410C centered on the interactive object intersects the object. In some embodiments, the coordinates of the center of the interactive object in space are recorded at intervals after the particular region centered on the interactive object intersects the object, regardless of whether the interactive object moves. For example, the coordinates of the center of the interactive object in space are recorded when the particular region 412C centered on the interactive object intersects the object after a time interval. As another example, the coordinates of the center of the interactive object in space are recorded when the particular region 414C centered on the interactive object intersects the object after a time interval. As yet another example, the coordinates of the center of the interactive object in space are recorded when the particular region 416C centered on the interactive object intersects the object after a time interval.

[0057] As shown at 400B, the interactive object moves toward the object 420B. Each time the particular region 410B centered on the interactive object intersects the object 420B, the coordinates of the center of the interactive object in space are recorded. For example, the coordinates of the center of the interactive object in space are recorded when the particular region 412B centered on the interactive object intersects the object 420B. As another example, the coordinates of the center of the interactive object in space are recorded when the particular region 414B centered on the interactive object intersects the object 420B. As yet another example, the coordinates of the center of the interactive object in space are recorded when the particular region 416B centered on the interactive object intersects the object 420B. Figure 4DAs shown, in 400D, when a collision occurs between the range 410D of a specific region centered on the interactor and an object, the coordinates of the center point of the interactor can be recorded. In some embodiments, when the center position of the interactor is removed from the region with radius R set in the previous frame after a collision (i.e., the distance moved is greater than radius R), the center position of the range 412D of the specific region centered on the interactor is recorded.

[0058] For example, in 428A, when a specific area centered on the interactor collides with an object, and the interactor's movement speed exceeds a certain limit, the center coordinates of the interactor can be recorded at regular intervals. For instance, when the interactor's movement speed exceeds 1 meter per second, the center coordinates of the interactor are recorded twice per second (2 times / second).

[0059] Continue to refer to Figure 4A In 430A, the probability distribution of the number of points and their trust levels within a certain radius of the interactive interface is calculated. In some embodiments, each time a specific area centered on the interactive interface collides with an object, the number of points within a certain radius of the interactive interface can be counted. For example, the number of points within a spherical region with radius R centered on the interactive interface can be counted. In some embodiments, the distance D from each point within these regions to the center of the interactive interface is calculated, and then D is divided by R to obtain the normalized distance. Next, the normalized distance is mapped to an S-shaped curve using a normal distribution and used as the weight W for each recorded center coordinate position. Vertices with values ​​close to 1 have a higher probability of trust.

[0060] like Figure 4E As shown, once all points have been recorded, a spherical region with radius R can be constructed, centered on the current location of the interactor. Within this spherical region, the trust level of the recorded points is calculated. In some embodiments, the distance between the recorded points within this spherical region and the current center point of the interactor can be calculated, and the weights of the recorded points within this region can be calculated using a normalization method. Then, based on these weights, the trust level of these points within this region is determined.

[0061] like Figure 4E As shown, points at 400E and 410E are considered to have high trust levels, point 420E is considered to have the second highest trust level, 430E is considered to have low trust levels, and 440E is considered to have the lowest trust level. It's understandable that the trust levels of these points are not constant. As time changes and users interact with the interactor, the coordinates of some new center points of the interactor can be recorded. Simultaneously, the trust levels of these points will be recalculated.

[0062] Continue to refer to Figure 4AAt 440A, based on the trust level of the settlement, it is determined whether the collision region is to be repeatedly warned. In some embodiments, when the trust level of the collision region is high, the user can not be warned and the real world can not be revealed to the user. When the trust level of the collision region is low, the user is warned and the real world is revealed to the user. With reference to Figure 4A At 450A, the higher the trust level, the less obvious the warning effect is. The lower the trust level, the more obvious the warning effect is, such as transparency, blur, and radius.

[0063] Referring to Figure 4F At 400F, as shown in 410F, when the trust level is the highest, the real world is not revealed. As shown in 420F, when the trust level is the second highest, the real world is slightly revealed. As shown in 430F, when the trust level is low, the real world is slightly clearly revealed. As shown in 440F, when the trust level is the lowest, the real world is more clearly revealed. In some embodiments, the size of the revealed region and the effect of the revelation change with the change of the trust level. The lower the trust level, the more obvious the size and effect of the revealed region are, and vice versa.

[0064] By dynamically counting the coordinates of the user's actions in the region that has been marked as a dangerous region, the region where the user has a high interaction demand can be dynamically determined, the region with the interaction demand is marked as a high trust level, and the warning is not performed, thereby improving the immersive experience of the user.

[0065] Figure 5A Another schematic diagram of a display flow 500A for a head-mounted display device is shown, which illustrates some embodiments of the present disclosure. Referring to Figure 5A At 510A, based on different collision behavior rules of the interactors, the region is controlled to be revealed and the user is warned. In some embodiments, the collision behavior rules of the interactors are divided into two types: hand and head. The hand type includes the hand itself and various interactors related to the hand, such as a handle, a bracelet, a medium, and a wearable device.

[0066] With reference to Figure 5A At 522A, when the movement speed of the head interactor reaches a certain threshold, a revelation prompt is given. For example, when the movement speed of the head reaches 1 m / s, the current real world can be revealed in the field of view of the user. Figure 5B As shown in 500B, when the head interactor 520B reaches a certain movement speed, the current real world is revealed in the field of view 510B of the head interactor 520B. For example, when the movement speed of the head reaches 1 m / s, the current real world can be revealed in the field of view of the user.

[0067] In some embodiments, the size and location of the break-through region can be determined based on the size of the head-interactor's collision volume, the distance between the head-interactor's center point and the collision center, and the speed of the head-interactor. For example, the greater the speed of the head-interactor, the larger the head-interactor's collision volume can be, and the more vertices of the intersection region with the real world can be affected in determining the collision center, which can affect the distance between the collision center and the head-interactor's center point, and ultimately the size of the break-through region.

[0068] With continued reference to Figure 5A At 524A, a break-through prompt is triggered when the head collision volume collides with the real world or when a particular orientation, such as the face, reaches a predetermined threshold distance from the real world. In some embodiments, a break-through prompt can be issued to the user when the outer boundary of the head collision volume is less than 0.1 meters or less than 0.3 meters from the real world. As shown in 500B, when the head-interactor's collision volume 510B reaches the threshold distance 520B, a break-through prompt is issued to the user. Figure 5C As shown in 500C, when the head-interactor's collision volume 510C reaches the threshold distance 520C, a break-through prompt is issued to the user. In some embodiments, a break-through prompt can be issued when the head collision volume intersects with the real world during motion. The break-through region can be determined based on the location of the intersection between the collision volume and the real world and the speed of the head-interactor. The faster the head-interactor, the larger the break-through region can be. The larger the collision volume, the larger the break-through region can be. In some embodiments, if the system identifies a sharp object or a physical attribute with a high risk in the real world, the break-through region can be larger and the break-through can be more intense, thereby increasing the user's alertness. Conversely, when the risk level is low, the break-through region can be set to be smaller.

[0069] With continued reference to Figure 5A At 532A, the required buffer distance for the hand-interactor at its current real-time speed is calculated. As shown in 500E, the required buffer distance 526E for the hand-interactor 520E to avoid colliding with the obstacle 510E is calculated based on the speed of the hand-interactor 520E. Figure 5D As shown in 500D, the required buffer distance 526D for the hand-interactor 520D to avoid colliding with the obstacle 510D is calculated based on the speed of the hand-interactor 520D. In some embodiments, the required buffer distance 526D can be calculated in advance when the speed of the hand-interactor 520D is known.

[0070] With continued reference to Figure 5AIn step 534A, the offset of the interactive collider is controlled. In some embodiments, the offset 524D of the interactive collider 522D based on the center of the interactive collider can be calculated based on the buffer distance 526D. Then, the position and size of the buffer collider are adjusted accordingly. In some embodiments, the greater the movement speed of the hand interactive collider, the greater the buffer distance and the greater the offset of the buffer collider. By adjusting the size of the buffer collider according to the movement speed of the hand interactive collider, it is possible to warn or display dangerous areas in advance, thus giving the user sufficient time to avoid or change direction.

[0071] In some embodiments, the visible area can be implemented in various ways. For example, adjusting the blur level, radius, transparency, softening the edges, or adding warning icons. This effectively alerts users to potential dangers without unduly interfering with their experience. Furthermore, this method can adjust the size and position of the visible area in real time as the user's actions and interaction states change, adapting to constantly changing scenarios and environments.

[0072] Figure 6 A block diagram of a display apparatus 600 for a head-mounted display device according to some embodiments of the present disclosure is shown. Figure 6 As shown, device 600 includes an information acquisition module 602 configured to acquire information about a physical object in response to an intersection between a first area of ​​the interactive element of the head-mounted display device and a second area of ​​a physical object in the physical environment. Device 600 also includes a transparency area determination module 604 configured to determine a transparency area to be displayed in the head-mounted display device based on the information about the physical object. Furthermore, device 600 includes a display module 606 configured to display at least a portion of the physical object within the transparency area of ​​the head-mounted display device.

[0073] Figure 7 Block diagrams of electronic devices 700 according to some embodiments of the present disclosure are shown. Device 700 may be the device or apparatus described in the embodiments of the present disclosure. Figure 7 As shown, device 700 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 702 or loaded from storage unit 708 into random access memory (RAM) 703. Various programs and data required for the operation of device 700 can also be stored in RAM 703. The CPU / GPU 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704. Although not shown in... Figure 7As shown in FIG. 7, the device 700 can also include a coprocessor.

[0074] A number of the components in the device 700 are connected to the I / O interface 705, including: an input unit 706, e.g., a keyboard, a mouse, etc.; an output unit 707, e.g., various types of displays, speakers, etc.; a storage unit 708, e.g., a magnetic disk, a magneto-optical disk, etc.; and a communication unit 709, e.g., a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices over a computer network, such as the Internet, and / or various telecommunication networks.

[0075] The various methods or processes described above can be performed by the CPU / GPU 701. For example, in some embodiments, the methods can be implemented as a computer software program tangibly embodied in a machine readable medium, e.g., the storage unit 708. In some embodiments, portions or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the CPU / GPU 701, one or more steps or actions of the methods or processes described above can be performed.

[0076] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith.

[0077] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a

[0078] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0079] Computer readable program instructions for carrying out operations of the present disclosure can be assembly-level instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages and conventional procedural programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0080] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including a manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0081] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0082] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of devices, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0083] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative of the embodiments and not restrictive. Many modifications and variations of the described embodiments are possible and are within the scope of the described embodiments. The selection of the terms to be used in the description is not intended to limit the scope of the embodiments described herein. Rather, the terms are chosen to best explain the principles of the embodiments, the practical application, or technical improvement over the technology found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0084] Some example implementations of the present disclosure are listed below.

[0085] Example 1. A method for display of a head-mounted display device, comprising:

[0086] in response to a first area of an interactor of the head-mounted display device intersecting a second area of a physical object in a physical environment, obtaining information of the physical object;

[0087] based on the information of the physical object, determining a see-through area for see-through display in the head-mounted display device; and

[0088] see-through displaying at least a portion of the physical object in the see-through area of the head-mounted display device.

[0089] Example 2. The method of example 1, wherein in response to a first region of an interactor of the head-mounted display device intersecting a second region of a physical object in a physical environment, obtaining information of the physical object comprises:

[0090] determining a plurality of grid points based on the first region of the interactor; and

[0091] determining a collision center of the physical object based on the plurality of grid points.

[0092] Example 3. The method of any one of examples 1-2, wherein determining a collision center of the physical object based on the plurality of grid points comprises:

[0093] determining a plurality of distances between the plurality of grid points and a center point of the interactor;

[0094] determining a plurality of weights of the plurality of grid points based on the plurality of distances; and

[0095] determining the collision center based on the plurality of weights of the plurality of grid points.

[0096] Example 4. The method of any one of examples 1-3, wherein determining a plurality of weights of the plurality of grid points based on the plurality of distances comprises:

[0097] determining a plurality of normalized values of the plurality of distances based on the plurality of distances; and

[0098] determining the plurality of weights of the plurality of grid points based on the plurality of normalized values.

[0099] Example 5. The method of any one of examples 1-4, wherein determining the collision center based on the plurality of weights of the plurality of grid points comprises:

[0100] determining a plurality of calculation points from the plurality of grid points based on the plurality of weights of the plurality of grid points; and

[0101] determining the collision center based on a plurality of weights of the plurality of calculation points.

[0102] Example 6. The method of any one of examples 1-5, wherein determining a pass-through region for pass-through display in the head-mounted display device based on the information of the physical object comprises:

[0103] determining the pass-through region based on a distance between the center point of the interactor and the collision center and a length of the center point of the interactor to an outer boundary of the first region of the interactor.

[0104] Example 7. The method of any one of examples 1-6, wherein in the see-through region of the head-mounted display device, see-through displaying at least a portion of the physical object comprises:

[0105] see-through displaying at least a portion of the physical object in a feathering manner based on the see-through region.

[0106] Example 8. The method of any one of examples 1-7, wherein determining the see-through region for see-through displaying in the head-mounted display device based on information of the physical object comprises:

[0107] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, determining a set of center point coordinates of the interactor; and

[0108] based on the set of center point coordinates, determining a plurality of trust degrees of a plurality of the center points.

[0109] Example 9. The method of any one of examples 1-8, wherein determining the set of center point coordinates of the interactor comprises at least one of:

[0110] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, recording a center point coordinate of the interactor;

[0111] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, recording a center point coordinate of the interactor based on a certain time interval;

[0112] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, and in response to a distance between a center point of the interactor at a current time and a center point of the interactor at a previous time being greater than a length from the center point of the interactor at the current time to an outer boundary of the first region, recording a center point coordinate of the interactor at the current time; or

[0113] in response to a moving speed of the interactor reaching a first speed threshold, recording a center point coordinate of the interactor based on a certain time interval.

[0114] Example 10. The method of any one of examples 1-9, wherein determining the plurality of trust degrees of the plurality of the center points based on the set of center point coordinates comprises:

[0115] in response to the first region of the interactor of the head-mounted display device intersecting the second region of the physical object in the physical environment, determining a subset of the set of center point coordinates of the interactor based on a length of a center point of the interactor to an outer boundary of the first region and the center point of the interactor;

[0116] based on the subset, determining distances of a plurality of points within the subset to the center point of the interactor;

[0117] based on the distances, determining a plurality of weights of the plurality of points within the subset; and

[0118] based on the plurality of weights of the plurality of points, determining a trustworthiness of each point of the plurality of points within the subset.

[0119] Example 11. The method of any one of examples 1-10, wherein determining, based on the information of the physical object, a see-through region for see-through display in the head-mounted display device comprises:

[0120] in response to the trustworthiness being in a first range, see-through displaying at least a portion of the physical object; and

[0121] in response to the trustworthiness being in a second range, ceasing see-through display of the physical object.

[0122] Example 12. The method of any one of examples 1-11, wherein determining, based on the information of the physical object, a see-through region for see-through display in the head-mounted display device comprises:

[0123] determining whether the interactor is located at a hand or a head;

[0124] in response to the interactor being located at the hand, determining a see-through region for the hand; and

[0125] in response to the interactor being located at the head, determining a see-through region for the head.

[0126] Example 13. The method of any one of examples 1-12, wherein determining, in response to the interactor being worn on a hand, a see-through region for the hand comprises:

[0127] in response to the interactor being located at a hand, determining a buffer distance based on a speed of motion of the interactor;

[0128] based on the buffer distance, determining an offset amount of the first region of the interactor;

[0129] based on the offset amount, determining a size of the first region of the interactor; and

[0130] determine the see-through region for the head based on a size of the first region, a distance between a center point of the interactor and the collision center, and a length from the center point of the interactor to an outer boundary of the first region of the interactor.

[0131] Example 14. The method of any one of examples 1-13, wherein in response to the interactor being worn on a head, determining a see-through region for the head comprises:

[0132] in response to the interactor being worn on a head, and in response to a speed of the interactor satisfying a second speed threshold, determining the see-through region for the head within a field of view range of the interactor.

[0133] Example 15. The method of any one of examples 1-14, further comprising:

[0134] in response to the interactor being worn on a head, and in response to a distance between an outer boundary of the first region of the interactor and the physical environment satisfying a distance threshold, determining the see-through region for the head within a field of view range of the interactor.

[0135] Example 16. The method of any one of examples 1-15, wherein determining the see-through region for the head comprises:

[0136] determining the see-through region for the head based on a location at which the first region of the interactor intersects the second region of the physical object in the physical environment and a speed of the interactor.

[0137] Example 17. An apparatus for display of a head-mounted display device, comprising:

[0138] an information obtaining module configured to, in response to a first region of an interactor of the head-mounted display device intersecting a second region of a physical object in a physical environment, obtain information of the physical object;

[0139] a see-through region determining module configured to determine, based on the information of the physical object, a see-through region for see-through display in the head-mounted display device; and

[0140] a display module configured to see-through display at least a portion of the physical object in the see-through region of the head-mounted display device.

[0141] Example 18. The apparatus of example 17, wherein the information obtaining module comprises:

[0142] a grid point determining module configured to determine, based on the first region of the interactor, a plurality of grid points; and

[0143] a collision center determination module configured to determine a collision center of the physical object based on the plurality of grid points.

[0144] Example 19. The apparatus of any one of examples 17-18, wherein the collision center determination module comprises:

[0145] a distance determination module configured to determine a plurality of distances between the plurality of grid points and a center point of the interactor;

[0146] a first weight determination module configured to determine a plurality of weights of the plurality of grid points based on the plurality of distances; and

[0147] a first collision center determination module configured to determine the collision center based on the plurality of weights of the plurality of grid points.

[0148] Example 20. The apparatus of any one of examples 17-19, wherein the weight determination module comprises:

[0149] a normalization value determination module configured to determine a plurality of normalization values of the plurality of distances based on the plurality of distances; and

[0150] a second weight determination module configured to determine the plurality of weights of the plurality of grid points based on the plurality of normalization values.

[0151] Example 21. The apparatus of any one of examples 17-20, wherein the first collision center determination module comprises:

[0152] a calculation point determination module configured to determine a plurality of calculation points from the plurality of grid points based on the plurality of weights of the plurality of grid points; and

[0153] a second collision center determination module configured to determine the collision center based on a plurality of weights of the plurality of calculation points.

[0154] Example 22. The apparatus of any one of examples 17-21, wherein the reveal region determination module comprises:

[0155] a first reveal region determination module configured to determine the reveal region based on a distance between the center point of the interactor and the collision center and a length of the center point of the interactor to an outer boundary of a first region of the interactor.

[0156] Example 23. The apparatus of any one of examples 17-22, wherein the display module comprises:

[0157] a first display module configured to display at least a portion of the physical object in a feathered manner based on the reveal region.

[0158] Example 24. The apparatus of any one of examples 17-23, wherein the breakthrough region determination comprises:

[0159] a center point coordinate set determination module configured to determine a center point coordinate set of the interactor of the head-mounted display device in response to the first region of the interactor intersecting the second region of the physical object in the physical environment; and

[0160] a trustworthiness determination module configured to determine a plurality of trustworthiness of a plurality of the center points based on the center point coordinate set.

[0161] Example 25. The apparatus of any one of examples 17-24, wherein the center point coordinate set determination module comprises at least one of:

[0162] a first center point coordinate recording module configured to record a center point coordinate of the interactor of the head-mounted display device in response to the first region of the interactor intersecting the second region of the physical object in the physical environment;

[0163] a second center point coordinate recording module configured to record a center point coordinate of the interactor of the head-mounted display device based on a certain time interval in response to the first region of the interactor intersecting the second region of the physical object in the physical environment;

[0164] a third center point coordinate recording module configured to record a center point coordinate of the interactor of the head-mounted display device at a current time in response to the first region of the interactor intersecting the second region of the physical object in the physical environment and in response to a distance between the center point of the interactor at the current time and the center point of the interactor at a previous time being greater than a length of the center point of the interactor to an outer boundary of the first region at the current time; or

[0165] a fourth center point coordinate recording module configured to record a center point coordinate of the interactor of the head-mounted display device based on a certain time interval in response to a moving speed of the interactor reaching a first speed threshold.

[0166] Example 26. The apparatus of any one of examples 17-25, wherein the trustworthiness determination module comprises:

[0167] a subset determination module configured to determine a subset of the center point coordinate set of the interactor of the head-mounted display device based on a length of the center point of the interactor to an outer boundary of the first region and the center point of the interactor in response to the first region of the interactor intersecting the second region of the physical object in the physical environment.

[0168] a first distance determination module configured to determine distances of the plurality of points within the subset from a center point of the interactor based on the subset;

[0169] a third weight determination module configured to determine a plurality of weights of the plurality of points within the subset based on the distances; and

[0170] a first trustworthiness determination module configured to determine a trustworthiness of each point of the plurality of points within the subset based on the plurality of weights of the plurality of points.

[0171] Example 27. The apparatus of any one of examples 17-26, wherein the see-through region display module comprises:

[0172] a second display module configured to display at least a portion of the physical object see-through in response to the trustworthiness being in a first range; and

[0173] a display stop module configured to stop displaying the physical object see-through in response to the trustworthiness being in a second range.

[0174] Example 28. The apparatus of any one of examples 17-27, wherein the see-through region determination module comprises:

[0175] a determination module configured to determine whether the interactor is located at a hand or a head;

[0176] a second see-through region determination module configured to determine a see-through region for the hand in response to the interactor being located at the hand; and

[0177] a third see-through region determination module configured to determine a see-through region for the head in response to the interactor being located at the head.

[0178] Example 29. The apparatus of any one of examples 17-28, wherein the second see-through region determination module comprises:

[0179] a buffer distance determination module configured to determine a buffer distance based on a motion speed of the interactor in response to the interactor being located at the hand;

[0180] an offset amount determination module configured to determine an offset amount of the first region of the interactor based on the buffer distance;

[0181] a size determination module configured to determine a size of the first region of the interactor based on the offset amount; and

[0182] a fourth see-through area determination module configured to determine the see-through area for the hand based on a size of the first area, a distance between a center point of the interactor and the collision center, and a length from the center point of the interactor to an outer boundary of the first area of the interactor.

[0183] Example 30. The apparatus of any one of examples 17-29, wherein the third see-through area determination module comprises:

[0184] a fifth see-through area determination module configured to determine the see-through area for the head within a field of view of the interactor in response to the interactor being worn on the head and in response to a speed of the interactor satisfying a second speed threshold.

[0185] Example 31. The apparatus of any one of examples 17-30, further comprising:

[0186] a sixth see-through area determination module configured to determine the see-through area for the head within a field of view of the interactor in response to the interactor being worn on the head and in response to a distance between an outer boundary of the first area of the interactor and the physical environment satisfying a distance threshold.

[0187] Example 32. The apparatus of any one of examples 17-31, wherein the determining see-through area determination module comprises:

[0188] a sixth see-through area determination module configured to determine the see-through area for the head based on a location where the first area of the interactor intersects the second area of the physical object in the physical environment and a speed of the interactor.

[0189] Example 33. An electronic device, comprising:

[0190] a processor; and

[0191] a memory coupled with the processor, the memory having instructions stored therein that, when executed by the processor, cause the electronic device to perform actions, the actions comprising:

[0192] in response to a first area of an interactor of a head-mounted display device intersecting a second area of a physical object in a physical environment, obtaining information of the physical object;

[0193] based on the information of the physical object, determining a see-through area for see-through display in the head-mounted display device; and

[0194] see-through displaying at least a portion of the physical object in the see-through area of the head-mounted display device.

[0195] Example 34. The electronic device of example 33, wherein in response to a first region of an interactor of the head-mounted display device intersecting a second region of a physical object in a physical environment, obtaining information of the physical object comprises:

[0196] determining a plurality of grid points based on the first region of the interactor; and

[0197] determining a collision center of the physical object based on the plurality of grid points.

[0198] Example 35. The electronic device of any of examples 33-34, wherein determining a collision center of the physical object based on the plurality of grid points comprises:

[0199] determining a plurality of distances between the plurality of grid points and a center point of the interactor;

[0200] determining a plurality of weights of the plurality of grid points based on the plurality of distances; and

[0201] determining the collision center based on the plurality of weights of the plurality of grid points.

[0202] Example 36. The electronic device of any of examples 33-35, wherein determining a plurality of weights of the plurality of grid points based on the plurality of distances comprises:

[0203] determining a plurality of normalized values of the plurality of distances based on the plurality of distances; and

[0204] determining the plurality of weights of the plurality of grid points based on the plurality of normalized values.

[0205] Example 37. The electronic device of any of examples 33-36, wherein determining the collision center based on the plurality of weights of the plurality of grid points comprises:

[0206] determining a plurality of calculation points from the plurality of grid points based on the plurality of weights of the plurality of grid points; and

[0207] determining the collision center based on a plurality of weights of the plurality of calculation points.

[0208] Example 38. The electronic device of any of examples 33-37, wherein determining a pass-through region for pass-through display in the head-mounted display device based on the information of the physical object comprises:

[0209] determining the pass-through region based on a distance between the center point of the interactor and the collision center and a length of the center point of the interactor to an outer boundary of the first region of the interactor.

[0210] Example 39. The electronic device of any of examples 33-38, wherein in the see-through region of the head-mounted display device, see-through displaying at least a portion of the physical object comprises:

[0211] see-through displaying at least a portion of the physical object in a feathering manner based on the see-through region.

[0212] Example 40. The electronic device of any of examples 33-39, wherein determining the see-through region for see-through displaying in the head-mounted display device based on information of the physical object comprises:

[0213] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, determining a set of center point coordinates of the interactor; and

[0214] based on the set of center point coordinates, determining a plurality of trust degrees of a plurality of the center points.

[0215] Example 41. The electronic device of any of examples 33-40, wherein determining the set of center point coordinates of the interactor comprises at least one of:

[0216] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, recording a center point coordinate of the interactor;

[0217] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, recording a center point coordinate of the interactor based on a certain time interval;

[0218] in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, and in response to a distance between a center point of the interactor at a current time and a center point of the interactor at a previous time being greater than a length from the center point of the interactor at the current time to an outer boundary of the first region, recording a center point coordinate of the interactor at the current time; or

[0219] in response to a moving speed of the interactor reaching a first speed threshold, recording a center point coordinate of the interactor based on a certain time interval.

[0220] Example 42. The electronic device of any of examples 33-41, wherein determining the plurality of trust degrees of the plurality of the center points based on the set of center point coordinates comprises:

[0221] in response to the first region of the interactor of the head-mounted display device intersecting the second region of the physical object in the physical environment, determining a subset of the set of center point coordinates of the interactor based on a length of a center point of the interactor to an outer boundary of the first region and the center point of the interactor;

[0222] based on the subset, determining distances of a plurality of points within the subset to the center point of the interactor;

[0223] based on the distances, determining a plurality of weights of the plurality of points within the subset; and

[0224] based on the plurality of weights of the plurality of points, determining a trustworthiness of each point of the plurality of points within the subset.

[0225] Example 43. The electronic device of any of Examples 33-42, wherein determining, based on the information of the physical object, a pass-through region for pass-through display in the head-mounted display device comprises:

[0226] in response to the trustworthiness being in a first range, pass-through displaying at least a portion of the physical object; and

[0227] in response to the trustworthiness being in a second range, stopping pass-through displaying the physical object.

[0228] Example 44. The electronic device of any of Examples 33-43, wherein determining, based on the information of the physical object, a pass-through region for pass-through display in the head-mounted display device comprises:

[0229] determining whether the interactor is located at a hand or a head;

[0230] in response to the interactor being located at the hand, determining a pass-through region for the hand; and

[0231] in response to the interactor being located at the head, determining a pass-through region for the head.

[0232] Example 45. The electronic device of any of Examples 33-44, wherein determining, in response to the interactor being worn on a hand, a pass-through region for the hand comprises:

[0233] in response to the interactor being located at a hand, determining a buffer distance based on a speed of motion of the interactor;

[0234] based on the buffer distance, determining an offset amount for the first region of the interactor;

[0235] based on the offset amount, determining a size of the first region of the interactor; and

[0236] determining the see-through region for the hand based on a size of the first region, a distance of a center point of the interactor to the collision center, and a length of the center point of the interactor to an outer boundary of the first region of the interactor.

[0237] Example 46. The electronic device of any of examples 33-45, wherein in response to the interactor being worn on a head, determining a see-through region for the head comprises:

[0238] in response to the interactor being worn on a head, and in response to a velocity of the interactor satisfying a second velocity threshold, determining the see-through region for the head within a field of view range of the interactor.

[0239] Example 47. The electronic device of any of examples 33-46, further comprising:

[0240] in response to the interactor being worn on a head, and in response to a distance of an outer boundary of the first region of the interactor to the physical environment satisfying a distance threshold, determining the see-through region for the head within a field of view range of the interactor.

[0241] Example 48. The electronic device of any of examples 33-47, wherein determining the see-through region for the head comprises:

[0242] determining the see-through region for the head based on a location at which the first region of the interactor intersects a second region of the physical object in the physical environment and a velocity of the interactor.

[0243] Example 49. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, perform a method in accordance with any of examples 1-16.

[0244] Example 50. A computer program product tangibly stored in a computer-readable medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform a method in accordance with any of examples 1-16.

[0245] Although the present disclosure has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject of the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for display of a head-mounted display device, comprising: in response to a first region of an interactor of the head-mounted display device intersecting with a second region of a physical object in a physical environment, obtaining information of the physical object; based on the information of the physical object, determining a pass-through region for pass-through display in the head-mounted display device; and passing through displaying at least a portion of the physical object in the pass-through region of the head-mounted display device; wherein in response to a first region of an interactor of the head-mounted display device intersecting with a second region of a physical object in a physical environment, obtaining information of the physical object comprises: based on the first region of the interactor, determining a plurality of grid points; and based on the plurality of grid points, determining a collision center of the physical object. 2.The method of claim 1, wherein based on the plurality of grid points, determining a collision center of the physical object comprises: determining a plurality of distances between the plurality of grid points and a center point of the interactor; based on the plurality of distances, determining a plurality of weights of the plurality of grid points; and based on the plurality of weights of the plurality of grid points, determining the collision center. 3.The method of claim 2, wherein based on the plurality of distances, determining a plurality of weights of the plurality of grid points comprises: based on the plurality of distances, determining a plurality of normalized values of the plurality of distances; and based on the plurality of normalized values, determining the plurality of weights of the plurality of grid points. 4.The method of claim 2, wherein based on the plurality of weights of the plurality of grid points, determining the collision center comprises: based on the plurality of weights of the plurality of grid points, determining a plurality of calculation points from the plurality of grid points; and based on a plurality of weights of the plurality of calculation points, determining the collision center. 5.The method of claim 2, wherein based on the information of the physical object, determining a pass-through region for pass-through display in the head-mounted display device comprises: based on a distance between the center point of the interactor and the collision center and a length from the center point of the interactor to an outer boundary of the first region of the interactor, determining the pass-through region. 6.The method of claim 5, wherein passing through displaying at least a portion of the physical object in the pass-through region of the head-mounted display device comprises: based on the pass-through region, passing through displaying at least a portion of the physical object in a feathering manner. 7.The method of claim 1, wherein based on the information of the physical object, determining a pass-through region for pass-through display in the head-mounted display device comprises: in response to the first region of the interactor of the head-mounted display device intersecting with the second region of the physical object in the physical environment, determining a set of center point coordinates of the interactor; and based on the set of center point coordinates, determining a plurality of trust degrees of a plurality of the center points. 8.The method of claim 7, wherein determining a set of center point coordinates of the interactor comprises at least one of: ​ ​ ​ ​ ​ ​ in response to the first area of the interactor of the head-mounted display device intersecting the second area of the physical object in the physical environment, recording a center point coordinate of the interactor; in response to the first area of the interactor of the head-mounted display device intersecting the second area of the physical object in the physical environment, recording a center point coordinate of the interactor based on a certain time interval; in response to the first area of the interactor of the head-mounted display device intersecting the second area of the physical object in the physical environment, and in response to a distance between the center point of the interactor at a current time and the center point of the interactor at a previous time being greater than a length from the center point of the interactor at the current time to an outer boundary of the first area, recording the center point coordinate of the interactor at the current time; or in response to a moving speed of the interactor reaching a first speed threshold, recording the center point coordinate of the interactor based on a certain time interval.

9. The method of claim 7, wherein determining a plurality of trust degrees of a plurality of the center points based on the set of center point coordinates comprises: in response to the first area of the interactor of the head-mounted display device intersecting the second area of the physical object in the physical environment, determining a subset of the set of center point coordinates of the interactor based on a length from the center point of the interactor to an outer boundary of the first area and the center point of the interactor; determining distances between a plurality of points in the subset and the center point of the interactor based on the subset; determining a plurality of weights of the plurality of points in the subset based on the distances; and determining a trust degree of each of the plurality of points in the subset based on the plurality of weights of the plurality of points.

10. The method of claim 7, wherein determining a see-through area for see-through display in the head-mounted display device based on the information of the physical object comprises: in response to the trust degree being in a first range, see-through displaying at least a portion of the physical object; and in response to the trust degree being in a second range, stopping see-through display of the physical object.

11. The method of claim 1, wherein determining a see-through area for see-through display in the head-mounted display device based on the information of the physical object comprises: determining whether the interactor is located at a hand or a head; in response to the interactor being located at the hand, determining a see-through area for the hand; and in response to the interactor being located at the head, determining a see-through area for the head.

12. The method of claim 11, wherein determining a see-through area for the hand in response to the interactor being worn on the hand comprises: in response to the interactor being located at the hand, determining a buffer distance based on a moving speed of the interactor; determining an offset of the first area of the interactor based on the buffer distance; determining a size of the first area of the interactor based on the offset; and determine the see-through region for the hand based on a size of the first region, a distance between a center point of the interactor and the collision center, and a length from the center point of the interactor to an outer boundary of the first region of the interactor. 13.The method of claim 11, wherein in response to the interactor being worn on a head, determining the see-through region for the head comprises: in response to the interactor being worn on a head and in response to a speed of the interactor satisfying a second speed threshold, determining the see-through region for the head within a field of view range of the interactor. 14.The method of claim 13, further comprising: in response to the interactor being worn on a head and in response to a distance between an outer boundary of the first region of the interactor and the physical environment satisfying a distance threshold, determining the see-through region for the head within a field of view range of the interactor. 15.The method of any one of claims 13-14, wherein determining the see-through region for the head comprises: determining the see-through region for the head based on a position at which the first region of the interactor intersects the second region of the physical object in the physical environment and a speed of the interactor. 16.An apparatus for display of a head-mounted display device, comprising: an information obtaining module configured to obtain information of a physical object in a physical environment in response to a first region of an interactor of the head-mounted display device intersecting a second region of the physical object; a see-through region determining module configured to determine a see-through region for see-through display in the head-mounted display device based on the information of the physical object; and a display module configured to see-through display at least a portion of the physical object in the see-through region of the head-mounted display device; wherein the information obtaining module comprises: a grid point determining module configured to determine a plurality of grid points based on the first region of the interactor; and a collision center determining module configured to determine a collision center of the physical object based on the plurality of grid points. 17.An electronic device, comprising: a processor; and a memory coupled with the processor, the memory having stored therein instructions which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 15. 18.A computer program product comprising computer executable instructions, wherein the computer executable instructions are executed by a processor to implement the method according to any one of claims 1 to 15. ​ ​ ​

Images