Head-mounted display device and anti-collision method

By obtaining the distance between VR glasses and handles and the user's movement information in real time, combining camera and ultrasonic sensors to obtain surrounding environment information, reminding users that they are about to exceed the safety boundary, solving the problem of bumps and falls during use of VR devices, and improving safety and experience .

CN118981248APending Publication Date: 2024-11-19GOERTEK INC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411008545.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

When using VR electronic products, users are prone to bumping and falling because they cannot detect the real environment around them, which affects their experience and safety.

Method used

The distance between the handle and the VR glasses is obtained in real time through VR glasses, and the user's moving distance is obtained by combining the inertial measurement component to determine whether it exceeds the preset safety boundary. When it exceeds the limit, the camera component and ultrasonic sensing component are activated to obtain the image of the surrounding environment. and obstacle information, determine the obstacle level and reminder methods, and provide anti-collision reminders.

Benefits of technology

Effectively prevent users from colliding with obstacles during movement, improve the safety and user experience of the headset, and ensure that the safe use of the actual environment does not affect the actual environment in the virtual reality environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981248A_ABST
    Figure CN118981248A_ABST
Patent Text Reader

Abstract

The invention provides a head-mounted display device and an anti-collision method thereof, and the anti-collision method comprises the steps: obtaining a first distance between a handle and VR glasses in real time through the VR glasses, and obtaining a movement distance of a user through an inertia measurement assembly; according to a preset safety boundary, the first distance and the moving distance, whether the handle and the VR glasses exceed the safety boundary or not is judged; when the handle or the VR glasses leave the safety boundary, the camera shooting assembly is started, image information of the surrounding environment of the user is obtained through the camera shooting assembly, and obstacle information of the surrounding environment is obtained through the ultrasonic sensing assembly; and according to the obstacle information and the image information, determining an obstacle grade and a reminding mode in a preset range, and carrying out anti-collision reminding on a user according to the corresponding reminding mode. The VR glasses and the handle can be effectively prevented from being collided and damaged in the using process, and therefore the wearing experience of a user is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent wearable technology, and more specifically, to a head-mounted display device and an anti-collision method. Background Art

[0002] Virtual Reality (VR) mainly refers to the use of modern high-tech means with computer technology as the core to generate a virtual environment. With the help of special input / output devices, users can interact naturally with objects in the virtual world and get the same experience as the real world through vision, hearing and touch. It can be seen that with the rapid development of 5G technology around the world, the VR market is becoming more and more hot.

[0003] At present, when users use VR electronic products, such as VR glasses, they are usually asked to set a safety boundary. The safety boundary can be a user-defined obstacle-free area or a circular area given by the system. For the circular area given by the system function, the user can set the size of the circle. However, when users play certain games, because the game settings require operations beyond the safety boundary, for example, they need to go to a slightly farther place to pick up props, because VR glasses cannot see the real environment around them during the game, it may cause bumps and falls.

[0004] Therefore, how to prevent users or devices from bumping and falling without affecting the user experience and ensure user safety is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a head-mounted display device and an anti-collision method to solve the problem that when users use VR electronic products, they are prone to bumps in the surrounding real environment, affecting the experience.

[0006] The head-mounted display device anti-collision method provided by the present invention includes: obtaining a first distance between a handle and the VR glasses in real time through VR glasses, and obtaining a moving distance of a user through an inertial measurement component; judging whether the handle and the VR glasses exceed the safety boundary according to a preset safety boundary, the first distance and the moving distance; when the handle or the VR glasses leave the safety boundary, starting a camera component, and obtaining image information of the user's surrounding environment through the camera component, and obtaining obstacle information of the surrounding environment through an ultrasonic sensor component; determining an obstacle level and a reminder method within a preset range according to the obstacle information and the image information, and providing an anti-collision reminder to the user according to the corresponding reminder method.

[0007] In addition, an optional technical solution is that when the handle or the VR glasses reaches the safety boundary, the virtual scene prompts through a display reminder or a sound reminder; wherein the display reminder includes displaying a red grid area in the virtual scene, prompting the user that the user is about to leave the safety boundary.

[0008] In addition, an optional technical solution is that when the handle or the VR glasses leave the safety boundary, it also includes: collecting the user's movement direction and speed information through the inertial measurement component; determining a preset range in the image information based on the movement direction and the speed information; and obtaining obstacle information within the preset range through the ultrasonic sensor component.

[0009] In addition, an optional technical solution is to obtain obstacle information within the preset range through the ultrasonic sensor component, including: obtaining the position, size and shape of the obstacle within the preset range through the ultrasonic sensor component.

[0010] In addition, an optional technical solution is that determining the obstacle level within the preset range includes determining whether the obstacles within the preset range are high-level or low-level; wherein high-level obstacles include: mobile obstacles; low-level obstacles include: static obstacles, and the obstacles are located on the user's moving path.

[0011] In addition, an optional technical solution is that when the obstacle level is high, the reminder method includes: generating a corresponding virtual obstacle based on the image information and the obstacle information, and displaying it in the current virtual scene, and providing anti-collision reminders to the user through the virtual obstacle.

[0012] In addition, an optional technical solution is that when the obstacle level is low, the reminder method includes: generating a corresponding virtual obstacle based on the image information and the obstacle information, and displaying it in the current virtual scene, and providing anti-collision reminders to the user through the virtual obstacle; or, adjusting the display angle of the virtual scene based on the image information and the obstacle information, and guiding the user to avoid the obstacle.

[0013] In addition, an optional technical solution is to use the virtual obstacle to remind the user of collision avoidance, including: displaying the virtual obstacle in a progressive, flashing or color-marked manner.

[0014] On the other hand, the present invention provides a head-mounted display device, including VR glasses and a control handle; wherein the VR glasses include a shell, an inertial measurement component arranged in the shell, and a camera component and an ultrasonic sensor component arranged on the surface of the shell; the head-mounted display device performs anti-collision reminders through the above-mentioned head-mounted display device anti-collision method.

[0015] In addition, an optional technical solution is that the camera assembly is provided with at least one group; wherein the camera assembly is used to collect image information in front of the moving direction according to the moving direction of the user.

[0016] By utilizing the above-mentioned head-mounted display device and anti-collision method, the position and distance information of the VR glasses and the handle can be obtained in real time. Combined with the preset safety boundary range, it is determined whether the handle and the VR glasses exceed the safety boundary. If the handle or the VR glasses leave the safety boundary, the camera component is started to collect image information of the surrounding environment, and the obstacle information of the surrounding environment is obtained through the ultrasonic sensor component. Based on this, the obstacle level and reminder method are determined to prevent the user from colliding with obstacles during movement, which can improve the safety of the head-mounted device and the user experience.

[0017] In order to achieve the above and related purposes, one or more aspects of the present invention include the features that will be described in detail later. The following description and the accompanying drawings describe some exemplary aspects of the present invention in detail. However, these aspects indicate only some of the various ways in which the principles of the present invention can be used. In addition, the present invention is intended to include all these aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings:

[0019] Figure 1 is a flowchart of a method for preventing a head mounted display device from collision according to an embodiment of the present invention;

[0020] Figure 2 Schematic diagram of the structure of a head mounted display device according to an embodiment of the present invention.

[0021] The reference numerals include: camera assembly 1, ultrasonic sensor assembly 2, inertial measurement assembly 3, front panel 4, and housing 5.

[0022] The same reference numerals throughout the drawings indicate similar or corresponding features or functions. DETAILED DESCRIPTION

[0023] In the following description, for the purpose of illustration, in order to provide a comprehensive understanding of one or more embodiments, many specific details are set forth. However, it is apparent that these embodiments may also be implemented without these specific details. In other examples, for ease of describing one or more embodiments, known structures and devices are shown in the form of block diagrams.

[0024] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0025] In order to describe in detail the head mounted display device and the anti-collision method and the use method thereof in the present invention, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 The schematic flow of the head mounted display device anti-collision method according to an embodiment of the present invention is shown.

[0027] like Figure 1 As shown, the head mounted display device anti-collision method of the embodiment of the present invention includes:

[0028] S100: acquiring a first distance between the handle and the VR glasses in real time through the VR glasses, and acquiring a moving distance of the user through an inertial measurement component;

[0029] S200: judging whether the handle and the VR glasses exceed the safety boundary according to the preset safety boundary, the first distance and the moving distance;

[0030] S300: When the handle or the VR glasses leave the safety boundary, start the camera component, and obtain image information of the user's surrounding environment through the camera component, and obtain obstacle information of the surrounding environment through the ultrasonic sensor component;

[0031] S400: Determine the obstacle level and reminder method within a preset range according to the obstacle information and the image information, and provide a collision avoidance reminder to the user according to the corresponding reminder method.

[0032] Specifically, during the startup or setting process of the head-mounted display device, a safety boundary can be provided for the user to select, or the user can set the range of the safety boundary by himself. For example, the safety boundary can be set to within 1m around the user, that is, a circular area with a radius of 1m centered on the user's current location (starting point). When the user moves and causes the VR glasses or handles to exceed the safety boundary, it indicates that the user may collide with other objects in the environment, so as to provide a collision warning to ensure the safe use of the device.

[0033] Among them, the first distance H between the handle and the VR glasses can be obtained in real time through the VR glasses, and the user's moving distance L from the starting point can be obtained through the inertial measurement component; when the user's moving distance L>1m-H, it indicates that the user's handle is about to leave the set safety boundary; when the user's moving distance L>1m, it indicates that the user is about to leave the set safety boundary. At this time, the user can be reminded in advance through the virtual scene, for example, by displaying reminders or sound reminders in the virtual scene; wherein, the display reminder includes displaying a red grid area in the virtual scene as an area beyond the safety boundary, prompting the user to leave the safety boundary, and the sound reminder can be a set reminder sound or reminder sentence, etc.

[0034] In addition, the first distance H between the handle and the VR glasses is obtained through the VR glasses, including photographing the infrared light on the handle through the camera module in the VR glasses, and then determining the distance between the handle and the VR glasses based on the photographed image; or, determining the distance between the two based on feedback from a signal transmitter between the VR glasses and the handle, and no specific restrictions are made here.

[0035] In the head-mounted display device anti-collision method of the present invention, when the handle or VR glasses leave the safety boundary, it may further include: collecting the user's movement direction and speed information through the inertial measurement component, and then determining the preset range in the image information according to the movement direction and speed information, and obtaining the obstacle information within the preset range through the ultrasonic sensor component. Since the user's movement is carried out in real time, the obstacle information also needs to be updated in real time. In order to achieve early warning, a safety warning interval can be set, that is, through the movement direction and speed information, the possible collision risks within the preset time or range in the future can be predicted in advance to ensure the reliability of the anti-collision solution.

[0036] Among them, obstacle information may include information such as the location, size and shape of the obstacle. In addition to facilitating the classification of obstacles and determining the corresponding reminder method, obtaining this information can also perform virtual reality on the obstacle information during the reminder process, realizing the combination of virtual and reality, and facilitating users to avoid obstacles.

[0037] Specifically, the obstacle level may include high level and low level; wherein, high level obstacles include: mobile obstacles, such as people or animals, whose positions are uncertain; low level obstacles include: static obstacles, and the obstacles are located on the user's moving path, such as chairs, tables, etc.; irrelevant obstacles include: static obstacles, and the obstacles are not located on the user's moving path, such as mats on the ground, etc., and thus will not pose a threat to the user, and such obstacles may not be processed as reminders.

[0038] Among them, when the obstacle level is high, the reminder method may include: generating a corresponding virtual obstacle based on the image information and obstacle information, and displaying it in the current virtual scene, and reminding the user to avoid collision through the virtual obstacle. When the obstacle level is low, the reminder method includes: generating a corresponding virtual obstacle based on the image information and obstacle information, and displaying it in the current virtual scene, and reminding the user to avoid collision through the virtual obstacle; or adjusting the display angle of the virtual scene based on the image information and obstacle information to guide the user to avoid the obstacle.

[0039] As an example, if the user is using a head-mounted display device and the VR glasses or handles exceed the safety boundary, the user may first be warned through sound or a red grid area. Then, if the user continues to move, it may further determine whether there are obstacles in the area ahead of the user's moving direction and determine the level of the obstacle. The corresponding virtual obstacle may then be generated and displayed in the virtual scene for the user to determine whether to continue moving, or the obstacles in the real world may be transferred to prevent collisions. In addition, the display angle of the current virtual scene may also be adjusted. For example, if the user moves to the left to pick up props or equipment, the entire virtual scene may be adjusted to the right by a certain angle so that the props or equipment appear in front of the user. The user may then pick up the required items in the virtual scene while avoiding obstacles, thereby avoiding collisions.

[0040] It should be noted that, since during the processing, objects in the real environment are formed into virtual obstacles and displayed, in order to distinguish between objects in the virtual scene and virtual obstacles corresponding to real obstacles, the virtual obstacles can be displayed in the virtual scene in a progressive, flashing or color-coded manner, thereby facilitating user distinction and serving as a certain reminder.

[0041] Corresponding to the above-mentioned head-mounted display device anti-collision method, the present invention also provides a head-mounted display device, Figure 2 The schematic structure of a head mounted display device according to an embodiment of the present invention is shown.

[0042] like Figure 2 As shown, the head-mounted display device of an embodiment of the present invention includes VR glasses and a corresponding control handle; wherein the VR glasses include a shell 5, an inertial measurement component 3 arranged in the shell 5, and a camera component 1 and an ultrasonic sensor component 2 arranged on the surface of the shell 5; the head-mounted display device performs anti-collision reminders through the above-mentioned head-mounted display device anti-collision method.

[0043] Among them, there is at least one group of camera components 1, which can be located on the front panel 4 of the shell 5, or can be set at other positions of the shell 5 according to movement requirements, so that the camera component 1 can collect image information in front of the moving direction according to the user's moving direction, thereby realizing reliable monitoring of movement in any direction.

[0044] It should be noted that the embodiments of the above-mentioned head-mounted display device can refer to the description in the embodiment of the head-mounted display device anti-collision method, which will not be repeated here.

[0045] According to the head-mounted display device and anti-collision method of the present invention, the position and distance information of the VR glasses and the handle can be obtained in real time, and combined with the preset safety boundary range, it is determined whether the handle and the VR glasses exceed the safety boundary, and when the handle or the VR glasses leave the safety boundary, the camera component and the ultrasonic sensor component are started to obtain obstacle information in the surrounding environment, thereby preventing the user from colliding with obstacles during movement, realizing the interaction between the virtual world and the real world, and improving the safety of the use of the head-mounted device and the user experience effect.

[0046] As described above, the head mounted display device and the anti-collision method according to the present invention are described by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the head mounted display device and the anti-collision method proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A head mounted display device anti-collision method, characterized in that: include: Acquire a first distance between the handle and the VR glasses in real time through the VR glasses, and acquire a moving distance of the user through the inertial measurement component; According to a preset safety boundary, the first distance, and the moving distance, determining whether the handle and the VR glasses exceed the safety boundary; When the handle or the VR glasses leave the safety boundary, the camera component is started, and image information of the user's surrounding environment is obtained through the camera component, and obstacle information of the surrounding environment is obtained through the ultrasonic sensor component; According to the obstacle information and the image information, the obstacle level and reminder method within the preset range are determined, and the user is reminded of the anti-collision according to the corresponding reminder method.

2. The head mounted display device anti-collision method according to claim 1, characterized in that: When the handle or the VR glasses reaches the safety boundary, the virtual scene prompts through display reminder or sound reminder; wherein, The display reminder includes displaying a red grid area in the virtual scene to remind the user that he is about to leave the safety boundary.

3. The head mounted display device anti-collision method according to claim 1, characterized in that: When the handle or the VR glasses leave the safety boundary, the method further includes: Collecting the user's moving direction and speed information through the inertial measurement component; Determining a preset range in the image information according to the moving direction and the speed information; Obstacle information within the preset range is obtained through the ultrasonic sensor component.

4. The head mounted display device anti-collision method according to claim 3, characterized in that: Obtaining obstacle information within the preset range through the ultrasonic sensor assembly includes: The position, size and shape of obstacles within the preset range are acquired through the ultrasonic sensor component.

5. The head mounted display device anti-collision method according to claim 1, characterized in that: Determining the level of obstacles within the preset range includes determining whether the obstacles within the preset range are high level or low level; wherein, Advanced obstacles include: moving obstacles; Low-level obstacles include: static obstacles, and the obstacles are located on the moving path of the user.

6. The head mounted display device anti-collision method according to claim 5, characterized in that: When the obstacle level is high, the reminder method includes: A corresponding virtual obstacle is generated according to the image information and the obstacle information, and is displayed in the current virtual scene, and the user is reminded of collision avoidance through the virtual obstacle.

7. The head mounted display device anti-collision method according to claim 5, characterized in that: When the obstacle level is low, the reminder method includes: Generate a corresponding virtual obstacle according to the image information and the obstacle information, and display it in the current virtual scene, and use the virtual obstacle to remind the user to avoid collision; or The display angle of the virtual scene is adjusted according to the image information and the obstacle information to guide the user to avoid the obstacle.

8. The head mounted display device anti-collision method according to claim 6 or 7, characterized in that: Providing anti-collision reminders to the user through the virtual obstacle includes: The virtual obstacle is displayed in a progressive, flashing or color-marking manner.

9. A head mounted display device, characterized in that: Including VR glasses and control handles; The VR glasses include a shell, an inertial measurement component arranged in the shell, and a camera component and an ultrasonic sensor component arranged on the surface of the shell; The head mounted display device performs anti-collision reminder by using the head mounted display device anti-collision method according to any one of claims 1 to 8.

10. The head mounted display device according to claim 9, characterized in that: The camera assembly is provided with at least one group; wherein, The camera assembly is used to collect image information in the direction of movement of the user.

Citation Information

Patent Citations

  • Virtual reality VR equipment and obstacle avoidance method and device thereof

    CN110503001A

  • Handle protection method and system and VR equipment

    CN111957034A

  • Method and device for avoiding real obstacles based on VR game

    CN115300896A

  • Collision early warning method and device and head-mounted display equipment

    CN115662095A

  • Virtual reality scene display method and related device

    CN116954362A