Virtual reality experience security area updating method and device

Abstract

This disclosure provides a method and apparatus for updating a safe area in a virtual reality experience, belonging to the field of virtual reality technology. The apparatus includes: a safe area setting module for setting an initial safe area; an obstacle detection module for detecting obstacles around the user; and a safe area updating module for updating the extent of the safe area based on the detected obstacles. Embodiments of this disclosure enable dynamic updating of the safe area, ensuring user safety.

Description

Technical Field

[0001] This disclosure relates to the field of virtual reality technology, and in particular to a method and apparatus for updating the security area of ​​a virtual reality experience. Background Technology

[0002] VR (Virtual Reality) devices utilize virtual reality technology to provide wearers with an immersive experience, placing them in a virtual environment. However, during this process, wearers cannot see the surrounding real world. When wearers interact with or move in the virtual scene, their bodies may shift. If other objects appear in the path of movement, it may create safety hazards and cause injury to the wearer. Summary of the Invention

[0003] The technical problem to be solved by this disclosure is to provide a method and apparatus for updating the safe zone of a virtual reality experience, which can ensure the safety of users.

[0004] To address the aforementioned technical problems, the embodiments of this disclosure provide the following technical solutions:

[0005] On the one hand, a virtual reality experience security area update device is provided, applied to VR devices, including:

[0006] The safe zone setting module is used to set the initial safe zone;

[0007] The obstacle detection module is used to detect obstacles around the user;

[0008] The safe zone update module is used to update the extent of the safe zone based on detected obstacles.

[0009] In some embodiments, the obstacle detection module includes:

[0010] A lidar module is used to emit laser signals in a 360° radius around the user and receive the reflected laser signals. Based on the received laser signals, the coordinates and outlines of obstacles around the user are determined.

[0011] In some embodiments, the obstacle detection module further includes:

[0012] A depth camera mounted on the VR headset of the VR device is used to capture images of the environment in the direction the VR headset is facing, and to determine the coordinates and outlines of obstacles around the user based on the captured depth images.

[0013] In some embodiments, the lidar module includes M lidars located on the same plane, the centers of the M lidars forming a regular M-gon, the scanning range of the M lidars being outside the regular M-gon, and the scanning angle of each lidar being not less than 360 / M degrees, where M is an integer greater than or equal to 3.

[0014] In some embodiments, the lidar module includes four lidars located on the same plane, the centers of the four lidars forming a square, the scanning range of the four lidars being outside the square, the scanning angle of each lidar being 90 degrees, and the scanning range of different lidars being different.

[0015] In some embodiments, the safe area update module is specifically used to convert the first coordinates of the obstacle detected by the obstacle detection module into second coordinates in a second coordinate system. If the detected obstacle is determined to be located within the safe area based on the coordinate range of the safe area in the second coordinate system and the second coordinates, the space occupied by the obstacle is calculated based on the second coordinates and outline of the obstacle to obtain the updated safe area. The second coordinate system is the coordinate system of the VR device or the world coordinate system, and the first coordinate is the coordinate in the first coordinate system, which is the coordinate system of the obstacle detection module.

[0016] In some embodiments, if the obstacle detection module includes a lidar module and a depth camera, the coordinates and contours of the first obstacle detected by the lidar module are not exactly the same as the coordinates and contours of the second obstacle detected by the depth camera.

[0017] The safe zone update module calculates the first space occupied by the first obstacle based on the position and outline of the first obstacle, calculates the second space occupied by the second obstacle based on the coordinates and outline of the second obstacle, takes the union of the first space and the second space as the space occupied by the obstacle, and removes the space occupied by the obstacle from the initial safe zone to obtain the updated safe zone.

[0018] In some embodiments, the obstacle detection module includes:

[0019] Multiple pressure sensing units are arranged in an array on the ground of the initial safe area. The areas where different pressure sensing units are located do not overlap, and the areas where all pressure sensing units are located constitute the initial safe area. Each pressure sensing unit is provided with at least one pressure sensor for sensing the pressure on the pressure sensing unit.

[0020] The processor is configured to receive pressure data from each of the pressure sensing units, detect candidate pressure sensing units whose pressure data is greater than a preset first threshold, determine whether the user is located in the area where the candidate pressure sensing unit is located, and if the user is not located in the area where the candidate pressure sensing unit is located, remove the area where the candidate pressure sensing unit is located from the initial safe area to obtain an updated safe area.

[0021] In some embodiments, it also includes:

[0022] The first alert module is used to issue an alarm if it detects that a user is approaching the boundary of the safe area or entering an area outside the safe area.

[0023] In some embodiments, it also includes:

[0024] The second prompt module is used to prompt the user to clear obstacles if the ratio of the area of ​​the updated safe area to the area of ​​the initial safe area is less than a preset second threshold.

[0025] This disclosure also provides a method for updating the security area of ​​a virtual reality experience, applied to a VR device, including:

[0026] Set the initial safe zone;

[0027] Obstacles around the user are detected; the extent of the safe zone is updated based on the detected obstacles.

[0028] Detecting obstacles within the safe area includes:

[0029] In some embodiments, detecting obstacles within the safe area includes:

[0030] Using a lidar module, laser signals are emitted in a 360° radius around the user, and the reflected laser signals are received. The coordinates and outlines of obstacles around the user are determined based on the received laser signals.

[0031] In some embodiments, detecting obstacles within the safe area further includes: using a depth camera mounted on the VR headset of the VR device to capture images of the environment in the direction the VR headset is facing, and determining the coordinates and outlines of obstacles around the user based on the captured depth images.

[0032] In some embodiments, updating the extent of the safe zone based on detected obstacles includes:

[0033] The first coordinates of the obstacle detected by the obstacle detection module are converted into second coordinates in the second coordinate system. If the detected obstacle is located within the safe area based on the coordinate range of the safe area in the second coordinate system and the second coordinates, the space occupied by the obstacle is calculated based on the second coordinates and outline of the obstacle. The space occupied by the obstacle is then removed from the initial safe area to obtain an updated safe area. The second coordinate system is the coordinate system of the VR device or the world coordinate system, and the first coordinate is the coordinate in the first coordinate system, which is the coordinate system of the obstacle detection module.

[0034] In some embodiments, if the coordinates and contours of the first obstacle detected by the lidar module are not exactly the same as the coordinates and contours of the second obstacle detected by the depth camera, updating the range of the safe area based on the detected obstacles specifically includes:

[0035] The first space occupied by the first obstacle is calculated based on its coordinates and outline. The second space occupied by the second obstacle is calculated based on its coordinates and outline. The union of the first space and the second space is taken as the space occupied by the obstacle. The space occupied by the obstacle is removed from the initial safe area to obtain the updated safe area.

[0036] In some embodiments, it also includes:

[0037] An alarm will be triggered if a user is detected approaching the boundary of the safe zone or entering an area outside the safe zone.

[0038] In some embodiments, it also includes:

[0039] If the ratio of the updated safe area to the initial safe area is less than a preset second threshold, the user is prompted to clear the obstacle.

[0040] This disclosure also provides a virtual reality experience security area update device, which is applied to a VR device. The virtual reality experience security area update device includes a memory and an actuator, the actuator being used to execute the method described above.

[0041] The embodiments disclosed herein have the following beneficial effects:

[0042] In the above scheme, after setting the initial safe zone, obstacles within the safe zone are detected. Once an obstacle is detected, the range of the safe zone is updated based on the detected obstacle. This allows the range of the safe zone to be updated in real time according to the obstacle situation, ensuring the safety of the wearer. Attached Figure Description

[0043] Figure 1 A schematic diagram illustrating the initial security zone set up according to an embodiment of this disclosure;

[0044] Figure 2 This is a schematic diagram illustrating the use of lidar and depth camera to detect obstacles according to an embodiment of this disclosure;

[0045] Figure 3 This is a schematic diagram of a lidar module according to an embodiment of the present disclosure;

[0046] Figure 4 This is a schematic diagram illustrating the updating of the security area according to an embodiment of this disclosure;

[0047] Figure 5 This is a schematic diagram of the pressure sensing unit according to an embodiment of the present disclosure;

[0048] Figure 6 This is a schematic diagram of the safety area formed by the pressure sensing unit in an embodiment of the present disclosure;

[0049] Figure 7 This is a flowchart illustrating the virtual reality experience security area update method according to an embodiment of this disclosure;

[0050] Figure 8 This is a schematic diagram illustrating the process of updating the security area according to a specific embodiment of this disclosure. Detailed Implementation

[0051] To make the technical problems, technical solutions and advantages to be solved by the embodiments of this disclosure clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0052] When using VR devices, users draw a closed area with VR controllers, which serves as the range of movement allowed while wearing the VR device—the safe zone. When the user approaches the boundary of this safe zone or is about to leave it, the VR headset generates a warning to remind them not to leave. However, this method has two problems: First, if static or dynamic obstacles appear within the safe zone, especially when the obstacle is behind the user, it cannot detect the obstacle or warn the user, resulting in blind spots; second, the safe zone cannot be updated.

[0053] The embodiments of this disclosure provide a method and apparatus for updating the safety zone in a virtual reality experience, which can dynamically update the safety zone to ensure the safety of the wearer.

[0054] Embodiments of this disclosure provide a virtual reality experience security area update device, applied to a VR device, comprising:

[0055] The safe zone setting module is used to set the initial safe zone;

[0056] The obstacle detection module is used to detect obstacles around the user;

[0057] The safe zone update module is used to update the extent of the safe zone based on detected obstacles.

[0058] In this embodiment, after setting an initial safe zone, obstacles within the safe zone are detected. Once an obstacle is detected, the range of the safe zone is updated based on the detected obstacle. This allows the range of the safe zone to be updated in real time according to the obstacle situation, ensuring the safety of the wearer.

[0059] In some embodiments, the security zone setting module includes:

[0060] The VR controller is used to draw a closed area in space as the safe area according to the user's operation instructions after receiving the user's instruction to initialize the safe area.

[0061] VR devices include VR headsets and VR controllers. VR headsets may contain an IMU (Inertial Measurement Unit) gyroscope, a depth camera, a data processing processor, and a rendering processor; VR controllers may contain an IMU inertial navigation system to obtain the real-time position and attitude of the VR controllers.

[0062] Setting up a safe zone requires using a VR controller. After activating the VR device, the user can click the "Initialize Safe Zone" button on the VR controller. Upon receiving the user's instruction to initialize the safe zone, the user draws a closed area in space using the VR controller. The VR controller then sets this closed area as the initial safe zone based on the user's command. This initial safe zone should not contain any obstacles that would impede the user's movement. However, if dynamic or static obstacles appear within the safe zone, they will affect the user's safety. Therefore, the safe zone needs to be dynamically updated based on the extent of any obstacles.

[0063] like Figure 1 As shown in the diagram, the area defined by the dashed line is the initial safe zone. The origin O of this initial safe zone is the point perpendicular to the starting position of the VR controller. The direction of the initial movement is the X-axis, and the opposite direction of the ending movement is the Y-axis. The Z-axis is perpendicular to the XY plane. The boundary of the safe zone in this coordinate system is represented by a line connecting a set of discrete points. This safe zone is a three-dimensional space, and the coordinate axes of the coordinate system of this safe zone are the X-axis, Y-axis, and Z-axis. This coordinate system is also the coordinate system of the VR device.

[0064] In some embodiments, such as Figure 2 As shown, the obstacle detection module includes at least one of the following:

[0065] A lidar module is used to emit laser signals in a 360° radius around the user and receive the reflected laser signals. Based on the received laser signals, the coordinates and outlines of obstacles around the user are determined.

[0066] In some embodiments, the lidar module may include M lidars located on the same plane, the centers of the M lidars forming a regular M-gon, the scanning range of the M lidars being outside the regular M-gon, and the scanning angle of each lidar being not less than 360 / M degrees, where M is an integer greater than or equal to 3.

[0067] Existing VR headsets are equipped with depth cameras, which can capture images of the environment in the direction the VR headset is facing. However, they cannot capture depth images in the direction the VR headset is facing away from, resulting in blind spots and the inability to detect obstacles behind the user. Considering the headset's structure and user comfort, it is not suitable to install more depth cameras in the headset. In this embodiment, a LiDAR module is used to detect obstacles, which can detect obstacles in a 360° radius around the user, avoiding blind spots.

[0068] The lidar module can be worn on the user's body, for example, by strapping it to the user's calf, and emits and receives laser signals in a 360-degree range centered on the user.

[0069] like Figure 3 As shown in a specific example, the lidar module includes four lidars (1, 2, 3, 4) located on the same plane. The four lidars are fixed at equal intervals on lidar straps, and their centers form a rectangle. The scanning range of each lidar is outside the rectangle, and the scanning angle of each lidar is 90 degrees. The scanning ranges of different lidars are different. Figure 3 As shown in the left-middle section, each lidar includes a laser transmitter / receiver 10 and a mirror structure 11. The mirror of the mirror structure 11 can rotate at a constant speed, such that the angle between the mirror and the laser emitted by the laser transmitter / receiver 10 is between 0 and 90 degrees. The mirror reflects the laser emitted by the laser transmitter / receiver 10 and then emits it back. When the laser encounters an obstacle, it is reflected back to the mirror and then back to the laser transmitter / receiver 10. The laser transmitter / receiver 10 can determine the coordinates and outline of the obstacle by receiving the laser. Each lidar can detect obstacles within an angle range of θ, where θ is 90°. Thus, with four lidars, obstacles within a 360° range can be detected for the user.

[0070] Of course, the number of lidars included in a lidar module is not limited to four; it can include more to improve the accuracy of obstacle detection. A lidar module can also include fewer lidars, such as three, as long as the lidar module can detect obstacles within a 360° range of the user.

[0071] When a lidar receives a returned laser signal at a specific angle, it calculates the distance *s* between the user and the obstacle at that angle using a distance formula. The user's position within the safe zone can be determined using an IMU gyroscope on the VR headset. Based on the user's position within the safe zone and the distance between the user and the obstacle at that angle, it can be determined whether the obstacle is within the initial safe zone. If the obstacle is within the initial safe zone, its position *P* is calculated and transmitted to the safe zone update module within the VR headset to alert the user; if the obstacle is outside the initial safe zone, it is ignored. The distance formula is:

[0072]

[0073] Where t1 is the time when the laser signal is received, t0 is the time when the laser signal is emitted, and v is the propagation speed of the laser.

[0074] In this embodiment, the position P of the obstacle is calculated only after it is determined that the obstacle is within the initial safe area. If the obstacle is outside the initial safe area, the position of the obstacle is not calculated, which can reduce the amount of calculation.

[0075] In some embodiments, the obstacle detection module further includes:

[0076] A depth camera mounted on the VR headset of the VR device is used to capture images of the environment in the direction the VR headset is facing, and to determine the coordinates and outlines of obstacles around the user based on the captured depth images. The depth camera can capture depth images of the scene in the user's direction of travel, and the depth images can capture obstacles and determine the distance between the obstacles and the user, the size of the obstacles, and their outlines.

[0077] In this embodiment, the obstacle detection module may include a lidar module and a depth camera, or it may only include a lidar module.

[0078] In some embodiments, the safe area update module is specifically used to convert the first coordinates of the obstacle detected by the obstacle detection module into second coordinates in a second coordinate system. If the detected obstacle is determined to be located within the safe area based on the coordinate range of the safe area in the second coordinate system and the second coordinates, the space occupied by the obstacle is calculated based on the second coordinates and outline of the obstacle to obtain the updated safe area. The second coordinate system is the coordinate system of the VR device or the world coordinate system, and the first coordinate is the coordinate in the first coordinate system, which is the coordinate system of the obstacle detection module.

[0079] like Figure 4 As shown, assuming an obstacle is detected and located within the initial safe area S0, the space S1 occupied by the obstacle is calculated based on its coordinates and outline. Space S1 is then removed from the safe area S0, resulting in an updated safe area of ​​S0-S1. When removing space S1, the boundaries of the safe area need to be redefined. These boundaries can be based on the boundaries of space S1, ensuring that the updated safe area does not include space S1.

[0080] In some embodiments, if the obstacle detection module only includes a lidar module, the first space occupied by the first obstacle can be calculated based on the coordinates and contour of the first obstacle detected by the lidar module, and the first space occupied by the obstacle can be removed from the initial safe area to obtain an updated safe area.

[0081] If the obstacle detection module includes both a LiDAR module and a depth camera, the coordinates and contours of the first obstacle detected by the LiDAR module may be exactly the same as or not exactly the same as the coordinates and contours of the second obstacle detected by the depth camera. If the coordinates and contours of the first obstacle detected by the LiDAR module are not exactly the same as the coordinates and contours of the second obstacle detected by the depth camera, the safe area update module is specifically used to calculate the first space occupied by the first obstacle based on the coordinates and contours of the first obstacle, calculate the second space occupied by the second obstacle based on the coordinates and contours of the second obstacle, take the union of the first space and the second space as the space occupied by the obstacle, and remove the space occupied by the obstacle from the initial safe area to obtain the updated safe area.

[0082] For example, the LiDAR module detects the coordinates and outline of a first obstacle, and calculates the first space S2 occupied by the first obstacle based on these coordinates and outline. The depth camera detects the coordinates and outline of a second obstacle, and calculates the second space S3 occupied by the second obstacle based on these coordinates and outline. The union of S3 and S2 is then taken, which includes both S3 and S2. This union is removed from the initial safe area to obtain the updated safe area. This method combines the detection results from the LiDAR module and the depth camera to update the safe area, improving detection accuracy, ensuring user safety, and providing a comfortable user experience.

[0083] If the coordinates and contour of the first obstacle detected by the lidar module are exactly the same as the coordinates and contour of the second obstacle detected by the depth camera, it means that the two detected the same obstacle. The safe area update module can calculate the first space occupied by the first obstacle based on the coordinates and contour of the first obstacle, and can also calculate the second space occupied by the second obstacle based on the coordinates and contour of the second obstacle, and remove the first space or the second space from the initial safe area to obtain the updated safe area.

[0084] In this embodiment, the LiDAR module scans the user's 360-degree range in real time, while the depth camera, typically facing the user's direction of travel, only captures images in that direction. Therefore, the LiDAR module can first scan for obstacles within the user's 360-degree range. If an obstacle is detected behind the user, the user can be prompted to turn towards it, and the depth camera can then collect further information about the obstacle. This way, the depth camera is only activated when an obstacle is detected behind the user, reducing its power consumption. The VR headset is equipped with an IMU gyroscope, which can collect the user's movement trajectory, orientation, and posture in real time.

[0085] In this embodiment, the LiDAR module, depth camera, and VR controller each have their own coordinate system. The data processing processor in the VR headset needs to transform the coordinate systems of the LiDAR module, depth camera, and VR controller to calculate the coordinate data of each part in a world coordinate system as a second coordinate system. For example, after the LiDAR module detects the coordinates and outline of an obstacle in its own coordinate system, the data processing processor in the VR headset needs to transform the detection data of the LiDAR module to obtain the coordinates and outline of the obstacle in the world coordinate system; after the depth camera detects the coordinates and outline of an obstacle in its own coordinate system, the data processing processor in the VR headset needs to transform the detection data of the depth camera to obtain the coordinates and outline of the obstacle in the world coordinate system; after setting a safe area in the VR controller coordinate system using the VR controller, the coordinates of the safe area need to be transformed to obtain the coordinates of the safe area in the world coordinate system.

[0086] In some embodiments, the virtual reality experience security area update device further includes:

[0087] The first alert module is used to issue an alarm if it detects that a user is approaching the boundary of the safe area or entering an area outside the safe area. The first alert module can alert the user by emitting an audible alert, a vibration alert, or by displaying an alarm message.

[0088] In some embodiments, the virtual reality experience security area update device further includes:

[0089] The second prompt module is used to prompt the user to clear the obstacle if the ratio of the updated safe area to the initial safe area is less than a preset second threshold. The second prompt module can alert the user with an audible alert, a vibration alert, or by displaying an alarm message.

[0090] In this embodiment, the safe zone can be updated using a lidar module and a depth camera, avoiding the problem that VR device users cannot see the surrounding environment and therefore cannot perceive surrounding obstacles, thus increasing user safety and improving user experience.

[0091] In some embodiments, such as Figure 5 and Figure 6 As shown, the obstacle detection module includes:

[0092] Multiple pressure sensing units are arranged in an array on the ground of the initial safety zone. Figure 6Each square in the diagram represents a pressure sensing unit. The areas containing different pressure sensing units do not overlap, and all the areas containing the pressure sensing units together form the initial safety area; for example... Figure 5 As shown, each pressure sensing unit is provided with at least one pressure sensor for sensing the pressure applied to the pressure sensing unit; it may also be provided with at least one vibration sensor for generating vibration to alert the user.

[0093] The processor is configured to receive pressure data from each of the pressure sensing units, detect candidate pressure sensing units whose pressure data is greater than a preset first threshold, determine whether the user is located in the area where the candidate pressure sensing unit is located, and if the user is not located in the area where the candidate pressure sensing unit is located, remove the area where the candidate pressure sensing unit is located from the initial safe area to obtain an updated safe area.

[0094] This embodiment eliminates the need for a LiDAR module and depth camera, and also avoids using VR controllers to draw an initial safe area. The initial safe area is formed by stitching together pressure-sensing units, and updates are performed on a unit-by-unit basis. This approach offers strong scalability, high accuracy, and eliminates blind spots. Figure 5 and Figure 6 As shown, a single pressure sensing unit is rectangular, such as a square; pressure sensors and vibration sensors are arranged on the pressure sensing unit. The pressure sensors are used to sense the pressure on the pressure sensing unit. There are splicing interfaces around the pressure sensing unit, which can be used for data transmission.

[0095] The pressure sensing units are spliced ​​together to form the initial safe area. The range of the safe area is the range covered by the pressure sensing units, that is, the closed area enclosed by s0-s1-s2-s3.

[0096] When updating the safe zone, the VR headset can be placed at a coordinate origin, which can be a point set on the pressure sensing unit. The position is initialized, and the position and attitude of the headset obtained by the VR headset IMU gyroscope are sent to the processor to calibrate the center position of the VR headset. In this way, the real-time position of the VR headset can be calculated based on the position of the coordinate origin. This real-time position is the coordinate in the coordinate system of the pressure sensing unit.

[0097] If an obstacle is located within the safe zone at the pressure sensing unit Pw, the pressure at Pw will increase. The pressure data sensed by the pressure sensor of pressure sensing unit Pw will exceed a preset first threshold, which can be set as needed. At this point, the VR headset's position information can be used to determine whether the user is located at pressure sensing unit Pw, and further, whether the pressure data sensed by the pressure sensor of pressure sensing unit Pw originates from the user's weight. If the VR headset is not located at pressure sensing unit Pw, it can be determined that the data from pressure sensing unit Pw does not originate from the user's weight but from an obstacle. Therefore, the obstacle can be identified as being located at pressure sensing unit Pw, and pressure sensing unit Pw can be removed from the initial safe zone. This process can be repeated to update the safe zone based on the pressure data from the pressure sensing units.

[0098] In some embodiments, the virtual reality experience security area update device further includes:

[0099] The first alert module is used to issue an alarm if it detects that a user is approaching the boundary of the safe zone or entering an area outside the safe zone. The first alert module can emit an audible alert, use a vibration sensor to emit vibrations, or display an alarm message to alert the user. For example, the vibration sensor might emit a prolonged vibration to warn the VR device user of nearby obstacles and to stop moving forward. If the VR device user reaches the boundary of the safe zone, it will emit a loud, persistent vibration to remind the user that they have reached the boundary and should step back.

[0100] In some embodiments, the virtual reality experience security area update device further includes:

[0101] The second prompt module is used to prompt the user to clear the obstacle if the ratio of the updated safe area to the initial safe area is less than a preset second threshold. The second prompt module can alert the user with an audible alert, a vibration alert, or by displaying an alarm message.

[0102] The second threshold can be set as needed, for example, to 50%. If the area occupied by the obstacle exceeds 50% of the initial safe zone, all vibration sensors will emit continuous vibrations, and the VR headset will issue a prompt to "remove the headset and clear the obstacle".

[0103] In this embodiment, the safety zone can be updated through the pressure sensing unit, avoiding the problem that VR device users cannot see the surrounding environment and therefore cannot feel the surrounding obstacles, thus increasing user safety and improving user experience.

[0104] This disclosure also provides a method for updating the security area of ​​a virtual reality experience, applicable to VR devices, such as... Figure 7 As shown, it includes:

[0105] Step 101: Set the initial safe zone;

[0106] Step 102: Detect obstacles around the user;

[0107] Step 103: Update the extent of the safe zone based on the detected obstacles.

[0108] In this embodiment, after setting an initial safe zone, obstacles within the safe zone are detected. Once an obstacle is detected, the range of the safe zone is updated based on the detected obstacle. This allows the range of the safe zone to be updated in real time according to the obstacle situation, ensuring the safety of the wearer.

[0109] In some embodiments, setting the initial security zone includes:

[0110] After receiving a user instruction to initialize the safe zone, the VR controller is used to draw a closed area in space as the safe zone according to the user's operation instructions.

[0111] VR devices include VR headsets and VR controllers. VR headsets may include an IMU gyroscope, a depth camera, a data processing processor, and a rendering processor; VR controllers may include an IMU inertial navigation system to obtain the real-time position and attitude of the VR controllers.

[0112] Setting up a safe zone requires using a VR controller. After activating the VR device, the user can click the "Initialize Safe Zone" button on the VR controller. Upon receiving the user's instruction to initialize the safe zone, the user draws a closed area in space using the VR controller. The VR controller then sets this closed area as the initial safe zone based on the user's command. This initial safe zone should not contain any obstacles that would impede the user's movement. However, if dynamic or static obstacles appear within the safe zone, they will affect the user's safety. Therefore, the safe zone needs to be dynamically updated based on the extent of any obstacles.

[0113] like Figure 1As shown in the diagram, the area defined by the dashed line is the initial safe zone. The origin O of this initial safe zone is the point perpendicular to the starting position of the VR controller. The direction of the initial movement is the X-axis, and the opposite direction of the ending movement is the Y-axis. The Z-axis is perpendicular to the XY plane. The boundary of the safe zone in this coordinate system is represented by a line connecting a set of discrete points. This safe zone is a three-dimensional space, and the coordinate axes of the coordinate system of this safe zone are the X-axis, Y-axis, and Z-axis. This coordinate system is also the coordinate system of the VR device.

[0114] In some embodiments, detecting obstacles within the safe area includes:

[0115] Using a lidar module, laser signals are emitted in a 360° radius around the user, and the reflected laser signals are received. The coordinates and outlines of obstacles around the user are determined based on the received laser signals.

[0116] In some embodiments, the lidar module may include M lidars located on the same plane, the centers of the M lidars forming a regular M-gon, the scanning range of the M lidars being outside the regular M-gon, and the scanning angle of each lidar being not less than 360 / M degrees, where M is an integer greater than or equal to 3.

[0117] Existing VR headsets are equipped with depth cameras, which can capture images of the environment in the direction the VR headset is facing. However, they cannot capture depth images in the direction the VR headset is facing away from, resulting in blind spots and the inability to detect obstacles behind the user. Considering the headset's structure and user comfort, it is not suitable to install more depth cameras in the headset. In this embodiment, a LiDAR module is used to detect obstacles, which can detect obstacles in a 360° radius around the user, avoiding blind spots.

[0118] The lidar module can be worn on the user's body, for example, by strapping it to the user's calf, and emits and receives laser signals in a 360-degree range centered on the user.

[0119] like Figure 3 As shown in a specific example, the lidar module includes four lidars (1, 2, 3, 4) located on the same plane. The four lidars are fixed at equal intervals on lidar straps, and their centers form a rectangle. The scanning range of each lidar is outside the rectangle, and the scanning angle of each lidar is 90 degrees. The scanning ranges of different lidars are different. Figure 3As shown in the left-middle section, each lidar includes a laser transmitter / receiver 10 and a mirror structure 11. The mirror of the mirror structure 11 can rotate at a constant speed, such that the angle between the mirror and the laser emitted by the laser transmitter / receiver 10 is between 0 and 90 degrees. The mirror reflects the laser emitted by the laser transmitter / receiver 10 and then emits it back. When the laser encounters an obstacle, it is reflected back to the mirror and then back to the laser transmitter / receiver 10. The laser transmitter / receiver 10 can determine the coordinates and outline of the obstacle by receiving the laser. Each lidar can detect obstacles within an angle range of θ, where θ is 90°. Thus, with four lidars, obstacles within a 360° range can be detected for the user.

[0120] Of course, the number of lidars included in a lidar module is not limited to four; it can include more to improve the accuracy of obstacle detection. A lidar module can also include fewer lidars, such as three, as long as the lidar module can detect obstacles within a 360° range of the user.

[0121] When a lidar receives a returned laser signal at a specific angle, it calculates the distance *s* between the user and the obstacle at that angle using a distance formula. The user's position within the safe zone can be determined using an IMU gyroscope on the VR headset. Based on the user's position within the safe zone and the distance between the user and the obstacle at that angle, it can be determined whether the obstacle is within the initial safe zone. If the obstacle is within the initial safe zone, its position *P* is calculated and transmitted to the safe zone update module within the VR headset to alert the user; if the obstacle is outside the initial safe zone, it is ignored. The distance formula is:

[0122]

[0123] Where t1 is the time when the laser signal is received, t0 is the time when the laser signal is emitted, and v is the propagation speed of the laser.

[0124] In this embodiment, the position P of the obstacle is calculated only after it is determined that the obstacle is within the initial safe area. If the obstacle is outside the initial safe area, the position of the obstacle is not calculated, which can reduce the amount of calculation.

[0125] In some embodiments, a depth camera mounted on the VR headset of the VR device can also be used to capture images of the environment in the direction the VR headset is facing, and the coordinates and outlines of obstacles around the user can be determined based on the captured depth images. The depth camera can capture depth images of the scene in the direction the user is traveling, and the depth images can capture obstacles and determine the distance between the obstacles and the user, the size of the obstacles, and their outlines.

[0126] In some embodiments, updating the extent of the safe zone based on detected obstacles includes:

[0127] The first coordinates of the obstacle detected by the obstacle detection module are converted into second coordinates in the second coordinate system. If the detected obstacle is located within the safe area based on the coordinate range of the safe area in the second coordinate system and the second coordinates, the space occupied by the obstacle is calculated based on the second coordinates and outline of the obstacle. The space occupied by the obstacle is then removed from the initial safe area to obtain an updated safe area. The second coordinate system is the coordinate system of the VR device or the world coordinate system, and the first coordinate is the coordinate in the first coordinate system, which is the coordinate system of the obstacle detection module.

[0128] like Figure 4 As shown, assuming an obstacle is detected and located within the initial safe area S0, the space S1 occupied by the obstacle is calculated based on its coordinates and outline. Space S1 is then removed from the safe area S0, resulting in an updated safe area of ​​S0-S1. When removing space S1, the boundaries of the safe area need to be redefined. These boundaries can be based on the boundaries of space S1, ensuring that the updated safe area does not include space S1.

[0129] In some embodiments, if the VR device only includes a LiDAR module, the first space occupied by the first obstacle can be calculated based on the coordinates and outline of the first obstacle detected by the LiDAR module, and the first space occupied by the obstacle can be removed from the initial safe area to obtain an updated safe area.

[0130] If the obstacle detection module includes both a lidar module and a depth camera, the coordinates and contours of the first obstacle detected by the lidar module may or may not be exactly the same as the coordinates and contours of the second obstacle detected by the depth camera. If the coordinates and contours of the first obstacle detected by the lidar module are not exactly the same as the coordinates and contours of the second obstacle detected by the depth camera, updating the range of the safe area based on the detected obstacles specifically includes:

[0131] The first space occupied by the first obstacle is calculated based on its coordinates and outline. The second space occupied by the second obstacle is calculated based on its coordinates and outline. The union of the first space and the second space is taken as the space occupied by the obstacle. The space occupied by the obstacle is removed from the initial safe area to obtain the updated safe area.

[0132] For example, the LiDAR module detects the coordinates and outline of a first obstacle, and calculates the first space S2 occupied by the first obstacle based on these coordinates and outline. The depth camera detects the coordinates and outline of a second obstacle, and calculates the second space S3 occupied by the second obstacle based on these coordinates and outline. The union of S3 and S2 is then taken, which includes both S3 and S2. This union is removed from the initial safe area to obtain the updated safe area. This method combines the detection results from the LiDAR module and the depth camera to update the safe area, improving detection accuracy, ensuring user safety, and providing a comfortable user experience.

[0133] If the coordinates and contour of the first obstacle detected by the lidar module are exactly the same as the coordinates and contour of the second obstacle detected by the depth camera, it means that the two detected the same obstacle. The first space occupied by the first obstacle can be calculated based on the coordinates and contour of the first obstacle, and the second space occupied by the second obstacle can be calculated based on the coordinates and contour of the second obstacle. The first space or the second space is then removed from the initial safe area to obtain the updated safe area.

[0134] In this embodiment, the LiDAR module scans the user's 360-degree range in real time, while the depth camera, typically facing the user's direction of travel, only captures images in that direction. Therefore, the LiDAR module can first scan for obstacles within the user's 360-degree range. If an obstacle is detected behind the user, the user can be prompted to turn towards it, and the depth camera can then collect further information about the obstacle. This way, the depth camera is only activated when an obstacle is detected behind the user, reducing its power consumption. The VR headset is equipped with an IMU gyroscope, which can collect the user's movement trajectory, orientation, and posture in real time.

[0135] like Figure 8 As shown, in one specific embodiment, the method for updating the safe area of ​​a virtual reality experience includes the following steps:

[0136] Step 201: Initialize the VR device and calibrate the positions of the VR controllers and LiDAR module;

[0137] Step 202: Set the initial safe zone using the VR controller;

[0138] Step 203: Obtain the VR headset posture and VR controller position, and perform scene rendering. If the user selects to render a prompt message, then render the prompt message.

[0139] Step 204: Use the lidar module to scan the user's information within a 360-degree range in real time;

[0140] Step 205: Use the scanning data from the lidar module to determine whether there are obstacles within the safe area. If there are, proceed to step 206; if not, proceed to step 204.

[0141] Step 206: Based on the obstacle's location information, prompt the user to turn their head towards the obstacle;

[0142] Step 207: Calculate the space occupied by the obstacle based on its coordinates and outline, and update the safe zone based on the space occupied by the obstacle;

[0143] Step 208: Determine whether the depth camera is enabled. If the depth camera is enabled, proceed to step 209.

[0144] Step 209: Turn on the depth camera, determine the coordinates and outline of the obstacle based on the depth image captured by the depth camera, determine the space occupied by the obstacle based on the coordinates and outline of the obstacle, and update the safe area based on the space occupied by the obstacle.

[0145] In this embodiment, the LiDAR module, depth camera, and VR controller each have their own coordinate system. The data processing processor in the VR headset needs to transform the coordinate systems of the LiDAR module, depth camera, and VR controller to calculate the coordinate data of each part in a world coordinate system as a second coordinate system. For example, after the LiDAR module detects the coordinates and outline of an obstacle in its own coordinate system, the data processing processor in the VR headset needs to transform the detection data of the LiDAR module to obtain the coordinates and outline of the obstacle in the world coordinate system; after the depth camera detects the coordinates and outline of an obstacle in its own coordinate system, the data processing processor in the VR headset needs to transform the detection data of the depth camera to obtain the coordinates and outline of the obstacle in the world coordinate system; after setting a safe area in the VR controller coordinate system using the VR controller, the coordinates of the safe area need to be transformed to obtain the coordinates of the safe area in the world coordinate system.

[0146] In this embodiment, the safe zone can be updated using a lidar module and a depth camera, avoiding the problem that VR device users cannot see the surrounding environment and therefore cannot perceive surrounding obstacles, thus increasing user safety and improving user experience.

[0147] In some embodiments, multiple pressure sensing units arranged in an array are disposed on the ground of the initial safe area. The areas where different pressure sensing units are located do not overlap, and the areas where all pressure sensing units are located constitute the initial safe area. Each pressure sensing unit is provided with at least one pressure sensor, and the detection of obstacles within the safe area includes:

[0148] Pressure data is obtained by sensing the pressure on the pressure sensing unit using a pressure sensor.

[0149] Candidate pressure sensing units whose pressure data exceeds a preset first threshold are detected. It is then determined whether the user is located in the area where the candidate pressure sensing unit is located. If the user is not located in the area where the candidate pressure sensing unit is located, the area where the candidate pressure sensing unit is located is removed from the initial safe area to obtain an updated safe area.

[0150] This embodiment eliminates the need for a LiDAR module and depth camera, and also avoids using VR controllers to draw an initial safe area. Instead, it utilizes interconnected pressure-sensing units to form the initial safe area, updating the safe area unit by unit. This approach offers strong scalability and high accuracy. Figure 5 and Figure 6 As shown, a single pressure sensing unit is rectangular, such as a square; pressure sensors and vibration sensors are arranged on the pressure sensing unit. The pressure sensors are used to sense the pressure on the pressure sensing unit. There are splicing interfaces around the pressure sensing unit, which can be used for data transmission.

[0151] The pressure sensing units are spliced ​​together to form the initial safe area. The range of the safe area is the range covered by the pressure sensing units, that is, the closed area enclosed by s0-s1-s2-s3.

[0152] When updating the safe zone, the VR headset can be placed at a coordinate origin, which can be a point set on the pressure sensing unit. The position is initialized, and the position and attitude of the headset obtained by the VR headset IMU gyroscope are sent to the processor to calibrate the center position of the VR headset. In this way, the real-time position of the VR headset can be calculated based on the position of the coordinate origin. This real-time position is the coordinate in the coordinate system of the pressure sensing unit.

[0153] If an obstacle is located within the safe zone at the pressure sensing unit Pw, the pressure at Pw will increase. The pressure data sensed by the pressure sensor of pressure sensing unit Pw will exceed a preset first threshold, which can be set as needed. At this point, the VR headset's position information can be used to determine whether the user is located at pressure sensing unit Pw, and further, whether the pressure data sensed by the pressure sensor of pressure sensing unit Pw originates from the user's weight. If the VR headset is not located at pressure sensing unit Pw, it can be determined that the data from pressure sensing unit Pw does not originate from the user's weight but from an obstacle. Therefore, the obstacle can be identified as being located at pressure sensing unit Pw, and pressure sensing unit Pw can be removed from the initial safe zone. This process can be repeated to update the safe zone based on the pressure data from the pressure sensing units.

[0154] Additionally, if the system detects that a user is approaching the boundary of the safe zone or entering an area outside the safe zone, an alarm will be triggered. This can be achieved by emitting an audible alert, using a vibration sensor to emit vibrations, or displaying an alarm message. For example, the vibration sensor might emit a prolonged vibration to alert the VR device user of an obstacle in their vicinity, warning them not to proceed. If the VR device user reaches the boundary of the safe zone, a loud, persistent vibration will be emitted to indicate that the boundary has been reached and the user should step back.

[0155] Furthermore, if the ratio of the updated safe zone area to the initial safe zone area is less than a preset second threshold, the user is prompted to clear the obstacle. The system can alert the user with an audible alert, a vibration alert, or by displaying an alarm message.

[0156] The second threshold can be set as needed, for example, to 50%. If the area occupied by the obstacle exceeds 50% of the initial safe zone, all vibration sensors will emit continuous vibrations, and the VR headset will issue a prompt to "remove the headset and clear the obstacle".

[0157] This disclosure also provides a virtual reality experience security area update device, which is applied to a VR device. The virtual reality experience security area update device includes a memory and an actuator, which is used to execute the method described above, and will not be repeated here.

[0158] In the various method embodiments of this disclosure, the sequence numbers of each step are not intended to limit the order of the steps. For those skilled in the art, any changes in the order of the steps are within the scope of protection of this disclosure without any creative effort.

[0159] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, since the embodiments are basically similar to the product embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the product embodiments.

[0160] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A device for updating a safe area for a virtual reality experience, characterized in that, Applied to VR devices, including: The safe zone setting module is used to set the initial safe zone; The obstacle detection module is used to detect obstacles around the user; A safe zone update module is used to update the extent of the safe zone based on detected obstacles; The obstacle detection module includes: A lidar module is used to emit laser signals in a 360° radius around the user and receive the reflected laser signals. Based on the received laser signals, the coordinates and outlines of obstacles around the user are determined. A depth camera mounted on the VR headset of the VR device is used to capture images of the environment in the direction the VR headset is facing, and to determine the coordinates and outlines of obstacles around the user based on the captured depth images. The obstacle detection module includes a lidar module and a depth camera. The coordinates and contours of the first obstacle detected by the lidar module are not exactly the same as the coordinates and contours of the second obstacle detected by the depth camera. The safe zone update module calculates the first space occupied by the first obstacle based on the position and outline of the first obstacle, calculates the second space occupied by the second obstacle based on the coordinates and outline of the second obstacle, takes the union of the first space and the second space as the space occupied by the obstacle, and removes the space occupied by the obstacle from the initial safe zone to obtain the updated safe zone. The system uses a lidar module to scan for obstacles within a 360-degree range of the user. Once an obstacle is detected behind the user, the system prompts the user to turn their head toward the obstacle and uses a depth camera to collect information about the obstacle.

2. The virtual reality experience security area update device according to claim 1, characterized in that, The lidar module includes M lidars located on the same plane. The centers of the M lidars form a regular M-gon. The scanning range of the M lidars is outside the regular M-gon. The scanning angle of each lidar is not less than 360 / M degrees, where M is an integer greater than or equal to 3.

3. The virtual reality experience security area update device according to claim 2, characterized in that, The lidar module includes four lidars located on the same plane. The centers of the four lidars form a square. The scanning range of each lidar is outside the square. The scanning angle of each lidar is 90 degrees, and the scanning range of different lidars is different.

4. The virtual reality experience security area update device according to claim 1, characterized in that, The safe area update module is specifically used to convert the first coordinates of the obstacle detected by the obstacle detection module into second coordinates in the second coordinate system. If the detected obstacle is determined to be located within the safe area based on the coordinate range of the safe area in the second coordinate system and the second coordinates, the space occupied by the obstacle is calculated based on the second coordinates and outline of the obstacle to obtain the updated safe area. The second coordinate system is the coordinate system of the VR device or the world coordinate system, and the first coordinate is the coordinate in the first coordinate system, which is the coordinate system of the obstacle detection module.

5. The virtual reality experience security area update device according to claim 1, characterized in that, The obstacle detection module also includes: Multiple pressure sensing units are arranged in an array on the ground of the initial safe area. The areas where different pressure sensing units are located do not overlap, and the areas where all pressure sensing units are located constitute the initial safe area. Each pressure sensing unit is provided with at least one pressure sensor for sensing the pressure on the pressure sensing unit. The processor is configured to receive pressure data from each of the pressure sensing units, detect candidate pressure sensing units whose pressure data is greater than a preset first threshold, determine whether the user is located in the area where the candidate pressure sensing unit is located, and if the user is not located in the area where the candidate pressure sensing unit is located, remove the area where the candidate pressure sensing unit is located from the initial safe area to obtain an updated safe area.

6. The virtual reality experience security area update device according to claim 1, characterized in that, Also includes: The first alert module is used to issue an alarm if it detects that a user is approaching the boundary of the safe area or entering an area outside the safe area.

7. The virtual reality experience security area update device according to claim 1, characterized in that, Also includes: The second prompt module is used to prompt the user to clear obstacles if the ratio of the area of ​​the updated safe area to the area of ​​the initial safe area is less than a preset second threshold.

8. A method for updating the safe area of ​​a virtual reality experience, characterized in that, Applied to VR devices, including: Set the initial safe zone; Detect obstacles around the user; update the extent of the safe zone based on the detected obstacles; Detecting obstacles within the safe area includes: Using a lidar module, laser signals are emitted in a 360° radius around the user, and the reflected laser signals are received. The coordinates and outlines of obstacles around the user are determined based on the received laser signals. The VR device uses a depth camera mounted on its VR headset to capture images of the environment in the direction the VR headset is facing, and determines the coordinates and outlines of obstacles around the user based on the captured depth images. If the coordinates and contour of the first obstacle detected by the lidar module are not exactly the same as the coordinates and contour of the second obstacle detected by the depth camera, updating the range of the safe area based on the detected obstacles specifically includes: The first space occupied by the first obstacle is calculated based on the coordinates and outline of the first obstacle, and the second space occupied by the second obstacle is calculated based on the coordinates and outline of the second obstacle. The union of the first space and the second space is taken as the space occupied by the obstacle, and the space occupied by the obstacle is removed from the initial safe area to obtain the updated safe area. The system uses a lidar module to scan for obstacles within a 360-degree range of the user. Once an obstacle is detected behind the user, the system prompts the user to turn their head toward the obstacle and uses a depth camera to collect information about the obstacle.

9. The virtual reality experience security area update method according to claim 8, characterized in that, The step of updating the extent of the safe zone based on detected obstacles includes: The first coordinates of the obstacle detected by the obstacle detection module are converted into second coordinates in the second coordinate system. If the detected obstacle is determined to be within the safe area based on the coordinate range of the safe area in the second coordinate system and the second coordinates, the space occupied by the obstacle is calculated based on the second coordinates and outline of the obstacle. The space occupied by the obstacle is then removed from the initial safe area to obtain an updated safe area. The second coordinate system is the coordinate system of the VR device or the world coordinate system, and the first coordinate is the coordinate in the first coordinate system. The first coordinate system is the coordinate system of the obstacle detection module.

10. The virtual reality experience security area update method according to claim 8, characterized in that, Also includes: An alarm will be triggered if a user is detected approaching the boundary of the safe zone or entering an area outside the safe zone.

11. The virtual reality experience security area update method according to claim 8, characterized in that, Also includes: If the ratio of the updated safe area to the initial safe area is less than a preset second threshold, the user is prompted to clear the obstacle.

12. A virtual reality experience safe area update device, the virtual reality experience safe area update device being applied to a VR device, the virtual reality experience safe area update device comprising: A memory and an actuator, the actuator being used to perform the method as described in any one of claims 8-11.

Images