Pose depth data output method and device, VR equipment and readable storage medium

Abstract

The invention discloses a pose depth data output method and device, VR equipment and a readable storage medium, and is applied to the technical field of virtual reality, and the method comprises the steps: outputting the real six-degree-of-freedom pose depth data of a user according to the position data and displacement data of the user in a safe boundary space region; according to the real six-degree-of-freedom pose depth data, judging whether the user triggers a first safety space boundary of the safety boundary space region or not; and if yes, collecting first swing arm image data of the user, and outputting simulated six-degree-of-freedom pose depth data of the user according to the step length data of the user and the first swing arm image data.

Description

Technical Field

[0001] The present invention belongs to the technical field of virtual reality technology, and in particular relates to a method and device for outputting posture depth data, a VR device, and a readable storage medium. Background Art

[0002] When users use VR (Virtual Reality) devices, for example, when moving in a virtual scene, they need to continue to move in one direction in an open space, so as to output continuous posture information moving in a certain direction, in order to achieve the visual effect of continuing to move in a certain direction in the virtual world. In the real world, a safe boundary space area needs to be constructed, such as a safe boundary space area defined in a room. In this safe boundary space area, if the user moves a certain distance in a certain direction, he will soon reach the safe boundary and cannot fully access the virtual space of the VR scene. At this time, the VR device will issue an alarm or enter the perspective function, thereby leaving the virtual scene and being unable to continue the experience.

[0003] To solve the above problems, usually in this case, one solution is to re-define the safe boundary space area, so as to continue to output continuous posture information to the VR application in the new safe boundary space area. However, this solution requires frequent demarcation of safe areas in a small space like a room, which reduces the experience of the VR device. Another solution is to use the remote sensing on the handle to move forward and backward and rotate in all directions, but this will cause dizziness to the user and reduce the experience of the VR device. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, device, VR device and readable storage medium for outputting posture depth data, which aims to improve the user experience of VR devices while continuously outputting posture depth data in a safe boundary space area.

[0005] In order to achieve the above-mentioned object, the present invention provides a posture depth data output method, and the posture depth data output method is applied to VR equipment;

[0006] The posture depth data output method comprises:

[0007] Outputting the user's real six-degree-of-freedom posture depth data according to the user's position data and displacement data within the safety boundary space area;

[0008] Determining, according to the real six-degree-of-freedom posture depth data, whether the user has triggered a first safety space boundary of the safety boundary space area;

[0009] If so, the first arm swing image data of the user is collected, and according to the step length data of the user and the first arm swing image data, the simulated six-degree-of-freedom posture depth data of the user is output.

[0010] Optionally, before the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data, the method includes:

[0011] Collecting second arm swing image data of the user moving from a first position to a second position in the safety boundary space area;

[0012] identifying the second arm swing image data by a preset arm swing algorithm to obtain a second number of steps of the user;

[0013] The step length data of the user is calculated according to the path distance from the first position to the second position and the second number of steps.

[0014] Optionally, the step of identifying the second arm swing image data by a preset arm swing algorithm to obtain the second number of steps of the user includes:

[0015] Identify whether there is an arm in each frame of the second arm swing image data according to the arm swing algorithm, and obtain the number of arm appearances in the second arm swing image data;

[0016] The second number of steps of the user is determined according to the number of times the arm appears in the second arm swing image data.

[0017] Optionally, the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data comprises:

[0018] Identify the first arm swing image data by using the arm swing algorithm to obtain the number of arm appearances in the second arm swing image data;

[0019] Calculating simulated displacement data of the user according to the step length data and the first number of steps;

[0020] Outputting simulated six-degree-of-freedom posture depth data of the user according to the current position data of the user and the simulated displacement data.

[0021] Optionally, according to the position data and displacement data of the user in the safety boundary space area, outputting the real six-degree-of-freedom posture depth data of the user, the method further includes:

[0022] Determine the safe boundary space area based on the collected external environment data;

[0023] A first safety space boundary and a second safety space boundary are divided in the safety boundary space area.

[0024] Optionally, after the step of determining whether the user triggers a first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data, the method further comprises:

[0025] If the first safety space boundary is triggered, prompting the user to stop moving forward and simulate moving forward by swinging the arms;

[0026] If the second safety space boundary is triggered, the output of the real six-degree-of-freedom posture depth data is stopped and the image of the external environment is displayed to the user in the display area of ​​the VR device.

[0027] Optionally, after the step of acquiring the first arm swing image data of the user, the method further comprises:

[0028] Determining whether the user is swinging his arms by identifying arm feature points in the first arm swinging image data;

[0029] If it is determined that the user is swinging his arms, then continue to execute the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swinging image data;

[0030] If it is determined that the user does not swing his arms, then outputting the user's real six-degree-of-freedom posture depth data is stopped.

[0031] In addition, to achieve the above-mentioned purpose, the present invention also provides a posture depth data output device, which is applied to VR equipment;

[0032] The posture depth data output device comprises:

[0033] A real posture output module, used for outputting the real six-degree-of-freedom posture depth data of the user according to the position data and displacement data of the user in the safety boundary space area;

[0034] A boundary judgment module, used for judging whether the user triggers a first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data;

[0035] The simulated posture output module is used to collect the first arm swing image data of the user, and output the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also provides a VR device, which includes: a memory, a processor, and a posture depth data output program stored in the memory and executable on the processor, and the posture depth data output program, when executed by the processor, implements the steps of the posture depth data output method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a pose depth data output program is stored, and when the pose depth data output program is executed by a processor, the steps of the pose depth data output method as described above are implemented.

[0038] The embodiments of the present invention propose a method, device, VR device and readable storage medium for outputting posture depth data. The method outputs the real six-degree-of-freedom posture depth data of the user according to the position data and displacement data of the user in the safe boundary space area; determines whether the user triggers the first safe space boundary of the safe boundary space area according to the real six-degree-of-freedom posture depth data; if so, collects the first arm swing image data of the user, and outputs the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data.

[0039] In this way, when the user triggers the first safety space boundary of the safety boundary space area, the embodiment of the present invention does not need to redefine the safety boundary space area, nor does it need to use remote control to adjust the front, back, left and right. The user only needs to swing his arms in situ to continuously output simulated six-degree-of-freedom posture depth data for simulating forward movement, thereby achieving the purpose of continuing to move forward in the virtual scene without being restricted by the safety space. The embodiment of the present invention improves the user experience of VR devices in a small space, that is, the embodiment of the present invention achieves continuous output of posture depth data in the safety boundary space while improving the user experience of VR devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the device structure of the VR device hardware operating environment involved in the embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the steps of an embodiment of a method for outputting posture depth data of the present invention;

[0042] Figure 3 A schematic diagram of data flow in the first embodiment of the method for outputting posture depth data of the present invention;

[0043] Figure 4 A schematic diagram of the step length data calculation principle of the first embodiment of the method for outputting posture depth data of the present invention;

[0044] Figure 5 A schematic diagram of the arm swing algorithm principle of an embodiment of the method for outputting posture depth data of the present invention;

[0045] Figure 6 This is a schematic diagram of the functional modules of an embodiment of a posture depth data output device of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure of the hardware operating environment of the VR device involved in the embodiment of the present invention.

[0049] like Figure 1 As shown, the VR device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art will understand that Figure 1 The VR device structure shown in the figure does not constitute a limitation of the VR device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0051] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a posture depth data output program.

[0052] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; and the processor 1001 can be used to call the posture depth data output program stored in the memory 1005 and perform the following operations:

[0053] Outputting the user's real six-degree-of-freedom posture depth data according to the user's position data and displacement data within the safety boundary space area;

[0054] Determining, according to the real six-degree-of-freedom posture depth data, whether the user has triggered a first safety space boundary of the safety boundary space area;

[0055] If so, the first arm swing image data of the user is collected, and according to the step length data of the user and the first arm swing image data, the simulated six-degree-of-freedom posture depth data of the user is output.

[0056] Optionally, the processor 1001 may also be used to call a posture depth data output program stored in the memory 1005, and before the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first swing arm image data, further perform the following operations:

[0057] Collecting second arm swing image data of the user moving from a first position to a second position in the safety boundary space area;

[0058] identifying the second arm swing image data by a preset arm swing algorithm to obtain a second number of steps of the user;

[0059] The step length data of the user is calculated according to the path distance from the first position to the second position and the second number of steps.

[0060] Optionally, the processor 1001 may also be used to call a posture depth data output program stored in the memory 1005, and perform the following operations:

[0061] Identify whether there is an arm in each frame of the second arm swing image data according to the arm swing algorithm, and obtain the number of arm appearances in the second arm swing image data;

[0062] The second number of steps of the user is determined according to the number of times the arm appears in the second arm swing image data.

[0063] Optionally, the processor 1001 may also be used to call a posture depth data output program stored in the memory 1005, and perform the following operations:

[0064] Identify the first arm swing image data by using the arm swing algorithm to obtain the number of arm appearances in the second arm swing image data;

[0065] Calculating simulated displacement data of the user according to the step length data and the first number of steps;

[0066] Outputting simulated six-degree-of-freedom posture depth data of the user according to the current position data of the user and the simulated displacement data.

[0067] Optionally, the processor 1001 may also be used to call a posture depth data output program stored in the memory 1005, and before collecting the position data and displacement data of the user in the safety boundary space area and outputting the real six-degree-of-freedom posture depth data, further perform the following operations:

[0068] Determine the safe boundary space area based on the collected external environment data;

[0069] A first safety space boundary and a second safety space boundary are divided in the safety boundary space area.

[0070] Optionally, the processor 1001 may also be used to call a posture depth data output program stored in the memory 1005, and after the step of determining whether the user triggers the first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data, further perform the following operations:

[0071] If the first safety space boundary is triggered, prompting the user to stop moving forward and simulate moving forward by swinging the arms;

[0072] If the second safety space boundary is triggered, the output of the real six-degree-of-freedom posture depth data is stopped and the image of the external environment is displayed to the user in the display area of ​​the VR device.

[0073] Optionally, the processor 1001 may also be used to call a posture depth data output program stored in the memory 1005, and after the step of collecting the first arm swing image data of the user, further perform the following operations:

[0074] Determining whether the user is swinging his arms by identifying arm feature points in the first arm swinging image data;

[0075] If it is determined that the user is swinging his arms, then continue to execute the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swinging image data;

[0076] If it is determined that the user does not swing his arms, then outputting the user's real six-degree-of-freedom posture depth data is stopped.

[0077] Based on the above-mentioned VR device, various embodiments of the method for outputting posture depth data of the present invention are proposed.

[0078] In each embodiment of the posture depth data output method of the present invention, the posture depth data output method of the present invention is applied to a VR device. In one use scenario of the posture depth data output method of the present invention, the VR device collects the position data and displacement data of the user in the safe boundary space area through a camera, thereby outputting the real six-degree-of-freedom posture depth data of the user, and then judges whether the user triggers the first safe space boundary of the safe boundary space area based on the real six-degree-of-freedom posture depth data. If the first safe space boundary is triggered, the first swing arm image data of the user is collected through the camera, and the simulated six-degree-of-freedom posture depth data of the user is output based on the step length data of the user and the first swing arm image data.

[0079] Please refer to Figure 2 , Figure 2 1 is a flow chart of the first embodiment of the method for outputting posture depth data of the present invention. In the first embodiment of the method for outputting posture depth data of the present invention, the method for outputting posture depth data of the present invention comprises:

[0080] Step S10, outputting the real six-degree-of-freedom posture depth data of the user according to the position data and displacement data of the user in the safety boundary space area;

[0081] In the embodiments of this application, refer to Figure 3, the raw environment data 3 (Raw Data 3) can be collected by the camera 1 (Camera) and IMU2 (Inertial Measurement Unit) installed on the VR device, wherein the raw environment data 3 includes the real six-degree-of-freedom pose depth data 4 (Generate Pose Data 4) of the user wearing the VR device in the divided safe boundary space area 5 (SafeBoundary Space 5), and the real six-degree-of-freedom pose depth data includes real-time six-degree-of-freedom pose data 6 (Real-time 6DOF Pose Data 6), and the real-time six-degree-of-freedom pose data 6 includes position data and displacement data. Among them, the position data includes the current position data in the X-axis direction, the position data in the Y-axis direction, and the position data in the Z-axis direction, which correspond to the current positions in the X-axis direction, the Y-axis direction, and the Z-axis direction, respectively, and the displacement data includes the displacement data in the X-axis direction, the displacement data in the Y-axis direction, and the displacement data in the Z-axis direction, which correspond to the current displacement conditions in the X-axis direction, the Y-axis direction, and the Z-axis direction, respectively. The real six-degree-of-freedom posture depth data refers to the user's current real-time 6DOF posture depth in reality, including actual posture depth data in the X-axis direction, actual posture depth data in the Y-axis direction, and actual posture depth data in the Z-axis direction, which correspond to the current 6DOF posture depth in the X-axis direction, the Y-axis direction, and the Z-axis direction, respectively.

[0082] It should be noted that, in the pose depth data output method, the pose depth data includes real six-degree-of-freedom pose depth data and simulated six-degree-of-freedom pose depth data. When the user moves in the safe boundary space area, the user's real six-degree-of-freedom pose depth data can be collected in real time, and the real six-degree-of-freedom pose depth data can be continuously output before the first safe space boundary is triggered.

[0083] Step S20, judging whether the user has triggered a first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data;

[0084] In this embodiment, refer to Figure 3When a user uses a VR device, he or she first needs to define a safe boundary area 5 (SafeBoundary Space 5). The defined safe boundary area 5 will be saved in the form of safe space six-degree-of-freedom posture depth data 16 (6DOFPose Data in Safe Space 16), and then generate a first safe space boundary 17 (Safe-Space Boundary Threshold1 17) and a second safe space boundary 18 (Safe-Space BoundaryThreshold2 18), wherein the two safe space boundaries include multi-dimensional boundary values ​​in the safe boundary space. When it is necessary to determine whether the user triggers the first safe space boundary of the safe boundary space area, a threshold information request 19 (Threshold Info Request 19) can be sent to the module corresponding to the safe boundary area 5, and then the threshold information 20 (Threshold Info 20) returned by the module corresponding to the safe boundary area 5 is received, wherein the threshold information includes multi-dimensional boundary values ​​of the first safe space boundary and the second safe space boundary. Finally, whether the user has touched the first safety space boundary of the safety boundary space area is determined by using the position data in the real six-degree-of-freedom posture depth data and the multi-dimensional boundary value of the first safety space boundary.

[0085] Step S30: If yes, collect the first arm swing image data of the user, and output the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data.

[0086] In this embodiment, it should be noted that when the user triggers the first safety space boundary of the safety boundary space area, it means that the user has moved to the safety boundary area. At this moment, the user is prompted to stop moving forward through the VR device, and the user is prompted to simulate moving forward through the swinging arm action if he needs to continue moving forward, so as to output the corresponding simulated six-degree-of-freedom posture data. Figure 3When the user swings his arms, the image recognition module 12 in the pose data processing unit 11 can be used to identify the arm swinging action in the camera image data 7 to obtain the first number of steps of the user, wherein the camera image data 1 includes the first arm swinging image data of the user, which is the image data of the arm swinging action made by the user when simulating forward movement. Then, the user's step length data and the first number of steps obtained by pre-training are combined for calculation to obtain the user's simulated displacement data. Finally, the simulated six-degree-of-freedom pose depth data is determined based on the simulated displacement data and the user's current position data.

[0087] It should be noted that Figure 3 The continuous 6DOF PoseData 23 is composed of real 6DOF pose depth data 4 and simulated 6DOF pose data, wherein when the user moves within the safety boundary, the continuous 6DOF pose depth data 23 is the superposition of the reference data 10 (Anchor Data 10) and the user's motion displacement on each coordinate axis (X axis, Y axis, Z axis). After the continuous 6DOF pose depth data 23 is generated, it is provided to other modules or applications 25 (Other modules / Applications 25) of the VR device, thereby realizing the continuous output of 6DOF pose depth information for the user within a small safety boundary space, so that the user can access the virtual space uninterruptedly within the safety boundary space, thereby improving the user's experience.

[0088] In an embodiment of the present application, the posture depth data output method of the present invention outputs the real six-degree-of-freedom posture depth data of the user according to the position data and displacement data of the user in the safe boundary space area, and then determines whether the user triggers the first safe space boundary of the safe boundary space area according to the real six-degree-of-freedom posture depth data. If so, the first arm swing image data of the user is collected, and the simulated six-degree-of-freedom posture depth data of the user is output according to the step data of the user and the first arm swing image data.

[0089] In this way, when the user triggers the first safety space boundary of the safety boundary space area, the embodiment of the present invention does not need to redefine the safety boundary space area, nor does it need to adjust the front, back, left and right through remote sensing. The user only needs to swing his arms in place to continuously output simulated six-degree-of-freedom posture depth data for simulating forward movement, thereby achieving the purpose of continuing to move forward in the virtual scene without being restricted by the safety space. The embodiment of the present invention improves the user experience of VR devices in a small space, that is, the embodiment of the present invention achieves continuous output of posture depth data in the safety boundary space while improving the user experience of VR devices.

[0090] Furthermore, based on the first embodiment of the method for outputting posture depth data of the present invention, a second embodiment of the method for outputting posture depth data of the present invention is proposed.

[0091] In the embodiment of the present application, before step S30, the method further includes:

[0092] Step A10, collecting second arm swing image data of the user moving from a first position to a second position in the safety boundary space area;

[0093] In this embodiment, a method for calculating the user's stride data before outputting the user's simulated six-degree-of-freedom posture depth data according to the user's stride data and the first arm swing image data is provided. Figure 3 , the step length data can be obtained by step length calculation module 8 (Step-Length calculation 8). The step length data calculation step can be performed by prompting the user to walk in the safe boundary space area when the user uses the VR device for the first time, and the step length data of the user can be obtained. The stored step length data can be directly called when using it later. Figure 4 The first position and the second position in the safety boundary space area can be prompted by the VR device for the user to walk from one end of the safety boundary space area to the other end, and the second arm swing image data of the whole process is collected through the camera during the walking process, wherein the second arm swing image data is the arm swing image data of the user during the actual walking process.

[0094] Step A20, identifying the second arm swing image data by a preset arm swing algorithm to obtain a second number of steps of the user;

[0095] In the present application, refer to Figure 3 The preset arm-swing algorithm 13 mainly analyzes the number of arm appearances in the second arm-swing image data to determine the second number of steps actually taken by the user between the first position and the second position. Figure 5After the user puts on the VR device, the VR device's perspective faces forward, and due to the swing of the arm, the arm will enter the VR device's perspective during action 1 (raising), and at this time, the arm image appears in the current image frame of the second arm swing image data; during action 2 (lowering), the arm disappears from the VR device's perspective, and at this time, there is no arm image in the current image frame of the second arm swing image data. After obtaining the image frames corresponding to each time point in the second arm swing image data, the number of steps actually taken by the user can be determined by the disappearance and appearance of the arm.

[0096] Step A30: Calculate the stride data of the user according to the path distance from the first position to the second position and the second number of steps.

[0097] For example, Figure 4 As shown, the VR device prompts the user to move from one end of the safe boundary area space to the other end. The moving direction in the figure is along the X-axis, that is, from position 1 to position 3. At this time, according to the pre-defined six-degree-of-freedom posture data of the safe space, the path distance X1-X3 of the user's movement along the X-axis direction can be obtained. Figure 3 , 6DOF Pose Data in Safe Space 15 is the 6DOF Pose Data in Safe Space 16 data requested by the module corresponding to the real 6DOF pose depth data 4 by sending a transfer request 14 to the module corresponding to the safe boundary area 5. After obtaining the second step number of the user and the path distance from the first position to the second position, the step length data of the user can be obtained by calculating the quotient of the path distance and the second step number.

[0098] Further, in a feasible embodiment, the step of identifying the second arm swing image data by a preset arm swing algorithm to obtain the second number of steps of the user may include:

[0099] Step A21, identifying whether there is an arm in each frame image of the second arm swing image data according to the arm swing algorithm, and obtaining the number of arm appearances in the second arm swing image data;

[0100] In this embodiment, the arm swing algorithm is a pre-trained recognition algorithm, which can identify whether each frame image in the second arm swing image data includes an arm image. In each frame image sorted by time, the arm image undergoes a "disappearance-appearance-disappearance" process, which can be recorded as the number of arm appearances + 1, and so on, to obtain the total number of arm appearances in each frame image in the second arm swing image data.

[0101] Step A22, determining the second number of steps of the user according to the number of times the arm appears in the second arm swing image data.

[0102] In this embodiment, the number of second steps actually taken by the user is calculated by the number of times the arm appears, because the swing of the arm and the step taken are positively correlated, and it can be generally considered that the user takes a step when the arm appears once. Therefore, after determining the number of times the arm appears in the second arm swing image data, the number of the user's second steps can be determined.

[0103] In an embodiment of the present application, the posture depth data output method of the present invention first collects the second arm swing image data of the user moving from the first position to the second position in the safety boundary space area, and then identifies the second arm swing image data through a preset arm swing algorithm to obtain the second step number of the user, and then calculates the user's stride data based on the path distance from the first position to the second position and the second step number.

[0104] In this way, when the user uses the VR device, the embodiment of the present invention can calculate the corresponding step length data based on the arm swinging image at that time and the actual forward path distance, so that when the user triggers the first safe space boundary, the user can achieve the purpose of moving forward in the virtual reality scene through the arm swinging action and the determined data without moving.

[0105] Furthermore, based on the above-mentioned second embodiment of the method for outputting posture depth data of the present invention, a third embodiment of the method for outputting posture depth data of the present invention is proposed.

[0106] In an embodiment of the present application, the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data comprises:

[0107] Step S31, identifying the first arm swing image data by the arm swing algorithm to obtain the number of arm appearances in the second arm swing image data;

[0108] Step S32, calculating the simulated displacement data of the user according to the step length data and the first number of steps;

[0109] Step S33: outputting simulated six-degree-of-freedom posture depth data of the user according to the current position data of the user and the simulated displacement data.

[0110] It should be noted that, in this embodiment, the specific implementation steps of obtaining the first number of steps by identifying the first arm swing data through the arm swing algorithm can refer to steps A21 to A22, which are not repeated here. In addition, the simulated displacement data can be obtained by calculating the product of the step length data and the first number of steps. Based on the user's current position data, the user's simulated six-degree-of-freedom posture depth data can be updated through the obtained simulated displacement data.

[0111] Exemplarily, the position data of the user when triggering the first safe space boundary is (x1, y1, z1), and the calculated simulated displacement data is that the user has moved a distance s in the x direction, that is, the output simulated six-degree-of-freedom posture depth data is (x1+s, y1, z1).

[0112] In an embodiment of the present application, the first arm swing image data is identified by the arm swing algorithm to obtain the number of arm appearances in the second arm swing image data, and then the user's simulated displacement data is calculated based on the step length data and the first number of steps, and then the user's simulated six-degree-of-freedom posture depth data is output based on the user's current position data and the simulated displacement data.

[0113] In this way, the embodiment of the present application realizes that the corresponding simulated displacement data can be simulated and generated by the acquired first swing arm image data without the user understanding it in situ, and then the simulated six-degree-of-freedom posture depth data of the user is output, thereby overcoming the constraints of the narrow safety boundary space area on the user's movement in the virtual scene and improving the user's VR device usage experience.

[0114] Furthermore, based on the first embodiment of the method for outputting posture depth data of the present invention, a fourth embodiment of the method for outputting posture depth data of the present invention is proposed.

[0115] In an embodiment of the present application, according to the position data and displacement data of the user in the safety boundary space area, the real six-degree-of-freedom posture depth data of the user is output, and the method further includes:

[0116] Step B10, determining the safety boundary space area according to the collected external environment data;

[0117] Step B20, dividing the safety boundary space area into a first safety space boundary and a second safety space boundary.

[0118] In an embodiment of the present application, a method is provided for collecting external environment data before a user uses a VR device, and dividing a safe boundary space area and a corresponding first safe space boundary and a second safe space boundary according to the collected external environment data.

[0119] For example, refer to Figure 3 , external environment data is collected through camera 1 and inertial sensor 2 to obtain raw environment data 3, and a safe boundary area 5 is obtained based on the raw data 3 and saved in the form of safe space six-degree-of-freedom posture depth data 16, thereby generating a first safe space boundary and a second safe space boundary.

[0120] Further, in a feasible embodiment, after the step of determining whether the user triggers the first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data, the method may further include:

[0121] Step B21, if the first safety space boundary is triggered, prompting the user to stop moving forward and simulate moving forward by swinging the arms;

[0122] Step B22: If the second safety space boundary is triggered, stop outputting the real six-degree-of-freedom posture depth data and display an image of the external environment to the user in the display area of ​​the VR device.

[0123] In this embodiment, it should be noted that when the first safety space boundary is triggered, the user can be prompted to stop moving forward by popping up a visual prompt box in the display area of ​​the VR device or by voice prompting.

[0124] In one possible embodiment, referring to Figure 3 The first safe space boundary is used when the user triggers the safe space boundary, by setting the processing enable state 9 (Processing Enable State 9), so as to output continuous 6DOF pose depth information within the limited safe boundary space. The second safe space boundary is used when the user exceeds the safe space boundary to a certain extent, triggering the out of bounds processing 6 (Out Of Bounds Processing 6). At this time, the VR device can implement the perspective function by calling the preset API 21, so that the user can clearly see the current external environment, thereby achieving the purpose of protecting the user's safety, wherein the degree of triggering the safe space boundary by the second safe space boundary is higher than the degree of triggering the safe space boundary by the first safe space boundary. And the safety protection mechanism after the second safe space boundary is triggered is higher than that of the first safe space. Processing enable state 9 is a mechanism for obtaining continuous 6DOF pose depth information, that is, after the safety protection mechanism is triggered, the mechanism for obtaining continuous 6DOF pose depth information will be closed, that is, the processing enable state 9 is set to a non-enabled state.

[0125] In this embodiment, the posture depth data output method of the present invention determines a safe boundary space area based on the collected external environment data, and divides the safe boundary space area into a first safe space boundary and a second safe space boundary, wherein the first safe space boundary and the second safe space boundary are respectively used to trigger different protection mechanisms.

[0126] In this way, the embodiment of the present invention allows the user to move and walk within a safe range according to the first safe space boundary and the second safe space boundary corresponding to the safe boundary area, thereby realizing movement in the virtual scene. When the first safe boundary is triggered, the user is prompted to simulate a forward movement by swinging his arms, so that the purpose of moving in the virtual scene can be achieved without moving. Compared with the existing technology, there is no need to redivide the safe boundary area or use remote sensing to adjust the viewing direction, which improves the user's experience of using VR equipment.

[0127] Furthermore, based on the first embodiment, the second embodiment, the third embodiment or the fourth embodiment of the pose depth data output method of the present invention, a fifth embodiment of the pose depth data output method of the present invention is proposed.

[0128] In this embodiment, after the step of collecting the first arm swing image data of the user, the method further includes:

[0129] Step C10, determining whether the user is swinging his arms by identifying arm feature points in the first arm swinging image data;

[0130] Step C20, if it is determined that the user is swinging his arms, then continue to execute the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swinging image data;

[0131] Step C30: If it is determined that the user does not swing his arms, stop outputting the user's real six-degree-of-freedom posture depth data.

[0132] In this embodiment, it should be noted that the arm feature points in the first arm swing image data can be identified by the image recognition module. The embodiment of the present application identifies whether an arm appears in the first arm swing image data to determine whether the user is simulating a forward motion and whether there is a need to continue moving forward in the virtual scene; if an arm appears in the first arm swing image data, it means that the user is swinging his arms, that is, simulating a forward motion, and there is a need to continue moving forward in the virtual scene. At this time, the steps in step S30 can be continued: and according to the user's step length data and the first arm swing image data, the simulated six-degree-of-freedom posture depth data of the user is output; if no arm appears in the first arm swing image data, it proves that the user has not made an arm swinging motion, that is, has not simulated a forward motion, and has no need to move forward. At this time, the output of the user's real six-degree-of-freedom posture depth data can be stopped, and the simulated six-degree-of-freedom posture depth data will not be output. At this time, the user's position in the virtual scene is fixed and has not changed, and the six-degree-of-freedom posture depth data will not be continuously output.

[0133] In this embodiment, the posture depth data output method of the present invention determines whether the user is swinging his arms by identifying the arm feature points in the first arm swing image data. If it is determined that the user is swinging his arms, the step of outputting the user's simulated six-degree-of-freedom posture depth data based on the user's stride data and the first arm swing image data is continued. If it is determined that the user is not swinging his arms, the step of outputting the user's real six-degree-of-freedom posture depth data is stopped.

[0134] In this way, the user's current intention is judged by the appearance of the arm in the first arm swing image data, so that the VR device can continue to output or stop outputting six-degree-of-freedom posture depth data according to the user's current wishes, thereby improving the intelligence of the VR device and the user's experience.

[0135] In addition, an embodiment of the present invention further provides a posture depth data output device, and the posture depth data output device of the present invention is applied to VR equipment.

[0136] Please refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of functional modules of an embodiment of a posture depth data output device of the present invention. Figure 6 As shown, the posture depth data output device of the present invention includes:

[0137] The posture depth data output device comprises:

[0138] A real posture output module 10, used to output the real six-degree-of-freedom posture depth data of the user according to the position data and displacement data of the user in the safety boundary space area;

[0139] A boundary judgment module 20, configured to judge whether the user triggers a first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data;

[0140] The simulated posture output module 30 is used to collect the first arm swing image data of the user, and output the simulated six-degree-of-freedom posture depth data of the user according to the step data of the user and the first arm swing image data.

[0141] Optionally, the posture depth data output device of the present invention further includes:

[0142] A step length calculation module, used for collecting second arm swing image data of the user moving from a first position to a second position in the safety boundary space area;

[0143] identifying the second arm swing image data by a preset arm swing algorithm to obtain a second number of steps of the user;

[0144] The step length data of the user is calculated according to the path distance from the first position to the second position and the second number of steps.

[0145] Optionally, the step length calculation module is further used to identify whether there is an arm in each frame image of the second arm swing image data according to the arm swing algorithm, and obtain the number of arm appearances in the second arm swing image data;

[0146] The second number of steps of the user is determined according to the number of times the arm appears in the second arm swing image data.

[0147] Optionally, the simulation position output module 30 is further used to identify the first arm swing image data through the arm swing algorithm to obtain the number of arm appearances in the second arm swing image data;

[0148] Calculating simulated displacement data of the user according to the step length data and the first number of steps;

[0149] Outputting simulated six-degree-of-freedom posture depth data of the user according to the current position data of the user and the simulated displacement data.

[0150] Optionally, the posture depth data output device of the present invention further includes:

[0151] The boundary division module is used to determine the safe boundary space area based on the collected external environment data;

[0152] A first safety space boundary and a second safety space boundary are divided in the safety boundary space area.

[0153] Optionally, the boundary judgment module 20 is further configured to prompt the user to stop moving forward and simulate moving forward by swinging the arm if the first safety space boundary is triggered;

[0154] If the second safety space boundary is triggered, the output of the real six-degree-of-freedom posture depth data is stopped and the image of the external environment is displayed to the user in the display area of ​​the VR device.

[0155] Optionally, the simulated posture output module 30 is further used to determine whether the user is swinging his arms by identifying arm feature points in the first arm swinging image data;

[0156] If it is determined that the user is swinging his arms, then continue to execute the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swinging image data;

[0157] If it is determined that the user does not swing his arms, then outputting the user's real six-degree-of-freedom posture depth data is stopped.

[0158] The specific implementations of the various functional modules of the posture depth data output device of the present invention during operation are basically the same as the various implementations of the posture depth data output method of the present invention mentioned above, and will not be elaborated here.

[0159] The present invention also provides a computer storage medium, on which a posture depth data output program is stored. When the posture depth data output program is executed by a processor, the steps of the posture depth data output program method described in any of the above embodiments are implemented.

[0160] The specific embodiments of the computer storage medium of the present invention are basically the same as the embodiments of the posture depth data output program method of the present invention mentioned above, and will not be described here in detail.

[0161] The present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the posture depth data output method of the present invention as described in any of the above embodiments, which will not be repeated here.

[0162] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0163] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, disk, optical disk) as described above, including a number of instructions for enabling a VR device (which can be a TWS headset, etc.) to execute the methods described in each embodiment of the present invention.

[0165] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for outputting posture depth data, characterized in that: The method for outputting posture depth data is applied to VR equipment; The posture depth data output method comprises: Outputting the user's real six-degree-of-freedom posture depth data according to the user's position data and displacement data within the safety boundary space area; Determining, according to the real six-degree-of-freedom posture depth data, whether the user has triggered a first safety space boundary of the safety boundary space area; If so, the first arm swing image data of the user is collected, and according to the step length data of the user and the first arm swing image data, the simulated six-degree-of-freedom posture depth data of the user is output.

2. The method for outputting posture depth data as claimed in claim 1, characterized in that: Before the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data, the method includes: Collecting second arm swing image data of the user moving from a first position to a second position in the safety boundary space area; identifying the second arm swing image data by a preset arm swing algorithm to obtain a second number of steps of the user; The step length data of the user is calculated according to the path distance from the first position to the second position and the second number of steps.

3. The method for outputting posture depth data as claimed in claim 2, characterized in that: The step of identifying the second arm swing image data by a preset arm swing algorithm to obtain the second number of steps of the user comprises: Identify whether there is an arm in each frame of the second arm-swing image data according to the arm-swing algorithm, and obtain the number of times the arm appears in the second arm-swing image data; The second number of steps of the user is determined according to the number of times the arm appears in the second arm swing image data.

4. The method for outputting posture depth data as claimed in claim 2, characterized in that: The step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data comprises: Recognizing the first arm swing image data by using the arm swing algorithm to obtain a first number of steps of the user; Calculating simulated displacement data of the user according to the step length data and the first number of steps; Outputting simulated six-degree-of-freedom posture depth data of the user according to the current position data of the user and the simulated displacement data.

5. The method for outputting posture depth data as claimed in claim 1, characterized in that: Before the step of outputting the user's real six-degree-of-freedom posture depth data according to the user's position data and displacement data in the safety boundary space area, the method further includes: Determine the safe boundary space area based on the collected external environment data; A first safety space boundary and a second safety space boundary are divided in the safety boundary space area.

6. The method for outputting posture depth data as claimed in claim 5, characterized in that: After the step of determining whether the user has triggered a first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data, the method further includes: If the first safety space boundary is triggered, prompting the user to stop moving forward and simulate moving forward by swinging the arms; If the second safety space boundary is triggered, the output of the real six-degree-of-freedom posture depth data is stopped and the image of the external environment is displayed to the user in the display area of ​​the VR device.

7. The method for outputting posture depth data according to any one of claims 1 to 6, characterized in that: After the step of collecting the first arm swing image data of the user, the method further includes: Determining whether the user is swinging his arms by identifying arm feature points in the first arm swinging image data; If it is determined that the user is swinging his arms, then continue to execute the step of outputting the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swinging image data; If it is determined that the user does not swing his arms, then outputting the user's real six-degree-of-freedom posture depth data is stopped.

8. A posture depth data output device, characterized in that: The device is applied to VR equipment; The posture depth data output device comprises: A real posture output module, used for outputting the real six-degree-of-freedom posture depth data of the user according to the position data and displacement data of the user in the safety boundary space area; A boundary judgment module, used for judging whether the user triggers a first safety space boundary of the safety boundary space area according to the real six-degree-of-freedom posture depth data; The simulated posture output module is used to collect the first arm swing image data of the user, and output the simulated six-degree-of-freedom posture depth data of the user according to the step length data of the user and the first arm swing image data.

9. A VR device, characterized in that: The VR device includes: a memory, a processor, and a posture depth data output program stored in the memory and executable on the processor. When the posture depth data output program is executed by the processor, the steps of the posture depth data output method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a pose depth data output program, and when the pose depth data output program is executed by the processor, the steps of the pose depth data output method as described in any one of claims 1 to 7 are implemented.

Images