Virtual reality collaboration method based on virtual reality technology

Abstract

The invention discloses a virtual reality technology-based virtual-real cooperation method, which comprises the following steps of: setting a first base station, a second base station and a trainer in a physical space, setting a position sensor on the trainer, acquiring initial distance data through the position sensor, and determining the position of a steel cable; constructing a three-dimensional space rectangular coordinate system by taking the first base station and the second base station as coordinates, and performing distance calculation and position calculation between the position sensor and the three-dimensional space rectangular coordinate system; mapping the coordinates of the position sensor to a virtual space to construct a virtual steel cable; applying vertical and horizontal tensile forces to the steel cable in the physical space to perform load-bearing simulation and swing simulation; and determining the position of the dragged rope through a real-time algorithm so as to obtain real-time data of the virtual steel cable. The invention has the beneficial effects that the authenticity of training is ensured, and the safety of training is also provided.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and specifically to a virtual-real collaboration method based on virtual reality technology. Background Art

[0002] During the virtual reality training process, although the pure digital virtual training can expand rich training scenarios and greatly reduce the training cost, it does not combine with the actual product and cannot provide feedback on the feelings generated by actual operations. For example, in aviation training, the descent speed and tension state of the rope hanging on the lifeguard after leaving the cabin need to be perceived during the process of grasping and dragging the steel cable by hand. Such perception cannot be achieved in a pure digital virtual software. The training that combines the virtual world and the real world is the solution we provide for aviation training. However, the virtual-real linkage faces the problems of steel cable positioning and accurate matching of virtual-real data, that is, the position of the steel cable in the virtual world is inconsistent with the position of the steel cable in the real world, resulting in the inability of the training personnel to accurately grasp the steel cable, and the picture of the steel cable being tightened seen during the dragging of the steel cable is inconsistent with the actual situation. To solve this problem, we adopt a combination of sensor positioning and algorithms to accurately position the static steel cable and accurately and real-time draw the deformed shape of the steel cable during the dragging process. Summary of the Invention

[0003] The purpose of the present invention is to provide a virtual-real collaboration method and device based on virtual reality technology to solve the problems proposed in the above background art.

[0004] To achieve the above purpose, the present invention provides the following technical solution: A virtual-real collaboration method based on virtual reality technology, including:

[0005] Set a first base station, a second base station and a trainer in the physical space, set a position sensor on the trainer, and obtain initial distance data through the position sensor to determine the position of the steel cable;

[0006] Construct a three-dimensional space rectangular coordinate system with the first base station and the second base station as coordinates, and perform distance calculation and position calculation between the position sensor and the three-dimensional space rectangular coordinate system;

[0007] Map the coordinates of the position sensor to the virtual space to construct a virtual steel cable;

[0008] Perform load simulation and swing simulation on the steel cable in the physical space by applying vertical and horizontal tensile forces;

[0009] Determine the position of the rope during dragging through a real-time algorithm to obtain real-time data of the virtual steel cable.

[0010] Preferably, the trainer is provided with a hatch, the steel cable is retractably arranged at the top of the hatch, and the initial distance data obtained by the position sensor includes the distances from the position sensor to the hatch, the steel cable, and the training personnel's headset.

[0011] Preferably, the calculation formula for the distance is: d = c × t, where t is the propagation time of the signal from the first base station or the second base station to the position sensor, and c is the propagation speed of the signal in space.

[0012] Preferably, the position calculation equation is:

[0013]

[0014] Among them, the coordinates of the first base station are (x1, y1), the coordinates of the second base station are (x2, y2), and d1 and d2 are the distances from the first base station and the second base station to the position sensor, respectively.

[0015] Preferably, the vertical pulling force is achieved by the power device pulling the steel cable downward, and the horizontal pulling force is achieved by the operator swinging the steel cable by hand.

[0016] Preferably, the position of the rope during dragging is determined by a real-time algorithm to obtain the real-time data of the virtual steel cable, including:

[0017] Obtain the two segments of the line formed by the horizontal pulling force on the steel cable;

[0018] Obtain the position of the operator's hand, and generate a virtual extension line with the position of the operator's hand as the vertex;

[0019] Obtain the swinging trajectory of the virtual extension line.

[0020] Preferably, obtaining the swinging trajectory of the virtual extension line includes:

[0021] Using the formula: Obtain the period of the simple pendulum of the operator's hand position, where T is the period, L is the pendulum length, and g is the acceleration due to gravity;

[0022] Using the formula: Calculate the angular frequency, and substitute to get

[0023] Through the formula: Calculate the law of change of the angle θ of the simple pendulum deviating from the equilibrium position with time, where θ 0 is the initial angle, is the initial phase;

[0024] Through the formula: Calculate the linear velocity of the simple pendulum;

[0025] Acceleration is calculated through the formula: Calculate the acceleration.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] 1. First, the present invention has a technological breakthrough. Currently, there is no dedicated equipment in China for designing aviation winch training using virtual reality technology;

[0028] 2. Such training also uses the method of combining hardware and software for the first time. By adopting the digital twin technology, the position of the real environment is determined using sensors, and then the real world and the virtual world are unified through software algorithms, so that there is a one-to-one correspondence between the virtual and the real. This not only ensures the authenticity of the training but also provides the safety of the training;

[0029] 3. After determining the one-to-one correspondence between the real world and the virtual world, the steel cable is a dynamically changing component, and this component needs to increase the corresponding ability to obtain dynamic data in the virtual and real corresponding worlds. Since the steel cable cannot use sensors for data synchronization, software algorithms are used to simulate the displacement, deformation and other data generated by the steel cable during training, so that the results of digital calculation are highly consistent with the results of real operation. Description of the Drawings

[0030] Figure 1 is the flowchart of the method of the present invention;

[0031] Figure 2 is the schematic diagram of the physical space of the present invention;

[0032] Figure 3 is the virtual steel cable diagram of the present invention. Detailed Embodiments

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] Please refer to Figure 1 , the present invention provides a technical solution: a virtual-real collaboration method based on virtual reality technology, including:

[0035] S100: Set up a first base station, a second base station and a trainer in the physical space, set up a position sensor on the trainer, and obtain the initial distance data through the position sensor to determine the position of the steel cable;

[0036] S200: Construct a three-dimensional rectangular coordinate system with the first base station and the second base station as coordinates, and calculate the distance and position between the position sensor and the three-dimensional rectangular coordinate system;

[0037] S300: Map the coordinates of the position sensor to the virtual space to construct a virtual cable;

[0038] S400: Conduct load-bearing simulation and swaying simulation on the cable in the physical space by applying vertical and horizontal tensile forces;

[0039] S500: Determine the position of the dragged rope through a real-time algorithm to obtain the real-time data of the virtual cable.

[0040] In the embodiments of the present invention, in order to make the rope in the digital world coincide with the rope on the real trainer in space, first, the coordinate systems between the real physical space and the virtual space are made consistent, that is, the digital world coincides with the physical world through the conversion relationship of the coordinate systems. To achieve this process, it is necessary to rely on the position sensor. In VR devices, there is a positioning device, lighthouse, and we also use a position sensor (Tracker) that can be used for positioning. Since the training device remains stationary throughout the training process, we can understand that the coordinates of the trainer are the coordinates of the virtual world. It is only necessary to unify the actual position of the trainer with the position of the helicopter in the virtual space.

[0041] In virtual reality technology, lighthouse refers to a laser-based positioning device used by HTC Vive. It determines the position and orientation of the device in space by emitting laser beams and detecting the reflected light. Its working principle is to use the time-of-flight (Time of Flight) technology or triangulation principle of laser, and work in coordination with supporting sensors (such as sensors on the head-mounted display, handle, etc.) to achieve high-precision positioning and tracking.

[0042] In this solution, lighthouse, as a positioning device in the VR device, is used together with the Tracker to determine the position information of the training device and related components, providing key position data support for achieving the consistency of the coordinate systems between the real physical space and the virtual space and subsequent functions such as cable positioning. It plays a basic positioning reference role in the entire virtual-real collaborative control process, helping the system accurately perceive the spatial positions of each component, thereby ensuring the accuracy and effectiveness of training.

[0043] In the embodiments of the present invention, the trainer is provided with a hatch, the cable is retractably arranged at the top of the hatch, and the initial distance data obtained by the position sensor includes the distances from the position sensor to the hatch, the cable, and the trainer's head-mounted display.

[0044] In an embodiment of the present invention, a hardware coordinate system is determined through a first base station and a second base station. A three-dimensional rectangular coordinate system is established with the line connecting the two base stations as one of the coordinate axes. The position of the position sensor in this physical space is calculated through the above method based on the triangulation principle and distance measurement and calculation.

[0045] In an embodiment of the present invention, when creating a VR space, a VR space coordinate system is set through SteamVR. The worn headset is in this space, and the positions of the two base stations in the virtual world can be obtained reversely inside the VR. According to the corresponding position relationship of the two existing base stations in the virtual space. Now, according to the calculated position of the position sensor in the physical space, a coordinate position can be mapped in the virtual space. Then, based on this position and the relative position between the actual measured sensor and the cable, the position of the cable in the virtual space is determined and displayed. Through actual operation, the position of the virtual cable is basically coincident with the position of the real cable.

[0046] In an embodiment of the present invention, based on the triangulation principle, first, the positions of the two base stations are fixed in the physical scene, and then the Tracker is fixed on the trainer. Relevant data such as the distances from the Tracker to the hatch, the cable, and the starting position of the trainer wearing the headset are measured. Given the position information of the base stations, by measuring information such as the distances or angles between the Tracker and the two base stations, the geometric relationship is used to determine the position of the Tracker, and then the position of the cable is determined. The distance measurement method uses a ranging method based on the signal propagation time.

[0047] In an embodiment of the present invention, the calculation formula for the distance is: d = c × t, where t is the propagation time of the signal from the first base station or the second base station to the position sensor, and c is the propagation speed of the signal in space.

[0048] In an embodiment of the present invention, the position calculation equation is:

[0049]

[0050] Among them, the coordinates of the first base station are (x1, y1), the coordinates of the second base station are (x2, y2), and d1 and d2 are the distances from the first base station and the second base station to the position sensor respectively. Solving this system of equations can obtain the coordinate position of the position sensor.

[0051] The problem of position synchronization between the static real cable and the digital cable is solved. Next, it is necessary to solve the problem that when the trainer drags the cable during the training process, the cable shows a stretched and taut effect in the virtual environment.

[0052] Since the cable itself does not carry positioning data, the cables in the digital world are all calculated in real time. The cable in a static state is a straight line, and the cable in the static state can be obtained through the reel position and the motor position. When performing the training task, the operator is constantly swinging, but the positions of the reel and the motor remain unchanged. Therefore, the position of the digital cable and the position of the real cable need to be shown as inconsistent.

[0053] As Figure 2 , Figure 3 shown, the cable is pulled out from the reel, and the motor acts as an external force to pull the cable to simulate the gravity generated by the rescued person hanging on the cable. When there is turbulence in the system during the training process, an additional external force is required to pull the cable. Thus, the angle formed by the cable in the real environment is a broken line of the two line segments formed by the reel - hand - motor. The end point of this broken line always remains at the position of the motor. On the display screen in the digital space, the person swings continuously along the extension direction of the cable of the reel - hand. When the winch operator pulls the cable, the rescued person should swing freely with the pull of the hand. The display of the digital cable should change continuously with the swing of the person. When the rescued person is lifted, a downward pull is generated due to their own gravity, and a horizontal pull is generated during the rising process due to the left - right swing. In the present invention, the downward pull is achieved through the motor, and the horizontal pull is achieved through the swing of the operator's hand.

[0054] In an embodiment of the present invention, the vertical pull is achieved by the power device pulling the cable downward, and the horizontal pull is achieved by the operator swinging the cable by hand.

[0055] In an embodiment of the present invention, determining the position of the dragged rope through a real - time algorithm to obtain real - time data of the virtual cable includes:

[0056] Obtaining the two line segments formed by the horizontal pull on the cable;

[0057] Obtaining the position of the operator's hand and generating a virtual extension line with the position of the operator's hand as the vertex;

[0058] Obtaining the swing trajectory of the virtual extension line.

[0059] In an embodiment of the present invention, obtaining the swing trajectory of the virtual extension line includes:

[0060] Using the formula: Obtaining the period of the simple pendulum of the operator's hand position, where \(T\) is the period, \(L\) is the pendulum length, and \(g\) is the acceleration due to gravity;

[0061] Using the formula: Calculating the angular frequency, substituting into it, and obtaining

[0062] By the formula: Calculate the law of change of the angle θ of the simple pendulum deviating from the equilibrium position with time, where θ 0 is the initial angle, is the initial phase;

[0063] By the formula: Calculate the linear velocity of the simple pendulum;

[0064] By the formula: Calculate the acceleration.

[0065] According to the above formulas, the position of the rescued person swinging at the end of the steel cable can be calculated. At this time, draw a line segment from the reel to the position of the operator's hand, and then draw a digital steel cable with the position of the operator's hand as the vertex and the position coordinates of the externally suspended rescued person as the end point, so that the correct position of the steel cable on the helicopter during the swing of the steel cable can be displayed in the digital world. Verified by training experiments, the position of this digital steel cable meets the training requirements.

[0066] The method provided by the present invention can also give different instructions to the motor through software, so that the motor generates different directions of pulling force to simulate different forces such as wind and resistance. At the same time, the movement trajectory of the digital human will be calculated according to different wind forces and resistances.

[0067] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A virtual-reality collaboration method based on virtual reality technology, characterized in that: A first base station, a second base station and a trainer are arranged in a physical space, a position sensor is arranged on the trainer, initial distance data is acquired through the position sensor, and the position of the steel cable is determined; A three-dimensional rectangular coordinate system is constructed with the first base station and the second base station as coordinates, and a distance calculation and a position calculation are performed between the position sensor and the three-dimensional rectangular coordinate system; Mapping the coordinates of the position sensor to the virtual space to construct a virtual cable; Apply vertical and horizontal tension to the steel cables in physical space to perform load simulation and sway simulation; The position of the towed rope is determined by a real-time algorithm to obtain real-time data of the virtual wire rope.

2. The virtual-reality collaboration method based on virtual reality technology according to claim 1, characterized in that: The trainer is provided with a door, the steel cable is retractably arranged at the top of the door, and the position sensor obtains initial distance data including the distance from the position sensor to the door, the steel cable and the head display of the trainee.

3. The virtual-reality collaboration method based on virtual reality technology according to claim 2, characterized in that: The calculation formula for distance calculation is: d=c×t, wherein t is the propagation time of the signal from the first base station or the second base station to the position sensor, and c is the propagation speed of the signal in space.

4. The virtual-reality collaboration method based on virtual reality technology according to claim 3, characterized in that: The position calculation equation is: The coordinates of the first base station are (x1, y1), the coordinates of the second base station are (x2, y2), and d1 and d2 are the distances from the first base station and the second base station to the position sensor, respectively.

5. The virtual-reality collaboration method based on virtual reality technology according to claim 4, characterized in that: The vertical pulling force is achieved by a power device pulling the steel cable downward, and the horizontal pulling force is achieved by an operator holding the steel cable and swinging it.

6. A virtual reality collaboration method and device based on virtual reality technology according to claim 5, characterized in that: The method of determining the position of the rope being dragged by a real-time algorithm to obtain real-time data of the virtual steel cable includes: Obtaining the horizontal tension to form two lines of the steel cable; The operator's hand position is obtained, and a virtual extension line is generated with the operator's hand position as the vertex; Get the virtual extension line swing trajectory.

7. A virtual reality collaboration method and device based on virtual reality technology according to claim 6, characterized in that: The step of obtaining the swing trajectory of the virtual extension line includes: Using the formula: Get the period of the operator's hand position simple pendulum, where T is the period, L is the pendulum length, and g is the acceleration due to gravity; Using the formula: Calculate the angular frequency and Bring in, get By formula: Calculate the law of the change of the angle θ of the pendulum's deviation from the equilibrium position over time, where θ0 is the initial angle, is the initial phase; By formula: Calculate the linear velocity of a simple pendulum; By formula: Calculate the acceleration.

Images