Virtual Reality Control System
The virtual reality control system addresses user collision issues by using sensors and controllers to manage safety areas and provide collision prevention guides, ensuring safe and seamless locomotion experiences.
Patent Information
- Application Number
- JP2023029925
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2023-01-04
- Filing Date
- 2023-02-28
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2043-02-28
AI Technical Summary
Existing virtual reality systems face issues with user collisions during locomotion modes due to the real position of users not being accurately reflected, leading to conflicts and safety concerns.
A virtual reality control system that includes sensors to detect optical signals, displays to output images, and controllers to manage safety areas and collision prevention guides, ensuring users remain within designated safe zones to prevent collisions.
The system effectively prevents collisions by providing real-time collision prevention guides, enhancing user safety and ensuring smooth locomotion experiences in virtual environments.
Smart Images

Figure 0007577904000001 
Figure 0007577904000002 
Figure 0007577904000003
Abstract
Description
[Technical field]
[0001] The embodiments relate to a virtual reality control system. [Background technology]
[0002] Virtual reality can be a technology that provides users with virtual information added to reality or the state of a target object in the real world as a programmatically generated virtual reality image.
[0003] Such a technology for providing virtual reality may include a technology for generating a virtual space, a virtual character, and a virtual object using a program based on information provided in the real world regarding the state of a target object such as a user or an object, etc. In this technology, information regarding the state of the target object may be obtained using various sensors.
[0004] Virtual reality may include a walk-through method, which tracks the participant's position and moves while matching the real-world movement of a target with the participant's character in the virtual space, and a locomotion method, which moves the character by using a separate controller without moving the participant.
[0005] In progressing the locomotion method in one play space, there is a problem that the characters may move through separate controllers, and accordingly, the real positions of the users are not reflected, resulting in collisions between users. Summary of the Invention [Problem to be solved by the invention]
[0006] The embodiments provide a virtual reality control system that more safely implements locomotion modes in an experience space. [Means for solving the problem]
[0007] The virtual reality control system comprises a sensor configured to detect an optical signal by emitting light at a target object and receiving light; a first display configured to output an image to a first user using a play space; a second display configured to output an image to a second user using the play space; and at least one or more controllers configured to control at least one of the first display and the second display, the controller being configured to assign a first safety area to the first user and a second safety area to the second user to prevent a collision between the first user and the second user; and to output a collision prevention guide if the second user leaves the second safety area. Effect of the Invention
[0008] A virtual reality control system according to an embodiment may prevent collisions between users by outputting collision prevention guides when a user leaves a safety area in locomotion mode, thereby allowing content to be provided more safely. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 illustrates a virtual reality control system according to a first embodiment.
[0010] [Diagram 2] FIG. 1 is a block diagram showing a detection device according to a first embodiment.
[0011] [Diagram 3] FIG. 2 is a block diagram illustrating a server according to the first embodiment.
[0012] [Figure 4] FIG. 2 is a block diagram illustrating an auxiliary computing device according to a first embodiment.
[0013] [Diagram 5] FIG. 1 illustrates a wearable display device according to a first embodiment.
[0014] [Figure 6] FIG. 1 illustrates an example of a virtual reality control system according to a first embodiment.
[0015] [Figure 7] FIG. 2 is a schematic diagram illustrating a method for tracking a target object according to a first embodiment;
[0016] [Figure 8] A figure showing an example of outputting a virtual environment image through a wearable display device according to the first embodiment.
[0017] [Figure 9] FIG. 2 is a diagram illustrating a control method of the virtual reality system according to the first embodiment.
[0018] [Figure 10] FIG. 2 is a diagram showing a locomotion conversion guide according to the first embodiment.
[0019] [Figure 11] FIG. 4 illustrates a conversion map according to the first embodiment.
[0020] [Figure 12] FIG. 2 is a diagram showing a waiting area according to the first embodiment.
[0021] [Figure 13] FIG. 11 is a diagram showing a mode conversion completion state according to the first embodiment.
[0022] [Figure 14] FIG. 2 is a diagram illustrating a control method of the virtual reality system according to the first embodiment.
[0023] [Figure 15]FIG. 2 illustrates a walk-through target area according to a first embodiment.
[0024] [Figure 16] FIG. 2 is a diagram showing a state in which all users have moved into the walk-through target area according to the first embodiment.
[0025] [Figure 17] FIG. 2 illustrates a locomotion transformation map.
[0026] [Figure 18] FIG. 11 is a diagram showing a locomotion conversion completed state according to the first embodiment.
[0027] [Figure 19] FIG. 11 illustrates a method of controlling a virtual reality control system in preparation for a case where a user leaves according to a second embodiment.
[0028] [Figure 20] FIG. 11 is a diagram showing a locomotion map according to the second embodiment.
[0029] [Figure 21] FIG. 13 illustrates a diagram showing a exited user according to a second embodiment.
[0030] [Figure 22] FIG. 11 is a diagram showing a collision prevention guide output on a user's display according to the second embodiment.
[0031] [Figure 23] FIG. 11 is a diagram showing a collision prevention guide output on the display of a user who has left according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0032] The above objects, features and advantages of the present invention will now be more fully described with reference to the accompanying drawings, in which: The invention may, however, be embodied in many different forms, and in the following description, specific embodiments of the invention are shown in the accompanying drawings and will be described in detail.
[0033] In the drawings, the thicknesses of layers and regions have been exaggerated for clarity. Also, when a layer is referred to as being "on" another layer or substrate, it is understood that it can be directly on the other layer or substrate, or that intervening layers may be present. Additionally, like reference numerals are used throughout the drawings to refer to like components having like functionality within the scope of this disclosure.
[0034] Detailed descriptions of related known functions that are deemed to unnecessarily obscure the gist of the present invention will be omitted. Terms including ordinal numbers such as "first", "second", etc. may be used to describe various components, but such components are not limited by these terms. The terms first and second should not be used to impart any order of importance, but are used to distinguish one element from another element.
[0035] Furthermore, in the following description, the use of terms such as "module" and "unit" used to refer to elements is provided merely to facilitate the description of the present invention and does not have any significant meaning in itself.
[0036] Unlike the real world, virtual reality can be an artificial environment that is generated by a program.
[0037] Such virtual reality can generally be classified into virtual reality (VR), which uses a program to generate a virtual space separated from reality and provides an image of the virtual space; augmented reality (AR), which provides a single image by overlaying a virtual image on the real world; and mixed reality (MR), which provides a virtual space and provides an image of the virtual space by mixing the real world and virtual reality.
[0038] In the following description of virtual reality, virtual reality may refer to virtual environments that provide various types of virtual spaces, as well as the VR, AR, and MR mentioned above.
[0039] The virtual reality control system comprises a sensor configured to detect an optical signal by emitting light at a target object and receiving light; a first display configured to output an image to a first user using a play space; a second display configured to output an image to a second user using the play space; and at least one or more controllers configured to control at least one of the first display and the second display, the controller being configured to assign a first safety area to the first user and a second safety area to the second user to prevent a collision between the first user and the second user; and to output a collision prevention guide if the second user leaves the second safety area. The collision prevention guide is displayed on the first display and the second display. The collision avoidance guide is displayed in different formats on the first display and the second display. The controller is configured to control the second display to output a first character corresponding to the first user even if the second user has left the second safety area. The collision prevention guide output on the second display includes a safety area guide that displays the second safety area. The collision avoidance guide output on the second display includes a first ghost image based on first position data of the first user. The collision prevention guide output on the second display includes an exit warning message. The exit warning message is output differently depending on the distance between the first position data and the second position data. The controller is configured to output an audio message when the second user exits the second secure area. The collision prevention guide output on the second display includes ghost images corresponding to position data of all users located in the play space. The controller is configured to output an image captured by a camera mounted on the second display on the second display when a distance between the second user and the second safety area is greater than or equal to a critical distance. The collision avoidance guide output on the first display includes a second ghost image based on second position data of the second user.
[0040] Referring now to FIG. 1, a virtual reality control system 10 for providing virtual reality according to one embodiment will be described.
[0041] FIG. 1 is a diagram illustrating the environment of a virtual reality control system 10 according to one embodiment.
[0042] Referring to FIG. 1 , the virtual reality control system may include a detection device 100, a server 200, an auxiliary computing device 300, a wearable display device 400, and an input device 500.
[0043] According to one embodiment, the detection device 100 may be connected to a server 200 .
[0044] The detection device 100 may obtain the detection data by tracking the target object.
[0045] In one embodiment, a target object may be an object that affects the image output through the wearable display device 400.
[0046] For example, the target object may include at least one of the wearable display device 400, the user, the input device 500, an object positioned near the user, and an object having a reference point or feature point.
[0047] Furthermore, tracking a target object according to one embodiment may mean obtaining data about the position of the target object in the real environment.
[0048] For example, the target object may be tracked to obtain data about a position that changes as the target object moves in the real environment. The target object position data may be obtained at predetermined intervals, but is not limited to this.
[0049] According to one embodiment, the detection device 100 may provide the detection data to the server 200 .
[0050] According to one embodiment, the server 200 may be connected to the detection device 100 and the auxiliary computing device 300 .
[0051] The server 200 may obtain data from devices connected to it.
[0052] According to one embodiment, the server 200 may obtain at least one of the following from the detection device 100: detection data, image data acquired by the detection device 100, and status data of the detection device 100.
[0053] Additionally, according to some embodiments described below, the server 200 may obtain various data.
[0054] According to one embodiment, the server 200 may control the devices connected to it.
[0055] According to one embodiment, the server 200 may control the auxiliary computing device 300 or the wearable display device 400.
[0056] In one example, the server 200 may control the running of a program or application installed on the auxiliary computing device 300. More specifically, the server 200 may control the starting and / or ending of a program or application installed on the auxiliary computing device 300.
[0057] In another example, the server 200 may provide various settings required for the operation of the detection device 100 .
[0058] Further, the server 200 may generate position data for the target object based on the detection data, or generate virtual position data corresponding to the position of the target object in the virtual environment.
[0059] The server 200 may also perform authentication of programs or applications running on the auxiliary computing device 300 .
[0060] The functionality of server 200 in one embodiment is not limited to the functions described above, and in some embodiments, server 200 may be provided to perform a variety of functions.
[0061] Furthermore, the server 200 according to one embodiment is not necessarily provided as a single physical device, but may be provided as multiple devices performing individual functions subdivided from the functions described above.
[0062] For example, the server 200 may be divided into a detection server connected to the detection device 100 and configured to obtain location data based on detection data, an operation server configured to control some of the devices provided in the system, and a licensing server configured to perform authentication of programs or applications executed in at least one of the devices of the virtual reality control system 10, and the relevant functions may be performed by each server.
[0063] Meanwhile, the server 200 may be provided with an input signal or input data based on an input signal obtained from the input device 500 by the auxiliary computing device 300 .
[0064] The input data may include user selection data relating to an object or the like, data relating to motion input via the input device 500 , and aiming data relating to the aiming direction of the input device 500 .
[0065] The auxiliary computing device 300 may be connected to at least one of the detection device 100, the server 200, the wearable display device 400, and the input device 500.
[0066] The auxiliary computing device 300 may calculate virtual location data based on the location data obtained from the server 200.
[0067] Alternatively, the auxiliary computing device 300 may calculate position data of the target object by processing detection data obtained from the detection device 100 or calculate virtual position data.
[0068] The auxiliary computing device 300 may provide images to the user via the wearable display device 400 through a pre-stored program or application.
[0069] Additionally, the auxiliary computing device 300 may provide sound data to be provided via the wearable display device 400.
[0070] According to one embodiment, the auxiliary computing device 300 may obtain images to be provided to the user based on the location data through a pre-installed program or application.
[0071] Furthermore, the auxiliary computing device 300 may obtain input data based on an input signal obtained from the input device 500.
[0072] Furthermore, the auxiliary computing device 300 may obtain an image to be provided to the user by taking into account the obtained input data.
[0073] The wearable display device 400 may be connected to the auxiliary computing device 300.
[0074] The wearable display device 400 may provide an image of the virtual environment to the user.
[0075] The wearable display device 400 may visually output the virtual environment image obtained from the auxiliary computing device 300 to the user.
[0076] Additionally, the wearable display device 400 may output sound data obtained from the auxiliary computing device 300.
[0077] The input device 500 may acquire signals related to a user's input that is to be reflected in the virtual environment.
[0078] The input device 500 may be connected to the auxiliary computing device 300 .
[0079] The input device 500 may provide input signals corresponding to user inputs to the auxiliary computing device 300 .
[0080] The input device 500 may include an acceleration sensor, a gyroscope, a gyro sensor, a microelectromechanical system (MEMS), a geomagnetic sensor, an inertial measurement sensor (IMIU), an optical sensor, a light sensor, a photo sensor, an infrared sensor, a color sensor, a depth sensor, an electromagnetic wave sensor, and the like, for acquiring signals corresponding to the user's movements.
[0081] Additionally, the input device 500 may include buttons, switches, jog shuttles, wheels, and the like for obtaining signals related to user selections.
[0082] Additionally, the input device 500 may be connected to the auxiliary computing device 300 through at least one of wired and wireless communication.
[0083] The input device 500 may also include a communication module for communicating with the auxiliary computing device 300 .
[0084] Although FIG. 1 illustrates input device 500 being connected to auxiliary computing device 300, embodiments are not so limited and input device 500 may be provided in a variety of connection formats as selected.
[0085] For example, the input device 500 may be connected to the server 200 and the wearable display device 400 to provide input signals thereto.
[0086] The above-described virtual reality control system 10 is merely an example for convenience of explanation. The virtual reality control system 10 according to one embodiment is not limited to the configuration and connections shown in FIG. 1 and may be provided in various forms as selected.
[0087] In one example, the auxiliary computing device 300 and the wearable display device 400 may be provided as a single device, in which case operations performed in the auxiliary computing device 300 may be implemented in the wearable display device 400.
[0088] However, in the following description of various embodiments, the above-described virtual reality control system 10 is described as an example for convenience of explanation.
[0089] In the following, with reference to FIG. 2, a detection device 100 according to one embodiment will be described.
[0090] FIG. 2 is a block diagram illustrating a detection device according to one embodiment.
[0091] Referring to FIG. 2, the detection device 100 may include a light emitting element 110 and a sensor 120 .
[0092] The light emitting elements 110 may project a signal onto or near the target object for tracking.
[0093] In one example, the light emitting element 110 may be provided as a light emitting device that projects an optical signal, such as visible light, infrared light, or the like.
[0094] More specifically, the light emitting elements may be provided as visible light emitting diodes (LEDs), infrared LEDs, or the like.
[0095] The sensor 120 may obtain a signal from an external source.
[0096] In one example, the sensor 120 may obtain a signal corresponding to a signal projected from the light emitting element 110 .
[0097] In another example, the sensor 120 may obtain a signal related to light reflected by a marker provided on the target object.
[0098] For example, the sensor 120 may be provided as an image sensor, an optical sensor, an illuminance sensor, a photo sensor, an infrared sensor, a color sensor, a depth sensor, an electromagnetic wave sensor, or the like.
[0099] FIG. 3 is a block diagram illustrating a server 200 according to one embodiment.
[0100] Referring to FIG. 3, the server 200 may include a server communicator 210 , a server storage 220 , a server inputter 230 , a server controller 240 , and a server display 250 .
[0101] The server communicator 210 may be connected to at least one of the detection device 100, the auxiliary computing device 300, the wearable display device 400, and the input device 500 to obtain data therefrom or provide data thereto.
[0102] The server communicator 210 may be connected to at least one of the detection device 100, the auxiliary computing device 300, the wearable display device 400, and the input device 500 through at least one of wired and wireless communication.
[0103] For example, wireless communication may include mobile communication networks such as Wi-Fi (registered trademark) networks, third generation (3G) networks, Long Term Evolution (LTE) networks, 5G networks, and Long Range (LoRA), Wireless Access in Vehicular Environments (WAVE), beacons, ZigBee (registered trademark), Bluetooth (registered trademark), Bluetooth low energy, or the like.
[0104] Additionally, wired communications may include twisted pair cables, coaxial cables, fiber optic cables, or the like.
[0105] The server communicator 210 may be provided as a communication module for providing at least one of wired and wireless communication.
[0106] The server storage 220 may store data therein.
[0107] The server storage 220 may store data obtained from external sources.
[0108] Furthermore, the server storage 220 may store data necessary for the operation of the server 200 .
[0109] For example, server storage 220 may be provided as a magnetic medium, such as a hard disk, a floppy disk, or a magnetic tape; an optical medium, such as a compact disk read-only memory (CD-ROM), a digital versatile disk, or the like; a magneto-optical medium, such as a floptical disk; a read-only memory (ROM), a random access memory (RAM), a flash memory, a solid-state drive (SSD), a CD-ROM, a DVD-ROM, a universal serial bus (USB), or the like.
[0110] Server inputter 230 may obtain a signal corresponding to a user's input.
[0111] The user input may be, for example, a button press, click, touch, or drag.
[0112] The server input 230 may be implemented, for example, as a keyboard, a keypad, a jog shuttle, or a wheel.
[0113] The server controller 240 may control the overall operation of the server 200 .
[0114] For example, the server controller 240 may control the operation of the devices included in the server 200 .
[0115] The server display 250 may output visual data.
[0116] A server display 250 may be provided as a monitor, TV, display panel, or the like, which outputs visual data.
[0117] Additionally, if the server display 250 is implemented as a touch screen, the server display 250 may perform the functions of the server inputter 230 .
[0118] FIG. 4 is a block diagram illustrating an auxiliary computing device 300 according to one embodiment.
[0119] Referring to FIG. 4, the auxiliary computing device 300 may include an auxiliary computing communicator 310 , an auxiliary computing storage 320 , an auxiliary computing inputter 330 , and an auxiliary computing controller 340 .
[0120] The auxiliary computing communicator 310 may be connected to at least one of the server 200, the wearable display device 400, and the input device 500.
[0121] The auxiliary computing communicator 310 may be connected to at least one of the server 200, the wearable display device 400, and the input device 500 through at least one of wired and wireless communication.
[0122] The auxiliary computing communicator 310 may exchange data with at least one of the connected server 200, the connected wearable display device, and the connected input device 500.
[0123] For example, wireless communications may include Wi-Fi networks, 3G networks, LTE networks, 5G networks, and mobile communication networks such as LoRA, WAVE, Beacon, ZigBee, Bluetooth, Bluetooth low energy, and the like.
[0124] Additionally, wired communications may include twisted pair cables, coaxial cables, fiber optic cables, or the like.
[0125] The auxiliary computing communicator 310 may be provided as a communication module for providing at least one of wired and wireless communication.
[0126] The auxiliary computing storage 320 may store data obtained from external sources.
[0127] Additionally, the auxiliary computing storage 320 may store data necessary for the operation of the auxiliary computing device 300 .
[0128] The auxiliary computing storage 320 may also store applications or programs thereon to provide a virtual experience to the user.
[0129] The auxiliary computing input 330 may obtain a signal corresponding to a user's input.
[0130] The user input may be, for example, a button press, click, touch, or drag.
[0131] For example, the auxiliary computing input 330 may be implemented as, for example, a keyboard, a keypad, a jog shuttle, or a wheel.
[0132] The auxiliary computing controller 340 may control the overall operation of the auxiliary computing device 300 .
[0133] FIG. 5 is a diagram illustrating a wearable display device 400 according to one embodiment.
[0134] Referring to FIG. 5, the wearable display device 400 may include a wearable display communicator 410, a wearable display storage 420, a wearable display sensor 430, a wearable display controller 440, a wearable display screen outputter 450, and a wearable display speaker 460.
[0135] The wearable display communicator 410 may be connected to the auxiliary computing device 300.
[0136] The wearable display communicator 410 may be connected to the auxiliary computing device 300 through at least one of wired and wireless communication.
[0137] The wearable display storage 420 may store data therein.
[0138] The wearable display storage 420 may store applications or programs necessary for the operation of the wearable display device 400.
[0139] Additionally, the wearable display storage 420 may store data obtained from external sources.
[0140] The wearable display sensor 430 may acquire signals corresponding to states of the wearable display device 400 and user inputs.
[0141] In one embodiment, the wearable display sensor 430 may include a wearable display motion sensor module 431 and a wearable display sound sensor module 432.
[0142] The wearable display motion sensor module 431 may obtain signals related to the state of the wearable display device 400.
[0143] In one example, the wearable display motion sensor module 431 may obtain rotational data related to the rotation of the wearable display device 400.
[0144] In another example, the wearable display motion sensor module 431 may obtain movement data related to positional movement of the wearable display device 400.
[0145] The wearable display motion sensor module 431 may include an acceleration sensor, a gyroscope, a gyro sensor, a MEMS, a geomagnetic sensor, an IMIU, an optical sensor, an illuminance sensor, a photo sensor, an infrared sensor, a color sensor, a depth sensor, an electromagnetic wave sensor, and the like.
[0146] The wearable display sound sensor module 432 may acquire a signal corresponding to sound input from the outside.
[0147] In one example, the wearable display sound sensor module 432 may be a microphone.
[0148] The wearable display controller 440 may control the overall operation of the wearable display device 400.
[0149] The wearable display screen outputter 450 may output visual data to the user.
[0150] In one example, the wearable display screen outputter 450 may output virtual reality images. In another example, the wearable display screen outputter 450 may output three-dimensional (3D) virtual reality images.
[0151] The wearable display screen outputter 450 may be provided as an image output device such as a liquid crystal display (LCD), ePaper, an LED display, an organic light emitting diode (OLED) display, a curved display, a stereoscope (a 3D display using binocular parallax), or the like.
[0152] The wearable display speaker 460 may output auditory data.
[0153] The wearable display speaker 460 may be implemented as a sound device such as a tuner, player, amplifier, speaker, or the like.
[0154] FIG. 6 is a diagram illustrating an example of a virtual reality control system 10 according to one embodiment.
[0155] Referring to FIG. 6, the virtual reality control system 10 may provide at least one user 800 with a tracking area 600 for a virtual experience.
[0156] Further, in the tracking area 600, the user 800 may be provided with at least one of an auxiliary computing device 300, a wearable display device 400, and an input device 500.
[0157] Furthermore, the target object to be provided to the user 800 may be provided with a marker M.
[0158] For example, when the target object is a wearable display device 400 and an input device 500, the wearable display device 400 and the input device 500 may be provided with markers M of different patterns.
[0159] The pattern in which the markers M are provided is described below.
[0160] Furthermore, the tracking area 600 may be provided with at least one detection device 100 .
[0161] For example, as shown in FIG. 6, a tracking area 600 may be provided with multiple detection devices 100.
[0162] The detection devices 100 may be positioned to be spaced apart from one another at predetermined intervals around the periphery of the tracking area 600 .
[0163] Furthermore, the detection devices 100 may be mounted spaced apart from each other at a predetermined height above the ground.
[0164] Further, the detection device 100 may be arranged to be aimed towards the tracking area 600 .
[0165] The detection device 100 may be fixedly mounted on a pre-mounted frame.
[0166] 6, a frame for installing the detection device 100 may be provided around the tracking area 600. Furthermore, the detection device 100 may be fixedly installed on the frame.
[0167] The detection device 100 may obtain detection data related to the tracking region 600 .
[0168] The sensor 120 included in the detection device 100 may obtain detection data related to at least a portion of the tracking area 600 .
[0169] The detection device 100 may provide the detection data to the server 200 or the auxiliary computing device 300 .
[0170] For example, the detection device 100 may provide the detection data acquired by the sensor 120 to the server 200 .
[0171] The server 200 may obtain real-time position data of the target object based on the detection data.
[0172] As shown in FIG. 6, when multiple detection devices 100 are provided in the tracking area 600, the server 200 or the auxiliary computing device 300 may acquire detection data from the multiple detection devices 100 and acquire current position data of the target object based on the acquired detection data.
[0173] Further, the server 200 or the auxiliary computing device 300 may obtain virtual position data of at least one target object based on the position data of the target object.
[0174] For example, the auxiliary computing device 300 may obtain coordinates in virtual reality that correspond to coordinates included in the position data of the user 800 in the real world as virtual position data of a character in virtual reality corresponding to the user 800.
[0175] The server 200 may provide at least one of the position data and the virtual position data of the target object to the auxiliary computing device 300.
[0176] The auxiliary computing device 300 may calculate virtual position data based on the obtained position data.
[0177] Additionally, the auxiliary computing device 300 may obtain a virtual environment image based on the virtual position data.
[0178] For example, a virtual environment required for the virtual experience may be constructed by a program or application stored in the auxiliary computing storage 320, and the auxiliary computing device 300 may obtain a virtual environment image region from the virtual environment based on the obtained virtual position data. The auxiliary computing device 300 may obtain a virtual environment image associated with the virtual environment image region.
[0179] The auxiliary computing device 300 may provide virtual environment images to the wearable display device 400.
[0180] The wearable display device 400 may output an image of the virtual environment to the user 800.
[0181] Additionally, the server 200 may provide the virtual environment images to the monitoring display device 700 .
[0182] The server 200 may provide the virtual environment images obtained from the auxiliary computing device 300 to a connected monitoring and display device 700 .
[0183] Further, when the server 200 is connected to multiple auxiliary computing devices 300, the server 200 may obtain a virtual environment image from at least one auxiliary computing device 300 among the multiple auxiliary computing devices 300 and provide the obtained virtual reality image to the connected monitoring display device 700.
[0184] For example, the server 200 may obtain, through the server input 230, a selection of the auxiliary computing devices 300 connected to the server 200 from which a virtual environment image is to be acquired, and may provide the virtual environment image acquired from the selected auxiliary computing device 300 to the monitoring display device 700.
[0185] Additionally, the server 200 may obtain virtual position data from the auxiliary computing device 300 and obtain a virtual environment image based on the obtained virtual position data and a pre-defined position of the virtual camera in the virtual environment.
[0186] Additionally, the server 200 may provide the captured virtual reality images to a connected monitoring display device 700 .
[0187] The monitoring and display device 700 may output virtual environment images obtained from the server 200.
[0188] Further, the input device 500 may be provided to be connected to at least one of the server 200, the auxiliary computing device 300, and the wearable display device 400.
[0189] Furthermore, the input device 500 may be provided with at least one marker M.
[0190] The input device 500 may be provided for each user 800 to hold.
[0191] For example, the user 800 may hold the input device 500 in his or her hand.
[0192] According to one embodiment, the server 200 may obtain position data of the input device 500 based on the detection data obtained from the detection device 100. Furthermore, the real-world position data of the input device 500 may include at least one of position data or orientation data of the input device 500 in the tracking area 600.
[0193] The auxiliary computing device 300 may determine an orientation direction of a virtual object corresponding to the input device 500 in the virtual environment based on the position data of the input device 500.
[0194] Furthermore, the auxiliary computing device 300 may obtain a virtual image that takes into account the orientation of a virtual object corresponding to the input device 500 in the virtual environment.
[0195] For example, the auxiliary computing device 300 may obtain a virtual image in which a gun corresponding to the input device 500 is pointed in a direction corresponding to the orientation of the input device 500 in the virtual environment.
[0196] Furthermore, the auxiliary computing device 300 may obtain a virtual image that takes into account the occurrence of an event in accordance with an event occurrence command of the user 800 through the input device 500 in the virtual environment.
[0197] For example, when the user 800 presses a switch on the input device 500, the auxiliary computing device 300 may obtain a virtual image showing a character corresponding to the user 800 firing a gun in a virtual environment.
[0198] FIG. 7 is a schematic diagram illustrating a method for tracking a target object according to one embodiment.
[0199] Referring to FIG. 7, a method for tracking a target object may include acquiring data related to the target object using an external sensor, and determining a position of the target object based on the acquired data related to the target object.
[0200] An example in which the target object is a wearable display device 400 will now be described with reference to FIG.
[0201] Referring to FIG. 7, the target object may be provided with a marker M for identifying the target object.
[0202] A marker M may be provided on the target object to serve as a reference for identifying and tracking the target object.
[0203] In order to track a target object, it is necessary to distinguish between the target object and other devices, and it may be possible to identify the target object by giving it a marker M.
[0204] Furthermore, when multiple target objects are provided, each of the target objects needs to be identified, and for this purpose a marker provided on one object may be distinguishable from another marker M provided on another object.
[0205] For example, markers M provided on one target object may be provided in a different pattern than another marker M provided on another target object.
[0206] Furthermore, the pattern may include various types of patterns, such as a pattern formed by multiple markers M provided at different positions, an optical pattern provided on one display panel, and the like.
[0207] The pattern may be formed by the marker coordinates of the markers M.
[0208] For example, three markers M may be tracked by the detection device 100, and the first marker coordinates MP1-1 to the third marker coordinates MP1-3 may form a triangular pattern, such that a first marker coordinate MP1-1, a second marker coordinate MP1-2, and a third marker coordinate MP1-3 may be obtained as detection data.
[0209] Furthermore, the marker M may be provided as a passive marker that reflects or absorbs the optical signal projected from the light emitting element 110, and an active marker that autonomously emits an optical signal.
[0210] For example, passive markers may include three-dimensional models with light reflective material applied to them, paper with a recognizable code printed on it, reflective tape, and the like.
[0211] Additionally, active markers may include LED modules, radio wave generators, and the like.
[0212] According to one embodiment, the target object may be provided with at least one marker M.
[0213] For example, if the virtual reality control system 10 is to track the position of only one object, then only one marker M may be provided on the target object.
[0214] Furthermore, multiple markers M may be provided on a target object even if the virtual reality control system 10 tracks the position of only one object.
[0215] Furthermore, where the virtual reality control system 10 tracks the positions of multiple target objects, a single target object may be provided with multiple markers M forming a pattern to identify each of the multiple target objects.
[0216] For example, if the target objects whose position is tracked by the virtual reality control system 10 are a wearable display device 400 and an input device 500, the wearable display device 400 may be provided with a first pattern of markers M, and the input device 500 may be provided with a second pattern of markers M.
[0217] The first pattern is different from the second pattern, and the first pattern detected during position tracking may be identified as the wearable display device 400 and the second pattern detected may be identified as the input device 500.
[0218] In the above description, when multiple target objects are provided, the markers M provided on each of the multiple objects are provided to form a pattern to identify each of the multiple objects. However, this embodiment is not limited thereto, and even when a single target object is provided, the markers M provided on the target object may be formed to form a pattern.
[0219] Furthermore, the pattern of markers M provided on the target object may be used to identify the user 800.
[0220] For example, a first pattern may be identified as a wearable display device 400 being worn by a first user and a second pattern may be identified as an input device 500 being held by the first user. Further, a third pattern may be identified as a wearable display device 400 being worn by a second user and a fourth pattern may be identified as an input device 500 being held by the second user.
[0221] To track the target object, the server 200 may obtain data related to the target object from the detection device 100, and obtain detection data related to the position of the target object based on the obtained data. Furthermore, the server 200 may calculate position data of the target object based on the detection data.
[0222] A description is given of a technique in which the detection device 100 provides data related to a target object to the server 200. The light emitting element 110 of the detection device 100 may project a signal into at least a portion of the tracking region 600.
[0223] For example, if the light emitting element 110 is an infrared LED, the light emitting element 110 may project an infrared signal onto at least a portion of the tracking area 600.
[0224] Additionally, the sensor 120 may provide data obtained from external sources to the server 200 .
[0225] In one example, if the sensor 120 is a camera, the sensor 120 may provide the server 200 with an image signal acquired from an external source.
[0226] 7 shows only one sensor 120, the embodiment is not limited thereto. As described in FIG. 6, multiple sensors 120 may be provided, and each of the multiple sensors 120 may provide acquired data to the server 200.
[0227] The server 200 may determine the location of the target object based on the data obtained from the sensor 120 .
[0228] The server 200 may determine whether the data obtained from the sensor 120 includes data related to the marker M. Furthermore, if it is determined that the data obtained from the sensor 120 includes data related to the marker M, the server 200 may identify the target object based on the pattern of the marker M.
[0229] For example, if the data obtained from the sensor 120 includes a first pattern, the server 200 may identify the target object as the wearable display device 400.
[0230] Multiple patterns may exist in the data obtained from one sensor 120, and the server 200 may identify the multiple patterns.
[0231] The patterns may be pre-stored in the server 200, and if the pre-stored pattern is present in the acquired data, the server 200 may determine that a corresponding pattern exists and may identify a target object that corresponds to the pattern.
[0232] The server 200 may determine the location of the target object based on the data obtained from the sensor 120 .
[0233] Meanwhile, a representative point RP associated with each of the pre-stored patterns may be set in the server 200 .
[0234] The representative point RP may be a point that represents the pattern.
[0235] The keypoint RP may lie outside the pattern.
[0236] For example, the representative point RP may be set at a point spaced a predetermined distance from the plane formed by the first marker coordinate MK1-1, the second marker coordinate MK1-2, and the third marker coordinate MK1-3.
[0237] When a pattern based on multiple markers M is provided, coordinate data related to the multiple markers M included in the pattern may be obtained, and the server 200 may obtain a representative point RP representing the pattern as position data of the target object on which the pattern is provided.
[0238] Thus, the server 200 can obtain the location data of the target object, thereby enabling the target object to be tracked.
[0239] The method for tracking the position of the target object is not limited to the above examples, and various types of methods for tracking the position may be used according to choice.
[0240] According to one embodiment, if the sensor 120 is provided as an image sensor, the sensor 120 may acquire an external image and acquire position data related to the target object based on the acquired image.
[0241] In one example, when the sensor 120 shown in FIG. 7 is provided on the wearable display device 400, the sensor 120 may be provided on one side of the wearable display device 400 and oriented in a direction from the inside to the outside of the wearable display device 400 to obtain image data related to the outside of the wearable display device 400.
[0242] Additionally, the wearable display device 400 may provide captured image data to the auxiliary computing device 300.
[0243] According to one embodiment, the wearable display device 400 may provide image data to the auxiliary computing device 300 at predetermined intervals.
[0244] For example, the wearable display device 400 may provide image data to the auxiliary computing device 300 at the same intervals that image data is acquired through the sensor 120.
[0245] The auxiliary computing device 300 may obtain at least one feature point from the acquired image data.
[0246] According to one embodiment, the auxiliary computing device 300 may obtain objects contained in the image data as feature points.
[0247] According to one embodiment, the auxiliary computing device 300 may acquire, as feature points, objects that are larger than a predetermined size from among the objects included in the image data.
[0248] The auxiliary computing device 300 may identify objects included in the image data and obtain, as feature points, objects larger than a predetermined size from among the identified objects. Furthermore, the auxiliary computing device 300 may determine the size of the object based on the number of pixels occupied by the object included in the image data.
[0249] According to one embodiment, the auxiliary computing device 300 may acquire, as feature points, objects of a preset type from among objects included in the image data.
[0250] For example, if a ball-type object is preset, the auxiliary computing device 300 may acquire a ball-type object, such as a baseball, a soccer ball, a basketball, or the like, contained in the image data as a feature point.
[0251] According to one embodiment, the auxiliary computing device 300 may acquire markers included in the image data as feature points.
[0252] The auxiliary computing device 300 may identify markers, such as bar codes, Quick Response (QR) codes, or the like, contained in the image data and capture the markers as feature points.
[0253] Additionally, the auxiliary computing device 300 may determine the locations of feature points contained in the image data.
[0254] The auxiliary computing device 300 may determine at least one of a position change and a size change of the feature point based on image data obtained from the wearable display device 400.
[0255] The auxiliary computing device 300 may determine the direction and distance of movement of the wearable display device 400 based on the direction of position change, position fluctuation, and size fluctuation of the feature points.
[0256] For example, the auxiliary computing device 300 may determine position changes of feature points based on image data obtained from the wearable display device 400.
[0257] The auxiliary computing device 300 may determine the direction and distance of movement of the wearable display device 400.
[0258] The auxiliary computing device 300 may determine the direction and distance of movement of the wearable display device 400 based on the direction of position change, position fluctuation, and size fluctuation of the feature points.
[0259] For example, the auxiliary computing device 300 may compare the positions of feature points included in first image data captured at a first time with the positions of feature points included in second image data captured at a second time later than the first time, and may determine that the wearable display device 400 has moved to the left if the comparison indicates that the feature points in the first image data have moved to the right in the second image data.
[0260] Additionally, the auxiliary computing device 300 may determine the distance a feature point has moved if its position has changed.
[0261] The auxiliary computing device may determine a distance traveled by the feature point based on a number of pixels between a location of the feature point in the first image data and a location of the feature point in the second image data.
[0262] Alternatively, the auxiliary computing device 300 may determine the movement distance of the feature point based on the coordinates of the feature point in the first image data and the coordinates of the feature point in the second image data.
[0263] Also, for example, the auxiliary computing device 300 may determine the direction and distance of movement of the wearable display device based on the size variation.
[0264] The auxiliary computing device 300 may compare the size of the feature points included in first image data captured at a first time with the size of the feature points included in second image data captured at a second time later than the first time, and may determine that the wearable display device 400 has moved to the left if the comparison indicates that the position of the feature points in the first image data has moved to the right in the second image data.
[0265] In response, the auxiliary computing device 300 may track the position of the target object based on the change in the position of the target object relative to a preset initial position.
[0266] FIG. 8 illustrates an example of outputting a virtual environment image 452 through a wearable display device according to one embodiment.
[0267] Referring to FIG. 8, the virtual reality control system 10 may provide a virtual environment image 452 relating to at least a portion of the virtual environment to a user 800 through a wearable display device 400.
[0268] The virtual environment may include a background, a terrain, virtual objects, characters, and the like.
[0269] For example, the virtual environment may be provided with a character corresponding to the user 800 .
[0270] In another example, the virtual environment may be provided with a virtual object, such as a hand or a gun, that corresponds to the input device 500 being held by the user 800.
[0271] In another example, a virtual object may include an object implemented in a virtual environment and may be used by the user 800 during a virtual experience.
[0272] The terrain may be provided at a pre-defined location in the virtual environment.
[0273] Additionally, the terrain may include accessible areas to which or into which the character may move, and inaccessible areas into which the character is not allowed to move.
[0274] The terrain is further described below.
[0275] Additionally, the characters may include non-player characters (NPCs) and user characters provided by applications or programs pre-stored on the auxiliary computing device 300 .
[0276] In a virtual environment, position data of a character or object may be presented as virtual position data.
[0277] On the other hand, the real-world position data may include at least one of the position coordinates and the orientation of the target object.
[0278] For example, the position data may be the position coordinates of a target object located in the tracking region 600 .
[0279] The server 200 may pre-store coordinate values associated with the tracking region 600 .
[0280] The server 200 may pre-store a coordinate system associated with the tracking region 600. The coordinate system may be at least one of a planar coordinate system, a Cartesian coordinate system, a polar coordinate system, a spatial coordinate system, a cylindrical coordinate system, and a spherical coordinate system.
[0281] The server 200 may obtain coordinate values of the target object within the tracking area 600 based on the detection data and a coordinate system associated with the tracking area 600. Furthermore, the server 200 may obtain the obtained coordinate values of the target object within the tracking area 600 as position data.
[0282] In one example, when the detection data is an infrared image, the server 200 may obtain coordinate values of the marker in the tracking area 600 based on the position of the marker corresponding to the target object in the infrared image and the installation position of the detection device 100 that provided the infrared image. Furthermore, the server 200 may determine a pattern formed by the marker based on the coordinate values of the marker in the tracking area 600, and may identify the target object corresponding to the pattern formed by the marker. Furthermore, the server 200 may obtain a representative point RP of the target object based on the pattern formed by the marker and the coordinate values of the marker in the tracking area 600, and obtain the coordinate value of the representative point RP of the target object as position data of the target object.
[0283] The server 200 may provide the location data to the auxiliary computing device 300 .
[0284] The auxiliary computing device 300 may pre-store coordinate values associated with the virtual environment.
[0285] The auxiliary computing device 300 may pre-store a coordinate system associated with the virtual environment, which may be at least one of a planar coordinate system, a Cartesian coordinate system, a polar coordinate system, a spatial coordinate system, a cylindrical coordinate system, and a spherical coordinate system.
[0286] The auxiliary computing device 300 may obtain coordinate values of the target object in the virtual environment based on the position data and a coordinate system associated with the virtual environment.
[0287] For example, the auxiliary computing device 300 may obtain coordinate values in the virtual environment that correspond to the coordinate values included in the real position data, and obtain the obtained coordinate values in the virtual environment as virtual position data.
[0288] The auxiliary computing device 300 may obtain a virtual environment image 452 to be output to the user 800 based on the virtual position data.
[0289] According to one embodiment, the auxiliary computing device 300 may obtain virtual position data of the wearable display device 400 as virtual position data of a virtual camera, and obtain the field of view 451 of the virtual camera based on the virtual position data of the virtual camera and the orientation direction of the virtual camera.
[0290] The auxiliary computing device 300 may obtain the orientation direction of the virtual camera based on the orientation direction included in the position data of the wearable display device 400.
[0291] Furthermore, the auxiliary computing device 300 may acquire a predetermined area in the direction of the virtual camera's orientation as the field of view 451 of the virtual camera.
[0292] Meanwhile, the field of view 451 of the virtual camera may be obtained based on specific virtual position data in the virtual environment as well as virtual position data of the wearable display device 400.
[0293] Additionally, the auxiliary computing device 300 may acquire a virtual environment image 452 corresponding to the field of view 451 of the virtual camera in the virtual environment.
[0294] The auxiliary computing device 300 may provide a virtual environment image 452 to the wearable display device 400.
[0295] The wearable display device 400 may output the captured virtual reality image 452 to the user 800 through the wearable display screen outputter 450.
[0296] Figure 9 is a diagram showing a control method of the virtual reality system according to the first embodiment, Figure 10 is a diagram showing a locomotion conversion guide according to the first embodiment, Figure 11 is a diagram showing a conversion map according to the first embodiment, Figure 12 is a diagram showing a waiting area according to the first embodiment, and Figure 13 is a diagram showing the mode conversion completion state according to the first embodiment.
[0297] 9 to 13, the controller of the virtual reality system 10 according to the first embodiment may control the virtual reality system in a locomotion mode and a walk-through mode. The controller may selectively control the virtual reality system in a locomotion mode and a walk-through mode.
[0298] As used herein, a controller may be at least one or more of server controller 240, auxiliary computing controller 340, and wearable display controller 440. That is, the functions of a controller described below may be performed independently in any one of server controller 240, auxiliary computing controller 340, and wearable display controller 440, or may be performed by a combination of two or more controllers.
[0299] Even if the virtual reality system 10 is controlled in either the locomotion mode or the walk-through mode, the above-described techniques for acquiring a position may be applied as they are.
[0300] In the walk-through mode, the play space may be used freely, and the controller may be controlled to obtain first position data and second position data of a first user and a second user located in the same play space, and then display a second character corresponding to the second user on a first display worn by the first user based on the first position data and the second position data.
[0301] The controller may display a second character on a region of the first display corresponding to the second position data.
[0302] A locomotion mode may also be implemented in the play space, but in the locomotion mode content not related to the first and second position data of a first and second user playing in the same play space may be displayed on the first and second displays, respectively.
[0303] In the locomotion mode, the controller may acquire first position data of a first user and second position data of a second user, but may not display the second character corresponding to the second user on the first display.
[0304] Alternatively, even if the controller displays a second character on the first display, the controller may display the second character as having a distance different from the distance between the first position data and the second position data, in other words, a distance different from the distance in the real play space may be displayed.
[0305] The controller may define a safety area in the locomotion mode. The safety area may be a virtual area in the play space, and multiple safety areas may be set to be separated from each other to prevent safety accidents when multiple users use the virtual reality system. In the locomotion mode, characters corresponding to other users may not be displayed in the play space, and the controller may define and control the safety area accordingly to prevent safety accidents due to collisions.
[0306] The controller may be controlled to allow each user to move freely within the safe area, such as in a walk-through mode, in which case the controller may control the progression of content based on the user's position in space within the safe area.
[0307] In the following, a method for converting from locomotion mode to walk-through mode will be described with reference to Figures 9 to 13.
[0308] The controller may proceed with the locomotion mode (S1100).
[0309] The controller may decide to convert from locomotion mode to walk-through mode based on the user's play.
[0310] When it is determined to convert from the locomotion mode to the walk-through mode, the controller may display a conversion area guide A shown in FIG. 9 (S1200).
[0311] The controller may display a transformation region guide on the locomotion map 2000. The transformation region guide may be displayed on the locomotion map 2000 for recognition by the user. The transformation region guide may be displayed on the locomotion map 2000 in different colors.
[0312] The locomotion map 2000 and the transformation region guide may be displayed on a display worn on the user. When a character moves into the transformation region, the controller may terminate output of the transformation region guide on the display corresponding to the moved character.
[0313] For example, if a first character of a first user has moved into the transformation domain but a second character of a second user has not yet moved into the transformation domain, the transformation domain guide may no longer be displayed on the first user's first display and the transformation domain guide may be displayed on the second user's second display.
[0314] The controller may determine whether or not all characters have moved into the transformation area (S1300).
[0315] Multiple users may move the character through their respective input devices 500. The users may move the character by manipulating the input device 500.
[0316] If all characters have not completed moving into the transformation region, the controller may continue to display the transformation region guide on the corresponding user's display.
[0317] When all characters have completed moving into the transformation area, the controller may display a transformation map 3000 on each user's display as shown in FIG. 10 (S1400).
[0318] The mode in which the controller displays the transformation map 3000 on the display of each user may be defined as a transformation mode, which may be the same mode as the walk-through mode.
[0319] Multiple users 3100 may be located in multiple safe regions 3200 on the transformation map 3000. In locomotion mode, the controller may control to allow users to be located in safe regions 3200 spaced apart from each other to prevent collisions between users, and accordingly, multiple users 3100 can be located apart from each other when the map is transformed into the transformation map 3000.
[0320] That is, multiple safety regions 3200 may be positioned so as to be spaced apart from each other on the transformation map.
[0321] For example, the first safety area 3201 and the second safety area 3203 may be positioned to be spaced apart from each other. The first user 3101 may be located in the first safety area 3201, and the second user 3103 may be located in the second safety area 3203. The first user 3101 and the second user 3103 may be located to be spaced apart from each other by a first distance.
[0322] The controller may display a waiting area 3300 on the transformation map 3000. The controller may display the waiting area 3300 on the displays of multiple users. The waiting area 3300 may be displayed on a particular area of the transformation map 3000.
[0323] The waiting area 3300 may be displayed to be distinguished from other components of the transformation map 3000. The waiting area 3300 may be displayed in a color different from that of the other components of the transformation map 3000. The waiting area 3300 may be a virtual space, or may be a space that exists in virtual reality.
[0324] The controller may display a waiting area guide on each display to enable multiple users to find their way to the waiting area 3300. The waiting area guide may be displayed differently depending on the user's current position. For example, if the user's current position is far away from the waiting area, a waiting area guide of a relatively small size may be displayed, and if the user's current position is close to the waiting area, a waiting area guide of a relatively large size may be displayed. If the user's current position is in a state where the waiting area can be displayed on the user's display, the controller may omit the waiting area guide.
[0325] The controller may display a waiting area guide on the displays of the users, thereby guiding the users to assemble in the waiting area.
[0326] The waiting area 3300 may have an area smaller than that of the transformation map 3000. The controller may display the transformation map 3000 and guide multiple users to the transformation map 3000. The controller may guide multiple users to the transformation map 3000, thereby reducing the distance between multiple users.
[0327] The controller may determine whether or not all characters have completed moving to the waiting area 3300 (S1600).
[0328] The controller may determine whether all users have completed moving to the waiting area 3300. The controller may continue to display the waiting area by using the display if any one of the users has not completed moving to the waiting area 3300.
[0329] The controller may display the waiting area 3300 on the display of a user who has not completed moving to the waiting area 3300, and may not display the waiting area 3300 on the display of a user who has completed moving to the waiting area 3300. The controller may display a guide for moving to the waiting area 3300 on the display of a user who has not completed moving to the waiting area 3300. The controller may display a guide for moving to the waiting area 3300 only for users who have not completed moving to the waiting area 3300, thereby guiding the users to move to the waiting area 3300.
[0330] The controller may reduce the distance between users in the waiting area 3300 by instructing the users to move to the waiting area 3300. When the distance between users in the waiting area 3300 is defined as the second distance, the second distance may be shorter than the first distance. That is, the distance between users located in the safety area 3200 transformed into the transformation map 3000 may be longer than the distance between users in the waiting area 3300.
[0331] The controller may guide the user to move into the waiting area 3300, thereby enabling a more efficient walk-through to be performed. That is, since the walk-through mode is a mode in which the user progresses through the virtual reality story while moving, the controller may prepare to efficiently progress through the walk-through mode by guiding the user to move into the waiting area 3300.
[0332] The controller may control the progression of the conversion mode to display the conversion map and to move the user to a waiting area, so that a realistic mode conversion can be performed and the safety of the user can be ensured when the locomotion mode is converted to the walk-through mode.
[0333] The controller may perform control so as to proceed with the walk-through mode when all characters have completed moving to the waiting area 3300 as shown in FIG. 13 (S1700).
[0334] When all characters have completed moving to the waiting area 3300, the controller may complete the mode change and perform control to proceed with the walk-through mode.
[0335] 14 to 18 are diagrams illustrating the conversion from walk-through mode to locomotion mode in a virtual reality system according to a first embodiment.
[0336] Figure 14 is a diagram showing a control method of the virtual reality system according to the first embodiment, Figure 15 is a diagram showing a walk-through target area according to the first embodiment, Figure 16 is a diagram showing a state in which all users have moved into the walk-through target area according to the first embodiment, Figure 17 is a diagram showing a locomotion transformation map, and Figure 18 is a diagram showing the locomotion transformation completed state according to the first embodiment.
[0337] 14 to 18, a controller of the virtual reality system 10 according to the first embodiment may control the virtual reality control system in a walk-through mode (S4100).
[0338] The controller may display a walk-through map 5000 as shown in Figure 15. The controller may display a target area 5200 on the walk-through map 5000.
[0339] The target area 5200 may be a part of the area of the walk-through map 5000. The target area 5200 may be a trigger area for progressing through a scenario in the walk-through mode.
[0340] The controller may display the target area 5200 on a display worn on the user 5100. The controller may display a guide for guiding the movement to the target area 5200 on a display worn on the user 5100 so that the user 5100 moves to the target area 5200.
[0341] The controller may determine whether all users 5100 have reached the target area 5200 as shown in FIG.
[0342] When all users 5100 are located in the target area 5200, the controller may transform the walk-through map 5000 into a locomotion transformation map 6000 as shown in FIG.
[0343] The controller may display a locomotion guide area 6200 on the locomotion transformation map 6000 (S4200).
[0344] The controller may display a plurality of locomotion guide regions 6200 assigned to each of the plurality of users 5100. The controller may output the locomotion guide region 6200 on the display of each of the plurality of users 5100.
[0345] The locomotion guide region 6200 may be part of the region of the locomotion transformation map 6000 .
[0346] For example, the controller may output a first locomotion guide region 6201 for a first user 5101. The controller may output a second locomotion guide region 6203 for a second user 5103.
[0347] The first locomotion guide region 6201 and the second locomotion guide region 6203 may be regions spaced apart from each other. The first locomotion guide region 6201 and the second locomotion guide region may have a third distance therebetween. The third distance may be longer than the distance between users located in the target region 5200.
[0348] The locomotion guide region 6200 may be assigned based on position data of the user 5100 located in the target region 5200. The locomotion guide region 6200 may be assigned to minimize collisions between users while the user 5100 is moving in the locomotion guide region 6200.
[0349] The controller may first assign the locomotion guide region 6200 to users located on the outer regions among users 5100 located in the target region 5200, and may later assign the locomotion guide region 6200 to users located in the central region.
[0350] The controller may first assign the locomotion guide region 6200 that is the furthest away from the target region 5200 among the locomotion guide regions 6200, and later assign the locomotion guide region 6200 that has a relatively short distance to the target region 5200.
[0351] The controller may assign the locomotion guide region 6200 that is furthest from the target region 5200 among the locomotion guide regions 6200 to the user that is closest to the corresponding locomotion guide region 6200, and may then assign the furthest locomotion guide region 6200 to the user that is closest to the corresponding locomotion guide region 6200.
[0352] For example, the controller may assign a first locomotion guide region 6201, which is the farthest from the target region 5200, to a first user 5101, which has the shortest distance from the first locomotion guide region 6201, and then assign a second locomotion guide region 6203, which is the farthest among the locomotion guide regions excluding the already assigned first locomotion guide region 6201, to a second user 5103, which has the shortest distance from the second locomotion guide region 6203.
[0353] Accordingly, collisions between users moving from the target region 5200 towards the locomotion guide region 6200 can be minimized.
[0354] The locomotion guide region 6200 may be displayed only on the display of the assigned user. The first locomotion guide region 6201 may be displayed only on the display of the first user 5101, and may not be displayed on the display of the second user 5103.
[0355] The locomotion guide region 6200 may no longer be output when the user has completed movement into the assigned locomotion guide region 6200 .
[0356] The controller may output the additional effect 6300 to enable the locomotion guide region 6200 to be more effectively recognized by the user. The additional effect 6300 may be output on the locomotion guide region 6200. In this case, the additional effect 6300 may be displayed on the locomotion guide region 6200 assigned to each user through the display of each user.
[0357] The additional effect 6300 may no longer be output when the user has completed movement into the assigned locomotion guide area 6200 .
[0358] Alternatively, the additional effect 6300 may be output only when the user completes the movement to the assigned locomotion guide region 6200. In this case, the additional effect 6300 may be displayed on the display of all users. Accordingly, the controller may enable the users who have completed the movement to the locomotion guide region 6200 to recognize.
[0359] The controller may identify whether all users have completed moving into the locomotion guide area 6200 (S4300).
[0360] If any one of the users has not completed moving into the locomotion guide region, the controller may continue to output the locomotion guide region 6200.
[0361] The controller may proceed with the locomotion mode when all users have completed moving into the locomotion guide area as shown in FIG. 18 (S4400).
[0362] When walk-through mode is converted to locomotion mode, the controller guides through the locomotion guide region to increase the distance between users greater than in walk-through mode, thereby preventing collisions between users in locomotion mode.
[0363] Figure 19 is a diagram showing a control method of the virtual reality control system in preparation for when a user leaves according to the second embodiment, Figure 20 is a diagram showing a locomotion map according to the second embodiment, Figure 21 is a diagram showing a state in which a user has left according to the second embodiment, Figure 22 is a diagram showing a collision prevention guide output on the user's display according to the second embodiment, and Figure 23 is a diagram showing a collision prevention guide output on the display of a user who has left according to the second embodiment.
[0364] The second embodiment is an additional embodiment related to a collision prevention guide that is output when the user leaves the safety area in the locomotion mode according to the first embodiment. Accordingly, in the description of the second embodiment, the same reference numerals are used for the same components as those in the first embodiment, and detailed descriptions thereof will be omitted.
[0365] Referring to FIG. 19, the controller may control the virtual reality system in locomotion mode (S7100).
[0366] The controller may configure a locomotion map 8000 and may define multiple safe regions 8200, as shown in FIG.
[0367] The safety regions 8200 may be defined as being separated from each other to prevent collisions between the users 8100. For example, a first safety region 8201 may be defined as being separated from a second safety region 8203.
[0368] A user 8100 may progress through content on a safety area 8200 in locomotion mode. For example, a first user 8101 may progress through a locomotion scenario in a first safety area 8201, and a second user 8103 may progress through a locomotion scenario in a second safety area 8203.
[0369] A first character 8101 corresponding to the first user may be output on the display of the second user 8103, and a second character 8103 corresponding to the second user may be output on the display of the first user 8101. However, the first character output on the display of the second user 8103 may not reflect the first position data of the first user 8101. The first character output on the display of the second user 8103 may only reflect data regarding the orientation of the first user 8101. That is, even if the first user 8101 moves within the first safe area 8201, the first character of the first user 8101 may still be shown as standing on the display of the second user 8103. However, if the first user 8101 rotates within the first safe area 8201, the first character may be displayed as rotating on the display of the second user 8103.
[0370] Since the safety area 8200 is an area configured to prevent collisions between the users 8100, there may be a risk of collision between the users if one of the users 8100 leaves the safety area 8200. Accordingly, collision prevention guides are needed to prevent collisions.
[0371] The controller may determine whether one of multiple users present in the play space has left the safe area (S7200).
[0372] For example, if the second user 8103 leaves the second secure area 8203 as shown in FIG. 21, the controller may detect that the second user 8203 has left.
[0373] The controller may continue the locomotion progress if all users 8100 are located within the safety area 8200.
[0374] If any one of all users 8100 leaves the safety area, the controller may output a collision prevention guide (S7300).
[0375] The collision prevention guide may be displayed on the display of the user 8100. The collision prevention guide may be displayed in different formats on the displays of the leaver and the non-leaver.
[0376] The collision prevention guide may be output in different formats depending on the distance of the departing person from the safety area.
[0377] When the second user 8103 leaves the secure area 8203, an image such as that shown in FIG. 22 may be displayed on the display of the first user 8101.
[0378] A character 8300 corresponding to multiple users located on the locomotion map 8000 may be displayed on a first display of a first user 8101. A second character 8303, a third character 8305, and a fourth character 8307 may be displayed on the first display of the first user 8101.
[0379] Even if the second user 8103 has left the second safety zone 8203, the second character 8303 may be output in its previous position.
[0380] The collision prevention guide output on the first display may be a second ghost image 8403 corresponding to the second user 8103. The second ghost image 8403 may be output in a format different from that of the second character 8303. The second ghost image 8403 may be output based on second position data of the second user 8103. The second ghost image 8403 may be output at a position that reflects the real-world position of the second user 8103.
[0381] If the second user 8103 leaves, the controller may output a second ghost image 8403 reflecting the position of the second user 8103 on the first display of the first user 8201 who has not left, thereby preventing a collision between the first user 8101 and the second user 8103.
[0382] The second ghost image 8403 may be output as an image with high brightness so as to be easily recognized by the first user 8101. In this case, the controller may output a voice or image message to the non-exiting users regarding the occurrence of a user leaving the safe area.
[0383] Additionally, if multiple users leave the safe area 8200, the controller may output ghost images on the first display corresponding to the positions of the multiple users who left.
[0384] When the second user 8103 leaves the second secure area 8203, an image such as that shown in FIG. 22 may be displayed on the second display of the second user 8103.
[0385] The controller may display the character 8300, the ghost image 8400, and the safe area guide 8213 on the second display.
[0386] The controller may display a ghost image 8400 and a safe area guide 8213 as collision prevention guides.
[0387] The first character 8301 and the third character 8305 may be output on the second display. The first character 8301 and the third character 8305 may be displayed on a position that does not reflect the user's current position.
[0388] A first ghost image 8401 and a third ghost image 8405 may be displayed on the second display. Ghost images 8400 corresponding to all users present in the play space may be output on the second display.
[0389] The first ghost image 8401 may be an image corresponding to the position data of the first user 8101, and the third ghost image may be an image corresponding to the position data of the third user. The first ghost image 8401 may be output at a position reflecting the real position of the first user 8101. All ghost images 8400 may be output at positions reflecting the respective positions of all users.
[0390] Since the second user 8103 who has left the safe area is likely to collide with other users, a ghost image 8400 reflecting the positions of all users may be output to prevent collisions between users.
[0391] The controller may output a safe area guide 8213 on the second display. The safe area guide 8213 may guide the second user 8103 as to a location to return to. The safe area guide 8213 may be displayed on a location corresponding to the second safety area 8203.
[0392] The controller may output an exit warning message to the second display notifying the second user 8103 that the second user 8103 has exited the safe area. Alternatively, the controller may output a message to the second user 8103 as audio notifying the second user 8103 that the second user 8103 has exited the safe area.
[0393] The controller may output the exit warning message in a different format depending on the second position data of the second user 8103 and the distance between the second safety area 8203. That is, when the second user 8103 leaves the second safety area 8203, the controller may output the exit warning message in a different format depending on a preset critical distance or a distance longer than that.
[0394] Further, the controller may output the impending collision message based on the location data of the second user 8103 and the other users. That is, the controller may output the impending collision message when a risk of collision exists because the second user 8103 is closer to another user within a predetermined range. The impending collision message may be output to users that are within a predetermined range from the second user 8103.
[0395] If the distance between the second user 8103 and the second safety area 8203 is greater than or equal to the critical distance, the controller may output an image different from the locomotion content image. In this case, the controller may control the display of all users to output an image different from the locomotion content image.
[0396] If a camera is installed on the second display of the second user 8103 to capture in the orientation direction of the second user, the controller may control to output the image captured by the camera on the second display. The image captured by the camera may be output on the second display, so that the second user 8103 can recognize the real space and collisions between users can be prevented.
[0397] The controller may determine whether a user who left the safe area has returned to the safe area (S7400).
[0398] The controller may determine whether the second user 8103 has returned to the second safe area 8203.
[0399] The controller may continue to output the collision prevention guide if the second user 8103 has not returned to the safety area 8203.
[0400] The controller may stop outputting the collision prevention guide when the second user 8103 has completed returning to the safety area 8203, thereby controlling the second user 8103 to resume locomotion mode.
[0401] The various embodiments described above may be implemented in a recording medium readable by a computer or similar device by using software, hardware, or a combination thereof. In some circumstances, the embodiments described herein may be implemented by the processor itself. When implemented by software, the embodiments, such as the procedures and functions described herein, may be implemented by separate software modules. Each of the software modules may perform one or more functions and operations described herein.
[0402] Computer instructions for performing the processing operations according to the various embodiments of the present disclosure described above may be stored on a non-transitory computer-readable medium. The computer instructions stored on the non-transitory computer-readable medium, when executed by a processor, may cause a particular device to perform the processing operations according to the various embodiments described above.
[0403] A non-transitory computer-readable medium refers to a medium that stores data semi-permanently, as opposed to storing data very briefly, such as a register, cache, memory, or the like, and is readable by a device. Examples of non-transitory computer-readable media may include a compact disc (CD), a digital versatile disc (DVD), a hard disk, a Blu-ray disc, a universal serial bus (USB), a memory card, a read-only memory (ROM), or the like.
[0404] The storage medium readable by the device may be provided in the form of a non-transitory storage medium. In this specification, the term "non-transitory storage medium" refers to a tangible device and means only that it does not contain a signal (e.g., radio waves), and the term does not distinguish between a case where data is semi-permanently stored in the storage medium and a case where data is temporarily stored. For example, the "non-transitory storage medium" may include a buffer in which data is temporarily stored.
[0405] The method according to various embodiments of the present disclosure may be provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a device-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)) or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). When distributed online, at least a portion of the computer program product (e.g., a downloadable app.) may be at least temporarily stored or temporarily generated in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0406] Although the preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the specific embodiments described above. Various modifications may be made by those skilled in the art without departing from the scope of the present disclosure as claimed in the claims, and the modified embodiments should not be understood as being separated from the technical idea or scope of the present disclosure.
Claims
1. a sensor configured to detect an optical signal by emitting light at a target object and receiving light; a first display configured to output images to a first user using the play space; a second display configured to output images to a second user using the play space; and At least one or more controllers configured to control at least one of the first display and the second display. Equipped with The controller: assigning a first safety region to the first user and a second safety region to the second user to prevent collisions between the first user and the second user; and outputting a collision prevention guide when the second user leaves the second safety area; It is structured as follows: the collision prevention guide is displayed in different formats on the first display and the second display; When the second user leaves the second safety area, the first display outputs a second character corresponding to the second user and a second ghost image as the collision prevention guide based on second position data of the second user; the second display outputs, when the second user leaves the second safety area, characters of all other users, ghost images of all other users, and a safety area guide as the collision prevention guide displaying the second safety area; A virtual reality control system, in which each ghost image of every other user corresponds to the position data of each user.
2. The virtual reality control system of claim 1 , wherein the collision avoidance guide is displayed on the first display and the second display.
3. The virtual reality control system of claim 1 , wherein the collision avoidance guide output on the second display includes an exit warning message.
4. The virtual reality control system of claim 3 , wherein the warning message is output differently depending on a distance between the first position data and the second position data of the first user.
5. The virtual reality control system of claim 1 , wherein the controller is configured to output an audio message if the second user exits the second safe area.
6. 2. The virtual reality control system of claim 1, wherein the controller is configured to output an image captured by a camera mounted on the second display on the second display when a distance between the second user and the second safety area is greater than or equal to a critical distance.
Citation Information
Patent Citations
Virtual reality VR equipment and obstacle avoidance method and device thereof
CN110503001A
Simulation system and program
JP2018081644A
Virtual reality control system
JP2021068209A
Virtual reality safety bounding box
US20180373412A1
Systems and Methods for Collision Avoidance in Virtual Environments
US20190227312A1
