An augmented reality (XR)-based hyper-realistic combat training method and system using a real firearm equipped with an XR controller.

Abstract

In an augmented reality (XR)-based hyperrealistic combat training method using a real firearm equipped with an XR controller, the steps include: a service provider server transmitting XR content to a splitter in the XR environment, the splitter transmitting the XR content to projectors in the XR environment so that multiple projectors illuminate the XR environment with the XR content; sensors in the XR environment determining the positions of multiple XR controllers and transmitting the positions of multiple XR controllers to the service provider server; and the service provider server receiving the interaction between the XR controllers and the real firearm from the multiple XR controllers. The process includes the steps of: a server correcting the XR content to match the interaction between the numerous XR controllers and the real gun and the positions of the numerous XR controllers, and transmitting the corrected XR content to a splitter in the XR environment; and the splitter in the XR environment transmitting the received corrected XR content to a projector so that the numerous projectors illuminate the XR environment with the corrected XR content, wherein the interaction between the XR controller and the real gun is characterized in that the XR controller recognizes at least one of the following: whether the trigger of the real gun to which the XR controller is attached is pressed, or whether the magazine has been changed.

Description

Technical Field

[0001] The present invention relates to an extended reality (XR)-based hyper-realistic combat training method and system using a real gun equipped with an XR controller. It is possible to conduct combat training in an XR environment by attaching an XR controller to a real gun used for actual combat training, and it is possible to adjust the difficulty according to the presence or absence of display of the aiming position of the real gun equipped with the XR controller, and to perform combat training. The present invention relates to an extended reality (XR)-based hyper-realistic combat training method and system using a real gun equipped with an XR controller.

Background Art

[0002] The Fourth Industrial Revolution era has arrived, and it has given impetus to the ICT national defense application promotion project in the military training field. In particular, due to the narrow land environment, the combat training space is insufficient, and it is impossible to conduct actual combat training similar to the actual battlefield environment. In the case of a high-risk situation, it is often not possible to actually conduct training. Therefore, combat training using virtual reality (VR) is necessary.

[0003] However, existing combat training systems using virtual reality (VR) require wearing an HMD (Head Mounted Display) for training or participating in the training system with the body fixed, so they cannot be said to be training similar to actual combat. In particular, wearing an HMD on the face not only reduces immersion but also has the disadvantage of requiring separate equipment. Also, the point that the equipment actually used cannot be used as it is must be improved.

[0004] Prior art includes Korean Published Patent No. 10-2022-0097352 (Augmented Reality-Based Virtual Training Provision Method and System), but it only provides a virtual training content provision method and a virtual training content provision system that include the steps of acquiring actual training content, processing virtual training content using the actual training content, transmitting the virtual training content to a user, and reproducing the virtual training content. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Korean Published Patent Publication No. 10-2022-0097352 [Overview of the project] [Problems that the invention aims to solve]

[0006] The problem that this invention aims to solve is that it was devised to solve the problems of the conventional technology described above. It enables combat training in an XR environment by attaching an XR controller to a real firearm that a trainee would normally use, and the difficulty of the combat training can be adjusted depending on whether or not the aiming position of the real firearm with the XR controller attached is displayed. [Means for solving the problem]

[0007] An augmented reality (XR)-based hyperrealistic combat training method using a real gun equipped with an XR controller according to an embodiment of the present invention includes the steps of: a service provider server transmitting XR content to a splitter in an XR environment, the splitter transmitting the XR content to projectors in the XR environment so that multiple projectors illuminate the XR environment with the XR content; sensors in the XR environment determining the positions of multiple XR controllers and transmitting the positions of multiple XR controllers to the service provider server; and the service provider server receiving the interaction between the XR controllers and the real gun from the multiple XR controllers. The process includes the steps of: a server providing XR controllers correcting the XR content to match the interaction between the numerous XR controllers and the real guns and the positions of the numerous XR controllers, and transmitting the corrected XR content to a splitter in the XR environment; and the splitter in the XR environment transmitting the received corrected XR content to a projector so that the numerous projectors illuminate the XR environment with the corrected XR content. The interaction between the XR controllers and the real guns is characterized in that the XR controllers recognize at least one of the following: whether the trigger of the real gun to which the XR controller is attached is pressed, or whether the magazine has been replaced.

[0008] An augmented reality (XR)-based hyperrealistic combat training system using a real firearm equipped with an XR controller according to an embodiment of the present invention includes an XR environment that can determine the locations of multiple XR controllers and emit XR content received from a service provider server, and a service provider server that generates and transmits XR content to the XR environment, receives the locations of multiple XR controllers from the XR environment, recognizes the interaction between the XR controllers and the real firearm, and corrects the XR content in accordance with the locations of the multiple XR controllers and the interaction between the XR controllers and the real firearm. [Effects of the Invention]

[0009] According to one embodiment of the present invention, trainees can attach an XR controller to a real firearm they were originally using, enabling them to conduct combat training in an XR environment.

[0010] Furthermore, the difficulty of combat training can be adjusted depending on whether or not the aiming position of the real firearm equipped with the XR controller is displayed. [Brief explanation of the drawing]

[0011] [Figure 1] This is a flowchart illustrating an augmented reality (XR)-based hyper-realistic combat training method using a real firearm equipped with an XR controller according to an embodiment of the present invention. [Figure 2] This is a diagram illustrating an augmented reality (XR)-based hyper-realistic combat training system using a real firearm equipped with an XR controller according to an embodiment of the present invention. [Figure 3] This is a diagram showing a real firearm with an XR controller attached. [Figure 4] This image shows a real firearm equipped with an XR controller being used to aim at a target within an XR environment. [Figure 5] This diagram shows a real firearm with an XR controller attached and the internal structure of the XR controller. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described in detail with reference to the drawings.

[0013] Figure 1 is a flowchart illustrating an augmented reality (XR)-based hyper-realistic combat training method using a real firearm equipped with an XR controller according to an embodiment of the present invention.

[0014] Referring to Figure 1, the augmented reality (XR)-based hyper-realistic combat training method using a real gun equipped with an XR controller first involves a service server 200 transmitting XR content to a splitter 130 in the XR environment 100, the splitter 130 transmitting the XR content to projectors 140 in the XR environment 100, and multiple projectors 140 illuminating the XR environment 100 with the XR content (S101). The splitter 130 is a device that separates specific signals according to their characteristics, and can separate and transmit the XR content received from the service server 200 to multiple projectors 140. Since the XR environment 100 consists of a total of five surfaces, including the floor and four surfaces, it is difficult to realistically represent 360° images on the five surfaces without using multiple projectors 140. However, the XR environment 100 is not necessarily limited to five surfaces, and in one embodiment it may consist of six surfaces including the ceiling.

[0015] The sensors 110 of the XR environment 100 determine the positions of multiple XR controllers 400 and transmit the positions of the multiple XR controllers 400 to the service provider server 200 (S103). The multiple XR controllers 400 can be attached to a real firearm 300 to indicate the position of the real firearm. The sensors 110 of the XR environment 100 may be numerous position sensors, and the multiple sensors 110 can calculate the position of each XR controller 400 as coordinate values. The sensors 110 can also determine the elevation of the XR controllers 400. Depending on the elevation of the XR controllers 400, it is possible to determine whether the trainee participating in the training, carrying the real firearm 300 with the XR controllers 400 attached, is sitting or standing and aiming at a target. In addition, when determining the position of the XR controllers 400, the lidar 120 of the XR environment 100 can be used to additionally determine the position of the XR controllers 400. The lidar 120 is a LiDAR (Light Detection And Ranging) and may mean a laser rangefinder that has the technology to scan by emitting laser pulses and measuring the time it takes for them to return. The lidars 120 of the XR environment 100 may, but are not limited to, be located on each of the five faces of the XR environment 100.

[0016] The service provider server 200 receives information about the interaction between the XR controllers 400 and the real guns 300 from a number of XR controllers 400 (S105). Here, the interaction between the XR controller 400 and the real guns 300 may be at least one of the following: whether the trigger of the real gun 300 to which the XR controller 400 is attached is pressed, or whether the magazine of the real gun 300 to which the XR controller 400 is attached has been replaced, but is not necessarily limited to these. In one embodiment, if the XR controller 400 recognizes that the trigger of the real gun 300 is pressed, the XR controller 400 can transmit the relevant interaction to the service provider server 200.

[0017] The service providing server 200 corrects the XR content according to the interaction between the multiple XR controllers 400 and the real firearm 300 and the positions of the multiple XR controllers 400 (S107), and transmits the corrected XR content to the splitter 130 of the XR environment 100 (S109). The splitter 130 of the XR environment 100 transmits the received corrected XR content to the projector 140, and the multiple projectors 140 irradiate the corrected XR content onto the XR environment 100 (S111).

[0018] FIG. 2 is a configuration diagram for explaining a virtual reality (VR)-based hyper-realistic combat training system using a real firearm equipped with an XR controller according to an embodiment of the present invention.

[0019] Referring to FIG. 2, a virtual reality (VR)-based hyper-realistic combat training system 10 using a real firearm equipped with an XR controller 400 includes an XR environment 100, a service providing server 200, a real firearm 300, and an XR controller 400.

[0020] [[ID=Id=12]]The XR environment 100 means an environment in which XR content is reproduced and at least one trainee participating in combat training can train. It may be composed of five surfaces including a floor and four sides, but is not necessarily limited thereto. The XR environment 100 includes a sensor 110, a lidar 120, a splitter 130, and a projector 140.

[0021] The sensor 110 refers to a position sensor, and a number of sensors are provided within the XR environment 100 to determine the position of the XR controller 400. The sensor 110 can calculate the positions of a number of XR controllers 400 as coordinate values respectively and transmit them to the service providing server 200. Also, since the sensor 110 can calculate the height and low of the XR controller 400 as coordinate values, it is possible to determine whether the trainer holding the real gun 300 with the XR controller 400 mounted is sitting or standing. The sensor 110 can not only determine the position of the XR controller 400 to grasp the position of the trainer, but also calculate the aiming direction and position of the real gun 300 with the XR controller 400 mounted.

[0022] The lidar 120 can be a laser rangefinder having a technology that emits laser pulses and measures the time it takes to return for scanning. The position of the XR controller 400 is estimated using the sensor 110, but additionally, the position of the XR controller 400 can be grasped more accurately using the lidar 120. The lidar 120 may be provided one by one on each of the five surfaces of the XR environment 100, but is not necessarily limited to this.

[0023] The splitter 130 is a device that separates a specific signal according to its characteristics, and can separate the XR content received from the service providing server 200 and transmit it to a number of projectors 140, and a number of splitters 130 can be used. Since the XR environment 100 can represent a 360° video using five surfaces, the XR content must be separated using the splitter 130 and transmitted to the projector 140. The projector 140 can irradiate the XR environment 100 with the XR content received from the splitter 130, and a number of projectors may be used.

[0024] The service provision server 200 consists of an XR content unit 210, an XR controller position unit 220, an XR controller interaction unit 230, a communication unit 240, and a control unit 250.

[0025] The XR content unit 210 consists of a content generation module 211 and a content correction module 212. The content generation module 211 can generate XR content for various situations similar to actual combat training. In one embodiment, it can generate XR content for a situation in which an enemy is lying in wait and being attacked, and it can also generate XR content for training in a situation where visibility is poor due to a lot of fog. Furthermore, the content generation module 211 can generate combat training content divided into three modes, a first mode, a second mode, and a third mode, according to difficulty level. Here, the difficulty level may be distinguished according to the XR content of the aiming position of the real gun 300 to which the XR controller 400 is attached, or whether or not it is displayed in the XR environment 100, and one XR content can be adjusted and generated for three difficulty levels. In one embodiment, when the content generation module 211 generates XR content A for hitting a target object 10m away, XR content A can be generated as XR content for a total of three modes: the first mode, the second mode, and the third mode.

[0026] The first mode is the easiest of the three modes, and it generates a sighting position within the XR content according to the muzzle direction of the real firearm 300 to which the XR controller 400 is attached. Using the first mode, trainees can understand exactly where they are aiming and can grasp the precise sighting position, making it suitable for use in the early stages of combat training. At this time, the sighting position can be calculated from the service provider server 200 based on the position of the XR controller 400 recognized by the sensor 110 of the XR environment 100.

[0027] The second mode is the intermediate difficulty of the three modes, and it is a mode in which a laser 340 attached to a real firearm 300, which is equipped with an XR controller 400, is generated within the XR environment 100, indicating the aiming position according to the direction of the muzzle of the real firearm 300. In the second mode, the light of the laser 340 can only be seen within the XR environment 100, and the aiming position is not generated within the XR content. Therefore, trainees using the second mode must turn on the laser 340 attached to the real firearm 300 before starting the training.

[0028] The third mode is the most difficult of the three modes, in which the aiming position corresponding to the muzzle direction of the real firearm 300 with the XR controller 400 attached is not generated within the XR content and XR environment 100. Trainees participate in the training without knowing whether they are aiming accurately, and because it is the most similar to a real situation, it is the mode for the final step in combat training.

[0029] The content correction module 212 can correct the XR content to match the interaction between the XR controller 400 and the real gun 300, as well as the positions of multiple XR controllers 400. The content correction module 212 can receive the interaction between the XR controller 400 and the real gun 300 from the XR controller interaction unit 230 and the position of the XR controller 400 from the XR controller position unit 220 to correct the XR content. In one embodiment, if the interaction received by the content correction module 212 is "magazine change required," the XR content can be corrected so that no more bullets are fired from the real gun 300 attached to the XR controller. In another embodiment, if the interaction received by the content correction module 212 is "trigger pressed," the XR content can be corrected so that a bullet is fired to the part corresponding to the aiming position of the real gun 300 attached to the XR controller. The aiming position can be determined based on the position of the XR controller 400. Furthermore, the content correction module 212 can correct the XR content to match the image of the XR content moving in the corresponding direction if the positions of the numerous XR controllers 400 are moving forward. The content correction module 212 can then transmit the corrected XR content to the splitter 130 of the XR environment 100.

[0030] The XR controller position unit 220 can receive the position of the XR controller 400 from the sensor 110. It can also receive additional positions of the XR controller 400 from the rider 120. The XR controller position unit 220 can receive the positions of multiple XR controllers 400 from the sensor 110 and the rider 120 as coordinate values ​​within the XR environment 100. The XR controllers 400 are attached to the real firearms 300 of multiple trainees, and one XR controller 400 may represent the position of one trainee. Since the XR content for combat training is content for training multiple trainees, and multiple XR controllers 400 can move simultaneously, the XR controller position unit 220 can store the positions received from the sensor 110 and the rider 120 according to each XR controller 400. In one embodiment, if the position of XR controller 400 No. 1, as received from sensor 110 and rider 120, is (30, 20, 50), the XR controller position unit 220 can accurately store the position of XR controller 400 No. 1. The XR controller position unit 220 can also receive the position of the XR controller relative to its height from sensor 110 and rider 120. Once the XR controller position unit 220 receives the position of the XR controller relative to its height, it can determine, based on a previously set height, whether the trainee holding the real gun 300 to which the XR controller 400 is attached is sitting or standing. In one embodiment, if the XR controller 400 is received to be located at a height of 85 cm or less, based on the average height of an adult male of 175 cm, the XR controller position unit 220 can determine that the trainee holding the real gun 300 is sitting. Furthermore, the XR controller position unit 220 can not only determine the trainee's position by understanding the position of the XR controller 400 received from the sensor 110, but can also determine the aiming direction and position of the real firearm 300 to which the XR controller 400 is attached. The XR controller position unit 220 can receive the position of the XR controller 400, determine its position, and transmit it to the XR content unit 210.

[0031] The XR controller interaction unit 230 can receive interactions between multiple XR controllers 400 and real firearms 300. Here, the interaction between the XR controller 400 and real firearms 300 may be at least one of the following: whether or not the trigger of the real firearm 300 to which the XR controller 400 is attached has been pressed, or whether or not the magazine of the real firearm 300 to which the XR controller 400 is attached has been replaced, but is not necessarily limited to these. In one embodiment, when the XR controller 400 recognizes that the trigger of the real firearm 300 has been pressed, the XR controller interaction unit 230 can receive the corresponding interaction from the XR controller 400. Furthermore, the interaction that the XR controller interaction unit 230 receives from the XR controller 400 may include not only whether or not the magazine has been replaced, but also whether or not a magazine replacement is necessary. In one embodiment, if the magazine can hold up to 20 rounds and the XR controller 400 recognizes that all 20 rounds have been fired, the XR controller interaction unit 230 can receive an interaction from the XR controller 400 indicating that a magazine change is necessary.

[0032] The communication unit 240 enables communication between the XR environment 100, the service provision server 200, and the XR controller 400 via the network. The network means may include, but is not limited to, at least one CDMA-based (or HSDPA-based) mobile communication network and / or an IEEE 802.16x-based ultra-high-speed wireless internet and / or an IEEE 802.11x-based wireless LAN communication network. The control unit 250 can control each of the configurations of the service provision server 200.

[0033] The real gun 300 consists of a muzzle 310, a trigger 320, a magazine 330, and a laser 340. The XR controller 400 is detachable from the real gun 300. When the XR controller 400 is attached to the real gun 300, it is connected to the trigger 320 and the magazine 330, and can recognize at least one of the following: whether the trigger 320 has been pulled, whether the magazine 330 needs to be replaced, or whether a magazine replacement is necessary, and transmit this information to the service server 200. The XR controller 400 may consist of module 1 321, module 2 331, and module 350. Module 1 321 is connected to the trigger push button of the real gun 300, and when the trigger button of the real gun 300 is pressed, module 1 of the XR controller 400 is activated and recognized as a trigger pull, but it is not necessarily limited to this configuration. Module 2 331 is connected to the magazine 330 mounting area of ​​the real gun 300, and when the magazine 330 is mounted, module 2 331 is pulled, allowing the system to recognize that the magazine 330 is mounted, although this is not necessarily the only way. If the magazine 330 is not mounted, module 2 322 is not activated, allowing the system to recognize that the magazine 330 is not mounted, although this is not necessarily the only way. Module 350 may be a ground wire. In one embodiment, when the trigger 320 of the real gun 300 is pressed, the XR controller 400 can recognize this and transmit the corresponding interaction to the service provider server 200. In addition, since the XR controller 400 is equipped with a GPS for positioning, the sensors 110 and rider 120 of the XR environment 100 can determine the position of the XR controller 400.

[0034] Figure 3 shows an actual real firearm with an XR controller attached.

[0035] Referring to Figure 3, the real gun 300 consists of a muzzle 310, a trigger 320, a magazine 330, and a laser 340, and the XR controller 400 is attached to the lower end of the real gun 300, but it is not limited to this and can be attached anywhere as long as it does not interfere with the movement of the real gun 300. Since the XR controller 400 is attached to the real gun 300, it can be said that the position of the XR controller 400 and the position of the real gun 300 are the same. In addition, since the XR controller 400 is connected to the trigger 320 and the magazine 330 of the real gun 300, the XR controller 400 can sense the pressure on the trigger 320 and whether or not the magazine 330 has been replaced.

[0036] Figure 4 shows a real firearm equipped with an XR controller being aimed at a target within an XR environment.

[0037] Referring to Figure 4, the diagram shows a trainee aiming at a target object using a real gun 300 equipped with an XR controller 400 within an XR environment 100, using the first mode. As it is the first mode, the currently aimed position is displayed as an arrow in the XR content, as shown in A. At this time, the aiming position is displayed according to the muzzle direction of the real gun 300 equipped with the XR controller 400. The sensor 110 measures the position and direction of movement of the XR controller 400 and transmits it to the service server 200, where the XR controller position unit 220 receives this information and can confirm the aiming position. Once the XR controller position unit 220 confirms the aiming position and transmits it to the XR content unit 210, the content correction module 212 corrects the XR content based on the received aiming position.

[0038] Figure 5 shows a real firearm with an XR controller attached and the internal structure of the XR controller.

[0039] Referring to Figure 5, Figure 5(a) shows the real firearm 300 with the XR controller 400 attached. The green wire is the wire connected to module 1 321, the red wire is the wire connected to module 2 331, and the blue wire is the ground wire. Figure 5(b) shows the internal structure of the XR controller 400. Module 1 321 is connected to the trigger push button of the real firearm 300, so when the trigger button of the real firearm 300 is pressed, module 1 321 of the XR controller 400 is activated and recognized as a trigger pull, but this is not necessarily the only way. Module 2 331 is connected to the magazine 330 mounting area of ​​the real firearm 300, so when the magazine 330 is mounted, module 2 322 is pulled and recognized as magazine 330 being mounted, but this is not necessarily the only way. If magazine 330 is not installed, module 322 (number 2) will not be activated, and the system will recognize that magazine 330 is not installed, but this is not necessarily the only case. Module 350 (number 3) may be a ground wire.

[0040] Although the invention has been described with reference to the embodiments shown in the drawings, these are merely illustrative, and a person with ordinary skill in the art will understand that a variety of modifications and equivalent other embodiments are possible therefrom. Therefore, the true scope of technical protection of the invention must be determined by the technical idea of ​​the appended claims.

Claims

1. In an augmented reality (XR)-based hyper-realistic combat training method using a real firearm equipped with an XR controller, The service provider server transmits XR content to a splitter in the XR environment, the splitter transmits the XR content to projectors in the XR environment, and multiple projectors project the XR content onto the XR environment; The XR environment sensors determine the locations of multiple XR controllers and transmit the locations of the multiple XR controllers to a service provider server; The service provider server receives the interaction between the XR controller and the real gun from multiple XR controllers; The service provider server corrects the XR content to match the interaction between the numerous XR controllers and the real gun and the positions of the numerous XR controllers, and transmits the corrected XR content to the splitter of the XR environment; and The process includes the step of a splitter in an XR environment transmitting the corrected XR content received to projectors, so that multiple projectors illuminate the XR environment with the corrected XR content. The interaction between the XR controller and the real gun is characterized in that the XR controller recognizes at least one of the following: whether the trigger of the real gun to which the XR controller is attached is pressed, or whether the magazine has been replaced. This is an augmented reality (XR) based hyper-realistic combat training method using a real gun to which an XR controller is attached.

2. The step of sensors in the XR environment determining the locations of multiple XR controllers and transmitting the locations of the multiple XR controllers to a service provider server is: The process includes the step of the LiDAR (Light Detection and Ranging) in the XR environment additionally determining the location of the XR controller and transmitting it to the service provider server. An augmented reality (XR)-based hyperrealistic combat training method using a real gun equipped with the XR controller described in claim 1, characterized in that the sensors and lidar of the XR environment calculate the positions of a number of XR controllers as coordinate values ​​within the XR environment.

3. The aforementioned XR content is for combat training and consists of three modes: Mode 1, Mode 2, and Mode 3, depending on whether or not the aiming position of the real gun to which the XR controller is attached is displayed. The first mode is a mode in which the aiming position is generated within the XR content according to the muzzle direction of the real gun to which the XR controller is attached. The second mode is a mode in which a laser attached to a real gun that indicates the aiming position according to the direction of the muzzle of the real gun to which the XR controller is attached is generated in the XR environment. The third mode is characterized in that no aiming position corresponding to the muzzle direction of the real gun to which the XR controller is attached is generated within the XR content and XR environment, as described in claim 1, an augmented reality (XR) based hyper-realistic combat training method using a real gun to which an XR controller is attached.

4. An XR environment capable of tracking the locations of multiple XR controllers and illuminating them with XR content received from a service provider server; and An augmented reality (XR) based hyper-realistic combat training system using a real firearm equipped with an XR controller, including a service server that generates XR content and transmits it to an XR environment, receives the positions of multiple XR controllers from the XR environment, recognizes the interaction between the XR controllers and real firearms, and corrects the XR content to match the positions of the multiple XR controllers and the interaction between the XR controllers and real firearms.

5. The aforementioned XR content is for combat training and consists of three modes: Mode 1, Mode 2, and Mode 3, depending on whether or not the aiming position of the real gun to which the XR controller is attached is displayed. The first mode is a mode in which the aiming position is generated within the XR content according to the muzzle direction of the real gun to which the XR controller is attached. The second mode is a mode in which a laser attached to a real gun that indicates the aiming position according to the direction of the muzzle of the real gun to which the XR controller is attached is generated in the XR environment. The third mode is characterized in that no aiming position corresponding to the muzzle direction of the real gun to which the XR controller is attached is generated within the XR content and XR environment, as described in claim 1, an augmented reality (XR) based hyper-realistic combat training system using a real gun to which an XR controller is attached.

Images