Extended reality (XR)-based super-vivid combat training method and system using real gun equipped with XR controller

Abstract

An augmented reality (XR)-based super-vivid combat training method using a real gun equipped with an XR controller, characterized by comprising the steps of: a server providing a service transmitting XR content to a diverter of an XR environment, the diverter transmitting the XR content to projectors of the XR environment, such that a plurality of projectors project the XR content to the XR environment; a sensor of the XR environment masters the positions of the plurality of XR controllers and sends the positions of the plurality of XR controllers to the server providing the service; the server providing the service receives the interaction between the XR controllers and the real gun from the plurality of XR controllers; the server providing the service corrects the XR content according to the interaction between the plurality of XR controllers and the real gun and the positions of the plurality of XR controllers, and sends the corrected XR content to a diverter of the XR environment; and transmitting the corrected XR content received by the diverter of the XR environment to the projectors, so that the plurality of projectors project the corrected XR content to the XR environment, and the interaction of the XR controller with the real gun is that the XR controller recognizes at least one of whether a trigger of the real gun equipped with the XR controller is pressed or not and whether a magazine is replaced or not.

Description

TECHNICAL FIELD

[0001] The present application relates to an extended reality (XR)-based hyper-realistic combat training method and system using a real gun installed with an XR controller, and more particularly to an extended reality (XR)-based hyper-realistic combat training method and system using a real gun installed with an XR controller, which can install an XR controller to a real gun used in actual combat training, thereby performing combat training in an XR environment, and can adjust the difficulty of combat training according to whether or not to display the aiming position of the real gun installed with the XR controller. BACKGROUND

[0002] With the advent of the fourth industrial revolution era, the military training field is also accelerating the ICT defense application business. In particular, since the land environment is narrow and the combat training space is insufficient, it is not possible to perform actual combat training similar to the actual battlefield environment, and in the case of a higher risk, there are not many cases where actual training can be performed, and thus it is necessary to perform combat training using virtual reality (VR).

[0003] However, the combat training system using the existing virtual reality (VR) can perform training after wearing a head-mounted display (HMD), or needs to participate in the training system by fixing the body, and thus cannot be said to be training that can be performed similar to actual combat. In particular, the HMD has the disadvantage of being worn on the face, and not only the immersion is reduced, but also an additional device is required. In addition, it is not possible to directly use the device actually used, and this point also needs to be improved.

[0004] As prior art, there is Korean Patent Publication No. 10-2022-0097352 (Virtual training content providing method and system based on extended reality), but it only provides a virtual training content providing method and a virtual training content providing system for executing the method, the virtual training content providing method including the steps of: acquiring actual training content; processing virtual training content using the actual training content; transmitting the virtual training content to a user; and reproducing the virtual training content. SUMMARY

[0005] The present application is proposed to solve the problems of the prior art as described above, by installing an XR controller on a real gun actually used by a trainer, so that combat training can be performed in an XR environment, and the difficulty of combat training can be adjusted according to whether or not to display the aiming position of the real gun installed with the XR controller.

[0006] According to an embodiment of the present invention, an extended reality (XR)-based hyper-realistic combat training method using a real gun equipped with an XR controller includes the following steps: a server providing a service sends XR content to a splitter of an XR environment, the splitter sends the XR content to projectors of the XR environment, so that multiple projectors project the XR content into the XR environment; a sensor of the XR environment grasps the positions of multiple XR controllers and sends the positions of the multiple XR controllers to the server providing the service; the server providing the service receives interactions between the XR controllers and the real gun from the multiple XR controllers; the server providing the service corrects the XR content based on the interactions between the multiple XR controllers and the real gun and the positions of the multiple XR controllers, and sends the corrected XR content to the splitter of the XR environment; and the corrected XR content received by the splitter of the XR environment is sent to the projectors, so that the multiple projectors project the corrected XR content into the XR environment, wherein the interaction between the XR controller and the real gun is identified by the XR controller as at least one of the following: whether the trigger of the real gun equipped with the XR controller is pressed or not, and whether the magazine is replaced or not.

[0007] According to an embodiment of the present invention, an extended reality (XR)-based ultra-realistic combat training system using a real gun equipped with an XR controller includes: an XR environment that grasps the positions of multiple XR controllers and projects XR content received from a server providing services; and a server providing services that generates XR content and sends it to the XR environment, receives the positions of multiple XR controllers from the XR environment, identifies the interaction between the XR controller and the real gun, and corrects the XR content according to the positions of the multiple XR controllers and the interaction between the XR controller and the real gun.

[0008] According to an embodiment of the present invention, combat training can be performed in an XR environment by installing an XR controller on a real gun originally used by the trainee.

[0009] In addition, the difficulty of combat training can be adjusted by displaying the aiming position of a real gun with an XR controller installed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a flowchart for describing an extended reality (XR)-based hyper-realistic combat training method using a real gun with an XR controller installed according to an embodiment of the present invention.

[0011] Figure 2 This is a diagram for describing a hyper-realistic combat training system based on extended reality (XR) using a real gun equipped with an XR controller according to an embodiment of the present invention.

[0012] Figure 3 This is a diagram showing what a real gun with an XR controller installed actually looks like.

[0013] Figure 4 is a schematic diagram of a real gun equipped with an XR controller used to aim at a target in an XR environment.

[0014] Figure 5 is a schematic diagram showing the internal structure of a real gun equipped with an XR controller and an XR controller. DETAILED DESCRIPTION

[0015] Embodiments of the present application are described in detail with reference to the accompanying drawings.

[0016] Figure 1 is a flowchart for describing an extended reality (XR)-based hyper-realistic combat training method using a real gun equipped with an XR controller according to an embodiment of the present application.

[0017] Referring to Figure 1 , the extended reality (XR)-based hyper-realistic combat training method using a real gun equipped with an XR controller, a server 200 providing a service first transmits XR content to a splitter 130 of an XR environment 100, and the splitter 130 transmits the XR content to a projector 140 of the XR environment 100, so that the plurality of projectors 140 project the XR content S101 to the XR environment 100. The splitter 130, which is a device that splits a specific signal according to a characteristic, can split and transmit the XR content received from the server 200 providing a service to the plurality of projectors 140. The XR environment 100 can be composed of a total of five surfaces including a floor and four walls, and thus a plurality of projectors 140 are required to be able to realistically display a 360° image on the five walls, but the XR environment 100 is not necessarily limited to five walls, and according to an embodiment, can also be composed of six walls including a ceiling.

[0018] The sensor 110 of the XR environment 100 grasps the positions of the plurality of XR controllers 400 and transmits the positions of the plurality of XR controllers 400 to the service-providing server 200 S103. The plurality of XR controllers 400 can be installed to the real gun 300, thereby displaying the positions of the real gun. The sensor 110 of the XR environment 100, as a position sensor, can be plural, and the plurality of sensors 110 can calculate the positions of the respective XR controllers 400 as coordinate values. Also, the sensor 110 can grasp the heights of the XR controllers 400. It can be judged whether the trainee who carries the real gun 300 on which the XR controller 400 is installed aims at a target while sitting or standing according to the heights of the XR controllers 400. Also, in grasping the positions of the XR controllers 400, the laser radar 120 of the XR environment 100 can be further used to grasp the positions of the XR controllers 400. The laser radar 120, as a Light Detection And Ranging (LiDAR), can refer to a laser rangefinder having a technology of emitting and scanning a laser pulse and measuring a time of return. The laser radar 120 of the XR environment 100 can be provided one in each of five faces of the XR environment 100, but is not necessarily limited thereto.

[0019] The service-providing server 200 receives the interactions of the XR controllers 400 and the real gun 300 from the plurality of XR controllers 400 S105. At this time, the interactions of the XR controllers 400 and the real gun 300 can be at least one of whether the trigger of the real gun 300 on which the XR controller 400 is installed is pressed and whether the magazine of the real gun 300 on which the XR controller 400 is installed is replaced, but are not necessarily limited thereto. As one embodiment, if the XR controller 400 recognizes that the trigger of the real gun 300 is pressed, the XR controller 400 can deliver the interaction to the service-providing server 200.

[0020] The service-providing server 200 corrects the XR content according to the interactions of the plurality of XR controllers 400 and the real gun 300 and the positions of the plurality of XR controllers 400 S107, and transmits the corrected XR content to the splitter 130 of the XR environment 100 S109. The corrected XR content received by the splitter 130 of the XR environment 100 is transmitted to the projectors 140, thereby the plurality of projectors 140 projecting the corrected XR content to the XR environment 100 S111.

[0021] Figure 2 is a configuration diagram for describing an extended reality (XR)-based hyper-realistic combat training system using a real gun on which an XR controller is installed according to an embodiment of the present application.

[0022] Reference Figure 2, an extended reality (XR)-based hyper realistic combat training system 10 using a real gun installed with an XR controller 400 is composed of an XR environment 100, a server 200 providing a service, a real gun 300, and the XR controller 400.

[0023] The XR environment 100 refers to an environment in which XR content is played and at least one trainer participating in a combat training can train, and is composed of five surfaces including a ground and four surfaces, but is not necessarily limited thereto. The XR environment 100 is composed of a sensor 110, a laser radar 120, a splitter 130, and a projector 140.

[0024] The sensor 110 can refer to a position sensor, and a plurality of sensors 110 can be provided within the XR environment 100 to grasp the position of the XR controller 400. The sensor 110 can calculate the positions of a plurality of XR controllers 400 as coordinate values, respectively, and transmit the same to the server 200 providing a service. In addition, the sensor 110 can calculate the height of the XR controller 400 as a coordinate value, so that it can also grasp whether the trainer holding the real gun 300 installed with the XR controller 400 is sitting or standing. The sensor 110 can grasp the position of the trainer not only by grasping the position of the XR controller 400, but also can calculate the aiming direction and position of the real gun 300 installed with the XR controller 400.

[0025] The laser radar 120, as an optical radar (LiDAR, Light Detection And Ranging), can be a laser range finder having a technology of measuring the time of laser pulse emission and return and scanning. The position of the XR controller 400 is estimated by using the sensor 110, but the position of the XR controller 400 can be more accurately grasped by using the laser radar 120. The laser radar 120 can be provided one each in the five surfaces of the XR environment 100, but is not necessarily limited thereto.

[0026] The splitter 130, as a device that splits a specific signal according to a characteristic, can split and transmit the XR content received from the server 200 providing a service to a plurality of projectors 140, and a plurality of splitters 130 can be used. The XR environment 100 can display a 360° image using five surfaces, and thus needs to split and transmit the XR content to the projector 140 using the splitter 130. The projector 140 can project the XR content received from the splitter 130 into the XR environment 100, and a plurality of projectors 140 can be used.

[0027] The server 200 providing a service is composed of an XR content part 210, an XR controller position part 220, an XR controller interaction part 230, a communication part 240, and a control part 250.

[0028] The XR content part 210 is composed of a content generation module 211 and a content correction module 212. The content generation module 211 can generate XR content similar to various situations of actual combat training. As one embodiment, a situation of finding an enemy in an ambush and attacking it can be generated as XR content, and a situation of training in which visibility cannot be ensured due to excessive fog can be generated as XR content. In addition, the content generation module 211 can generate combat training content in first, second, and third modes according to difficulty levels. At this time, the difficulty levels can be distinguished according to whether the aiming position of the real gun 300 installed with the XR controller 400 is displayed within the XR content or the XR environment 100, and can be generated by adjusting the three difficulty levels in one XR content. As one embodiment, if the content generation module 211 generates XR content A aiming at a target 10 meters away, the XR content A can be generated as XR content in three modes of first, second, and third modes.

[0029] The first mode is a mode in which the aiming position is generated in the XR content according to the muzzle direction of the real gun 300 installed with the XR controller 400, and is the lowest in difficulty level among the three modes. When the first mode is used, the trainee can grasp whether he aims at a certain place and the exact position of the aiming position, and thus can be used at the initial stage of combat training. At this time, the aiming position can be calculated based on the position of the XR controller 400 recognized by the sensor 110 of the XR environment 100 by the server 200 providing the service.

[0030] The second mode is a mode in which the laser 340 installed to the real gun 300 is generated to display the aiming position within the XR environment 100 according to the muzzle direction of the real gun 300 installed with the XR controller 400, and is the middle in difficulty level among the three modes. The second mode can only confirm the light of the laser 340 within the XR environment 100 and does not generate the aiming position within the XR content. Therefore, the trainee using the second mode must turn on the laser 340 installed on the real gun 300 before the training starts.

[0031] The third mode is a mode in which the aiming position according to the muzzle direction of the real gun 300 installed with the XR controller 400 is not generated in the XR content and the XR environment 100, and is the highest in difficulty level among the three modes. The trainee participates in the training without knowing whether he accurately aims, and is closest to the actual situation, and thus is a mode used for the final stage of combat training.

[0032] The content modification module 212 can modify the XR content according to the interaction of the XR controller 400 with the real gun 300 and the positions of the plurality of XR controllers 400. The content modification module 212 can receive the interaction of the XR controller 400 with the real gun 300 from the XR controller interaction section 230 and can receive the positions of the XR controller 400 from the XR controller position section 220, thereby modifying the XR content. As one embodiment, if the interaction received by the content modification module 212 is "need to replace the magazine", the XR content is modified such that the bullets of the real gun 300 mounted to the XR controller are no longer fired in the XR content. As another embodiment, if the interaction received by the content modification module 212 is "press the trigger", the XR content is modified such that the bullets are fired in the XR content to the position aimed by the real gun 300 mounted to the XR controller. The aimed position can be grasped based on the position of the XR controller 400. In addition, when the positions of the plurality of XR controllers 400 move forward, the content modification module 212 can modify the XR content to be an image in which the XR content moves in the direction. The content modification module 212 can transmit the modified XR content to the shuffler 130 of the XR environment 100.

[0033] The XR controller position part 220 can receive the position of the XR controller 400 from the sensor 110. Also, the position of the XR controller 400 can be further received from the laser radar 120. The XR controller position part 220 can receive coordinate values of the positions of a plurality of XR controllers 400 within the XR environment 100 from the sensor 110 and the laser radar 120. One XR controller 400 can mean the position of one trainer, as the XR controller 400 is mounted to the real gun 300 of the plurality of trainers. The XR content for the battle training is content for training of the plurality of trainers, and thus the plurality of XR controllers 400 can move at the same time, and thus the XR controller position part 220 can store the positions received from the sensor 110 and the laser radar 120 according to each XR controller 400. As one embodiment, if the position of the 1st XR controller 400 received from the sensor 110 and the laser radar 120 is (30, 20, 50), the XR controller position part 220 can accurately store the position of the 1st XR controller 400. Also, the XR controller position part 220 can receive the height position of the XR controller from the sensor 110 and the laser radar 120. If the XR controller position part 220 receives the height position of the XR controller, the XR controller position part 220 can determine whether the trainer who carries the real gun 300 on which the XR controller 400 is mounted is sitting or standing, based on a preset height. As one embodiment, based on the average height of an adult male, 175 cm, if the position of the XR controller 400 is received as being below 85 cm, the XR controller position part 220 can determine that the trainer who carries the real gun 300 is in a sitting state. Also, the XR controller position part 220 can not only grasp the position of the XR controller 400, and thus the position of the trainer, according to the XR controller 400 received from the sensor 110, but also determine the aiming direction and position of the real gun 300 on which the XR controller 400 is mounted. The XR controller position part 220 can receive the position of the XR controller 400, determine the position, and transmit the same to the XR content part 210.

[0034] The XR controller interaction part 230 can receive an interaction of the XR controller 400 with the real gun 300 from the plurality of XR controllers 400. At this time, the interaction of the XR controller 400 with the real gun 300 can be at least one of whether the trigger of the real gun 300 installed with the XR controller 400 is pressed, and whether the magazine of the real gun 300 installed with the XR controller 400 is replaced, but is not necessarily limited thereto. As one embodiment, if the XR controller 400 recognizes that the trigger of the real gun 300 is pressed, the XR controller interaction part 230 can receive the interaction from the XR controller 400. Also, the interaction received by the XR controller interaction part 230 from the XR controller 400 can include not only whether the magazine is replaced, but also whether the magazine needs to be replaced. As one embodiment, if the magazine has a maximum of 20 bullets, and the XR controller 400 recognizes that all of the 20 bullets have been fired, the XR controller interaction part 230 can receive the interaction that the magazine needs to be replaced from the XR controller 400.

[0035] The communication part 240 can enable communication of the XR environment 100, the components of the server 200 providing a service, and the XR controller 400 through a network. The network device can include at least one of a mobile communication network based on CDMA (or based on HSDPA), and / or a super high-speed wireless Internet based on IEEE 802.16x, and / or a wireless LAN based on IEEE 802.11x, but is not necessarily limited thereto. The control part 250 can control the components of the server 200 providing a service.

[0036] The real gun 300 is composed of a muzzle 310, a trigger 320, a magazine 330, and a laser 340. The XR controller 400 is detachably mounted to the real gun 300, and if the XR controller 400 is mounted to the real gun 300, it is connected to the trigger 320 and the magazine 330, thereby being able to recognize at least one of whether the trigger 320 is pulled, whether the magazine 330 is replaced, and whether the magazine 330 needs to be replaced, and deliver it to the server 200 providing a service. The XR controller 400 can be composed of a No. 1 module 321 connected to the trigger button of the real gun 300, a No. 2 module 331 connected to the magazine mounting part of the real gun 300, and a No. 3 module 350. When the trigger button of the real gun 300 is pressed, the No. 1 module 321 of the XR controller 400 is activated, and it can be recognized that the trigger is pulled, but it is not necessarily limited thereto. When the magazine 330 is mounted, the No. 2 module 331 is pulled, and it can be recognized that the magazine 330 is mounted, but it is not necessarily limited thereto. When the magazine 330 is not mounted, the No. 2 module 322 is not activated, and it can be recognized that the magazine 330 is not mounted, but it is not necessarily limited thereto. The No. 3 module 350 can be a ground wire. As an embodiment, when the trigger 320 of the real gun 300 is pressed, the XR controller 400 can recognize and deliver the interaction to the server 200 providing a service. In addition, the XR controller 400 is mounted with a GPS based on a location, thereby enabling the sensor 110 and the laser radar 120 of the XR environment 100 to grasp the location of the XR controller 400.

[0037] Figure 3 is a schematic view of a real gun actually mounted with an XR controller.

[0038] Referring to Figure 3 , the real gun 300 is composed of a muzzle 310, a trigger 320, a magazine 330, and a laser 340, and the XR controller 400 is mounted to the lower end of the real gun 300, but it is not necessarily limited thereto, and can be mounted at any position without hindering the movement of the real gun 300. Since the XR controller 400 is mounted to the real gun 300, it can be said that the location of the XR controller 400 is the same as that of the real gun 300. In addition, since the XR controller 400 is connected to the trigger 320 and the magazine 330 of the real gun 300, the XR controller 400 can sense whether the trigger 320 is pressed, whether the magazine 330 is replaced, and the like.

[0039] Figure 4 is a schematic view of a real gun actually mounted with an XR controller.

[0040] Referring to Figure 4This is a schematic diagram of the trainer using the first mode to aim at the target in the XR environment 100 using the real gun 300 installed with the XR controller 400. Because it is the first mode, the current aiming position is displayed in the XR content in the shape of an arrow as A. At this time, the aiming position can be indicated according to the muzzle direction of the real gun 300 installed with the XR controller 400, and when the sensor 110 measures the position and moving direction of the XR controller 400 and transmits it to the server 200 providing the service, the XR controller position part 220 of the server 200 providing the service can receive it and confirm the aiming position. When the XR controller position part 220 confirms the aiming position and transmits it to the XR content part 210, the XR content is corrected based on the aiming position received by the content correction module 212.

[0041] Figure 5 This is a schematic diagram showing the internal structure of the real gun installed with the XR controller and the XR controller.

[0042] Referring to Figure 5 , Figure 5 (a) of FIG. 1 shows the appearance of the real gun 300 installed with the XR controller 400. The green line is the line connecting the No. 1 module 321, the red line is the line connecting the No. 2 module 331, and the blue line is the ground line. Figure 5 (b) of FIG. 1 shows the internal structure of the XR controller 400. The No. 1 module 321 is connected to the trigger button of the real gun 300, and when the trigger button of the real gun 300 is pressed, the No. 1 module 321 of the XR controller 400 is activated and can be recognized as the trigger pull, but is not necessarily limited thereto. The No. 2 module 331 is connected to the magazine 330 installation part of the real gun 300, and when the magazine 330 is installed, the No. 2 module 322 is pulled and can be recognized as the magazine 330 installation, but is not necessarily limited thereto. When the magazine 330 is not installed, the No. 2 module 322 is not activated and can be recognized as the magazine 330 not installed, but is not necessarily limited thereto. The No. 3 module 350 can be a ground line.

[0043] The present application has been described with reference to the embodiments illustrated in the drawings, but it should be understood that this is only exemplary, and various modifications and other embodiments equivalent thereto can be implemented by those skilled in the art. Therefore, the true technical scope of the present application should be determined by the technical idea of the appended claims.

Claims

1. A method of extended reality (XR) based hyper realistic combat training using a real gun mounted with an XR controller, characterized in that, comprising the steps of: a server providing a service sends XR content to a splitter of an XR environment, the splitter sends the XR content to projectors of the XR environment, whereby the projectors project the XR content to the XR environment; a sensor of the XR environment grasps positions of the plurality of XR controllers and sends the positions of the plurality of XR controllers to the server providing a service; the server providing a service receives interactions of the XR controllers with the real gun from the plurality of XR controllers; the server providing a service corrects the XR content according to the interactions of the XR controllers with the real gun and the positions of the plurality of XR controllers, and sends the corrected XR content to the splitter of the XR environment; and the splitter of the XR environment receives the corrected XR content and sends the corrected XR content to the projectors, whereby the projectors project the corrected XR content to the XR environment, the interactions of the XR controllers with the real gun are at least one of whether a trigger of the real gun to which the XR controller is mounted is pressed and whether a magazine is replaced, which are recognized by the XR controller. 2.The extended reality (XR) -based hyper realistic combat training method using a real gun to which an XR controller is mounted according to claim 1, wherein the step in which the sensor of the XR environment grasps the positions of the plurality of XR controllers and sends the positions of the plurality of XR controllers to the server providing a service comprises the steps of: a laser radar of the XR environment further grasps the positions of the XR controllers and sends the positions to the server providing a service; the sensor and the laser radar of the XR environment can respectively calculate the positions of the plurality of XR controllers as coordinate values within the XR environment. 3.The extended reality (XR) -based hyper realistic combat training method using a real gun to which an XR controller is mounted according to claim 1, wherein the XR content is content for combat training, and is composed of a first mode, a second mode, and a third mode according to whether a real gun to which an XR controller is mounted is displayed to have a sight position, the first mode is a mode in which a sight position is generated in the XR content according to a direction of a muzzle of the real gun to which the XR controller is mounted, the second mode is a mode in which a laser that displays the sight position is generated in the XR environment according to the direction of the muzzle of the real gun to which the XR controller is mounted, the laser being mounted to the real gun, the third mode is a mode in which the sight position is not generated in the XR content and the XR environment, the sight position being a sight position according to the direction of the muzzle of the real gun to which the XR controller is mounted.

4. An extended reality (XR) based hyper realistic combat training system using a real gun mounted with an XR controller, characterized in that, comprising: an XR environment that grasps positions of a plurality of XR controllers and projects XR content received from a server providing a service; and a server providing a service that generates the XR content and sends the XR content to the XR environment, receives the positions of the plurality of XR controllers from the XR environment, identifies interactions of the XR controllers with a real gun, and corrects the XR content according to the positions of the plurality of XR controllers and the interactions of the XR controllers with the real gun. 5.The extended reality (XR) -based hyper realistic combat training system using a real gun to which an XR controller is mounted according to claim 4, wherein The XR content is content for a battle training, and is composed of a first mode, a second mode, and a third mode according to whether a real gun aiming position is displayed with an XR controller mounted, The first mode is a mode of generating an aiming position in the XR content according to a real gun muzzle direction with the XR controller mounted, The second mode is a mode of generating a laser display aiming position in an XR environment according to a real gun muzzle direction with the XR controller mounted, the laser being mounted to the real gun, The third mode is a mode of not generating an aiming position in the XR content and the XR environment, the aiming position being according to a real gun muzzle direction with the XR controller mounted.

Images