A hyper-realistic combat training method and system based on extended reality (XR) using real guns equipped with XR controllers.

Abstract

An extended reality (XR) based ultra-realistic combat training method using a real gun mounted with an XR controller, characterized by comprising the following steps: a server providing a service sends XR content to a splitter of an XR environment, the splitter sends the XR content to a projector of the XR environment, so that multiple projectors project the XR content to the XR environment; a sensor of the XR environment masters positions of multiple XR controllers and sends the positions of the multiple XR controllers to the server providing the service; the server providing the service receives interactions of the XR controllers with the real gun from the multiple XR controllers; the server providing the service corrects the XR content according to the interactions of the multiple XR controllers with the real gun and the positions of the multiple XR controllers, and sends the corrected XR content to the splitter of the XR environment; and the splitter of the XR environment receives the corrected XR content and sends the corrected XR content to the projector, so that the multiple projectors project the corrected XR content to the XR environment, the interactions of the XR controllers with the real gun being identified by the XR controller as at least one of whether a trigger of the real gun mounted with the XR controller is pressed or not and whether a magazine is replaced or not.

Description

Technical Field

[0001] This invention relates to a hyper-realistic combat training method and system based on extended reality (XR) using a real gun equipped with an XR controller. More specifically, it relates to a hyper-realistic combat training method and system based on extended reality (XR) using a real gun equipped with an XR controller, which can install the XR controller on a real gun used in actual combat training, thereby conducting combat training in an XR environment, and can adjust the difficulty of combat training according to whether the aiming position of the real gun equipped with the XR controller is displayed. Background Technology

[0002] With the advent of the Fourth Industrial Revolution, the field of military training is also accelerating the application of ICT in defense. In particular, due to limited land and insufficient space for combat training, it is impossible to conduct live-fire training that resembles the actual battlefield environment. Furthermore, there are few opportunities to conduct actual training in highly dangerous situations. Therefore, it is necessary to utilize virtual reality (VR) for combat training.

[0003] However, existing virtual reality (VR) combat training systems can be conducted while wearing a head-mounted display (HMD) or require a fixed body position to participate in the training system, thus they cannot be considered training similar to actual combat. In particular, the disadvantages of HMDs are that they are worn on the face, which not only reduces immersion but also requires additional equipment. Furthermore, the inability to directly use actual equipment needs improvement.

[0004] As prior art, there is Korean Patent No. 10-2022-0097352 (Method and System for Providing Virtual Training Based on Extended Reality), but it only provides a method for providing virtual training content and a system for providing virtual training content that executes the method. The method for providing virtual training content includes the following steps: obtaining actual training content; processing virtual training content using the actual training content; sending the virtual training content to the user; and reproducing the virtual training content. Summary of the Invention

[0005] The problem to be solved by the present invention is to address the issues of the prior art as described above by installing an XR controller on the actual gun used by the trainee, thereby enabling combat training in an XR environment, and adjusting the difficulty of combat training based on whether the aiming position of the actual gun with the XR controller installed is displayed.

[0006] An embodiment of the present invention provides a hyper-realistic combat training method based on extended reality (XR) using a live gun equipped with an XR controller, comprising the following steps: a service server sends XR content to a splitter in an XR environment, the splitter sending the XR content to a projector in the XR environment, thereby projecting the XR content onto the XR environment by multiple projectors; sensors in the XR environment detect the positions of multiple XR controllers and send these positions to the service server; the service server receives interactions between the XR controllers and the live gun from the multiple XR controllers; the service server corrects the XR content based on the interactions between the multiple XR controllers and the live gun and the positions of the multiple XR controllers, and sends the corrected XR content to the splitter in the XR environment; and the service server sends the corrected XR content received by the splitter in the XR environment to the projector, thereby projecting the corrected XR content onto the XR environment by multiple projectors, wherein the interactions between the XR controllers and the live gun are identified by the XR controllers as at least one of the following: whether the trigger of the live gun equipped with the XR controller is pressed, or whether the magazine has been changed.

[0007] An embodiment of the present invention provides an extended reality (XR)-based hyper-realistic combat training system using a live gun equipped with an XR controller, comprising: an XR environment that knows the positions of multiple XR controllers and projects XR content received from a service server; and a service server that generates XR content and sends it to the XR environment, receives the positions of the multiple XR controllers from the XR environment, identifies the interaction between the XR controllers and the live gun, and modifies the XR content based on the positions of the multiple XR controllers and the interaction between the XR controllers and the live gun.

[0008] According to an embodiment of the present invention, combat training can be conducted in an XR environment by installing an XR controller on the actual firearm used by the trainee.

[0009] In addition, the difficulty of combat training can be adjusted based on whether or not the aiming position of a real gun equipped with an XR controller is displayed. Attached Figure Description

[0010] Figure 1 This is a flowchart describing a hyper-realistic combat training method based on extended reality (XR) using a real gun equipped with an XR controller, according to an embodiment of the present invention.

[0011] Figure 2 This is a configuration diagram for describing a hyper-realistic combat training system based on extended reality (XR) using a real gun equipped with an XR controller, according to an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram showing what a real gun with an XR controller looks like.

[0013] Figure 4 This is a diagram illustrating the use of a real gun equipped with an XR controller to aim at a target in an XR environment.

[0014] Figure 5 This is a schematic diagram showing a real gun equipped with an XR controller and the internal structure of the XR controller. Detailed Implementation

[0015] The embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0016] Figure 1 This is a flowchart describing a hyper-realistic combat training method based on extended reality (XR) using a real gun equipped with an XR controller, according to an embodiment of the present invention.

[0017] Reference Figure 1 A hyper-realistic combat training method based on extended reality (XR) using real guns equipped with XR controllers involves the following steps: First, the service server 200 sends XR content to a splitter 130 in the XR environment 100. The splitter 130 then sends the XR content to projectors 140 in the XR environment 100, thereby projecting the XR content S101 onto the XR environment 100 from multiple projectors 140. The splitter 130, as a device for splitting specific signals according to their characteristics, can split the XR content received from the service server 200 and send it to multiple projectors 140. The XR environment 100 can consist of a total of five surfaces, including the ground and four sides. Therefore, multiple projectors 140 are needed to realistically display 360° images on the five surfaces. However, the XR environment 100 is not necessarily limited to five surfaces, and according to one embodiment, it can also consist of six surfaces, including the ceiling.

[0018] The XR environment 100's sensor 110 detects the positions of multiple XR controllers 400 and sends these positions (S103) to the service server 200. The multiple XR controllers 400 can be mounted on a live gun 300 to display its position. The XR environment 100's sensor 110 acts as a position sensor, and multiple sensors 110 can calculate the positions of each XR controller 400 as coordinate values. Furthermore, the sensor 110 can also detect the elevation of the XR controllers 400. The elevation of the XR controllers 400 can be used to determine whether a trainee carrying a live gun 300 equipped with XR controllers 400 is aiming at a target while seated or standing. In addition, when detecting the position of the XR controllers 400, the XR environment 100's lidar 120 can be used to further determine their position. The lidar 120, as an optical radar (LiDAR, Light Detection and Ranging), can refer to a laser rangefinder, which has the technology to measure and scan the emission and return time of laser pulses. The LiDAR 120 of the XR environment 100 can be set in each of the five faces of the XR environment 100, but is not necessarily limited to this.

[0019] The service server 200 receives interactions S105 between the XR controllers 400 and the real gun 300 from multiple XR controllers 400. At this time, the interaction between the XR controller 400 and the real gun 300 can be at least one of whether the trigger of the real gun 300 equipped with the XR controller 400 is pressed, and whether the magazine of the real gun 300 equipped with the XR controller 400 is changed, but is not necessarily limited to these. As an example, if the XR controller 400 detects that the trigger of the real gun 300 has been pressed, the XR controller 400 can transmit this interaction to the service server 200.

[0020] The server 200, providing the service, modifies the XR content S107 based on the interaction between the plurality of XR controllers 400 and the real gun 300, as well as the positions of the plurality of XR controllers 400, and sends the modified XR content to the splitter 130 of the XR environment 100 (S109). The splitter 130 of the XR environment 100 sends the modified XR content received to the projector 140, thereby the plurality of projectors 140 project the modified XR content onto the XR environment 100 S111.

[0021] Figure 2 This is a configuration diagram for describing a hyper-realistic combat training system based on extended reality (XR) using a real gun equipped with an XR controller, according to an embodiment of the present invention.

[0022] Reference Figure 2The extended reality (XR) based hyper-realistic combat training system 10, which uses real guns equipped with XR controllers 400, consists of an XR environment 100, a service server 200, real guns 300, and XR controllers 400.

[0023] XR Environment 100 refers to an environment that plays XR content and allows at least one trainee participating in combat training to train. It consists of five surfaces, including the ground and four sides, but is not necessarily limited to these. XR Environment 100 is composed of sensor 110, lidar 120, splitter 130, and projector 140.

[0024] The sensor 110 can be a position sensor, and multiple sensors can be installed within the XR environment 100 to determine the position of the XR controller 400. The sensor 110 can calculate the positions of the multiple XR controllers 400 as coordinate values ​​and send them to the service server 200. Furthermore, the sensor 110 can calculate the elevation of the XR controllers 400 as coordinate values, thus determining whether the trainee holding the live gun 300 equipped with the XR controller 400 is sitting or standing. The sensor 110 can not only determine the trainee's position by knowing the position of the XR controllers 400, but also calculate the aiming direction and position of the live gun 300 equipped with the XR controller 400.

[0025] The lidar 120, as a light detection and ranking (LiDAR) system, can be a laser rangefinder, which has the technology to measure and scan the emission and return time of laser pulses. While the position of the XR controller 400 is inferred using sensor 110, the lidar 120 provides a more accurate determination of its position. The lidar 120 can be positioned on each of the five faces of the XR environment 100, but is not necessarily limited to this.

[0026] The splitter 130, as a device for splitting specific signals according to their characteristics, can split XR content received from the service server 200 and send it to multiple projectors 140, and multiple splitters 130 can be used. The XR environment 100 can display 360° images using five surfaces, therefore, splitters 130 are needed to split the XR content and send it to the projectors 140. The projectors 140 can project the XR content received from the splitters 130 into the XR environment 100, and multiple projectors can be used.

[0027] The server 200 that provides services consists of an XR content unit 210, an XR controller location unit 220, an XR controller interaction unit 230, a communication unit 240, and a control unit 250.

[0028] The XR content unit 210 consists of a content generation module 211 and a content correction module 212. The content generation module 211 can generate XR content similar to various scenarios in actual combat training. For example, it can generate XR content depicting finding and attacking an ambush enemy, or training scenarios where visibility is limited by heavy fog. Furthermore, the content generation module 211 can generate combat training content in three modes: a first mode, a second mode, and a third mode, based on difficulty. The difficulty level can be differentiated by whether the aiming position of the real gun 300 equipped with the XR controller 400 is displayed in the XR content or XR environment 100, and can be generated by adjusting the three difficulty levels within a single XR content. For example, if the content generation module 211 generates XR content named A, which aims at a target 10 meters away, then XR content A can be generated in three modes: the first mode, the second mode, and the third mode.

[0029] The first mode generates the aiming position within the XR content based on the muzzle direction of the real gun 300 equipped with the XR controller 400. It is the easiest of the three modes. Using the first mode, trainees can determine whether they are aiming at a specific location and the precise location of that aiming position, making it suitable for use in the early stages of combat training. In this mode, the aiming position can be calculated by the server 200 providing the service, based on the position of the XR controller 400 identified by the sensor 110 of the XR environment 100.

[0030] The second mode generates a laser 340 displaying the aiming position within the XR environment 100 based on the muzzle direction of the real gun 300 equipped with an XR controller 400. This mode represents a middle level of difficulty among the three modes, with the laser 340 mounted on the real gun 300. In the second mode, the laser 340 light can only be detected within the XR environment 100, and the aiming position is not generated within the XR content. Therefore, trainees using the second mode must turn on the laser 340 mounted on the real gun 300 before starting training.

[0031] The third mode, which does not generate an aiming position in either the XR content or the XR environment 100, is the most difficult of the three modes. The aiming position is based on the aiming position of the muzzle of a real gun 300 equipped with an XR controller 400. Trainees participate in training without knowing whether they are aiming accurately, which is closest to reality, and therefore is the mode used for the final stage of combat training.

[0032] The content correction module 212 can correct the XR content based on the interaction between the XR controller 400 and the real gun 300, as well as the positions of the multiple XR controllers 400. The content correction module 212 can receive the interaction between the XR controller 400 and the real gun 300 from the XR controller interaction unit 230, and can receive the position of the XR controller 400 from the XR controller position unit 220, thereby correcting the XR content. As one embodiment, if the interaction received by the content correction module 212 is "magazine needs to be changed," the XR content is corrected so that the bullets from the real gun 300 mounted on the XR controller no longer fire. As another embodiment, if the interaction received by the content correction module 212 is "trigger pressed," the XR content is corrected so that the bullets fire towards the aiming position of the real gun 300 mounted on the XR controller. The aiming position can be determined based on the position of the XR controller 400. Furthermore, when the positions of the multiple XR controllers 400 move forward, the content correction module 212 can correct the XR content to an image that reflects the XR content moving in that direction. The content correction module 212 can then send the corrected XR content to the splitter 130 of the XR environment 100.

[0033] The XR controller location unit 220 can receive the position of the XR controller 400 from the sensor 110. Furthermore, it can further receive the position of the XR controller 400 from the lidar 120. The XR controller location unit 220 can receive the coordinate values ​​of the positions of multiple XR controllers 400 within the XR environment 100 from the sensor 110 and the lidar 120. The XR controllers 400 are mounted on the live guns 300 of multiple trainees, and one XR controller 400 can represent the position of one trainee. The XR content used for combat training is for multiple trainees; therefore, multiple XR controllers 400 may move simultaneously. Thus, the XR controller location unit 220 can store the positions received by the sensor 110 and lidar 120 according to each XR controller 400. As an example, if the position of XR controller 400 1 received from the sensor 110 and lidar 120 is (30, 20, 50), then the XR controller location unit 220 can accurately store the position of XR controller 400 1. Furthermore, the XR controller position unit 220 can receive the elevation position of the XR controller from the sensor 110 and the lidar 120. If the XR controller position unit 220 receives the elevation position of the XR controller, it can determine whether the trainee carrying the live gun 300 equipped with the XR controller 400 is sitting or standing, based on a preset height. As an example, using the average height of an adult male of 175cm as a reference, if the XR controller 400 is received at a position below 85cm, the XR controller position unit 220 can determine that the trainee carrying the live gun 300 is sitting. In addition, the XR controller position unit 220 can not only determine the position of the XR controller 400 based on the position received from the sensor 110, thereby determining the position of the trainee, but also determine the aiming direction and position of the live gun 300 equipped with the XR controller 400. The XR controller position unit 220 can receive the position of the XR controller 400, determine the position, and send it to the XR content unit 210.

[0034] The XR controller interaction unit 230 can receive interactions between the XR controller 400 and the real gun 300 from multiple XR controllers 400. In this case, the interaction between the XR controller 400 and the real gun 300 can be at least one of whether the trigger of the real gun 300 equipped with the XR controller 400 is pressed, and whether the magazine of the real gun 300 equipped with the XR controller 400 is changed, but it is not necessarily limited to these. As an example, if the XR controller 400 detects that the trigger of the real gun 300 is pressed, the XR controller interaction unit 230 can receive this interaction from the XR controller 400. Furthermore, the interactions received by the XR controller interaction unit 230 from the XR controller 400 can include not only whether the magazine is changed, but also whether the magazine needs to be changed. As an example, if the magazine contains a maximum of 20 bullets and the XR controller 400 recognizes that all 20 bullets have been fired, the XR controller interaction unit 230 can receive an interaction from the XR controller 400 indicating that the magazine needs to be replaced.

[0035] The communication unit 240 enables communication between the XR environment 100, the service-providing server 200, and the XR controller 400 via a network. The network device may include at least one of the following: a CDMA-based (or HSDPA-based) mobile communication network, and / or an IEEE 802.16x-based ultra-high-speed wireless internet, and / or an IEEE 802.11x-based wireless local area network, but is not necessarily limited to these. The control unit 250 can control the components of the service-providing server 200.

[0036] The live gun 300 consists of a muzzle 310, a trigger 320, a magazine 330, and a laser 340. An XR controller 400 is detachably mounted on the live gun 300. If the XR controller 400 is mounted on the live gun 300, it connects to the trigger 320 and the magazine 330, thereby identifying at least one of whether the trigger 320 has been pulled, whether the magazine 330 has been replaced, and whether the magazine 330 needs to be replaced, and transmitting this information to the service server 200. The XR controller 400 can consist of module 1 321, module 2 331, and module 350. Module 1 321 is connected to the trigger button of the live gun 300. When the trigger button of the live gun 300 is pressed, module 1 321 of the XR controller 400 is activated and can be identified as a trigger pull, but this is not necessarily the case. Module 2 331 connects to the magazine 330 mounting area of ​​the real gun 300. When the magazine 330 is installed, module 2 331 is pulled, and this can be recognized as the magazine 330 being installed, but it is not necessarily limited to this. When the magazine 330 is not installed, module 2 322 is not activated, and this can be recognized as the magazine 330 not being installed, but it is not necessarily limited to this. Module 3 350 can be a ground wire. As an example, when the trigger 320 of the real gun 300 is pressed, the XR controller 400 can recognize this interaction and transmit it to the server 200 providing the service. In addition, the XR controller 400 is equipped with a location-based GPS, which allows the sensors 110 and LiDAR 120 of the XR environment 100 to determine the location of the XR controller 400.

[0037] Figure 3 This is a schematic diagram showing the actual appearance of a real gun equipped with an XR controller.

[0038] Reference Figure 3 The real gun 300 consists of a muzzle 310, trigger 320, magazine 330, and laser 340. An XR controller 400 is mounted at the lower end of the real gun 300, but it is not limited to this location and can be mounted anywhere that does not obstruct the movement of the real gun 300. Since the XR controller 400 is mounted on the real gun 300, it can be said that the position of the XR controller 400 is the same as the position of the real gun 300. Furthermore, since the XR controller 400 is connected to the trigger 320 and magazine 330 of the real gun 300, the XR controller 400 can sense whether the trigger 320 is pressed and whether the magazine 330 has been changed, etc.

[0039] Figure 4 This is a diagram illustrating the use of a real gun equipped with an XR controller to aim at a target in an XR environment.

[0040] Reference Figure 4This is a diagram illustrating a trainee aiming at a target using a live gun 300 equipped with an XR controller 400 in an XR environment 100 in the first mode. Because it is the first mode, like in mode A, the current aiming position is displayed as an arrow in the XR content. At this time, the aiming position can be represented by the muzzle direction of the live gun 300 equipped with the XR controller 400. When the sensor 110 determines the position and direction of movement of the XR controller 400 and transmits it to the service server 200, the XR controller position unit 220 of the service server 200 can receive it and confirm the aiming position. After the XR controller position unit 220 confirms the aiming position and sends it to the XR content unit 210, the XR content is corrected based on the aiming position received by the content correction module 212.

[0041] Figure 5 This is a schematic diagram showing a real gun equipped with an XR controller and the internal structure of the XR controller.

[0042] Reference Figure 5 , Figure 5 (a) represents the appearance of the XR controller 400 installed on the real gun 300. The green wire is the wire connecting to module 1 321, the red wire is the wire connecting to module 2 331, and the blue wire is the ground wire. Figure 5 (b) indicates the internal structure of the XR controller 400. Module 1 321 is connected to the trigger button of the real gun 300. When the trigger button of the real gun 300 is pressed, module 1 321 of the XR controller 400 is activated and can be recognized as a trigger pull, but this is not necessarily the case. Module 2 331 is connected to the magazine 330 mounting part of the real gun 300. When the magazine 330 is installed, module 2 322 is pulled and can be recognized as a magazine 330 being installed, but this is not necessarily the case. When the magazine 330 is not installed, module 2 322 is not activated and can be recognized as a magazine 330 not being installed, but this is not necessarily the case. Module 3 350 can be a ground wire.

[0043] While the present invention has been described with reference to the embodiments shown in the accompanying drawings, it should be understood that these are merely exemplary, and various modifications and equivalent embodiments can be implemented by those skilled in the art. Therefore, the true scope of protection of the present invention should be determined by the technical concept of the appended claims.

Claims

1. A method of extended reality (XR) based hyper realistic combat training using a real gun mounted with an XR controller, characterized in that, comprises the steps of: a server providing a service transmits XR content to a splitter of an XR environment, the splitter transmits the XR content to projectors of the XR environment, whereby the projectors project the XR content to the XR environment; a sensor of the XR environment grasps positions of a plurality of XR controllers and transmits the positions of the plurality of XR controllers to the server providing a service; the server providing a service receives interactions of the XR controllers with real guns from the plurality of XR controllers; the server providing a service corrects the XR content according to the interactions of the XR controllers with the real guns and the positions of the plurality of XR controllers and transmits the corrected XR content to the splitter of the XR environment; and the splitter of the XR environment receives the corrected XR content and transmits the corrected XR content to the projectors, whereby the projectors project the corrected XR content to the XR environment, the interactions of the XR controllers with the real guns are at least one of whether a trigger of the real gun installed with the XR controller is pressed or not and whether a magazine is replaced or not, which are recognized by the XR controller, the XR content is content for a battle training, and is composed of a first mode, a second mode, and a third mode according to whether a sighting position of the real gun installed with the XR controller is displayed or not, the first mode is a mode of generating the sighting position in the XR content according to a barrel direction of the real gun installed with the XR controller, the second mode is a mode of generating a laser display showing the sighting position in the XR environment according to the barrel direction of the real gun installed with the XR controller, the laser being installed to the real gun, the third mode is a mode of not generating the sighting position in the XR content and the XR environment, the sighting position being the sighting position according to the barrel direction of the real gun installed with the XR controller.

2. The extended reality (XR)-based hyper realistic battle training method using a real gun installed with an XR controller according to claim 1, characterized in that: the step of the sensor of the XR environment grasping the positions of the plurality of XR controllers and transmitting the positions of the plurality of XR controllers to the server providing a service comprises the steps of: a laser radar of the XR environment further grasps the positions of the XR controllers and transmits to the server providing a service; the sensor and the laser radar of the XR environment can calculate the positions of the plurality of XR controllers as coordinate values in the XR environment, respectively.

3. An extended reality (XR) based hyper realistic combat training system using a real gun mounted with an XR controller, characterized in that, comprises: an XR environment grasping positions of a plurality of XR controllers and projecting XR content received from a server providing a service; and a server providing a service generating the XR content and transmitting to the XR environment, receiving the positions of the plurality of XR controllers from the XR environment, recognizing interactions of the XR controllers with real guns, and correcting the XR content according to the positions of the plurality of XR controllers and the interactions of the XR controllers with the real guns, the XR content is content for a battle training, and is composed of a first mode, a second mode, and a third mode according to whether a sighting position of the real gun installed with the XR controller is displayed or not, the first mode is a mode of generating the sighting position in the XR content according to a barrel direction of the real gun installed with the XR controller, The second mode is a mode in which a laser that displays a sighting position is generated in the XR environment according to the direction of the muzzle of the real gun in which the XR controller is installed, and the laser is installed in the real gun, The third mode is a mode in which a sighting position is not generated in the XR content and the XR environment, and the sighting position is according to the direction of the muzzle of the real gun in which the XR controller is installed.

Images