Training system

Abstract

In training conducted in a virtual space, the ability to respond to different situations is objectively evaluated. [Solution] The training system evaluates the trainee's actions in a virtual space. The training system's display device 10 includes a storage 13, a sensor unit 19, a identification unit 101, a display unit 16, and an evaluation unit 104. The storage 13 stores a field of view image from the trainee's position in the virtual space. The sensor unit 19 detects the trainee's field of view direction in the real space. The identification unit 101 identifies the trainee's field of view in the virtual space based on the trainee's position and field of view direction in the virtual space. The display unit 16 displays the portion of the field of view image that is included in the field of view. The evaluation unit 104 evaluates the trainee's actions toward a person in the virtual space that is included in the field of view, depending on whether or not it is a situation that requires action toward that person.

Description

[Technical Field]

[0001] The present invention relates to a technique for performing training in a virtual space. [Background Art]

[0002] Patent Document 1 describes a technique for performing training in which a trainee checks the state of targets such as equipment and suspicious objects using virtual reality. [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2020-052290 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] In the security industry, training for countermeasures against persons for whom dangerous acts such as intruders like robbers and violent criminals are foreseen (hereinafter referred to as suspicious persons in this specification) is carried out. Generally, in training for countermeasures against suspicious persons, a trainee performs predetermined actions such as stances and practice swings using a warning baton, and an evaluator evaluates the actions. Here, when performing training using a virtual space in order to achieve more realistic training, it is not easy for an evaluator to perform an evaluation based on actions in the virtual space. The technique described in Patent Document 1 evaluates a user's actions in training using a virtual space, but it is aimed at checking the state of stationary objects such as equipment and suspicious objects, and therefore cannot appropriately evaluate the judgment ability and response ability according to changing situations, which are required for security work for responding to suspicious persons.

[0005] An object of the present invention is to objectively evaluate response ability according to situations in training performed in a virtual space. [Means for Solving the Problem]

[0006] One aspect of the present invention provides a training system for evaluating the actions of a trainee in a virtual space, comprising: a storage unit for storing a field of view image from the trainee's position in the virtual space; a detection unit for detecting the trainee's field of view direction in real space; a specification unit for identifying the trainee's field of view in the virtual space based on the trainee's position and the field of view direction in the virtual space; a display unit for displaying the portion of the field of view image that is included in the field of view; and an evaluation unit for evaluating the trainee's actions toward a person in the virtual space that is included in the field of view, depending on whether or not it is necessary to respond to the person in question.

[0007] The evaluation unit may evaluate the trainee's actions in accordance with each of the multiple situations that have been set as situations requiring action.

[0008] The situation requiring the aforementioned response includes at least a first situation in which a person who is the target of vigilance appears within the field of view, and the evaluation unit may evaluate the trainee's actions in the first situation depending on whether or not a predetermined vigilance action was performed against the person.

[0009] The aforementioned prescribed warning actions may include at least one of the following: taking distance from the person, raising a warning baton, and calling out to the person.

[0010] The situation requiring the aforementioned response includes at least a second situation in which an attack from the person is imminent, and the evaluation unit may evaluate the trainee's actions in the second situation depending on whether or not an attack countermeasure action was performed against the person.

[0011] The evaluation unit may evaluate the trainee's actions based on at least one of the following: the part of the body to which the trainee strikes with the baton during the attack response action, the magnitude of the swing of the baton, and the positional relationship between the trainee and the person before the attack response action is performed. [Effects of the Invention]

[0012] According to the present invention, in training conducted in a virtual space, the ability to respond to situations can be objectively evaluated. [Brief explanation of the drawing]

[0013] [Figure 1] A diagram showing an example of the configuration of the training system according to the embodiment. [Figure 2] A diagram showing an example of a display device configuration. [Figure 3] A diagram showing an example of an evaluation table. [Figure 4] A flowchart illustrating an example of the operation of a display device. [Figure 5] A floor plan showing an example of a virtual space. [Figure 6] This diagram shows an example of a video that appears on the display when a person appears in front of the trainee in a virtual space. [Figure 7] A diagram showing an example of the evaluation results screen. [Modes for carrying out the invention]

[0014] (composition) Figure 1 shows an example of the configuration of training system 1 according to this embodiment. Training system 1 utilizes VR (Virtual Reality) and is used for training in dealing with suspicious individuals in security operations. This "training" refers to educational activities aimed at achieving a certain goal. Training also includes internships and learning. Training system 1 simulates a situation in which a trainee encounters a suspicious person in a virtual space and evaluates the trainee's actions in that situation.

[0015] A training system 1 includes a plurality of controllers 5, a plurality of display devices 10, and a management device 20. The plurality of controllers 5 and the plurality of display devices 10 each form one-to-one pairs and are communicatively connected wirelessly. The plurality of display devices 10 and the management device 20 are communicatively connected via a communication line 30. The communication line 30 transmits communications between these devices. A private network such as a LAN (Local Area Network) may be used as the communication line 30, for example.

[0016] The controller 5 has a rod-like shape imitating a warning stick and is held by a trainee. In training for responding to suspicious persons, a trainee uses the controller 5 to take a posture of holding a warning stick, or performs a striking motion against a suspicious person. The controller 5 tracks the movements of these trainees in real time, and wirelessly transmits movement information to the paired display device 10. This movement information includes posture information indicating a posture of holding a warning stick, and striking motion information indicating a striking motion.

[0017] The controller 5 includes a sensor unit 51 that detects the trainee's movements. The sensor unit 51 includes, for example, a gyro sensor and an acceleration sensor. The gyro sensor detects the angular velocity of the controller 5. The acceleration sensor detects the acceleration of the controller 5 in three axial directions. The sensor unit 51 measures the angle and vertical position of the controller 5 based on the angular velocity detected by the gyro sensor and the acceleration detected by the acceleration sensor, and outputs posture information indicating these. Further, the sensor unit 51 measures the range of the swinging motion and the striking motion range of a striking action using the controller 5 based on the angular velocity detected by the gyro sensor and the acceleration detected by the acceleration sensor, and outputs striking motion information indicating these.

[0018] (Configuration of Display Device) FIG. 2 is a diagram showing an example of the configuration of the display device 10. The display device 10 provides a virtual space by displaying an image simulating the visual field of a trainee, and evaluates the trainee's behavior when encountering a suspicious person in the virtual space. The display device 10 is, for example, a head-mounted wearable terminal, and is worn on the head of the trainee.

[0019] The display device 10 includes a processor 11, a memory 12, a storage 13, a communication unit 14, an operation unit 15, a display unit 16, a sound output unit 17, a sound collection unit 18, and a sensor unit 19. The processor 11 is connected to the memory 12, the storage 13, the communication unit 14, the operation unit 15, the display unit 16, the sound output unit 17, the sound collection unit 18, and the sensor unit 19 via a bus.

[0020] The processor 11 executes various processes by reading a program into the memory 12 and executing the program. For example, a CPU (Central Processing Unit) is used as the processor 11. The memory 12 stores a program to be executed by the processor 11. For example, a ROM (Read Only Memory) or a RAM (Random Access Memory) may be used as the memory 12. The storage 13 stores various data and programs. For example, a flash memory may be used as the storage 13. The storage 13 is an example of the storage unit according to the present invention. The communication unit 14 is connected to a communication line 30, and performs data communication via the communication line 30. The operation unit 15 is used for inputting various types of information. For example, operation keys may be used as the operation unit 15. The display unit 16 displays various types of information. For example, a liquid crystal display may be used as the display unit 16. The sound output unit 17 converts an audio signal into sound and outputs the sound. For example, a speaker may be used as the sound output unit 17. The sound collection unit 18 collects sound, converts the sound into an audio signal, and outputs the audio signal. For example, a microphone may be used as the sound collection unit 18.

[0021] The sensor unit 19 detects the movement of the trainee's head in a coordinate system in real space. This movement includes the orientation of the head. In this embodiment, the orientation of the trainee's head in real space is detected as the direction of the trainee's field of view in real space. The sensor unit 19 is an example of a detection unit according to the present invention. The sensor unit 19 may include, for example, a gyro sensor, an accelerometer, and a geomagnetic sensor. The gyro sensor measures the angle or angular velocity of the trainee's head. The accelerometer measures the acceleration of the trainee's head. The geomagnetic sensor measures the orientation of the trainee's head. The sensor unit 19 also measures the distance the trainee has traveled from a reference position based on the angular velocity detected by the gyro sensor, the acceleration detected by the accelerometer, and the orientation detected by the geomagnetic sensor.

[0022] The display unit 16 displays an image of the virtual space corresponding to the trainee's field of view, which is determined based on the orientation of the head detected by the sensor unit 19. In other words, by turning the trainee's head left and right in the real space, the trainee can view the entire virtual space.

[0023] Furthermore, the storage 13 pre-stores operational data showing a video for training on dealing with suspicious individuals. This video shows the scenario necessary for training, and the trainee performs the training according to this scenario. The video data is pre-recorded using an imaging device (not shown) and is associated with a coordinate system in the virtual space. In this embodiment, the trainee's position in the virtual space is predetermined for each time period and changes over time. Therefore, this video simulates the view from the trainee at a predetermined position in the virtual space for each time period, that is, it is a field of view image of the range visible from each position of the trainee in the virtual space. This video is composed of multiple panoramic images called frames. The panoramic images have a wider field of view than the human field of view. The field of view of the panoramic image may be, for example, 360 degrees. The video may be recorded using, for example, a single video camera equipped with a wide-angle lens, or it may be generated by combining videos recorded using multiple video cameras that record in different directions. Alternatively, a 360-degree panoramic image of all directions (up, down, left, and right) may be used to construct the video, allowing training to be conducted in a situation closer to real space.

[0024] Furthermore, by setting the 0-degree direction as a specific direction in the panoramic image that constitutes the virtual space, the range from -180 degrees to 180 degrees in the panoramic image is defined. In this embodiment, the direction of movement at the trainee's position is used as the specific direction.

[0025] The display device 10 may also consist of a housing that is attached to the trainee's head and a smartphone attached to the housing. In this case, the smartphone includes the processor 11, memory 12, storage 13, communication unit 14, operation unit 15, display unit 16, sound output unit 17, sound pickup unit 18, and sensor unit 19. The smartphone is attached to the housing in a position such that the display unit 16 is visible to the trainee when the housing is attached to the trainee's head.

[0026] The display device 10 functions as a specific unit 101, a display control unit 102, an acquisition unit 103, an evaluation unit 104, and an output unit 105. These functions are realized through the cooperation of a program stored in memory 12 and a processor 11 that executes this program, with the processor 11 performing calculations or controlling communication by the communication unit 14.

[0027] The identification unit 101 identifies the trainee's field of view in the virtual space based on the trainee's position in the virtual space and the trainee's field of view direction in the real space detected by the sensor unit 19. An example of how to identify the field of view is described below. The orientation of the trainee's head in the real space when video playback starts is used as the specific direction in the panoramic image, and the orientation of the trainee's head (coordinate system in the real space) is associated with the orientation in the panoramic image (coordinate system in the virtual space). Then, the range between one end that forms a predetermined angle (e.g., 80 degrees) clockwise from the specific direction in the virtual space and the other end that forms a predetermined angle (e.g., 80 degrees) counterclockwise is identified as the trainee's field of view in the virtual space. That is, the field of view when video playback starts is in the range of -80 degrees to 80 degrees, and a part of the panoramic image corresponding to this field of view (-80 degrees to 80 degrees) is displayed on the display unit 16. Subsequently, the trainee's current field of view in the virtual space is identified according to the change in the trainee's head orientation detected by the sensor unit 19. For example, if the field of view is defined as being in the range of -80 to 80 degrees, and the sensor unit 19 detects that the orientation of the trainee's head has changed by 20 degrees counterclockwise, the identification unit 101, based on this detection result, identifies the range of -60 to 100 degrees (the entire field of view changed by 20 degrees counterclockwise) as the trainee's current field of view. The size of the field of view may be, for example, the size of a typical person's field of view.

[0028] The display control unit 102 displays on the display unit 16 the portion of the virtual space video that falls within the trainee's field of view, as identified by the identification unit 101, based on the video data stored in the storage unit 13. Specifically, the display control unit 102 extracts the field of view portion corresponding to the range of the trainee's field of view from each panoramic image included in the video shown by the video data, and displays it on the display unit 16. Since the trainee's field of view in the virtual space is updated in real time, the trainee can view the entire virtual space by rotating their head left and right in the real space.

[0029] The acquisition unit 103 acquires information used for evaluation by the evaluation unit 104 from various parts of the display device 10 and the controller 5. The information used for evaluation includes the trainee's movement distance, audio signals, posture information, and striking motion information. For example, the acquisition unit 103 acquires the trainee's movement distance from the sensor unit 19. The acquisition unit 103 also acquires the trainee's audio signals from the sound collection unit 18. Furthermore, the acquisition unit 103 receives the trainee's posture information and striking motion information from the controller 5 via the communication unit 14.

[0030] The evaluation unit 104 evaluates the trainee's actions toward a person appearing in the virtual space, depending on whether or not a response is required. Situations requiring a response include when a person appears in the virtual space, when this person is a target of vigilance, and when an attack from this person is imminent. In security work, different responses are required depending on these situations. Therefore, the evaluation unit 104 evaluates the trainee's actions depending on whether or not the appropriate actions for each situation were taken. The evaluation by the evaluation unit 104 includes a first to third evaluation, depending on the situation.

[0031] (First evaluation) The first evaluation assesses the trainee's reaction when a person appears in the virtual space. In security work, it is essential to quickly notice when a person appears. Therefore, in the first evaluation, the faster the trainee notices the person appearing in the virtual space, the higher their score will be. It should be noted that in the first evaluation, it is not necessary to judge whether the person appearing in the virtual space is a target of concern or not; the evaluation is limited to whether or not the trainee reacted to the person's appearance.

[0032] Specifically, the evaluation unit 104 measures the reaction time from when a person enters the trainee's field of view until the trainee brings the person to the center of their field of view, based on the trainee's field of view identified by the identification unit 101 and the direction of the trainee's field of view detected by the sensor unit 19. The timing when a person enters the trainee's field of view is obtained by identifying the timing when the person moves from outside to inside the range of the trainee's field of view identified by the identification unit 101 in the virtual space. The timing when the trainee brings the person to the center of their field of view is obtained by identifying the timing when the trainee's line of sight coincides with the person's position in the virtual space. The trainee's line of sight is, for example, the direction from the trainee's position in the virtual space toward the center of the trainee's field of view. Subsequently, the evaluation unit 104 performs an evaluation according to the reaction time based on the evaluation table 131 pre-stored in the storage 13.

[0033] Figure 3 shows an example of an evaluation table 131. The evaluation table 131 includes an evaluation table 131A for the first evaluation, an evaluation table 131B for the second evaluation, and an evaluation table 131C for the third evaluation. Each evaluation table 131 stores evaluation items, conditions, and evaluation points in association.

[0034] In the first evaluation, evaluation table 131A is used. Evaluation table 131A for the first evaluation includes "reaction time" as an evaluation item. This evaluation item is associated with the following conditions: "within 3 seconds" and a score of "10 points", "3 seconds to 7 seconds" and a score of "5 points", and "7 seconds or more" and a score of "0 points". This means that if the reaction time is within 3 seconds, the score is 10 points, if the reaction time is between 3 seconds and 7 seconds, the score is 5 points, and if the reaction time is 7 seconds or more, the score is 0 points. Thus, the shorter the reaction time, the faster the person appears in the virtual space, and therefore the higher the score.

[0035] In the example shown in Figure 3, the evaluation results are shown as points, but the evaluation results may also be shown in formats other than points, such as ranks like "A," "B," and "C," or "Pass" or "Fail" against the criteria.

[0036] (Second rating) The second evaluation assesses the trainee's actions when a person appearing in the virtual space is a person of concern, i.e., a suspicious person. The situation in which a person appearing in the virtual space is a person of concern is, in other words, a situation in which a person of concern appears within the trainee's field of vision in the virtual space, and is therefore an example of the first situation according to the present invention. In security work, when a person who appears is a person of concern, it is required to perform predetermined security actions. Therefore, in the second evaluation, the evaluation will be higher if the trainee performs predetermined security actions when a person appearing in the virtual space is a person of concern.

[0037] (Target of concern) A person who warrants vigilance is someone who should be kept on guard to ensure safety. Whether or not a person appearing in the virtual space is a person who warrants vigilance can be determined by their actions and appearance. A person exhibiting suspicious behavior or appearance may be considered a person who warrants vigilance. Examples of actions that warrant vigilance include appearing from behind an object or shouting. Examples of appearances that warrant vigilance include carrying a weapon or concealing one's face with a balaclava or full-face helmet. However, the conditions for someone to be considered a person who warrants vigilance are not limited to these, and other conditions may also be used.

[0038] (Warning action) A vigilance action refers to actions taken to ensure one's own safety while remaining vigilant against suspicious individuals and being able to respond quickly if necessary. Vigilance actions include maintaining distance, raising a baton, and calling out.

[0039] (Action to create distance) Maintaining distance is the action of securing a predetermined distance from a suspicious person. If the distance from a suspicious person is too close, safety cannot be ensured, and if the distance is too far, it is not possible to respond quickly when necessary. Therefore, in security work, it is necessary to maintain an appropriate distance from suspicious people.

[0040] The evaluation unit 104 measures the distance between the trainee and the suspicious person based on the trainee's movement distance acquired by the acquisition unit 103. The distance is obtained by measuring the distance between the trainee's position in the virtual space and the suspicious person's position after movement, in a state where the trainee's movement distance in the real space is reflected in the virtual space. Subsequently, the evaluation unit 104 performs an evaluation according to the distance based on the evaluation table 131 stored in the storage 13.

[0041] In the second evaluation, the evaluation table 131B for the second evaluation, shown in Figure 3, is used. The evaluation items in the evaluation table 131B for the second evaluation include "distance". This evaluation item is associated with the following conditions: "3m to 5m" and an evaluation score of "10 points", "1 to 3m, 5 to 10m" and an evaluation score of "5 points", and "within 1m, 10m or more" and an evaluation score of "0 points". This means that if the distance is between 3m and 5m, the evaluation score is 10 points; if the distance is between 1m and less than 3m, or between 5m and less than 10m, the evaluation score is 5 points; and if the distance is within 1m or 10m or more, the evaluation score is 0 points. Thus, the evaluation score is high when the distance is within the appropriate distance range, and decreases as it moves away from the appropriate distance range.

[0042] (A gesture of raising a baton) The action of holding a baton involves crouching down and assuming a stance with the baton held in front of one's body. To ensure one's own safety while intimidating suspicious individuals, the angle of the baton and the width of the feet are determined according to standards when holding a baton. In security work, it is required that the angle of the baton and the width of the feet when holding a baton are close to the standard values.

[0043] The evaluation unit 104 calculates the difference between the angle of the baton held by the trainee and the angle of the reference baton, and the difference between the width of the trainee's feet and the width of the reference feet, based on the trainee's posture information acquired by the acquisition unit 103. The angle of the baton is indicated by the angle of the controller 5, which is shown by the trainee's posture information. The width of the trainee's feet is estimated from the vertical position of the controller 5, which is shown by the trainee's posture information. When the trainee adopts a posture with their feet spread apart and their hips lowered, the position of the hands holding the controller 5 also naturally lowers. By utilizing this relationship between the position of the controller 5 and the width of the feet, the width of the trainee's feet can be estimated from the vertical position of the controller 5. Subsequently, the evaluation unit 104 performs an evaluation according to the movement of holding the baton, based on the evaluation table 131B for the second evaluation.

[0044] As shown in Figure 3, the evaluation items in the evaluation table 131B for the second evaluation include "stance with the baton." This evaluation item is associated with two conditions: "the difference between the angle of the baton and the reference angle is within a predetermined range" and an evaluation score of "5 points," and "the difference between the width of the feet and the reference width is within a predetermined range" and an evaluation score of "5 points." This means that if the difference between the angle of the trainee's baton and the angle of the reference baton is within a predetermined range, the evaluation score will be 5 points, and if the difference between the width of the trainee's feet and the reference width of the feet is within a predetermined range, the evaluation score will be 5 points. If the difference between the angle of the trainee's baton and the angle of the reference baton is outside the predetermined range, the evaluation score may be 0 points. Similarly, if the difference between the width of the trainee's feet and the width of the feet is outside the predetermined range, the evaluation score may be 0 points. Thus, when the angle of the trainee's baton and the width of their feet are close to the reference values, the evaluation score will be high, and when they deviate from the reference values, the evaluation score will be low.

[0045] (Speaking / Action) The act of calling out to a suspicious person involves asking them about their identity and clearly indicating that one is a security guard. In security work, it is necessary to speak loudly enough so that the suspicious person and those around can hear clearly.

[0046] The evaluation unit 104 measures the volume of the voice spoken to the trainee based on the voice signal acquired by the acquisition unit 103. The volume of the voice is indicated by the amplitude of the voice signal. Subsequently, the evaluation unit 104 performs an evaluation based on the evaluation table 131B, according to the volume of the voice spoken to the trainee.

[0047] As shown in Figure 3, the evaluation items in the evaluation table 131B for the second evaluation include "calling out." This evaluation item is associated with the following conditions: "voice volume is above the first threshold" and an evaluation score of "10 points," "voice volume is above the second threshold but below the first threshold" and an evaluation score of "5 points," and "voice volume is below the second threshold" and an evaluation score of "0 points." This means that if the voice volume during calling out is above the first threshold, the evaluation score is 10 points; if it is above the second threshold but below the first threshold, the evaluation score is 5 points; and if it is below the second threshold, the evaluation score is 0 points. Here, the first threshold is greater than the second threshold. Thus, the louder the voice during calling out, the higher the evaluation score.

[0048] (unlawful defense) In evaluation table 131B for the second evaluation, the evaluation item "unlawful self-defense" is associated with the condition "when an attack response action is taken" and the evaluation score "-10 points". This indicates that if a trainee takes an attack response action against a suspicious person appearing in the virtual space when there is no imminent attack from the suspicious person, the evaluation score will be -10 points. For example, if a person simply appears from behind an object, this person is merely a target of caution, and the situation does not reach one where an attack from a suspicious person is imminent. If an attack response action is taken when there is no imminent attack from a suspicious person, it may be considered unlawful self-defense. Therefore, if a trainee takes an attack response action in such a situation during training, their evaluation will be low.

[0049] (comprehensive evaluation) The evaluation unit 104 performs a second evaluation by comprehensively evaluating the evaluation items described above: "distance," "stance of the baton," "verbal communication," and "unjustified self-defense." For example, the evaluation unit 104 may use the sum or average of the evaluation scores for these evaluation items as the result of the second evaluation. Furthermore, if an attack response action is taken in a situation where an attack from a suspicious person is not imminent, the evaluation unit 104 may set the score for the second evaluation to 0, regardless of the evaluation results of the other evaluation items.

[0050] (3rd rating) The third evaluation assesses the trainee's actions in response to an imminent attack from a person appearing in a virtual space. An example of the second scenario according to the present invention is an imminent attack from a person appearing in a virtual space. In security work, when an attack from a suspicious person is imminent, it is necessary to perform appropriate attack response actions. Therefore, in the third evaluation, a higher score is given when the trainee performs appropriate attack response actions in response to an imminent attack from a person appearing in a virtual space.

[0051] (An imminent attack) An imminent attack situation refers to a situation in which a suspicious person is about to attack or is actually attacking. Whether or not an attack is imminent can be determined by the suspicious person's actions. Examples of an imminent attack situation include when a suspicious person approaches rapidly or when a suspicious person approaches while brandishing a weapon. However, an imminent attack situation is not limited to these situations and may include other situations as well.

[0052] (Action taken to deal with an attack) An offensive response action refers to an action taken out of necessity to protect oneself from imminent danger. Offensive response actions include striking actions for self-defense. For example, a trainee might use a baton to strike a suspicious person as an offensive response action. Evaluation criteria for offensive response actions include the point of impact, the size of the baton swing, and the relative positions of the trainee and the suspicious person before the offensive response action is performed.

[0053] (Area of ​​impact) The target area refers to the part of the suspicious person's body that the trainee strikes with a baton. In security operations, striking dangerous areas may be considered excessive force, so it is required to strike areas other than dangerous areas. These dangerous areas include the head and neck. On the other hand, striking the hand or shoulder holding the weapon is preferable because it can reduce the attacker's ability to attack.

[0054] The evaluation unit 104 identifies the striking area with the baton based on the traininger's striking motion information acquired by the acquisition unit 103. The striking area is obtained by identifying the part of the suspicious person's body that comes into contact with the baton within the striking motion range, in a state where the traininger's striking motion in real space is reflected in the virtual space. Subsequently, the evaluation unit 104 performs an evaluation according to the striking area based on the evaluation table 131 stored in the storage unit 13.

[0055] In the third evaluation, the evaluation table 131C for the third evaluation, shown in Figure 3, is used. The evaluation items in the evaluation table 131C for the third evaluation include "target of strike." This evaluation item is associated with the following conditions: "strike on a dangerous area" and a score of "-10 points," "strike on the shoulder" and a score of "5 points," and "strike on the hand holding the weapon" and a score of "10 points." This means that if the target of the strike with the baton is a dangerous area, the score will be -10 points; if it is the shoulder, the score will be 5 points; and if it is the hand holding the weapon, the score will be 10 points. Thus, the score will be lower if the target of the strike with the baton is a dangerous area, and the score will be higher the more effective the target is in reducing the opponent's offensive ability.

[0056] (size of the wind-up) The "size of the swing" refers to the range of motion a trainee takes when striking with a baton, pulling it back. In security work, it is necessary to swing the baton widely when responding to an attack in order to intimidate a suspicious person. However, an excessively large swing is an unnecessary movement and may cause delays in response depending on the situation, so it is preferable to be close to the standard value that indicates an appropriate swing size.

[0057] The evaluation unit 104 calculates the difference between the magnitude of the trainee's swing of the baton and the standard swing magnitude, based on the trainee's striking motion information acquired by the acquisition unit 103. The magnitude of the baton swing is indicated by the maximum angle of the swing range of the controller 5 shown by the striking motion information. Subsequently, the evaluation unit 104 performs an evaluation according to the magnitude of the baton swing based on the evaluation table 131C.

[0058] As shown in Figure 3, the evaluation items in the evaluation table 131C for the third evaluation include "size of swing". This evaluation item is associated with the condition "the difference between the size of the baton swing and the standard is within a predetermined range" and an evaluation score of "5 points". This indicates that if the difference between the size of the baton swing and the standard swing size is within a predetermined range, the evaluation score will be 5 points. If the difference between the size of the baton swing and the standard swing size is outside the predetermined range, the evaluation score may be 0 points. Thus, the evaluation score will be high when the size of the baton swing is close to the standard value, and low when it deviates from the standard value.

[0059] Alternatively, instead of evaluating the size of the wind-up, or in addition to it, the batting motion from the wind-up position (angle, distance, speed) may be evaluated. Similar to the evaluation of the size of the wind-up, the evaluation is performed based on whether the difference from the reference angle, distance, and speed is within a predetermined range, etc.

[0060] (Positional relationship) In security work, when performing an attack response, it is necessary to face the suspicious person directly before taking action, in order to aim at them and then strike. This means that the trainee and the suspicious person are aligned in a straight line and facing each other directly.

[0061] The evaluation unit 104 determines whether the trainee is facing the intruder directly in the virtual space before performing an attack response action, based on the trainee's field of view direction detected by the sensor unit 19. The determination of whether the trainee is facing the intruder directly is made by identifying the trainee's line of sight direction in the virtual space based on the trainee's field of view direction in the real space, and calculating the angle difference between the trainee's line of sight direction before performing the attack response action and the intruder's frontal direction. If this angle difference is within a predetermined range indicating that they are facing almost directly opposite each other, the security guard indicates that they are facing the intruder directly. Subsequently, the evaluation unit 104 performs an evaluation based on the evaluation table 131C, according to the positional relationship between the trainee and the intruder before performing the attack response action.

[0062] As shown in Figure 3, the evaluation items in evaluation table 131C for the third evaluation include "positional relationship." This evaluation item is associated with the condition "facing the suspect directly before performing an attack response action" and an evaluation score of "5 points." This indicates that if the trainee is facing the suspect directly before performing an attack response action, the evaluation score will be 5 points. If the trainee is not facing the suspect directly before performing an attack response action, the evaluation score may be 0 points. Thus, the evaluation score will be higher if the trainee is facing the suspect directly before performing an attack response action, and lower if they are not.

[0063] (unlawful defense) In evaluation table 131C for the third evaluation, the evaluation item "unlawful self-defense" is associated with the conditions "attack response action is taken when there is no imminent attack" and "attack response action is taken three or more times" and the evaluation score "-10 points". This indicates that if a trainee takes an attack response action when there is no imminent attack from a suspicious person, or if a trainee takes an attack response action against a suspicious person three or more times, the evaluation score will be -10 points. Such attack response actions may be considered unlawful self-defense. Therefore, if a trainee takes such an attack response action, the evaluation will be low.

[0064] (comprehensive evaluation) The evaluation unit 104 performs a third evaluation by comprehensively evaluating the evaluation items of "point of impact," "size of wind-up," "positional relationship," and "unlawful self-defense." For example, the evaluation unit 104 may use the sum or average of the evaluation scores for these evaluation items as the result of the third evaluation. In addition, if the trainee performs an attack response action in a situation where there is no imminent attack from a suspicious person, or if the trainee performs an attack response action three or more times, the evaluation unit 104 may set the score for the third evaluation to 0 points, regardless of the evaluation results of the other evaluation items.

[0065] Returning to Figure 1, the management device 20 collects and outputs the trainee's evaluation results from each display device 10. For example, a general-purpose computer may be used for the management device 20. The management device 20 may have the same configuration as the processor 11, memory 12, storage 13, communication unit 14, operation unit 15, and display unit 16 described above.

[0066] (operation) Figure 4 is a flowchart illustrating an example of the operation of the display device 10. This operation begins, for example, when a signal instructing the display device 10 to start training is input. This signal may be input, for example, by operation using the operation unit 15, or it may be transmitted from the management device 20.

[0067] In step S11, the display device 10 starts training on how to deal with suspicious individuals in a virtual space. Specifically, the display control unit 102 starts playing a video of the content for dealing with suspicious individuals based on the video data stored in the storage 13. The identification unit 101 identifies the trainee's field of view in the virtual space based on the trainee's position in the virtual space and the direction of the trainee's field of view in the real space detected by the sensor unit 19. The display control unit 102 extracts the field of view portion corresponding to the range of the trainee's field of view from each panoramic image included in the video shown by the video data stored in the storage 13 and displays it on the display unit 16. This video simulates the field of view of a trainee moving along a predetermined path. Therefore, in the video, the position of the camera viewpoint automatically moves along the predetermined path. Also, when the trainee moves their gaze, the field of view in the virtual space moves in the corresponding direction accordingly.

[0068] Figure 5 is a plan view showing an example of a virtual space. This virtual space simulates a building 60 such as an office building, and the trainee 61 automatically moves along a predetermined route within the virtual space, patrolling the building 60 or performing inspections in response to any abnormalities. The training scenario also includes a scenario in which the trainee encounters a suspicious person during patrol or inspection in response to an abnormality.

[0069] In step S12, a person appears in front of the trainee in the virtual space. Specifically, when the video displayed on the display unit 16 reaches a scene in which a person appears, the person appears in front of the trainee in the virtual space. For example, as shown in Figure 5, when trainee 61 is moving in the direction of arrow A in the figure, person 62 appears in front of trainee 61 from behind an object.

[0070] Figure 6 shows an example of a video displayed on the display unit 16 when a person 62 appears in front of the trainee 61 in the virtual space. The person 62 is included in the foreground of this video. Also included in the foreground of this video is a baton 63 corresponding to the controller 5.

[0071] In step S13, when a person appears in the virtual space, the display device 10 performs a first evaluation to determine whether or not the user has reacted to the appearance of this person. Specifically, the evaluation unit 104 measures the reaction time from when the person 62 shown in Figure 5 enters the trainee's field of view until the trainee 61 focuses on the person 62 in the center of their field of view, based on the trainee's field of view identified by the identification unit 101 and the direction of the trainee's field of view in the real space detected by the sensor unit 19, and performs an evaluation according to this reaction time. For example, if the reaction time is 2 seconds, the evaluation unit 104 determines an evaluation score of "10 points" associated with the condition "within 3 seconds" in the evaluation table 131A shown in Figure 3.

[0072] In step S14, the display device 10 determines whether the person who appeared in the virtual space is a person of interest. If the actions and appearance of the person who appeared in the virtual space do not meet the conditions for being a person of interest, then this person is not a person of interest (the determination in step S14 is NO), and the process returns to step S12. On the other hand, if, for example, a person appears from behind an object as shown in Figure 5, then the conditions for being a person of interest are met. In this case, this person is a person of interest (the determination in step S14 is YES), and the process proceeds to the next step S15.

[0073] In step S15, the display device 10 performs a second evaluation to assess whether a predetermined alert action has been performed against a person who has appeared in the virtual space and is a target of alert. Specifically, the acquisition unit 103 acquires the trainee's movement distance from the sensor unit 19, the trainee's voice signal from the sound pickup unit 18, and the trainee's posture information from the controller 5. First, the evaluation unit 104 measures the distance between the trainee 61 and the person 62 shown in Figure 5 based on the trainee's movement distance acquired by the acquisition unit 103. For example, if this distance is 4m, the evaluation unit 104 determines an evaluation score of "10 points" associated with the condition "3m~5m" in the evaluation table 131B shown in Figure 3.

[0074] Next, the evaluation unit 104 calculates the difference between the angle of the baton held by the trainee and the angle of the reference baton, and the difference between the width of the trainee's feet and the width of the reference feet, based on the trainee's posture information acquired by the acquisition unit 103. For example, if the difference between the angle of the controller 5 indicated by the trainee's posture information and the angle of the reference baton is within a predetermined range, the evaluation unit 104 determines an evaluation score of "5 points" in the evaluation table 131B shown in Figure 3, associated with the condition that "the difference between the angle of the baton and the reference angle is within a predetermined range." Also, if the difference between the width of the trainee's feet indicated by the trainee's posture information and the width of the reference feet is within a predetermined range, the evaluation unit 104 determines an evaluation score of "5 points" in the evaluation table 131B shown in Figure 3, associated with the condition that "the difference between the width of the feet and the reference width is within a predetermined range."

[0075] Furthermore, the evaluation unit 104 measures the volume of the voice spoken to the trainee based on the voice signal acquired by the acquisition unit 103. For example, if the volume of the voice is below the second threshold, the evaluation unit 104 determines an evaluation score of "0 points" in the evaluation table 131B shown in Figure 3, which is associated with the condition "volume of voice is below the second threshold".

[0076] The evaluation unit 104 then calculates 10 + 5 + 5 + 0 = 20 points, which is the sum of the evaluation points determined in the second evaluation, and uses this as the evaluation score for the second evaluation.

[0077] At this point, the situation has not yet reached a point where an attack from the person appearing in the virtual space is imminent. Therefore, if the trainee were to perform an attack response action against the person at this point, the evaluation unit 104 would determine an evaluation score of "-10" based on the evaluation table 131B shown in Figure 3.

[0078] In step S16, the display device 10 determines whether an attack from the person who has appeared in the virtual space is imminent. For example, if the person who has appeared in the virtual space remains in place, an attack from this person is not imminent (the determination in step S16 is NO), and the determination in step S16 is repeated. On the other hand, if, for example, the person who has appeared in the virtual space runs towards the trainee while brandishing a weapon, this corresponds to an imminent attack from this person. In this case, since an attack from this person is imminent (the determination in step S16 is YES), the process proceeds to the next step S17.

[0079] In step S17, the display device 10 performs a third evaluation to assess whether the trainee performed appropriate attack response actions in response to an imminent attack from a person appearing in the virtual space. Specifically, the acquisition unit 103 acquires information on the trainee's striking actions from the controller 5. The evaluation unit 104 first identifies the part of the body the trainee struck with the baton based on the information on the trainee's striking actions acquired by the acquisition unit 103. For example, if the body was struck on the head, the head is a dangerous area, so the evaluation unit 104 determines an evaluation score of "-10 points" associated with the condition "striked a dangerous area" in the evaluation table 131C shown in Figure 3.

[0080] Next, the evaluation unit 104 calculates the difference between the magnitude of the trainee's swing of the baton and the standard swing magnitude, based on the trainee's striking motion information acquired by the acquisition unit 103. For example, if the difference between the magnitude of the baton swing and the standard swing magnitude is within a predetermined range, the evaluation unit 104 determines an evaluation score of "5 points" in the evaluation table 131C shown in Figure 3, associated with the condition "the difference between the magnitude of the baton swing and the standard is within a predetermined range."

[0081] Furthermore, the evaluation unit 104 calculates the angular difference between the trainee's line of sight before performing the striking motion and the frontal direction of the person appearing in the virtual space, based on the trainee's line of sight direction in the real space detected by the sensor unit 19. If this angular difference is within a predetermined range, the evaluation unit 104 determines an evaluation score of "5 points" in the evaluation table 131C shown in Figure 3, which is associated with the condition "facing directly before performing the striking motion".

[0082] The evaluation unit 104 then calculates -10 + 5 + 5 = 0 points, which is the sum of the evaluation points determined in the third evaluation, and uses this as the evaluation score for the third evaluation.

[0083] Furthermore, if a trainee performs offensive actions against a suspicious person more than necessary, it may constitute excessive self-defense. Therefore, if a trainee performs offensive actions against a suspicious person three or more times, the evaluation unit 104 determines an evaluation score of "-10" based on the evaluation table 131C shown in Figure 3.

[0084] Furthermore, it is preferable for the trainee to perform a defensive action against a suspicious person appearing in the virtual space, and then perform a vigilant action again. Therefore, the evaluation unit 104 performs a second evaluation to assess whether the trainee performed a predetermined vigilant action against the person appearing in the virtual space, similar to step S15 described above, after performing a defensive action against the person. However, if the trainee has already spoken to the person during the aforementioned vigilant action, the trainee does not need to speak to the person again. Therefore, the evaluation regarding the speaking action may be omitted in this second evaluation.

[0085] In step S18, the playback of the training video for dealing with suspicious individuals is completed. Specifically, the display control unit 102 plays the training video for dealing with suspicious individuals to the end, and when the training scenario is completed, the playback of the training video for dealing with suspicious individuals is completed.

[0086] In step S19, the display device 10 outputs the trainee's evaluation results. Specifically, the output unit 105 displays an evaluation results screen showing the evaluation results of the first to third evaluations on the display unit 16. The output unit 105 may also transmit the evaluation results of the first to third evaluations to the management device 20 or the training administrator's terminal device via the communication unit 14. Once the trainee's evaluation results are output, the suspicious person countermeasure training ends.

[0087] Figure 7 shows an example of the evaluation results screen 161. This evaluation results screen 161 includes the evaluation score of "10 points" and evaluation comments for the first evaluation, the evaluation score of "20 points" and evaluation comments for the second evaluation, and the evaluation score of "0 points" and evaluation comments for the third evaluation. In the example shown in Figure 7, the evaluation results for the first to third evaluations are presented separately, but it is also possible to present only an overall evaluation result that combines the evaluation results of the first to third evaluations.

[0088] According to the embodiment described above, a first evaluation is conducted when a person appears in the virtual space, a second evaluation is conducted if this person is a target of concern, and a third evaluation is conducted if an attack from this person becomes imminent. In the first to third evaluations, it is evaluated whether the trainee accurately judged the situation requiring a response and took appropriate action according to that situation. Therefore, in training on dealing with suspicious persons conducted in a virtual space, it is possible to objectively evaluate the ability to respond to situations required in real security work. Furthermore, through this evaluation, trainees can improve their judgment and response abilities required in real security work.

[0089] Furthermore, in the second evaluation, if a person appearing in the virtual space is a target of vigilance, the trainee's actions are evaluated based on whether or not a predetermined vigilance action was performed towards that person. By utilizing these evaluation results, trainees will be able to perform the predetermined vigilance action when encountering a suspicious person in actual security work.

[0090] Furthermore, in the third evaluation, the trainee's actions are assessed based on whether or not they took attack countermeasures against a person appearing in the virtual space, in response to an imminent attack from that person. By utilizing these evaluation results, trainees will be able to take appropriate attack countermeasures when faced with an imminent attack from a suspicious person in actual security work.

[0091] (modified version) The present invention is not limited to the embodiments described above. Various modifications may be made to the embodiments described above. Furthermore, the following modifications may be implemented in combination.

[0092] (Variation 1) In the embodiment described above, when a person appears in the virtual space, the sound output unit 17 may output a sound, and the evaluation unit 104 may measure the time from when the sound is output until the trainee places the person in the center of their field of vision as the reaction time. In this modified configuration as well, the first evaluation can assess whether the trainee reacted appropriately to the appearance of the person.

[0093] (Modification 2) In the embodiment described above, the second evaluation criterion may include the content of the verbal communication. Preferably, the verbal communication includes a phrase that asks the suspicious person for their identity and a phrase that clearly indicates that the person is a security guard. Therefore, the evaluation unit 104 recognizes the content of the verbal communication by applying speech recognition processing to the trainee's voice signal acquired by the acquisition unit 103, and may give a higher evaluation if the trainee's verbal communication includes these phrases, and a lower evaluation if at least one of these phrases is not included. According to this modification, the trainee's behavior can be evaluated taking into account the content of the verbal communication.

[0094] (Variation 3) In the embodiment described above, the second evaluation criterion may include the positional relationship between the trainee, the suspicious person, and the civilian. In security work, if a civilian is near a suspicious person, the trainee may be required to move between the suspicious person and the civilian to ensure the civilian's safety. Therefore, the evaluation unit 104 identifies the positional relationship between the trainee, the suspicious person, and the civilian after movement, based on the trainee's movement distance acquired by the acquisition unit 103, in a state where the trainee's movement distance in real space is reflected in the virtual space. The evaluation unit 104 may give a higher evaluation if the trainee is positioned between the suspicious person and the civilian. According to this modification, the trainee's actions can be evaluated taking into account the positional relationship between the trainee, the suspicious person, and the civilian.

[0095] (Modification 4) In the embodiment described above, the third evaluation criterion may include the strength of the strike in the attack response action. In an attack response action, if the strike is too weak, it may not be possible to stop the intruder's actions, while if the strike is too strong, it may be considered excessive self-defense. Therefore, the evaluation unit 104 may, for example, use the strength of the strike in the attack response action of a model instructor as a standard, and give a higher evaluation if the difference between the strength of the trainee's strike and the standard strike strength is within a predetermined range, and a lower evaluation if it is outside the predetermined range. Furthermore, the evaluation unit 104 may use AI (Artificial Intelligence) to learn the standard strike strength through machine learning. According to this modification, the trainee's actions can be evaluated taking into account the strength of the strike in the attack response action.

[0096] (Variation 5) In the embodiments described above, the first, second, and third evaluations do not necessarily have to be performed in their entirety; only one or two of these evaluations may be performed. For example, the first evaluation may not be performed, and only the second and third evaluations may be performed.

[0097] (Experimental variation 6) In the embodiments described above, the evaluation items for the first to third evaluations are not limited to those shown in Figure 3. The evaluation items for the first to third evaluations may be some of those shown in Figure 3, or other evaluation items may be included. For example, the warning action in the second evaluation does not have to include all of the actions of creating distance, holding the warning baton, and calling out, and may include only one or two of these actions. In this case, the evaluation items for the second evaluation may include only one or two evaluation items from "distance," "holding the warning baton," and "calling out" that correspond to the warning action. Similarly, the evaluation items for the third evaluation do not have to include all of "striking area," "size of swing," and "positional relationship," and may include only one or two of these evaluation items.

[0098] (Example 7) In the embodiment described above, the management device 20 may perform the process of evaluating the trainee's actions when encountering a suspicious person in the virtual space, instead of the display device 10. In this case, the management device 20 has the same configuration as the acquisition unit 103, evaluation unit 104, and output unit 105 described above. The management device 20 acquires the information necessary for the first to third evaluations from the display device 10, performs the first to third evaluations using this information, and transmits the evaluation results to the display device 10. Even with this modified configuration, the trainee's actions when encountering a suspicious person can be evaluated.

[0099] (Variation 8) In the embodiments described above, the trainee's movements may be detected using well-known motion capture technology. In this case, the trainee may move within the virtual space by performing walking movements. That is, the trainee's position in the virtual space may change in accordance with the trainee's walking movements. In other examples, the trainee's position in the virtual space may change through operations using the operation unit 15.

[0100] (Extreme variation 9) In the embodiments described above, the display unit 16 may be a device for projecting images. For example, the display unit 16 may be a projection unit such as a projector and a display surface on which images are displayed, or it may be a projection unit that projects images directly onto the retina. Furthermore, the display device 10 is not limited to a head-mounted wearable terminal. For example, the display device 10 may be a glasses-type wearable terminal.

[0101] (Variation 10) In the embodiment described above, video data may be transmitted from the management device 20 to the display device 10. In this case, for example, the video data may be distributed from the management device 20 to multiple display devices 10 used in different locations via a wide-area network such as the Internet.

[0102] (Variation 11) In the embodiments described above, the management device 20 is not necessarily required. For example, the display device 10 may be used on its own.

[0103] (Example 12) In the embodiments described above, the training system 1 may be used not only for training using VR, but also for training using AR (Augmented Reality) or MR (Mixed Reality).

[0104] (Example 13) In the embodiments described above, the processing steps performed in the training system 1 are not limited to the examples described in the embodiments above. These processing steps may be replaced as long as they do not contradict each other. The present invention may also be provided as a method comprising the processing steps performed in the training system 1.

[0105] (Variation 14) The present invention may be provided as a program executed in the controller 5, display device 10, and management device 20. This program may be downloaded via a communication line such as the Internet. Alternatively, this program may be provided recorded on a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or a semiconductor memory. [Explanation of symbols]

[0106] 1: Training system, 5: Controller, 10: Display device, 11: Processor, 12: Memory, 13: Storage, 14: Communication unit, 15: Operation unit, 16: Display unit, 17: Sound output unit, 18: Sound collection unit, 19: Sensor unit, 20: Management device, 51: Sensor unit, 101: Identification unit, 102: Display control unit, 103: Acquisition unit, 104: Evaluation unit, 105: Output unit

Claims

1. A training system that evaluates the behavior of trainees in a virtual space, A storage unit that stores field-of-view images from the trainee's position in the virtual space, A detection unit for detecting the field of view direction of the trainee in real space, A identifying unit that identifies the trainee's field of view in the virtual space based on the trainee's position and field of view direction in the virtual space, A display unit that displays the portion of the aforementioned field of view image that is included in the field of view, An evaluation unit evaluates the trainer's actions toward a person in the virtual space included in the field of view, depending on whether or not it is necessary to respond to the person in question. A training system equipped with the following features.

2. The evaluation unit evaluates the trainee's actions in accordance with each of the multiple situations that have been set as situations requiring action. The training system according to claim 1.

3. The situations requiring the aforementioned response include at least the first situation in which a person of interest appears within the field of view, The evaluation unit evaluates the trainer's actions in the first situation based on whether or not a predetermined alert action was performed against the person in question. The training system according to claim 1 or 2.

4. The aforementioned prescribed warning action includes at least one of the following: taking distance from the person, raising a baton, and calling out to the person. The training system according to claim 3.

5. The situations requiring the aforementioned response include at least the second situation in which an attack from the aforementioned person is imminent, The evaluation unit evaluates the trainer's actions in the second situation depending on whether or not an attack response action was performed against the person in question. The training system according to claim 1 or 2.

6. The evaluation unit evaluates the trainee's actions based on at least one of the following: the part of the body the trainee strikes with the baton during the attack response action, the magnitude of the swing of the baton, and the positional relationship between the trainee and the person before the attack response action is performed. The training system according to claim 5.

Images