Training support apparatus, training support method, and computer-readable recording medium

Abstract

To enable efficient education support.SOLUTION: The training support apparatus 100 constructs a virtual space on the basis of the acquired image data and the history information of the motion of the first user included in the image data. The education support device 100 collects history information on the action of the second user in the constructed virtual space. The education support device 100 outputs feedback to the second user on the basis of the collected history information of the motion of the user.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an education support device, an education support method, and an education support program. [Background technology]

[0002] Education and guidance may be provided in various fields such as medicine, industry, sports, etc. Examples of education and guidance include classroom education using textbooks and videos, and practical training in which the educational content is put into practice.

[0003] Therefore, as a technology for efficiently providing education and guidance, for example, a technology is known in which an education screen is automatically displayed according to the student's movements obtained from measurement information for grasping the student's movements and photographing information taken by a photographing unit (see, for example, Patent Document 1). The above-mentioned conventional technology makes it possible to provide highly effective education to students. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2024-070364 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional technologies, it is sometimes difficult to provide efficient educational support. For example, when training is provided on dangerous tasks such as rescue operations at fire scenes, it is difficult with conventional technologies to accurately reproduce the fire scene and rescue operations at the fire scene so that users can practice rescue operations. [Means for solving the problem]

[0006] Therefore, in order to solve the above-mentioned problems and achieve the objectives, the educational support device of the present invention is characterized by having a construction unit that constructs a virtual space based on acquired image data and historical information of a first user's actions contained in the image data, a collection unit that collects historical information of a second user's actions in the virtual space constructed by the construction unit, and an output unit that outputs feedback to the second user based on the historical information of the user's actions collected by the collection unit. [Effects of the Invention]

[0007] The present invention has the effect of enabling efficient educational support. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an overall view of the processing of an education support device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a configuration of the education support device according to the embodiment. [Figure 3] FIG. 3 is a table illustrating an example of image data according to the embodiment. [Figure 4] FIG. 4 is a table diagram illustrating an example of sensor data according to the embodiment. [Figure 5] FIG. 5 is a table diagram illustrating an example of operation history information according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of educational support according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating the processing performed by the education support device according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating IOWN. [Figure 9] FIG. 9 is a diagram illustrating an example of a computer that realizes the education support device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention (hereinafter referred to as "embodiments") will be described with reference to the drawings. Note that the embodiments are not limited to the following description.

[0010] <Overview> (background) When education or instruction is provided in various fields such as medicine, industry, and sports, it is done through classroom education using textbooks, videos, etc., or practical education in which the educational content is put into practice, etc. Therefore, as a technology for efficiently providing education or instruction, for example, a reference technology is known that automatically displays an educational screen in accordance with the user's (student's) movements detected using measurement information on the user's (student's) movements and video information captured by a camera, etc.

[0011] The above-mentioned reference technology can provide highly effective education to trainees (users), but it may be difficult to provide effective educational support in fields related to dangerous work such as rescue operations at disaster sites. For example, education may be provided on dangerous work such as rescue operations at fire sites, but the reference technology has difficulty in accurately reproducing the fire site and rescue operations at the fire site so that users can practice rescue operations.

[0012] (Processing by the education support device 100) Therefore, the education support device 100 according to this embodiment displays to the user a virtual space constructed using still images and moving images (hereinafter, sometimes simply referred to as "image data") captured by an imaging device such as a camera. The education support device 100 then collects historical information about the user's actions in the virtual space and outputs feedback about the actions to the user.

[0013] Here, an overview of the processing by the education support device 100 will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an overview of the processing by the education support device 100 according to an embodiment. The education support device 100 shown in Fig. 1 is an example of a computer that executes the processing described below.

[0014] First, the education support device 100 constructs a virtual space based on the acquired image data and the history information of the first user's actions contained in the image data ((1) in FIG. 1). Then, the education support device 100 displays the virtual space including the three-dimensional structure to the user ((2) in FIG. 1).

[0015] Next, the educational support device 100 collects historical information on the second user's actions in the constructed virtual space based on image data of the user's actions or sensor data acquired by sensors attached to the user himself or to tools, machines, etc. used by the user ((3) in Figure 1).

[0016] The action history information is information about the trajectory of the user's action in the virtual space, the action speed, the amount of force applied to the action, etc. Hereinafter, "action history information" may be referred to as "action history information."

[0017] Then, the education support device 100 outputs feedback to the second user based on the collected user action history information ((4) in FIG. 1).

[0018] In this way, the education support device 100 according to this embodiment creates a virtual space and displays it to the user, and generates feedback using the user's action history information in the virtual space.

[0019] As a result, the education support device 100 enables the user to perform more realistic simulated movements when learning simulated movements, and also enables feedback of the differences in the simulated movements between the user and experienced users, movements that the user is weak at, etc. As a result, the education support device 100 has the effect of enabling effective education support.

[0020] <Explanation of the Educational Support Device 100> Next, the configuration of the education support device 100 according to this embodiment will be described. Fig. 2 is a diagram showing the configuration of the education support device 100 according to this embodiment. As shown in Fig. 2, the education support device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Although not shown in Fig. 2, the education support device 100 can also have an input unit such as a keyboard or a mouse for receiving input such as operations by an administrator or the like.

[0021] (Communication unit 110) The communication unit 110 performs data communication related to input of information such as image data captured by the imaging device 200, user operation history information collected by the sensor 300, and predetermined user commands input from the terminal device 400. The communication unit 110 also performs data communication related to output of information related to the virtual space and the constructed three-dimensional structure.

[0022] The communication unit 110 is realized by a NIC (Network Interface Card) or the like, and controls communication via an electric communication line such as a LAN (Local Area Network), the Internet, etc. The communication unit 110 is connected to a network by wire or wirelessly as necessary, and can transmit and receive information bidirectionally with the imaging device 200, the sensor 300, the terminal device 400, etc.

[0023] (Storage unit 120) The storage unit 120 stores data and programs used for various processes by the control unit 130, and various data acquired by the operation of the control unit 130. The storage unit 120 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 2 , the storage unit 120 has an image data DB 121, a sensor data DB 122, a virtual space DB 123, a three-dimensional structure DB 124, and an operation history information DB 125.

[0024] (Image data DB121) The image data DB 121 is a database that stores image data including still images and moving images captured by the imaging device 200. An example of image data stored in the image data DB 121 will now be described with reference to FIG.

[0025] Fig. 3 is a table diagram showing an example of image data according to the embodiment. As shown in Fig. 3, the image data DB 121 stores imaging targets and imaging data in association with "No.", which is information for identifying individual image data. For example, as shown in Fig. 3, the image data DB 121 stores imaging target "A" and imaging data "video data a" in association with No. "1." Note that "A," "video data a," and the like shown in Fig. 3 are legends for the data.

[0026] The "image capture subject" refers to the target space or object that is captured by the image capture device 200, and includes, for example, actual surgery scenes, sports play scenes, rescue scenes at disaster sites, and dangerous work scenes using power saws.

[0027] "Imaging data" refers to still image data or moving image data captured by imaging device 200, and includes, for example, JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), PNG (Portable Network Graphics), etc. Moving image data also includes data saved in formats such as MP4 (MPEG-4 Part 14), AVI (Audio Video Interleave), MOV, WMV (Windows (registered trademark) Media Video), FLV, WebM, MPEG (Moving Picture Experts Group), MPEG2 (Generic coding of moving pictures and associated audio information), and MKV (Matroska Video File).

[0028] (Sensor data DB122) The sensor data DB 122 is a database that stores sensor data relating to user actions acquired by the sensor 300 etc. An example of the sensor data stored in the sensor data DB 122 will now be described with reference to FIG.

[0029] Fig. 4 is a table diagram showing an example of sensor data according to the embodiment. As shown in Fig. 4, the sensor data DB 122 stores a sensor name and acquired data in association with "No." This "No." is information that identifies individual sensor data. For example, as shown in Fig. 4, the sensor data DB 122 stores a sensor name "C" and acquired data "c" in association with No. "1." Note that the letters "C" and "c" shown in Fig. 3 are legends for the data.

[0030] "Sensor name" is information that identifies a sensor attached to a user or an appliance or machine operated by the user, and includes, for example, the name or identification symbol of an acceleration sensor (e.g., a 3-axis acceleration sensor), a gyro sensor (e.g., a 3-axis gyro sensor), a magnetic sensor, a pressure sensor, a temperature sensor, a motion sensor, etc. "Acquired data" is individual sensor data acquired by the various sensors described above. Note that the data format of the acquired data is not particularly limited.

[0031] (Virtual Space DB123) The virtual space DB 123 is a database that stores data on the virtual space constructed by the construction unit 132. For example, the virtual space data stores information on the size of the virtual space, the functions of the virtual space, the materials that make up the virtual space, the interactions between the materials, and the like.

[0032] The virtual space data is not particularly limited and may include various information used to configure a virtual space constructed using known technology. The virtual space may also include a digital twin constructed based on Digital Twin Computing (DTC) of the Innovative Optical and Wireless Network (IOWN), which will be described later.

[0033] (Three-dimensional structure DB124) The three-dimensional structure DB 124 is a database that stores data of three-dimensional structures to be placed in a virtual space constructed by the construction unit 132. For example, the data of the three-dimensional structures may include, but is not limited to, data on a 3DCG (3 Dimensional Computer Graphics) model constructed by a well-known technique such as photogrammetry.

[0034] (Operation history information DB125) The operation history information DB 125 is a database that stores history information of user operations (operation history information) collected by the later-described collection unit 134. Here, an example of image data stored in the operation history information DB 125 will be described with reference to FIG.

[0035] 5 is a table diagram showing an example of operation history information according to an embodiment. As shown in FIG. 5, the operation history information DB 125 stores user identification information and history information in association with "No." This "No." is information that identifies individual operation history information. For example, as shown in FIG. 5, the operation history information DB 125 stores user identification information "E" and history information "e" in association with No. "1." Note that the letters "E" and "e" shown in FIG. 5 are legends for the data.

[0036] "User identification information" is information for identifying a user, and includes, for example, a username, a combination of unique numbers, text, symbols, etc. assigned to a user, and the like.

[0037] "History information" refers to history information related to the user's actions collected by the collection unit 134, which will be described later. For example, the history information includes information such as the coordinates (starting point) from which the user's body moved to which coordinates (ending point) in the target space, and at what speed. The history information also includes information such as the amount of force with which the user applied the action to the target three-dimensional structure.

[0038] (control unit 130) Now, returning to Fig. 2, the explanation will be continued. The control unit 130 has an internal memory for temporarily storing programs defining various processing procedures and the like of the education support device 100 and processing data, and is realized by electronic circuits such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). As shown in Fig. 2, the control unit 130 has an acquisition unit 131, a construction unit 132, a display unit 133, a collection unit 134, a calculation unit 135, and an output unit 136.

[0039] (Acquisition part 131) The acquisition unit 131 acquires predetermined information from the imaging device 200, the sensor 300, etc. Specifically, the acquisition unit 131 acquires image data including still image data and moving image data captured by the imaging device 200, and stores the image data in the image data DB 121. For example, the acquisition unit 131 acquires imaging data "moving image data a" about imaging target "A," associates it with No. "1," and stores it in the image data DB 121.

[0040] Furthermore, the acquiring unit 131 acquires sensor data from the sensor 300 and stores it in the sensor data DB 122. For example, the acquiring unit 131 acquires sensor data "c" of a sensor name "C", associates it with No. "1", and stores it in the sensor data DB 122.

[0041] (Construction Section 132) The construction unit 132 constructs a virtual space and a three-dimensional structure, and stores them in the virtual space DB 123 and the three-dimensional structure DB 124, respectively.

[0042] Specifically, the construction unit 132 constructs a virtual space based on image data captured by the imaging device 200, image data input via an external device, and the first user's action history information included in the image data. Furthermore, the construction unit 132 can use the acquired image data to construct a three-dimensional structure to be placed in the virtual space based on photogrammetry, a well-known technology.

[0043] (Display section 133) The display unit 133 displays to the user the virtual space and the three-dimensional structure constructed by the construction unit 132. Specifically, the display unit 133 displays the virtual space and the three-dimensional structure to the user via a head-mounted display worn by the user, a space reproduction display, or the like.

[0044] In addition to the head-mounted display and spatial reproduction display described above, the display unit 133 can display virtual spaces and three-dimensional structures to the user via devices that output information via the five human senses, such as visual devices and tactile devices.

[0045] (Collection Department 134) The collection unit 134 collects operation history information of the first user and the second user in the virtual space constructed by the construction unit 132. For example, the collection unit 134 collects information collected by a sensor attached to a tool operated by the user, or operation history information of a predetermined machine operated by the user, as the user operation history information.

[0046] Specifically, the collection unit 134 collects the user's operation history information based on the sensor data obtained by the sensor 300 or the like in the image data DB 121. Then, the collection unit 134 stores the collected user's operation history information in the operation history information DB 125.

[0047] For example, the collection unit 134 inputs image data stored in the image data DB 121 into a model trained to extract the movements of a target person included in the image data, and extracts the user's movements included in the image data. Then, the collection unit 134 collects user movement history information based on the extracted user movements. Note that the method by which the collection unit 134 extracts the movement history information from the image data is not particularly limited.

[0048] For example, the collection unit 134 uses the sensor data stored in the sensor data DB 122 to collect user movement history information, such as the distance the user's body has moved, the moving speed, and the amount of force applied to a three-dimensional structure, based on known technology.

[0049] For example, the collection unit 134 extracts user actions included in the sensor data by inputting the actions of a target person included in the sensor data into a model trained to extract the actions, and then collects user action history information based on the extracted user actions.

[0050] (Calculation unit 135) The calculation unit 135 compares the action history information of the first user and the action history information of the second user collected by the collection unit 134 to calculate a difference. Specifically, the calculation unit 135 extracts the action of the first user from the action history information of the first user. The calculation unit 135 also extracts the action of the second user from the action history information of the second user. Then, the calculation unit 135 compares the action of the first user with the action of the second user to calculate a difference in the actions between the users.

[0051] Furthermore, the calculation unit 135 calculates specific actions of the user, such as actions that the second user is not good at or actions that differ significantly from other users, from the action history information of the second user collected by the collection unit 134.

[0052] An example of the difference calculated by the calculation unit 135 and the specific user action will be described later in the section of FIG.

[0053] (output unit 136) The output unit 136 outputs feedback to the second user, including differences in behavior among multiple users and behaviors that the user is not good at, based on the user's behavior history information collected by the collection unit 134, to a terminal device (wearable terminal, laptop computer, etc.) operated by the user.

[0054] Furthermore, when outputting feedback including a difference between the movements of the first user and the second user, the output unit 136 outputs the target movement with emphasis if the difference related to the second user calculated by the calculation unit 135 satisfies a predetermined condition. For example, if the difference between the movements exceeds a predetermined threshold, the output unit 136 can output the movement to the user after applying a predetermined color or highlighting to the trajectory of the movement or an area including the movement in the image data.

[0055] Furthermore, the output unit 136 highlights and outputs the specific action calculated by the calculation unit 135. For example, when the user's action is an action (specific action) that satisfies a predetermined condition, the output unit 136 can output the image data to the user after applying a predetermined color or highlighting to an area in the image data that includes the specific action. Note that the above-mentioned "predetermined condition" is an arbitrarily determined condition, and may be a condition such as "the action is determined to be the same or similar by a machine learning model" or "the trajectory of the specific action is the same or similar."

[0056] (imaging device 200) The imaging device 200 is a device that captures still images or moving images in a target space. For example, it may be a digital camera, a digital video camera, or other device fixed at a predetermined position. The imaging device 200 is not particularly limited as long as it is a device that can acquire still images or moving images used by the education support device 100 for processing.

[0057] (Sensor 300) The sensor 300 is a device that is attached to a user or a machine operated by the user, and acquires operation history information of the user or the machine. For example, the sensor 300 includes an acceleration sensor (e.g., a three-axis acceleration sensor), a gyro sensor (e.g., a three-axis gyro sensor), a magnetic sensor, a pressure sensor, a temperature sensor, a motion sensor, etc. Note that the sensor 300 is not particularly limited as long as it is a device that can acquire operation history information of the user that the education support device 100 uses for processing.

[0058] (Terminal device 400) The terminal device 400 is an information processing terminal device operated by a user. Specifically, the terminal device 400 includes a visual device (for example, VR (Virtual Reality) goggles, etc.) worn by the user, a tactile device (for example, a haptic device, etc.), a thermal device (for example, a thermo device, etc.), etc.

[0059] In addition to a wearable terminal, the terminal device 400 may include a spatial reproduction display that displays a three-dimensional shape without wearing special goggles, etc. The terminal device 400 may also be a smartphone, a tablet, a PDA (Personal Digital Assistant), a PC (Personal Computer), a notebook PC, etc.

[0060] (Examples of educational support) An example of education support by the education support device 100 according to this embodiment will now be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of education support according to the embodiment. Fig. 6 shows an example in which a junior firefighter 11 (second user) practices disaster rescue using a disaster scene (virtual space) reproduced based on image data relating to rescue activities (tasks) by a veteran firefighter 10 (first user).

[0061] First, the education support device 100 (acquisition unit) acquires image data relating to rescue activities at an actual disaster site, captured by an imaging device or the like worn by a veteran firefighter 10 ((1) in FIG. 6). Note that in addition to image data of rescue activities at an actual disaster site, image data created using CG (Computer Graphics), VFX (Visual Effects), or the like may also be acquired.

[0062] The educational support device 100 (construction unit) constructs a virtual disaster site (virtual space) using the acquired image data related to rescue activities at the actual disaster site ((2) in FIG. 6). Note that the "virtual disaster site (virtual space)" is a virtual disaster site constructed by the educational support device 100 (construction unit), and enables a user to experience being in the virtual disaster site via a wearable device or the like, and to perform rescue operations at the disaster site within the virtual space.

[0063] Next, the educational support device 100 (display unit) transmits information for displaying the constructed virtual disaster scene to the terminal device 400 (VR goggles, etc.) worn by the young firefighter 11 ((3-1) in Figure 6).

[0064] Then, the terminal device 400 (VR goggles or the like) worn by the junior firefighter 11 uses the information for displaying the virtual disaster scene transmitted by the education support device 100 (display unit) to display the virtual disaster scene to the junior firefighter 11 ((3-2) in FIG. 6). At this time, the terminal device 400 may display the virtual disaster scene to the junior firefighter 11 as XR (cross reality).

[0065] Next, the education support device 100 (collection unit) collects action history information of the veteran firefighter 10 (first user) from the acquired image data related to rescue activities at the actual disaster site ((4-1) in FIG. 6). The education support device 100 (collection unit) also collects action history information of the junior firefighter 11 (second user) based on the displayed virtual disaster site ((4-2) in FIG. 6).

[0066] For example, the educational support device 100 (collection unit) can collect action history information related to rescue activity actions at disaster sites, etc., obtained by sensors attached to the bodies of veteran firefighters 10 and junior firefighters 11 or to tools operated by veteran firefighters 10 and junior firefighters 11.

[0067] The training support device 100 (calculation unit) uses the collected action history information to calculate differences and weak actions ((5) in FIG. 6). Specifically, the training support device 100 (calculation unit) compares the collected action history information of the veteran firefighter 10 with that of the junior firefighter 11 to calculate the difference between their actions. For example, the training support device 100 (calculation unit) can compare action history information of the veteran firefighter 10 and the junior firefighter 11, including information on the movement route, movement speed, attention to surroundings, rescue procedures for rescue victims, firefighting activities, etc. at the fire scene, to calculate the difference between their actions.

[0068] Furthermore, the education support device 100 (calculation unit) compares the collected action history information of the veteran firefighter 10 with the action history information of the junior firefighter 11, and calculates, as "actions that the junior firefighter 11 is weak at," any part of the action where there is a large difference between the action of the veteran firefighter 10 and the action of the junior firefighter 11. For example, the education support device 100 (calculation unit) compares action history information of the veteran firefighter 10 and the junior firefighter 11, including information on movement routes, movement speeds, attention to surroundings, rescue procedures for rescue victims, firefighting activities, etc. at the disaster site, and can calculate any action where the difference exceeds a threshold (for example, "paying attention to surroundings") as an action that the junior firefighter 11 is weak at.

[0069] The education support device 100 (output unit) outputs the movement information including the calculated difference and information about the weak movements to the junior firefighter 11 as feedback ((6) in FIG. 6). For example, the education support device 100 (output unit) can output to the junior firefighter 11 a list of movements that show differences compared to the veteran firefighter 10, a comparison video of movements, etc. Furthermore, the education support device 100 (output unit) can color or highlight movements with particularly large differences (weak movements) and output them to the junior firefighter 11.

[0070] (Processing Procedure by Education Support Device 100) Next, the procedure of the process realized by the education support device 100 according to this embodiment will be described with reference to Fig. 9. Fig. 7 is a diagram showing a flowchart of the process performed by the education support device 100 according to this embodiment.

[0071] The acquisition unit 131 acquires image data from the imaging device 200 (S101). Next, the construction unit 132 constructs a virtual space using the acquired image data (S102). The display unit 133 displays the virtual space to the user (S103).

[0072] If the user has performed an action (Yes in S104), the collection unit 134 collects the action history information (S105). On the other hand, if the user has not performed an action (No in S104), the education support device 100 waits for processing.

[0073] The calculation unit 135 calculates the action information using the collected action history information (S106). Next, the output unit 136 outputs the action information (S107). Then, the education support device 100 ends the process.

[0074] (effect) Next, a description will be given of the effects achieved by the education support device 100 according to this embodiment. When education or guidance on rescue activities at disaster sites is provided, it may be difficult to provide appropriate education using conventional technology.

[0075] Therefore, the construction unit 132 of the education support device 100 according to this embodiment constructs a virtual space based on the acquired image data and the action history information of the first user contained in the image data. The collection unit 134 of the education support device 100 collects the action history information of the second user in the virtual space constructed by the construction unit 132. The output unit 136 of the education support device 100 outputs feedback to the second user based on the user action history information collected by the collection unit 134.

[0076] As described above, the education support device 100 can construct a virtual space that reproduces rescue operations and dangerous work at disaster sites, which are difficult to reproduce with high accuracy using conventional technologies. The education support device 100 can then collect user actions (action history information) based on the constructed virtual space and output feedback to the user. As a result, the education support device 100 according to this embodiment has the effect of enabling efficient education support.

[0077] Furthermore, the education support device 100 according to this embodiment achieves predetermined effects by executing the processes described below.

[0078] The collection unit 134 collects the action history information of the first user and the action history information of the second user in the virtual space constructed using image data related to the work performed by the first user. The calculation unit 135 compares the action history information of the first user and the action history information of the second user collected by the collection unit 134 and calculates the difference.

[0079] Specifically, the calculation unit 135 extracts the actions of the first user from the action history information of the first user. The calculation unit 135 also extracts the actions of the second user from the action history information of the second user. Next, the calculation unit 135 compares the actions of the first user with the actions of the second user to calculate the difference in actions between the users.

[0080] Through the above-described processing, the education support device 100 can collect action history information related to user actions in the collected virtual space and calculate differences in actions, accuracy of actions, etc. between multiple users. As a result, the education support device 100 has the effect of enabling effective education support, such as allowing users to improve their own actions based on the differences in actions.

[0081] When the difference related to the second user calculated by the calculation unit 135 satisfies a predetermined condition, the output unit 136 highlights and outputs the target action. By performing the above-described process, when the user's action is significantly different from the actions of other users (i.e., the difference is large, the accuracy is low, etc.), the education support device 100 highlights the action, thereby achieving the effect of enabling effective education support, such as enabling the user to improve their own action by themselves.

[0082] The calculation unit 135 calculates the specific action of the user from the action history information of the second user collected by the collection unit 134. The output unit 136 outputs the specific action calculated by the calculation unit 135 in an emphasized manner.

[0083] Through the above-described processing, the education support device 100 enables the user to easily grasp actions that the user is not good at or actions that the user is less proficient at than other users, etc. As a result, the education support device 100 has the effect of enabling effective education support, such as allowing the user to improve their own actions based on the output highlighting of the specific actions.

[0084] The collection unit 134 collects, as user operation history information, information collected by a sensor attached to a tool operated by the user or operation history information of a specific machine operated by the user. Through the above-described processing, the education support device 100 can easily collect user operation history information.

[0085] For example, the education support device 100 can collect action history information based on sensors attached to tools and machines, and can calculate the differences in actions between users and weak actions using accurately traced action history information.

[0086] <Modification> The following describes modified examples realized by the education support device 100 according to this embodiment.

[0087] (Using IOWN) The education support device 100 according to this embodiment can realize education support processing based on the IOWN() technology. The IOWN technology will now be described. Figure 8 is a diagram for explaining IOWN.

[0088] As shown in Figure 8, IOWN technology consists of three main technology areas: All-Photonics Network (APN), Digital Twin Computing (DTC), and Cognitive Foundation (CF).

[0089] (APN (All-Photonics Network)) The APN related to IOWN technology is a technology that enables the construction of high-speed networks by processing all network transfer functions in the optical domain. Specifically, the APN related to IOWN technology is a technology that realizes low-power, high-quality, large-capacity, and low-latency communications based on optical-based (photonics-based) technologies such as "photonics-electronic convergence technology," "large-capacity optical transmission system and device technology," "optical Ising machine," and "optical lattice clock network."

[0090] For example, the construction unit 132 constructs a three-dimensional structure based on photogrammetry using image data with high image quality and a large amount of data acquired by multiple imaging devices connected via the APN of the IOWN. The display unit 133 can then display the highly accurate three-dimensional structure constructed by the construction unit 132 to the user via the APN.

[0091] By the above-described processing, the education support device 100 according to this embodiment can use the APN of the IOWN described above to construct a three-dimensional structure with higher accuracy than conventionally possible and display it to the user.

[0092] (Digital Twin Computing) DTC, which is related to IOWN technology, is a technology that maps individual objects in the real world onto a virtual space using the vast amount of data collected by devices connected to the APN described above.

[0093] Conventional digital twin frameworks are used by mapping individual objects, such as automobiles and robots, into a virtual space, performing analysis and predictions on them, and then mapping the results of the analysis and predictions back onto the real world.

[0094] On the other hand, DTC related to IOWN technology expands on the conventional concept of digital twins, freely combining digital twins of various industries, objects, and people to perform calculations, thereby reproducing with high accuracy the combination of multiple objects, such as people and automobiles in a city. Furthermore, DTC related to IOWN technology enables not only the expression of a person's external appearance, but also the digital expression of their internal state, such as consciousness and thoughts, by combining technologies that enable "speech recognition," "speech synthesis," "understanding of emotions and intentions," etc. to collect information and build a digital twin environment.

[0095] In this way, DTC related to IOWN technology is a technology that enables the creation of digital twins that do not exist in the real world by combining multiple entities that are single in the real world and replicating them as digital twins in a virtual space, or by exchanging or merging some of the components between multiple digital twins.

[0096] For example, the construction unit 132 constructs a virtual space as a digital twin based on IOWN's DTC (Digital Twin Computing). The display unit 133 can display the digital twin constructed by the construction unit 132 and the three-dimensional structure of the digital twin to the user via the APN.

[0097] Through the above-described processing, the education support device 100 according to this embodiment can construct a virtual space and a three-dimensional structure based on the DTC of IOWN and display it to the user.

[0098] (Cognitive Foundation) CF related to IOWN technology is a technology that centrally performs the deployment, configuration, linkage, management, and operation of ICT (Information and Communication Technology) resources at different layers, from the cloud to edge computers, network services, user equipment, etc. Specifically, CF related to IOWN technology treats various targets as a group of virtualized ICT resources, and optimally integrates multiple resources at different layers using multi-orchestration functions as a hub.

[0099] Furthermore, as shown in FIG. 8, the IOWN technology provides high-value-added services by linking the above-mentioned APN, DTC, and network services provided by operators.

[0100] For example, as shown in (1) of Figure 8, IOWN technology provides a technology for transmitting information collected via APN to other terminal devices at high speed and with low latency. Also, as shown in (2) of Figure 8, IOWN technology provides a technology for collecting large amounts of information from terminal devices and outputting information such as analysis results from the service provided by the operator at high speed and with low latency in services such as information analysis. Also, as shown in (3) of Figure 8, IOWN technology provides a technology for transmitting large amounts of information at high speed and with low latency, using information obtained from surveillance cameras, automobile sensors, etc. to build a digital twin environment, make future predictions, and output the prediction results to the user.

[0101] By using the above-described IOWN technology, the education support device 100 can construct three-dimensional structures and virtual spaces with higher accuracy than conventional ones. As a result, the education support device 100 displays highly accurate three-dimensional structures to the user, enabling the user to perform rescue operations or practice dangerous work at disaster sites with a greater sense of reality.

[0102] (Data, etc.) The image data, three-dimensional structures, virtual spaces, sensor data, disaster sites, rescue operations, veteran firefighters, young firefighters, names of functional parts of the educational support device 100, steps, processes, names of steps or processes, etc. used in the description of the above embodiments are merely examples and can be changed as desired.

[0103] For example, the image data DB 121 stores imaging targets and imaging data in association with "No.", which is information identifying individual image data, but the items and contents to be stored are not particularly limited. The sensor data DB 122 stores sensor names and acquired data in association with "No.", which is information identifying individual sensor data, but the items and contents to be stored are not particularly limited. The operation history information DB 125 stores user identification information and history information in association with "No.", which is information identifying individual operation history information, but the items and contents to be stored are not particularly limited.

[0104] (Example of dangerous work) The education support device 100 according to this embodiment can create virtual spaces that reproduce various situations. For example, the education support device 100 can create virtual spaces related to cutting wood with an electric saw, heavy snow, difficult-to-return-home areas, collapsed houses, breached levees, cliff collapses, liquefaction, tornadoes, typhoons, landslides, volcanic eruptions, underwater work, nuclear disasters, etc.

[0105] Therefore, the education support device 100 has an effect of enabling a user to easily experience situations that can only be experienced under actual circumstances, and as a result, the education support device 100 can realize effective education support that has been difficult to achieve with conventional technologies.

[0106] (Flowcharts, etc.) The steps in the flowcharts may be interchanged as long as there is no contradiction, and some steps may not be performed. In addition, conjunctions such as "next," "continue," "further," "at this time," and "on this occasion" used in the explanation of the flowcharts do not limit the order or timing of the execution of the processes in the flowcharts.

[0107] <Hardware configuration> The components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of each device can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0108] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can also be performed manually using known methods. In addition, the information including the processing procedures, control procedures, specific names, various data, and parameters shown in the drawings can be changed as desired unless otherwise specified.

[0109] <Program> In one embodiment, the various devices constituting the education support device 100 can be implemented by installing an education support program as package software or online software on a desired computer. For example, by executing the education support program on an information processing device, the various devices constituting the education support device 100 can function. The information processing device referred to here includes desktop and notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones and mobile phones, and even slate terminals such as PDAs (Personal Digital Assistants).

[0110] 9 is a diagram showing an example of a computer that realizes the education support device 100 according to the embodiment. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0111] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0112] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, application programs 1092, program modules 1093, and program data 1094. That is, programs that define the processes of the various devices that make up the education support device 100 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, program modules 1093 for executing processes similar to those of the functional configurations of the various devices that make up the education support device 100 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0113] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0114] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a LAN or a WAN (Wide Area Network)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0115] <Other> Although the present embodiment has been described above, the present embodiment is not limited by the descriptions and drawings that form part of the disclosure. In other words, other embodiments, examples, operational techniques, etc. that are made by those skilled in the art based on the present embodiment are all included in the scope of the present embodiment. [Explanation of symbols]

[0116] 100 Educational support equipment 110 Communications Department 120 Storage section 121 Image Data DB 122 Sensor Data DB 123 Virtual Space DB 124 Three-dimensional structure DB 125 Operation History Information DB 130 control section 131 Acquisition Department 132 Construction Department 133 Display section 134 Collection Department 135 Calculation Unit 136 Output section 200 Imaging device 300 sensors 400 Terminal Equipment

Claims

1. a construction unit that constructs a virtual space based on the acquired image data and history information of the first user's actions included in the image data; a collection unit that collects history information of actions of a second user in the virtual space constructed by the construction unit; an output unit that outputs feedback to the second user based on the history information of the user's actions collected by the collection unit; An educational support device comprising:

2. The collecting unit collecting history information of the first user's actions and history information of the second user's actions in the virtual space constructed using image data related to the work performed by the first user; The system further includes a calculation unit that compares the history information of the first user's actions collected by the collection unit with the history information of the second user's actions and calculates a difference between the history information of the first user's actions and the history information of the second user's actions collected by the collection unit.

2. The educational support device according to claim 1.

3. The calculation unit extracting a first user's action from history information of the first user's action; extracting a second user's action from history information of the second user's action; comparing the first user's motion with the second user's motion to calculate a difference in motion between the users; The output unit When the difference related to the second user calculated by the calculation unit satisfies a predetermined condition, a target motion is emphasized and output.

3. The educational support device according to claim 2.

4. a calculation unit that calculates a specific action of the second user from history information of the actions of the second user collected by the collection unit; The output unit the specific action calculated by the calculation unit is output in an emphasized manner; 2. The educational support device according to claim 1.

5. The collecting unit Collecting information collected by a sensor attached to an appliance operated by the user or history information of the operation of a predetermined machine operated by the user as history information of the user's operation; 5. The educational support device according to claim 1, wherein the educational support device comprises: a computer;

6. The construction unit A three-dimensional structure is constructed based on photogrammetry using image data acquired by multiple imaging devices connected via an APN (All-Photonics Network) of an IOWN (Innovative Optical and Wireless Network). The system further includes a display unit that displays the three-dimensional structure constructed by the construction unit to the user via the APN.

5. The educational support device according to claim 1, wherein the educational support device comprises: a computer;

7. The construction unit Based on IOWN's DTC (Digital Twin Computing), the virtual space is constructed as a digital twin, The display unit The digital twin constructed by the construction unit and a three-dimensional structure in the digital twin are displayed to the user via the APN.

7. The educational support device according to claim 6.

8. An education support method executed by an education support device, a construction step of constructing a virtual space based on the acquired image data and history information of the first user's actions included in the image data; a collection step of collecting history information of actions of a second user in the virtual space constructed by the construction step; an output step of outputting feedback to the second user based on the history information of the user's actions collected by the collection step; An educational support method comprising:

9. a construction step of constructing a virtual space based on the acquired image data and history information of the first user's actions included in the image data; a collection step of collecting history information of actions of a second user in the virtual space constructed by the construction step; an output step of outputting feedback to the second user based on the history information of the user's actions collected by the collection step; An educational support program that runs on a computer.

Images