Simulation system and simulation program
The simulation system addresses the inefficiency in existing training methods by dynamically setting a higher target ability, enabling workers to learn from a model with superior skills, thus enhancing their performance.
Patent Information
- Application Number
- JP2023083878
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-12-04
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing learning support systems fail to account for a learner's potential for growth, leading to inefficient training of workers performing predetermined tasks.
A simulation system that dynamically sets a target ability higher than the current ability of a worker, using a simulation program to superimpose a model worker's motion in a virtual space, allowing the worker to imitate and learn from a model with higher abilities.
This approach enables efficient and effective training by leveraging the worker's potential, as they can easily imitate the model's motions, thereby enhancing their skills effectively.
Smart Images

Figure 2024167655000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a technique for simulating a predetermined task in a virtual space. [Background technology]
[0002] Patent Document 1 discloses a learning support system. The learning support system displays a model video of an instructor, which serves as a model for the learner's movements, superimposed on a video of the learner's field of vision. Basic information of the learner includes the learner's age, dominant hand, eyesight, grip strength, etc. The learning support system changes the display content of the model video according to the learner's basic information. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2020-144233 A Summary of the Invention [Problem to be solved by the invention]
[0004] According to the technology described in Patent Document 1, the model video is customized based on the learner's current basic information, but the learner's potential (room for growth) is not taken into consideration.
[0005] One object of the present disclosure is to provide a technique that enables efficient and effective training of a target worker who performs a predetermined task. [Means for solving the problem]
[0006] One aspect of the present disclosure relates to a simulation system that simulates a predetermined task in a virtual space. The simulation system includes one or more processors. The one or more processors acquire information indicating a current capability for a given task of a target worker who is to perform the given task. The one or more processors dynamically set a target capacity higher than the current capacity of the target worker in accordance with the current capacity of the target worker. The one or more processors set, as a target motion, a model motion of a worker having a target capability when performing a predetermined task. The one or more processors render a model worker performing a target action in the virtual space when a target worker performs a predetermined task in the virtual space.
[0007] Another aspect of the present disclosure relates to a simulation program that is executed by a computer and that simulates a predetermined task in a virtual space. The simulation program is Obtaining information indicating a current capability for a predetermined task of a target worker performing the predetermined task; Dynamically setting a target capability higher than the current capability of the target worker in accordance with the current capability of the target worker; Setting a model motion of a worker having a target ability performing a predetermined task as a target motion; When a target worker performs a predetermined task in the virtual space, a model worker performing a target action is drawn in the virtual space. to be executed by the computer. Effect of the Invention
[0008] According to the present disclosure, the target ability is dynamically set in conjunction with the current ability of the target worker so as to be higher than the current ability of the target worker. Then, a model action of a worker having the target ability performing a predetermined task is set as the target action. Therefore, the target worker can naturally imitate the target action performed by the model worker. Furthermore, since the ability of the model worker is higher than the current ability of the target worker, the potential of the target worker can be brought out. In other words, it is possible to efficiently and effectively train the target worker to perform the predetermined task. [Brief description of the drawings]
[0009] [Figure 1] 1 is a conceptual diagram for explaining an overview of a simulation system according to an embodiment; [Diagram 2] 1 is a block diagram showing an example of a configuration of a simulation system according to an embodiment; [Diagram 3] FIG. 2 is a conceptual diagram for explaining a process performed by a simulation system according to an embodiment. [Figure 4] FIG. 13 is a conceptual diagram for explaining setting of a target capability according to an embodiment. [Diagram 5] FIG. 13 is a conceptual diagram for explaining a modified example of the process performed by the simulation system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] 1. Overview of the Simulation System Consider a predetermined task performed by a worker in a real space. For example, the predetermined task is performed in a real factory. The predetermined task is, for example, assembling parts. According to this embodiment, a simulation is utilized to efficiently train (study) a worker who performs the predetermined task.
[0011] FIG. 1 is a conceptual diagram for explaining an overview of a simulation system 100 according to this embodiment. The simulation system 100 performs a simulation of a predetermined task performed by a worker in a virtual space. To achieve this, the simulation system 100 virtually reproduces, in the virtual space, a work environment in which the worker performs the predetermined task. For example, the simulation system 100 reproduces (draws) the work environment in the virtual space based on DigitalTwin technology. For example, the simulation system 100 reproduces, in the virtual space, a virtual factory (DigitalTwin factory) that simulates an actual factory.
[0012] A worker to be trained (hereinafter, referred to as "target worker TW") can experience a predetermined task in a virtual space by using an experience device. For example, the experience device is a wearable device such as a head mounted display (HMD). The head mounted display displays the virtual space reproduced by the simulation system 100. The motion of the target worker TW in the real space (e.g., hand motion) is recognized by a motion capture technique. For example, the target worker TW is photographed by a camera, and the motion of the target worker TW is recognized. As another example, an inertial sensor (gyro sensor, acceleration sensor, etc.) may be attached to the body of the target worker TW, and the motion of the target worker TW may be recognized based on the detection result of the inertial sensor. The simulation system 100 draws the recognized motion of the target worker TW by superimposing it on the virtual space. For example, the simulation system 100 draws the recognized hand motion of the target worker TW by superimposing it on the virtual space. This allows the target worker TW to get the feeling that he or she is performing a predetermined task in the virtual space. That is, the target worker TW can experience a predetermined task in the virtual space.
[0013] Furthermore, the simulation system 100 may draw a virtual model worker MW performing a model action superimposed on the virtual space. The model action is an action that serves as a model when the target worker TW performs a specified task. In the virtual space, the virtual model worker MW performs the model action. The model worker MW may also be called a "ghost worker." In the virtual space, the target worker TW performs a specified task by imitating the model action performed by the model worker MW. This enables efficient and effective training.
[0014] 2 is a block diagram showing an example of the configuration of a simulation system 100 according to this embodiment. The simulation system 100 includes a sensor 110, an input device 120, an output device 130, and an experience device 140.
[0015] The sensor 110 is disposed in the real space and detects various information. For example, the sensor 110 detects the motion of a worker in the real space. Examples of the sensor 110 that detects the motion of a worker include a camera, an infrared sensor, an inertial sensor, and the like. For example, the motion of a worker can be detected by photographing the worker with a camera. As another example, an inertial sensor (gyro sensor, acceleration sensor, etc.) may be attached to the body of the worker, and the motion of the worker may be detected by the inertial sensor.
[0016] As another example, the sensor 110 may detect physical characteristics of the worker in real space. Examples of physical characteristics include handedness, hand size, grip strength, height, muscle mass, eyesight, and the like. For example, the handedness of the worker is detected based on the movement of the worker captured by a camera. As another example, the hand size, height, and the like are detected based on an image of the worker captured by a camera. In addition, the sensor 110 may include a height gauge, a weight scale, a body composition scale, a grip strength meter, an eyesight meter, and the like.
[0017] As yet another example, the sensor 110 may include a biosensor that detects bioinformation of the worker in the real space. Examples of the bioinformation include body temperature, heart rate, fatigue level, stress level, and the like.
[0018] Examples of the input device 120 include a touch panel, a keyboard, a mouse, a microphone, etc. Examples of the output device 130 include a display, a touch panel, a speaker, etc.
[0019] The experience device 140 is used to allow the target worker TW to experience the work environment and a predetermined task in a virtual space. For example, the experience device 140 is a wearable device such as a head mounted display (HMD). The head mounted display displays the virtual space reproduced by the simulation system 100.
[0020] The simulation system 100 further includes one or more processors 150 (hereinafter simply referred to as "processor 150") and one or more storage devices 160 (hereinafter simply referred to as "storage device 160"). The processor 150 executes various processes. Examples of the processor 150 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The storage device 160 stores various information. Examples of the storage device 160 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory.
[0021] The simulation program 170 is a computer program for performing a simulation in the above-mentioned virtual space, and is executed by the processor 150. Various processes by the simulation system 100 may be realized by cooperation between the processor 150 executing the simulation program 170 and the storage device 160. The simulation program 170 is stored in the storage device 160. The simulation program 170 may be recorded in a computer-readable recording medium.
[0022] The work environment information 180 is information about a work environment (e.g., an actual factory) reproduced in a virtual space. For example, the work environment information 180 indicates a three-dimensional arrangement of structures (lines, machines, walls, pillars, etc.) in the work environment. For example, the three-dimensional arrangement of structures is expressed by CAD data. The work environment information 180 is stored in the storage device 160.
[0023] The storage device 160 further stores a database 200. The database 200 includes a worker characteristic database 210, a work performance database 220, and a model operation database 230.
[0024] The worker characteristic database 210 is a database showing the physical characteristics of each worker. Examples of the physical characteristics include handedness, hand size, grip strength, height, muscle mass, eyesight, etc. For example, the physical characteristics of each worker are detected by the above-mentioned sensor 110. As another example, each worker may input his / her own physical characteristics using the input device 120. Physical characteristics expressed by numerical values such as height may be grouped according to a predetermined range.
[0025] The work history database 220 is a database showing the results of predetermined work performed by various workers. For example, the work history database 220 includes video of the worker performing the predetermined work. The video of the worker is captured by a camera included in the above-mentioned sensor 110. As another example, the work history database 220 may show the content of the action performed by the worker during the predetermined work. As yet another example, the work history database 220 may show the time it took for the worker to complete the predetermined work. Such a work history database 220 is generated based on the action of the worker detected by the above-mentioned sensor 110.
[0026] The model action database 230 is a database related to model actions that serve as a model when the target worker TW performs a predetermined task. The model actions related to the predetermined task may be created in advance based on the task record database 220.
[0027] The processor 150 reproduces, in a virtual space, a work environment in which a worker performs a predetermined work, based on the work environment information 180. For example, the processor 150 reproduces (draws) the work environment in the virtual space based on DigitalTwin technology.
[0028] Moreover, the processor 150 recognizes the motion of the target worker TW, who is the training target, using the sensor 110. The motion of the target worker TW in the real space is recognized by a motion capture technique. For example, the motion of the target worker TW is recognized by photographing the target worker TW with a camera. As another example, an inertial sensor may be attached to the body of the target worker TW, and the motion of the target worker TW may be recognized based on the detection result of the inertial sensor. The processor 150 draws the recognized motion of the target worker TW by superimposing it on the virtual space. For example, the processor 150 draws the recognized hand movement of the target worker TW by superimposing it on the virtual space. The target worker TW can get a sense of performing a predetermined task in the virtual space via the experience device 140. That is, the target worker TW can experience a predetermined task in the virtual space via the experience device 140.
[0029] Furthermore, the processor 150 determines a model action to be a model for the target worker TW to perform a predetermined task based on the model action database 230. Then, the processor 150 draws a virtual model worker MW performing the model action by superimposing it on the virtual space. In the virtual space, the target worker TW performs the predetermined task by imitating the model action performed by the model worker MW. This enables efficient and effective training.
[0030] 2. Training methods that take into account the current capabilities of workers FIG. 3 is a conceptual diagram for explaining the process performed by the simulation system 100 according to the present embodiment.
[0031] Consider the "ability for a given task" of a worker. For example, the ability for a given task is defined by the output amount of the given task within a certain time. The greater the output amount, the higher the ability for the given task is determined to be. As another example, the ability for a given task is defined by the task time required to complete the given task. The shorter the task time, the higher the ability for the given task is determined to be. The output amount and task time for the given task are obtained from the task record database 220. As yet another example, if the given task requires strength, the ability for the given task may include the muscle mass of the worker as a factor. The greater the muscle mass of the worker, the higher the ability for the given task is determined to be. The muscle mass of the worker is obtained from the worker characteristic database 210.
[0032] According to this embodiment, the "model actions when performing a specified task" are not fixed uniformly, but are set stepwise according to the "ability for the specified task". For example, when the "ability for the specified task" is at a low stage, the "model actions when performing the specified task" may be slow. As the "ability for the specified task" increases, the "model actions when performing the specified task" may become faster. The model action database 230 according to this embodiment shows the correspondence between such "ability for the specified task" and "model actions when performing the specified task".
[0033] The work record database 220 shows the results of the predetermined work performed by various workers with different abilities. The results of the predetermined work include the time taken by various workers to complete the predetermined work. Therefore, the simulation system 100 (processor 150) can estimate the abilities of each of the various workers based on the work record database 220. The simulation system 100 classifies the abilities of various workers into N stages (N is an integer of 2 or more). The results of the predetermined work also include images of the workers performing the predetermined work and the contents of the actions performed by the workers during the predetermined work. Therefore, the simulation system 100 can set the average actions of workers with abilities of each stage as model actions corresponding to the abilities based on the work record database 220. That is, the simulation system 100 can generate and update the model action database 230 that shows the correspondence between "ability for the predetermined work" and "model actions when performing the predetermined work" based on the work record database 220.
[0034] The training for the target worker TW is as follows:
[0035] FIG. 4 shows a growth curve of a target worker TW. The vertical axis represents the ability of the target worker TW for a specified task. The horizontal axis represents the number of times the target worker TW has performed the specified task. It is believed that the ability of the target worker TW for the specified task will gradually improve as the target worker TW repeatedly performs the specified task. According to this embodiment, the following process is performed to effectively bring out the potential of the target worker TW.
[0036] First, the simulation system 100 (processor 150) acquires information indicating the "current ability" of the target worker TW for a predetermined task. For example, the "current ability" of the target worker TW is obtained from the task record database 220. As another example, the target worker TW may perform the predetermined task once. The simulation system 100 uses the sensor 110 to monitor the latest predetermined task performed by the target worker TW and estimates the "current ability" of the target worker TW.
[0037] Next, the simulation system 100 (processor 150) dynamically sets a "target ability" in consideration of the "current ability" of the target worker TW. More specifically, as shown in Fig. 4, the simulation system 100 dynamically sets the target ability in conjunction with the current ability of the target worker TW so as to be slightly higher than the current ability of the target worker TW. For example, if the current ability of the target worker TW is at stage X (X = 1 to N-1), the target ability is set to stage X+1.
[0038] Next, the simulation system 100 (processor 150) sets a model motion of a worker having the target ability performing a predetermined task as a "target motion." More specifically, the simulation system 100 sets a model motion associated with the target ability as the "target motion" based on the above-mentioned model motion database 230. That is, according to this embodiment, the target motion for the predetermined task is not fixed uniformly, but changes dynamically according to the current ability of the target worker TW.
[0039] When a target worker TW performs a predetermined task in the virtual space, the simulation system 100 (processor 150) renders a model worker MW performing a target action in the virtual space. The target worker TW performs the predetermined task by imitating the target action performed by the model worker MW.
[0040] As a first comparative example, consider the case where a target motion is set according to the current ability of the target worker TW. In this case, a model motion associated with the current ability of the target worker TW is set as the target motion. The target worker TW can easily imitate the target motion performed by the model worker MW, whose ability is equivalent to his / her own. The ability of the target worker TW may increase slightly as the number of times the predetermined motion is performed increases. However, the effect of training cannot be said to be large.
[0041] As a second comparative example, consider the case where the target ability is fixed to the highest level (ideal level) from the beginning without considering the current ability of the target worker TW. In this case, the ability of the model worker MW is far from the current ability of the target worker TW, so the target worker TW cannot easily imitate the target movement performed by the model worker MW. As a result, the effect of training is not obtained.
[0042] On the other hand, according to this embodiment, the target ability is dynamically set in conjunction with the current ability of the target worker TW so as to be higher than the current ability of the target worker TW. Then, a model action of a worker having the target ability performing a specified task is set as the target action. Therefore, the target worker TW can naturally imitate the target action performed by the model worker MW. Moreover, since the ability of the model worker MW is higher than the current ability of the target worker TW, the potential of the target worker TW can be brought out. In other words, it is possible to efficiently and effectively train the target worker TW to perform the specified task. By repeating such training, continuous growth of the target worker TW is expected.
[0043] 3. Variations FIG. 5 is a conceptual diagram for explaining the modified example. In the modified example, the physical characteristics of the worker are further taken into consideration. Examples of the physical characteristics include handedness, height, grip strength, and the like. The model movement database 230 shows the correspondence between "physical characteristics", "ability for a specific task", and "model motion when performing a specific task". Physical characteristics expressed by numerical values such as height may be grouped by a specific range. The physical characteristics of each worker are obtained from the worker characteristic database 210. Therefore, the simulation system 100 can generate and update the model movement database 230 according to the modified example based on the worker characteristic database 210 and the task record database 220.
[0044] The simulation system 100 (processor 150) acquires information indicating the physical characteristics of the target worker TW from the worker characteristic database 210. The simulation system 100 then sets a model motion of a worker having the same physical characteristics as the target worker TW and a target ability when performing a predetermined task as a "target motion". In the case of physical characteristics expressed by a numerical value such as height, the same physical characteristics as the target worker TW means physical characteristics of the same group as the target worker TW. More specifically, the simulation system 100 sets a model motion associated with the target ability and the physical characteristics of the target worker TW as a "target motion" based on the above-mentioned model motion database 230. The rest is the same as the above-mentioned embodiment.
[0045] According to the modified example, it is possible to set a target motion taking into consideration the physical characteristics (e.g., handedness) of the target worker TW. This makes it possible to efficiently and effectively perform training suited to the physical characteristics of the target worker TW. [Explanation of symbols]
[0046] 100...simulation system, 110...sensor, 120...input device, 130...output device, 140...experience device, 150...processor, 160...storage device, 170...simulation program, 180...work environment information, 200...database, 210...worker characteristic database, 220...work performance database, 230...model operation database
Claims
1. A simulation system for simulating a predetermined task in a virtual space, comprising: One or more processors; The one or more processors: acquiring information indicating a current capability for the predetermined task of a target worker who is to perform the predetermined task; Dynamically setting a target capability higher than the current capability of the target worker in conjunction with the current capability of the target worker; A model motion of a worker having the target ability performing the predetermined task is set as a target motion; When the target worker performs the predetermined task in the virtual space, a model worker performing the target action is drawn in the virtual space. It was configured as Simulation system.
2. 2. The simulation system according to claim 1, The system further includes one or more storage devices for storing a database indicating a correspondence between a capability for the predetermined task and the model operation when performing the predetermined task, The one or more processors set the model action associated with the target capability as the target action based on the database. Simulation system.
3. 2. The simulation system according to claim 1, The one or more processors further include: Acquire information indicating physical characteristics of the target worker; The model motion when a worker having the same physical characteristics as the target worker and the target ability performs the predetermined work is set as the target motion. Simulation system.
4. The simulation system according to claim 3, one or more storage devices for storing a database showing a correspondence between abilities for the predetermined task, physical characteristics, and the model motion when performing the predetermined task; The one or more processors set the model motion associated with the target ability and the physical characteristics of the target worker as the target motion based on the database. Simulation system.
5. A simulation program that is executed by a computer and that simulates a predetermined task in a virtual space, acquiring information indicating a current capability for the predetermined task of a target worker who is to perform the predetermined task; Dynamically setting a target capability higher than the current capability of the target worker in conjunction with the current capability of the target worker; setting a model motion of a worker having the target ability when performing the predetermined work as a target motion; When the target worker performs the predetermined task in the virtual space, a model worker performing the target action is drawn in the virtual space. causing the computer to execute Simulation program.
Citation Information
Patent Citations
Human motion trajectory learning device in virtual environment
JP2000504854A
Methods and systems for automated personal training
JP2014502178A
Experienced operation teaching system
JP2019020670A
Work support device, work support method, and work support program
JP2020034849A
Learning assisting system, learning assisting device, and program
JP2020144233A
