Simulation system and simulation program

Abstract

To improve learning effects and learning efficiency when a worker learns prescribed work.SOLUTION: A simulation system performs simulation of prescribed work in a virtual space. A comment database indicates a correspondence between the comment created by a reference worker regarding the prescribed work and a characteristic of the reference worker. The simulation system acquires information of a characteristic of a subject worker performing the prescribed work. The simulation system extracts a comment associated to the characteristic of the reference worker equivalent to the characteristic of a subject worker on the basis of the comment database. When the subject worker performs the prescribed work in the virtual space, the simulation system draws a model worker who performs a model operation in the virtual space and presents the extracted comment to the subject worker.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a technique for simulating a predetermined task in a virtual space. [Background technology]

[0002] Patent Document 1 discloses an information processing device that provides learning content for learning physical movements. The information processing device displays an overlaid image of a teacher's digital twin and a student's digital twin on a student's device. The information processing device also displays instruction information on the student's device based on the evaluation value of the student's digital twin. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 230101 Summary of the Invention [Problem to be solved by the invention]

[0004] When a worker learns a specific task, it is conceivable to present the worker with comments (advice, etc.) about the task. However, the tricks and weak points when mastering a task may differ from person to person. Presenting comments that are inappropriate for the worker or that are not helpful to the worker will not produce the expected effect. On the other hand, presenting all comments to the worker will only confuse the worker, and in the end, the worker will not know which comments are helpful.

[0005] One object of the present disclosure is to provide a technique that can improve the learning effect and learning efficiency when a worker learns a predetermined task. [Means for solving the problem]

[0006] One aspect of the present disclosure relates to a simulation system that simulates a predetermined task in a virtual space. The simulation system is one or more processors; one or more storage devices for storing a comment database; Equipped with. The comment database indicates the correspondence between comments made by a reference worker regarding a predetermined task and the characteristics of the reference worker. One or more processors acquire information on a model operation that serves as a model for performing a predetermined task. The one or more processors acquire information on the characteristics of a target worker who performs a predetermined task. The one or more processors extract comments associated with characteristics of the reference worker that correspond to the characteristics of the target worker, based on the comment database. When the target worker performs a predetermined task in the virtual space, one or more processors draw the model worker performing the model action in the virtual space and present the extracted comments to the target worker.

[0007] Another aspect of the present disclosure relates to a simulation program that is executed by a computer and that simulates a predetermined task in a virtual space. The comment database indicates the correspondence between comments made by a reference worker regarding a predetermined task and the characteristics of the reference worker. The simulation program is Obtaining information on a model operation that serves as a model for performing a predetermined task; Acquiring information on characteristics of a target worker who will perform a predetermined task; extracting comments associated with characteristics of a reference worker that correspond to the characteristics of the target worker based on the comment database; When the target worker performs a predetermined task in the virtual space, a model worker performing a model action is drawn in the virtual space, and the extracted comment is presented to the target worker. to be executed by the computer. [Effects of the Invention]

[0008] According to the present disclosure, a comment database is prepared. The comment database indicates the correspondence between comments made by a reference worker regarding a predetermined task and the characteristics of the reference worker. Based on the comment database, comments associated with the characteristics of the reference worker that correspond to the characteristics of a target worker are extracted. The extracted comments are then selectively presented to the target worker. Since not all comments are presented indiscriminately, the target worker is not confused. On the other hand, appropriate comments that match the characteristics of the target worker are presented. In other words, the target worker can learn the predetermined task by referring to appropriate comments that match his or her characteristics. Therefore, the learning effect and learning efficiency of the target worker are improved. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a conceptual diagram for explaining an overview of a simulation system according to an embodiment; [Figure 2] 1 is a block diagram illustrating an example of the configuration of a simulation system according to an embodiment. [Figure 3] FIG. 2 is a conceptual diagram illustrating an example of a comment database according to the embodiment. [Figure 4] FIG. 10 is a conceptual diagram for explaining a comment filter function according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] 1. Overview of the simulation system Consider a predetermined task performed by a worker in a real space. For example, the predetermined task is performed in a real factory. The predetermined task is, for example, assembling parts. According to this embodiment, simulation is utilized to efficiently train (learn) the worker who performs the predetermined task.

[0011] FIG. 1 is a conceptual diagram for explaining an overview of a simulation system 100 according to this embodiment. The simulation system 100 performs a simulation of a predetermined task performed by a worker in a virtual space. To achieve this, the simulation system 100 virtually reproduces, in the virtual space, a work environment in which the worker performs the predetermined task. For example, the simulation system 100 reproduces (draws) the work environment in the virtual space based on DigitalTwin technology. For example, the simulation system 100 reproduces, in the virtual space, a virtual factory (DigitalTwin factory) that simulates a real factory.

[0012] A worker to be trained (hereinafter referred to as the "target worker TW") can experience a predetermined task in a virtual space by using an experiential device. For example, the experiential device is a wearable device such as a head-mounted display (HMD). The head-mounted display displays the virtual space reproduced by the simulation system 100. The motions (e.g., hand motions) of the target worker TW in the real space are recognized using motion capture technology. For example, the target worker TW is photographed with a camera, and the motions of the target worker TW are recognized. As another example, an inertial sensor (a gyro sensor, an acceleration sensor, etc.) may be attached to the body of the target worker TW, and the motions of the target worker TW may be recognized based on the detection results of the inertial sensor. The simulation system 100 renders the recognized motions of the target worker TW superimposed on the virtual space. For example, the simulation system 100 renders the recognized hand movements of the target worker TW superimposed on the virtual space. This allows the target worker TW to feel as if he or she is performing a predetermined task in the virtual space. In other words, the target worker TW can experience the predetermined task in the virtual space.

[0013] Furthermore, the simulation system 100 may superimpose a virtual model worker MW performing a model action in the virtual space. The model action is an action that serves as a model for the target worker TW when performing a predetermined task. In the virtual space, the virtual model worker MW performs the model action. The model worker MW can also be called a "ghost worker." In the virtual space, the target worker TW performs a predetermined task by imitating the model action performed by the model worker MW. This enables efficient and effective training.

[0014] 2 is a block diagram showing an example of the configuration of a simulation system 100 according to this embodiment. The simulation system 100 includes a sensor 110, an input device 120, an output device 130, and an experience device 140.

[0015] The sensor 110 is placed in real space and detects various information. For example, the sensor 110 detects the movement of a worker in real space. Examples of the sensor 110 that detects the movement of a worker include a camera, an infrared sensor, an inertial sensor, and the like. For example, the movement of a worker can be detected by capturing an image of the worker with a camera. As another example, an inertial sensor (a gyro sensor, an acceleration sensor, etc.) may be attached to the body of the worker, and the movement of the worker may be detected by the inertial sensor.

[0016] As another example, the sensor 110 may detect physical characteristics of a worker in real space. Examples of physical characteristics include handedness, hand size, grip strength, height, muscle mass, eyesight, etc. For example, the handedness of a worker is detected based on the worker's movements captured by a camera. As another example, hand size, height, etc. are detected based on an image of the worker captured by a camera. In addition, the sensor 110 may include a height gauge, a weight scale, a body composition scale, a grip strength meter, an eyesight meter, etc.

[0017] As yet another example, the sensor 110 may include a biosensor that detects biometric information of the worker in real space. Examples of biometric information include body temperature, heart rate, fatigue level, stress level, and the like.

[0018] Examples of the input device 120 include a touch panel, a keyboard, a mouse, a microphone, etc. Examples of the output device 130 include a display, a touch panel, a speaker, etc.

[0019] The experience device 140 is used to allow the target worker TW to experience the work environment and a predetermined task in a virtual space. For example, the experience device 140 is a wearable device such as a head-mounted display (HMD). The head-mounted display displays the virtual space reproduced by the simulation system 100.

[0020] The simulation system 100 further includes one or more processors 150 (hereinafter simply referred to as "processors 150") and one or more storage devices 160 (hereinafter simply referred to as "storage devices 160"). The processors 150 execute various processes. Examples of the processors 150 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The storage device 160 stores various information. Examples of the storage device 160 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory.

[0021] The simulation program 170 is a computer program for performing the simulation in the above-described virtual space, and is executed by the processor 150. Various processes by the simulation system 100 may be realized by cooperation between the processor 150 that executes the simulation program 170 and the storage device 160. The simulation program 170 is stored in the storage device 160. The simulation program 170 may be recorded on a computer-readable recording medium.

[0022] The work environment information 180 is information about a work environment (e.g., an actual factory) reproduced in a virtual space. For example, the work environment information 180 indicates the three-dimensional arrangement of structures (lines, machines, walls, pillars, etc.) in the work environment. For example, the three-dimensional arrangement of structures is expressed by CAD data. The work environment information 180 is stored in the storage device 160.

[0023] The storage device 160 further stores a database 200. The database 200 includes a worker characteristics database 210, a work performance database 220, and a model operation database 230.

[0024] The worker characteristic database 210 is a database that indicates the physical characteristics of each worker. Examples of physical characteristics include handedness, hand size, grip strength, height, muscle mass, eyesight, etc. For example, the physical characteristics of each worker are detected by the above-mentioned sensor 110. As another example, each worker may input their own physical characteristics using the input device 120. Physical characteristics expressed as numerical values, such as height, may be grouped according to a predetermined range.

[0025] The work history database 220 is a database showing the results of predetermined work performed by various workers. For example, the work history database 220 includes video of the worker performing the predetermined work. The video of the worker is captured by a camera included in the sensor 110 described above. As another example, the work history database 220 may show the details of the actions performed by the worker during the predetermined work. As yet another example, the work history database 220 may show the time it took the worker to complete the predetermined work. Such a work history database 220 is generated based on the actions of the worker detected by the sensor 110 described above.

[0026] The model action database 230 is a database relating to model actions that serve as a model when the target worker TW performs a predetermined task. The model actions relating to the predetermined task may be created in advance based on the task record database 220.

[0027] The processor 150 reproduces, in a virtual space, a work environment in which a worker performs a predetermined work, based on the work environment information 180. For example, the processor 150 reproduces (draws) the work environment in the virtual space based on DigitalTwin technology.

[0028] The processor 150 also uses the sensor 110 to recognize the movements of the target worker TW who is the training target. The movements of the target worker TW in real space are recognized using motion capture technology. For example, the movements of the target worker TW are recognized by capturing an image of the target worker TW with a camera. As another example, an inertial sensor may be attached to the body of the target worker TW, and the movements of the target worker TW may be recognized based on the detection results of the inertial sensor. The processor 150 draws the recognized movements of the target worker TW by superimposing them on the virtual space. For example, the processor 150 draws the recognized hand movements of the target worker TW by superimposing them on the virtual space. The target worker TW can get the feeling that he or she is performing a predetermined task in the virtual space via the experience device 140. In other words, the target worker TW can experience a predetermined task in the virtual space via the experience device 140.

[0029] Furthermore, the processor 150 determines a model action to be used as a model when the target worker TW performs a predetermined task, based on the model action database 230. Then, the processor 150 draws a virtual model worker MW performing the model action, superimposed on the virtual space. In the virtual space, the target worker TW performs the predetermined task by imitating the model action performed by the model worker MW. This enables efficient and effective training.

[0030] 2. Comments Database FIG. 3 is a conceptual diagram illustrating an example of the comment database 240 according to this embodiment. The comment database 240 is a database of comments related to a predetermined task, and is stored in the storage device 160. Examples of comments related to the predetermined task include advice, tips, and points of observation. Such comments are created in advance by workers (predecessors) who have previously performed the predetermined task. A worker (predecessor) who has previously performed the predetermined task will hereinafter be referred to as a "reference worker." Each reference worker can input comments related to the predetermined task by using the input device 120. The simulation system 100 registers the comments input by each reference worker in the comment database 240.

[0031] According to this embodiment, comments on a predetermined task are associated with the characteristics of the reference worker who wrote the comment. The characteristics of the reference worker are hereinafter referred to as "reference characteristics." In other words, the comment database 240 indicates the correspondence between the reference characteristics of the reference worker and the comments written by the reference worker regarding the predetermined task.

[0032] A first example of a reference characteristic is the difference between the actual movement of a reference worker when performing a predetermined task and the model movement. The actual movement of a reference worker when performing a predetermined task is recognized by a sensor 110 (e.g., a camera). The actual movement of a reference worker when performing a predetermined task can also be acquired from the task record database 220. The model movement related to a predetermined task is acquired from the model movement database 230. The difference between the actual movement of a reference worker when performing a predetermined task and the model movement is hereinafter referred to as a "reference difference." For example, the reference difference is an incorrect movement that is not included in the model movement. Another example of a reference difference is the delay time between the model movement and the actual movement of the reference worker. Another example of a reference difference is the distance between the hand position of the model worker MW and the hand position of the reference worker. The simulation system 100 acquires information about the reference difference and registers the correspondence between the reference difference and the comment in the comment database 240.

[0033] A second example of the reference characteristics is the physical characteristics of the reference worker (e.g., dominant hand, hand size, height, etc.). The physical characteristics of the reference worker are obtained from the worker characteristic database 210. The simulation system 100 acquires information on the physical characteristics of the reference worker and registers the correspondence between the physical characteristics and comments in the comment database 240.

[0034] 3. Comment filter function As described above, when the target worker TW performs a predetermined task in the virtual space, the simulation system 100 renders the model worker MW performing the model action in the virtual space. At the same time, the simulation system 100 may obtain comments related to the predetermined task from the comment database 240 and present the obtained comments to the target worker TW. For example, the simulation system 100 renders the comments near the model worker MW in the virtual space. As another example, the simulation system 100 may render the comments in a speech bubble coming out of the model worker MW in the virtual space. As yet another example, the simulation system 100 may output audio comments from a speaker.

[0035] However, the tricks and weak points when learning a task may differ from person to person. Presenting comments that are inappropriate for the target worker TW or that are not helpful to the target worker TW will not produce the expected effect. For example, if comments created by a right-handed reference worker are presented to a left-handed target worker TW, those comments will not be helpful to the target worker TW. On the other hand, presenting all comments to the target worker TW will only confuse the target worker TW, and he or she will not know which comments are helpful. These situations are undesirable from the perspective of the learning effect and learning efficiency when a worker learns a specified task.

[0036] Therefore, the simulation system 100 according to this embodiment has a "comment filter function" for presenting comments that are appropriate for the target worker TW.

[0037] 4 is a conceptual diagram for explaining the comment filter function according to this embodiment. The simulation system 100 acquires information on the characteristics of a target worker TW who performs a predetermined task. The characteristics of the target worker TW are hereinafter referred to as "target characteristics."

[0038] A first example of the target characteristic is the difference between the actual movement of the target worker TW when performing a predetermined task and the model movement. For example, the actual movement of the target worker TW is the actual movement of the target worker TW the previous time the target worker TW performed the predetermined task. The previous actual movement of the target worker TW is obtained from the work performance database 220. As another example, the actual movement of the target worker TW may be a real-time movement. The real-time movement of the target worker TW is recognized by a sensor 110 (e.g., a camera). The model movement for the predetermined task is obtained from the model movement database 230. The difference between the actual movement of the target worker TW when performing the predetermined task and the model movement is hereinafter referred to as a "first difference." For example, the first difference is an incorrect movement that is not included in the model movement. As another example, the first difference is the delay time of the actual movement of the target worker TW from the model movement. As yet another example, the first difference is the distance between the hand position of the model worker MW and the hand position of the target worker TW.

[0039] A second example of the target characteristics is the physical characteristics of the target worker TW (e.g., handedness, hand size, height, etc.). The physical characteristics of the target worker TW are obtained from the worker characteristic database 210.

[0040] Next, the simulation system 100 extracts comments associated with the reference characteristics corresponding to the target characteristics of the target worker TW based on the comment database 240 described above.

[0041] In a first example, the reference characteristic of the reference worker includes a reference difference between the actual movement of the reference worker when performing a predetermined task and the model movement. The target characteristic of the target worker TW is a first difference between the actual movement of the target worker TW when performing a predetermined task and the model movement. The simulation system 100 extracts comments associated with reference differences including the first difference based on the comment database 240. For example, if the first difference is an erroneous movement not included in the model movement, the reference difference including the first difference is a reference difference including the same erroneous movement. As another example, if the first difference is a delay time, the reference difference including the first difference is a reference difference that is equal to or greater than that delay time. As yet another example, if the first difference is a distance, the reference difference including the first difference is a reference difference that is equal to or greater than that distance.

[0042] The simulation system 100 may extract, based on the comment database 240, a comment associated with the reference difference that matches the first difference.

[0043] In the second example, the reference characteristics of the reference worker include the physical characteristics of the reference worker. The target characteristics of the target worker TW include the physical characteristics of the target worker TW. The simulation system 100 extracts comments associated with the physical characteristics of the reference worker that match the physical characteristics of the target worker TW based on the comment database 240. For example, if the target worker TW is left-handed, comments made by left-handed reference workers are extracted. Physical characteristics expressed as numerical values, such as height, are grouped by a predetermined range. In this case, physical characteristics that match the physical characteristics of the target worker TW mean physical characteristics in the same group as the target worker TW.

[0044] Then, when the target worker TW performs a predetermined task in the virtual space, the simulation system 100 draws the model worker MW and presents the extracted comments to the target worker TW.

[0045] As described above, according to this embodiment, the comment database 240 is prepared. The comment database 240 indicates the correspondence between comments made by reference workers (predecessors) regarding a predetermined task and the characteristics of the reference workers. Based on the comment database 240, comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker TW are extracted. Then, the extracted comments are selectively presented to the target worker TW.

[0046] Since not all comments are presented indiscriminately, the target worker TW does not get confused. On the other hand, appropriate comments that match the characteristics of the target worker TW are presented. In other words, the target worker TW can learn a given task by referring to appropriate comments that match his or her own characteristics. For example, the target worker TW can refer to comments made by a reference worker who made the same mistake as the target worker TW. As another example, the target worker TW can refer to comments made by a reference worker who has the same physical characteristics as the target worker TW. Therefore, the learning effect and learning efficiency of the target worker TW are improved. [Explanation of symbols]

[0047] 100...Simulation system, 110...Sensor, 120...Input device, 130...Output device, 140...Experience device, 150...Processor, 160...Storage device, 170...Simulation program, 180...Work environment information, 200...Database, 210...Worker characteristic database, 220...Work performance database, 230...Model operation database, 240...Comment database

Claims

1. A simulation system that simulates a predetermined task in a virtual space, one or more processors; one or more storage devices for storing a comment database; Equipped with the comment database indicates a correspondence between comments made by a reference worker regarding the predetermined work and characteristics of the reference worker; the one or more processors: acquiring information on a model motion that serves as a model for performing the predetermined task; Acquire information on the characteristics of the target worker who will perform the predetermined work; extracting the comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker based on the comment database; When the target worker performs the predetermined work in the virtual space, a model worker performing the model action is drawn in the virtual space, and the extracted comment is presented to the target worker. It is configured as follows: the characteristics of the reference worker include a reference difference that is a difference between the motion of the reference worker when performing the predetermined work and the model motion; the characteristics of the target worker include a first difference that is a difference between a motion of the target worker when performing the predetermined task and the model motion; The comment associated with the reference difference of the reference worker, which includes the first difference of the target worker, is extracted and presented to the target worker. Simulation system.

2. A simulation system according to claim 1, the reference difference of the reference worker includes at least one of an erroneous movement not included in the model movement, a delay time of an actual movement of the reference worker from the model movement, and a distance between a hand position of the model worker and a hand position of the reference worker; the first difference of the target worker includes at least one of an erroneous motion not included in the model motion, a delay time of the actual motion of the target worker from the model motion, and a distance between a hand position of the model worker and a hand position of the target worker; When the first difference of the target worker is the erroneous action, the comment associated with the reference difference including the same erroneous action as the erroneous action of the target worker is extracted and presented to the target worker; When the first difference of the target worker is the delay time from the model motion, the comment associated with the reference difference that is a delay time equal to or greater than the delay time of the target worker is extracted and presented to the target worker; When the first difference of the target worker is the distance, the comment associated with the reference difference that is equal to or greater than the distance of the target worker is extracted and presented to the target worker. Simulation system.

3. A simulation system that simulates a predetermined task in a virtual space, one or more processors; one or more storage devices for storing a comment database; Equipped with the comment database indicates a correspondence between comments made by a reference worker regarding the predetermined work and physical characteristics of the reference worker; the one or more processors: acquiring information on a model motion that serves as a model for performing the predetermined task; Acquire information on the physical characteristics of the target worker who will perform the predetermined work; extracting the comments associated with the physical characteristics of the reference worker that match the physical characteristics of the target worker based on the comment database; When the target worker performs the predetermined work in the virtual space, a model worker performing the model action is drawn in the virtual space, and the extracted comment is presented to the target worker. It was configured as Simulation system.

4. A simulation program that is executed by a computer and that simulates a predetermined task in a virtual space, the comment database indicates a correspondence between comments made by a reference worker regarding the predetermined work and characteristics of the reference worker; The simulation program acquiring information on a model action that serves as a model for performing the predetermined task; acquiring information on characteristics of a target worker who will perform the predetermined work; extracting the comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker based on the comment database; When the target worker performs the predetermined work in the virtual space, a model worker performing the model action is drawn in the virtual space, and the extracted comment is presented to the target worker. causing the computer to execute the characteristics of the reference worker include a reference difference that is a difference between the motion of the reference worker when performing the predetermined work and the model motion; the characteristics of the target worker include a first difference that is a difference between a motion of the target worker when performing the predetermined task and the model motion; The comment associated with the reference difference of the reference worker, which includes the first difference of the target worker, is extracted and presented to the target worker. Simulation program.

5. A simulation program that is executed by a computer and that simulates a predetermined task in a virtual space, the comment database indicates a correspondence between comments made by a reference worker regarding the predetermined work and physical characteristics of the reference worker; The simulation program acquiring information on a model action that serves as a model for performing the predetermined task; acquiring information on physical characteristics of a target worker who will perform the predetermined work; extracting the comments associated with the physical characteristics of the reference worker that match the physical characteristics of the target worker based on the comment database; When the target worker performs the predetermined work in the virtual space, a model worker performing the model action is drawn in the virtual space, and the extracted comment is presented to the target worker. causing the computer to execute Simulation program.

Images