Simulation system and simulation program
The simulation system addresses the issue of personalized comment provision by correlating comments with worker characteristics, enhancing learning efficiency through targeted feedback.
Patent Information
- Application Number
- JP2023083867
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-12-04
- Estimated Expiration
- 2043-05-22
AI Technical Summary
Existing learning systems fail to provide personalized comments that cater to individual worker characteristics, leading to confusion and reduced learning efficiency when learning a predetermined task.
A simulation system and program that utilize a comment database correlating comments with worker characteristics, selectively presenting appropriate comments based on the target worker's characteristics through a comment filter function.
Enhances learning effect and efficiency by providing tailored comments that align with the target worker's characteristics, reducing confusion and improving task learning outcomes.
Smart Images

Figure 2024167648000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a technique for simulating a predetermined task in a virtual space. [Background technology]
[0002] Patent Document 1 discloses an information processing device that provides learning content for learning physical movements. The information processing device displays a superimposed image of a teacher's digital twin and a student's digital twin on a student's device. The information processing device also displays instruction information on the student's device based on an evaluation value of the student's digital twin. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] International Publication No. 2021 / 230101 Summary of the Invention [Problem to be solved by the invention]
[0004] When a worker learns a given task, it is conceivable to present the worker with comments (advice, etc.) regarding the given task. However, the tricks and weak points in mastering a task may differ from person to person. Presenting comments that are not suitable for the worker or that are not helpful to the worker will not produce the expected effect. On the other hand, presenting all comments to the worker will only confuse the worker, and he or she will not know which comments are helpful in the end.
[0005] An object of the present disclosure is to provide a technique capable of improving the learning effect and learning efficiency when a worker learns a given task. [Means for solving the problem]
[0006] One aspect of the present disclosure relates to a simulation system that simulates a predetermined task in a virtual space. The simulation system is one or more processors; one or more storage devices for storing a comment database; Equipped with. The comment database indicates the correspondence between comments made by a reference worker regarding a given task and the characteristics of the reference worker. The one or more processors acquire information on a model operation that serves as a model for performing a given task. The one or more processors acquire information on characteristics of a target worker who performs a predetermined task. The one or more processors extract comments associated with characteristics of a reference worker that correspond to the characteristics of the target worker, based on the comment database. When the target worker performs a predetermined task in the virtual space, the one or more processors render the model worker performing the model action in the virtual space and present the extracted comments to the target worker.
[0007] Another aspect of the present disclosure relates to a simulation program that is executed by a computer and that simulates a predetermined task in a virtual space. The comment database indicates the correspondence between comments made by a reference worker regarding a given task and the characteristics of the reference worker. The simulation program is Obtaining model motion information that serves as a model for performing a predetermined task; Acquiring information on characteristics of a target worker who performs a predetermined task; extracting comments associated with characteristics of a reference worker that correspond to the characteristics of the target worker based on the comment database; When the target worker performs a predetermined task in the virtual space, a model worker performing a model action in the virtual space is drawn, and the extracted comment is presented to the target worker. to be executed by the computer. Effect of the Invention
[0008] According to the present disclosure, a comment database is prepared. The comment database indicates a correspondence relationship between comments made by a reference worker regarding a specified task and the characteristics of the reference worker. Based on the comment database, comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker are extracted. Then, the extracted comments are selectively presented to the target worker. Since not all comments are presented indiscriminately, the target worker is not confused. Meanwhile, appropriate comments that match the characteristics of the target worker are presented. That is, the target worker can learn the specified task by referring to appropriate comments that match his or her own characteristics. Therefore, the learning effect and learning efficiency of the target worker are improved. [Brief description of the drawings]
[0009] [Figure 1] 1 is a conceptual diagram for explaining an overview of a simulation system according to an embodiment; [Diagram 2] 1 is a block diagram showing an example of a configuration of a simulation system according to an embodiment; [Diagram 3] FIG. 2 is a conceptual diagram for explaining an example of a comment database according to the embodiment. [Figure 4] FIG. 13 is a conceptual diagram for explaining a comment filter function according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] 1. Overview of the Simulation System Consider a predetermined task performed by a worker in a real space. For example, the predetermined task is performed in a real factory. The predetermined task is, for example, assembling parts. According to this embodiment, a simulation is utilized to efficiently train (study) a worker who performs the predetermined task.
[0011] FIG. 1 is a conceptual diagram for explaining an overview of a simulation system 100 according to this embodiment. The simulation system 100 performs a simulation of a predetermined task performed by a worker in a virtual space. To achieve this, the simulation system 100 virtually reproduces, in the virtual space, a work environment in which the worker performs the predetermined task. For example, the simulation system 100 reproduces (draws) the work environment in the virtual space based on DigitalTwin technology. For example, the simulation system 100 reproduces, in the virtual space, a virtual factory (DigitalTwin factory) that simulates an actual factory.
[0012] A worker to be trained (hereinafter, referred to as "target worker TW") can experience a predetermined task in a virtual space by using an experience device. For example, the experience device is a wearable device such as a head mounted display (HMD). The head mounted display displays the virtual space reproduced by the simulation system 100. The motion of the target worker TW in the real space (e.g., hand motion) is recognized by a motion capture technique. For example, the target worker TW is photographed by a camera, and the motion of the target worker TW is recognized. As another example, an inertial sensor (gyro sensor, acceleration sensor, etc.) may be attached to the body of the target worker TW, and the motion of the target worker TW may be recognized based on the detection result of the inertial sensor. The simulation system 100 draws the recognized motion of the target worker TW by superimposing it on the virtual space. For example, the simulation system 100 draws the recognized hand motion of the target worker TW by superimposing it on the virtual space. This allows the target worker TW to get the feeling that he or she is performing a predetermined task in the virtual space. That is, the target worker TW can experience a predetermined task in the virtual space.
[0013] Furthermore, the simulation system 100 may draw a virtual model worker MW performing a model action superimposed on the virtual space. The model action is an action that serves as a model when the target worker TW performs a specified task. In the virtual space, the virtual model worker MW performs the model action. The model worker MW may also be called a "ghost worker." In the virtual space, the target worker TW performs a specified task by imitating the model action performed by the model worker MW. This enables efficient and effective training.
[0014] 2 is a block diagram showing an example of the configuration of a simulation system 100 according to this embodiment. The simulation system 100 includes a sensor 110, an input device 120, an output device 130, and an experience device 140.
[0015] The sensor 110 is disposed in the real space and detects various information. For example, the sensor 110 detects the motion of a worker in the real space. Examples of the sensor 110 that detects the motion of a worker include a camera, an infrared sensor, an inertial sensor, and the like. For example, the motion of a worker can be detected by photographing the worker with a camera. As another example, an inertial sensor (gyro sensor, acceleration sensor, etc.) may be attached to the body of the worker, and the motion of the worker may be detected by the inertial sensor.
[0016] As another example, the sensor 110 may detect physical characteristics of the worker in real space. Examples of physical characteristics include handedness, hand size, grip strength, height, muscle mass, eyesight, and the like. For example, the handedness of the worker is detected based on the movement of the worker captured by a camera. As another example, the hand size, height, and the like are detected based on an image of the worker captured by a camera. In addition, the sensor 110 may include a height gauge, a weight scale, a body composition scale, a grip strength meter, an eyesight meter, and the like.
[0017] As yet another example, the sensor 110 may include a biosensor that detects bioinformation of the worker in the real space. Examples of the bioinformation include body temperature, heart rate, fatigue level, stress level, and the like.
[0018] Examples of the input device 120 include a touch panel, a keyboard, a mouse, a microphone, etc. Examples of the output device 130 include a display, a touch panel, a speaker, etc.
[0019] The experience device 140 is used to allow the target worker TW to experience the work environment and a predetermined task in a virtual space. For example, the experience device 140 is a wearable device such as a head mounted display (HMD). The head mounted display displays the virtual space reproduced by the simulation system 100.
[0020] The simulation system 100 further includes one or more processors 150 (hereinafter simply referred to as "processor 150") and one or more storage devices 160 (hereinafter simply referred to as "storage device 160"). The processor 150 executes various processes. Examples of the processor 150 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The storage device 160 stores various information. Examples of the storage device 160 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory.
[0021] The simulation program 170 is a computer program for performing a simulation in the above-mentioned virtual space, and is executed by the processor 150. Various processes by the simulation system 100 may be realized by cooperation between the processor 150 executing the simulation program 170 and the storage device 160. The simulation program 170 is stored in the storage device 160. The simulation program 170 may be recorded in a computer-readable recording medium.
[0022] The work environment information 180 is information about a work environment (e.g., an actual factory) reproduced in a virtual space. For example, the work environment information 180 indicates a three-dimensional arrangement of structures (lines, machines, walls, pillars, etc.) in the work environment. For example, the three-dimensional arrangement of structures is expressed by CAD data. The work environment information 180 is stored in the storage device 160.
[0023] The storage device 160 further stores a database 200. The database 200 includes a worker characteristic database 210, a work performance database 220, and a model operation database 230.
[0024] The worker characteristic database 210 is a database showing the physical characteristics of each worker. Examples of the physical characteristics include handedness, hand size, grip strength, height, muscle mass, eyesight, etc. For example, the physical characteristics of each worker are detected by the above-mentioned sensor 110. As another example, each worker may input his / her own physical characteristics using the input device 120. Physical characteristics expressed by numerical values such as height may be grouped according to a predetermined range.
[0025] The work history database 220 is a database showing the results of predetermined work performed by various workers. For example, the work history database 220 includes video of the worker performing the predetermined work. The video of the worker is captured by a camera included in the above-mentioned sensor 110. As another example, the work history database 220 may show the content of the action performed by the worker during the predetermined work. As yet another example, the work history database 220 may show the time it took for the worker to complete the predetermined work. Such a work history database 220 is generated based on the action of the worker detected by the above-mentioned sensor 110.
[0026] The model action database 230 is a database related to model actions that serve as a model when the target worker TW performs a predetermined task. The model actions related to the predetermined task may be created in advance based on the task record database 220.
[0027] The processor 150 reproduces, in a virtual space, a work environment in which a worker performs a predetermined work, based on the work environment information 180. For example, the processor 150 reproduces (draws) the work environment in the virtual space based on DigitalTwin technology.
[0028] Moreover, the processor 150 recognizes the motion of the target worker TW, who is the training target, using the sensor 110. The motion of the target worker TW in the real space is recognized by a motion capture technique. For example, the motion of the target worker TW is recognized by photographing the target worker TW with a camera. As another example, an inertial sensor may be attached to the body of the target worker TW, and the motion of the target worker TW may be recognized based on the detection result of the inertial sensor. The processor 150 draws the recognized motion of the target worker TW by superimposing it on the virtual space. For example, the processor 150 draws the recognized hand movement of the target worker TW by superimposing it on the virtual space. The target worker TW can get a sense of performing a predetermined task in the virtual space via the experience device 140. That is, the target worker TW can experience a predetermined task in the virtual space via the experience device 140.
[0029] Furthermore, the processor 150 determines a model action to be a model for the target worker TW to perform a predetermined task based on the model action database 230. Then, the processor 150 draws a virtual model worker MW performing the model action by superimposing it on the virtual space. In the virtual space, the target worker TW performs the predetermined task by imitating the model action performed by the model worker MW. This enables efficient and effective training.
[0030] 2. Comments Database FIG. 3 is a conceptual diagram for explaining an example of the comment database 240 according to this embodiment. The comment database 240 is a database of comments related to a predetermined task, and is stored in the storage device 160. Examples of comments related to the predetermined task include advice, tips, and points of observation. Such comments are created in advance by workers (predecessors) who have performed the predetermined task in the past. A worker (predecessor) who has performed the predetermined task in the past is hereinafter referred to as a "reference worker." Each reference worker can input comments related to the predetermined task by using the input device 120. The simulation system 100 registers the comments input by each reference worker in the comment database 240.
[0031] According to this embodiment, a comment on a predetermined task is associated with the characteristics of a reference worker who created the comment. The characteristics of a reference worker are hereinafter referred to as "reference characteristics." In other words, the comment database 240 indicates the correspondence between the reference characteristics of a reference worker and the comments created by the reference worker on a predetermined task.
[0032] A first example of the reference characteristic is a difference between an actual motion of a reference worker when performing a predetermined task and a model motion. The actual motion of a reference worker when performing a predetermined task is recognized by a sensor 110 (e.g., a camera). The actual motion of a reference worker when performing a predetermined task can also be acquired from the task record database 220. The model motion related to the predetermined task is acquired from the model motion database 230. The difference between an actual motion of a reference worker when performing a predetermined task and a model motion is hereinafter referred to as a "reference difference." For example, the reference difference is an incorrect motion that is not included in the model motion. As another example, the reference difference is a delay time of the actual motion of the reference worker from the model motion. As yet another example, the reference difference is a distance between the hand position of the model worker MW and the hand position of the reference worker. The simulation system 100 acquires information on the reference difference and registers a correspondence between the reference difference and a comment in the comment database 240.
[0033] A second example of the reference characteristic is the physical characteristics of the reference worker (e.g., handedness, hand size, height, etc.). The physical characteristics of the reference worker are obtained from the worker characteristic database 210. The simulation system 100 acquires information on the physical characteristics of the reference worker, and registers the correspondence between the physical characteristics and comments in the comment database 240.
[0034] 3. Comment filter function As described above, when the target worker TW performs a predetermined task in the virtual space, the simulation system 100 renders the model worker MW performing the model action in the virtual space. At the same time, the simulation system 100 may obtain a comment related to the predetermined task from the comment database 240 and present the obtained comment to the target worker TW. For example, the simulation system 100 renders the comment near the model worker MW in the virtual space. As another example, the simulation system 100 may render the comment in a speech bubble coming out of the model worker MW in the virtual space. As yet another example, the simulation system 100 may output an audio comment from a speaker.
[0035] However, the tricks and weak points in mastering a task may differ from person to person. Presenting comments that are not suitable for the target worker TW or that are not helpful to the target worker TW will not produce the expected effect. For example, even if comments made by a right-handed reference worker are presented to a left-handed target worker TW, the comments will not be helpful to the target worker TW. On the other hand, presenting all comments to the target worker TW will only confuse the target worker TW, and he or she will not know which comments are helpful in the end. These things are not desirable from the viewpoint of the learning effect and learning efficiency when a worker learns a specified task.
[0036] Therefore, the simulation system 100 according to the present embodiment has a "comment filter function" for presenting appropriate comments to the target worker TW.
[0037] 4 is a conceptual diagram for explaining the comment filter function according to the present embodiment. The simulation system 100 acquires information on the characteristics of a target worker TW who performs a predetermined task. The characteristics of the target worker TW are hereinafter referred to as "target characteristics."
[0038] A first example of the target characteristic is a difference between the actual motion of the target worker TW when performing a predetermined task and the model motion. For example, the actual motion of the target worker TW is the actual motion of the target worker TW when performing the predetermined task last time. The previous actual motion of the target worker TW is obtained from the task record database 220. As another example, the actual motion of the target worker TW may be a real-time motion. The real-time motion of the target worker TW is recognized by the sensor 110 (e.g., a camera). The model motion related to the predetermined task is obtained from the model motion database 230. The difference between the actual motion of the target worker TW when performing the predetermined task and the model motion is hereinafter referred to as a "first difference". For example, the first difference is an incorrect motion that is not included in the model motion. As another example, the first difference is a delay time of the actual motion of the target worker TW from the model motion. As yet another example, the first difference is a distance between the hand position of the model worker MW and the hand position of the target worker TW.
[0039] A second example of the target characteristic is the physical characteristics of the target worker TW (e.g., handedness, hand size, height, etc.). The physical characteristics of the target worker TW are obtained from the worker characteristic database 210.
[0040] Next, the simulation system 100 extracts comments associated with the reference characteristic corresponding to the target characteristic of the target worker TW, based on the comment database 240 described above.
[0041] In a first example, the reference characteristic of the reference worker includes a reference difference between the actual motion of the reference worker when performing a predetermined task and the model motion. The target characteristic of the target worker TW is a first difference between the actual motion of the target worker TW when performing a predetermined task and the model motion. The simulation system 100 extracts a comment associated with the reference difference including the first difference based on the comment database 240. For example, if the first difference is an erroneous motion that is not included in the model motion, the reference difference including the first difference is a reference difference including the same erroneous motion. As another example, if the first difference is a delay time, the reference difference including the first difference is a reference difference equal to or greater than the delay time. As yet another example, if the first difference is a distance, the reference difference including the first difference is a reference difference equal to or greater than the distance.
[0042] The simulation system 100 may extract, based on the comment database 240, a comment associated with a reference difference that matches the first difference.
[0043] In the second example, the reference characteristics of the reference worker include the physical characteristics of the reference worker. The target characteristics of the target worker TW include the physical characteristics of the target worker TW. The simulation system 100 extracts comments associated with the physical characteristics of the reference worker that match the physical characteristics of the target worker TW based on the comment database 240. For example, if the target worker TW is left-handed, comments made by a left-handed reference worker are extracted. Physical characteristics expressed by numerical values such as height are grouped by a predetermined width. In this case, physical characteristics that match the physical characteristics of the target worker TW mean physical characteristics in the same group as the target worker TW.
[0044] Then, when the target worker TW performs a predetermined task in the virtual space, the simulation system 100 renders the model worker MW and presents the extracted comments to the target worker TW.
[0045] As described above, according to this embodiment, the comment database 240 is prepared. The comment database 240 indicates the correspondence between comments made by reference workers (predecessors) regarding a specific task and the characteristics of the reference workers. Based on the comment database 240, comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker TW are extracted. Then, the extracted comments are selectively presented to the target worker TW.
[0046] Since not all comments are presented randomly, the target worker TW does not become confused. On the other hand, appropriate comments that match the characteristics of the target worker TW are presented. In other words, the target worker TW can learn a specific task by referring to appropriate comments that match his / her own characteristics. For example, the target worker TW can refer to comments made by a reference worker who made the same mistake as the target worker TW. As another example, the target worker TW can refer to comments made by a reference worker who has the same physical characteristics as the target worker TW. Therefore, the learning effect and learning efficiency of the target worker TW are improved. [Explanation of symbols]
[0047] 100...simulation system, 110...sensor, 120...input device, 130...output device, 140...experience device, 150...processor, 160...storage device, 170...simulation program, 180...work environment information, 200...database, 210...worker characteristic database, 220...work performance database, 230...model operation database, 240...comment database
Claims
1. A simulation system for simulating a predetermined task in a virtual space, comprising: one or more processors; One or more storage devices for storing a comment database; Equipped with the comment database indicates a correspondence between comments made by a reference worker regarding the predetermined work and characteristics of the reference worker; The one or more processors: Acquire information on a model motion that serves as a model for performing the predetermined task; Acquire information on characteristics of a target worker who will perform the predetermined work; extracting the comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker based on the comment database; When the target worker performs the predetermined work in the virtual space, a model worker performing the model action is drawn in the virtual space, and the extracted comment is presented to the target worker. It was configured as Simulation system.
2. 2. The simulation system according to claim 1, the characteristics of the reference worker include a reference difference that is a difference between the motion of the reference worker when performing the predetermined task and the model motion, the characteristics of the target worker include a first difference that is a difference between a motion of the target worker when performing the predetermined task and the model motion; The one or more processors extract the comments associated with the reference difference including the first difference based on the comment database. Simulation system.
3. 3. The simulation system according to claim 2, The one or more processors extract, based on the comment database, the comments associated with the reference difference that matches the first difference. Simulation system.
4. 2. The simulation system according to claim 1, The characteristics of the reference worker include physical characteristics of the reference worker; The characteristics of the target worker include physical characteristics of the target worker; The one or more processors extract, based on the comment database, the comments associated with the physical characteristics of the reference worker that match the physical characteristics of the target worker. Simulation system.
5. A simulation program that is executed by a computer and that simulates a predetermined task in a virtual space, The comment database indicates a correspondence between comments made by a reference worker regarding the predetermined work and characteristics of the reference worker, The simulation program is acquiring information on a model motion that serves as a model for performing the predetermined task; acquiring information on characteristics of a target worker who will perform the predetermined work; extracting the comments associated with the characteristics of the reference worker that correspond to the characteristics of the target worker based on the comment database; When the target worker performs the predetermined work in the virtual space, a model worker performing the model action is drawn in the virtual space, and the extracted comment is presented to the target worker. causing the computer to execute Simulation program.
Citation Information
Patent Citations
Sensor Assisted Head Mounted Displays For Welding
CN113192374A
Content presentation system, and content presentation method
JP2020003708A
Learning assisting system, learning assisting device, and program
JP2020144233A
Work support system, work support device, and work support method
JP2020144776A
Weld training simulations using mobile devices, modular workpieces, and simulated welding equipment
US20210158716A1
