Skill transmission support system, skill transmission support method, and program

Abstract

To provide a skill transmission support system, method and program for providing a technology for efficiently and electronically recording a skill.SOLUTION: In a skill transmission support system, a skill transmission support device includes an operation extracting unit that extracts one or a plurality of operations of a worker, an operation list generating unit that generates a photographing operation list including the operations extracted by the operation extracting unit, an actual operation video image acquiring unit that acquires an actual operation video image, and an actual operation video image determining unit that determines whether or not an operation included in the photographing operation list is photographed in the actual operation video image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a skill transfer support system, a skill transfer support method, and a program. [Background technology]

[0002] Various techniques exist for electronically recording skills in order to transfer the skills of an expert to others.

[0003] Patent Document 1 discloses a technology related to a motion information generating device. Paragraph

[0018] of the document states, "The motion information generating device according to the present invention is a device for automatically generating optimal motions (model motions) for a certain task based on data of a person's motions performed multiple times." Paragraph

[0036] also states, "A worker is made to perform a certain task multiple times, and the task data for each task is recorded by the task data recording unit 204 (step S400)." Paragraph

[0040] states, "The motion classification unit 206 analyzes the task samples selected by the motion selection unit 205, thereby dividing and classifying the overall motion from the start to the end of the task into multiple partial motions (step S402)." Paragraph

[0045] states, "The motion synthesis unit 207 selects the best partial motion for each partial motion from the multiple good samples, and synthesizes the selected best partial motions to generate optimal motion data for the entire task (step S403)." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6583537 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, in business fields such as manufacturing, there is a wide range of skills to be passed down, so efficient and highly accurate archiving is required. The technology disclosed in Patent Document 1 lacks efficiency because workers must perform the work multiple times and select the best samples from those.

[0006] The present invention has been made in view of the above points, and has as its object to provide a technique for efficiently recording skills electronically. [Means for solving the problem]

[0007] The present application includes a number of means for solving the above problems, examples of which are as follows.

[0008] In order to solve the above problem, a skill transfer support system according to one embodiment of the present invention is characterized by comprising an action extraction unit that extracts one or more actions of a worker, an action list generation unit that generates a filmed action list including the actions extracted by the action extraction unit, an actual work video acquisition unit that acquires an actual work video, and an actual work video determination unit that determines whether or not an action included in the filmed action list is filmed in the actual work video.

[0009] The action extraction unit may extract the action using a sensor that acquires action data of the worker, and the actual work video judgment unit may analyze an image of the worker included in the actual work video to detect the action included in the shooting action list.

[0010] The skill transfer support system may include a memory unit that stores a candidate action list including candidate actions of the worker and names of the candidate actions, the action extraction unit extracts the action of the worker when the action of the worker is included in the candidate action list, and the action list generation unit includes the action extracted by the action extraction unit and the name of the candidate action related to the action in the candidate action list in the shooting action list.

[0011] The movement extracting unit may be configured to, upon detecting a predetermined preparatory movement, extract the movement performed following the preparatory movement.

[0012] The skill transfer support system may include a string extraction unit that extracts strings indicating a subject from audio data acquired together with images used by the action extraction unit to extract actions, a subject list generation unit that generates a subject list including the strings extracted by the string extraction unit, and a material image management unit that acquires material images corresponding to the strings of the subject and associates them with the strings included in the subject list, and the actual work video determination unit may be characterized in that it determines whether the subject is filmed in the actual work video using the material images.

[0013] The material image management section may be configured to, when the action extraction section extracts a predetermined action, acquire a portion of the image corresponding to the predetermined action as the material image.

[0014] In addition, in order to solve the above-mentioned problems, a skill transfer support method according to another aspect of the present invention is a skill transfer support method using a skill transfer system, characterized in that it comprises an action extraction step for extracting one or more actions of a worker, an action list generation step for generating an action list including the actions extracted in the action extraction step, an actual work video acquisition step for acquiring an actual work video, and an actual work video judgment step for determining whether or not the actions included in the action list are captured in the actual work video.

[0015] In addition, in order to solve the above-mentioned problems, a program according to another aspect of the present invention is a program that causes a processing unit of a computer to execute a skill transfer support method, and is characterized by executing an action extraction procedure that extracts one or more actions of a worker, an action list generation procedure that generates an action list including the actions extracted in the action extraction procedure, an actual work video acquisition procedure that acquires an actual work video, and an actual work video judgment procedure that judges whether or not the actions included in the action list are captured in the actual work video. [Effects of the Invention]

[0016] According to the present invention, a technique for efficiently recording skills electronically can be provided.

[0017] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 2 is a diagram illustrating an example of functional blocks of the skill transfer support system according to the first embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of a data structure of a candidate action list. [Figure 3] FIG. 10 is a diagram illustrating an example of the data structure of a shooting operation list. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a skill transfer assistance device. [Figure 5] 10 is a flowchart illustrating an example of a shooting operation list generation process. [Figure 6] 10 is a flowchart illustrating an example of an actual work video determination process. [Figure 7] FIG. 10 is a diagram showing an example of a shooting operation list display screen. [Figure 8] 10 is a flowchart showing an example of an actual work moving image determination process in the first modified example. [Figure 9] FIG. 10 is a diagram showing an example of functional blocks of a skill transfer support system according to a second modified example. [Figure 10] FIG. 10 is a diagram showing an example of functional blocks of a skill transfer support system according to a third modified example. [Figure 11] FIG. 10 is a diagram illustrating an example of functional blocks of a skill transfer support system according to a second embodiment. [Figure 12] FIG. 10 is a diagram showing an example of the data structure of a subject list 123. [Figure 13] 10 is a flowchart illustrating an example of a subject list generation process. [Figure 14]10 is a flowchart showing an example of subject moving image determination processing in the second embodiment. [Figure 15] FIG. 10 is a diagram showing an example of an imaging list display screen. [Figure 16] 10 is a flowchart showing an example of a subject list generation process in a modified example of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] First Embodiment Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of functional blocks of a skill transfer support system 1 in a first embodiment. The skill transfer support system 1 generates an actual work video in which the skill is filmed in order to transfer the skill of an expert. The skill transfer support system 1 in this embodiment is configured to generate a filming action list that lists the actions to be filmed in the actual work video, so as to more reliably include the actions of the expert in the actual work image.

[0020] The skill transfer support system 1 includes a skill transfer support device 100, one or more sensors 300, and one or more cameras 400.

[0021] The skill transfer support device 100 is an information processing device such as a PC (Personal Computer). The skill transfer support device 100 includes a processing unit 110, a storage unit 120, an input unit 130, an output unit 140, and a communication unit 150. The processing unit 110 controls the entire skill transfer support device 100. The storage unit 120 stores information required for processing by the processing unit 110. The input unit 130 accepts information input to the skill transfer support device 100 from an input device connected via an input IF 14 (described later). The output unit 140 outputs information stored in the skill transfer support device 100 from an output device connected via an output IF 15 (described later). The communication unit 150 controls transmission and reception of information to and from other information processing devices connected for communication via a network.

[0022] The processing unit 110 includes an action extraction unit 111, an action list generation unit 112, a video acquisition unit 113, and an actual work video determination unit 114. The action extraction unit 111 extracts one or more pieces of action data of the worker using the sensor 300. For example, the action extraction unit 111 acquires data measured using the inertial sensor 300 attached to the worker's body. As an example, the action extraction unit 111 acquires data of actions performed while wearing gloves to which the sensor 300 is attached. The data of the sensor 300 acquired by the action extraction unit 111 is treated as data for generating an action list.

[0023] The movement extraction unit 111 converts the movement of the worker into data by replacing the movement list generation data with the movement of the virtual skeletal model. The movement data of the skeletal model obtained by the movement extraction unit 111 using data from the sensor 300 will be described as primary movement data.

[0024] The primary action data extracted by the action extraction unit 111 may include multiple actions. As an example, when a worker wearing the sensor 300 on his or her body performs the action of "loosening a screw" and then the action of "opening a cover," the primary action data includes the action data of "loosening a screw" and the action data of "opening the cover."

[0025] The action extraction unit 111 divides the primary action data of the worker into action units. For example, the action extraction unit 111 refers to a candidate action list 121 (described below) that includes types of the worker's actions, and if a candidate action is included in the primary action data, extracts the action, thereby dividing the primary action data. Using the above example, the candidate action list 121 includes, as candidate actions, action data for the action of "loosening (a screw)" and action data for the action of "opening (a cover)." The action extraction unit 111 divides the primary action data into action units by determining whether or not a candidate action is included in the primary action data. One or more pieces of action data obtained by dividing will be described as secondary action data.

[0026] The method by which the action extraction unit 111 extracts the secondary action data from the primary action data is not limited to this example. For example, the action extraction unit 111 may extract the secondary action data by determining whether a predetermined preparatory action has been performed and extracting the action performed following the preparatory action. The preparatory action includes, for example, a pointing action and an action that has not been performed for a certain period of time.

[0027] The action list generation unit 112 generates a shooting action list 122 (described later) that includes the actions extracted by the action extraction unit 111. When the action extraction unit 111 extracts secondary action data using a candidate action list 121 (described later), the action list generation unit 112 includes the name of the candidate action in the shooting action list 122. Note that the action list generation unit 112 may include information indicating the timing at which the extracted secondary action data starts in the shooting action list 122.

[0028] The video acquisition unit 113 acquires images captured by the camera 400. For example, the images acquired by the video acquisition unit 113 include actual work videos of work performed by a worker for skill transfer. Note that the video acquisition unit 113 may acquire images from other devices (not shown) that are communicably connected via a network.

[0029] The actual work video determination unit 114 determines whether or not an action included in a captured action list 122 (described later) has been captured in the actual work video. As a first example, the actual work video determination unit 114 analyzes an image of the worker included in the actual work video to extract primary action data of the worker and compares it with the captured action list 122. In this way, it is determined whether or not the worker has performed the action included in the captured action list 122.

[0030] For example, the actual work video determination unit 114 extracts a skeletal model from the silhouette of the worker's hand captured in the actual work video, and extracts primary action data from the movement of the skeletal model. By extracting the primary action data simultaneously with capturing the actual work video, the actual work video determination unit 114 can instantly determine whether the worker has performed an action included in the captured action list 122. Note that the actual work video determination unit 114 may track a portion of a predetermined brightness captured in the actual work video, thereby replacing the worker's action with the action of the skeletal model and extracting primary action data.

[0031] As a second example, the actual work video determination unit 114 can directly monitor the motion data of the worker to be filmed, thereby determining whether the worker has performed the motion included in the filmed motion list 122. For example, the actual work video determination unit 114 attaches a marker that reflects infrared light to the worker's body and uses a camera 400 that captures the reflected infrared light to directly extract primary motion data. Also, for example, the actual work video determination unit 114 extracts primary motion data using a sensor 300, similar to the motion extraction unit 111.

[0032] The storage unit 120 stores a candidate action list 121 and a shooting action list 122. The candidate action list 121 includes candidate actions of the worker and the names of the candidate actions. The shooting action list 122 is information generated by a shooting action list generation process described below, and is information that lists target actions to be shot, which are actions of the worker that should be included in the actual work video.

[0033] The sensor 300 is, for example, an inertial sensor that detects the movement of the worker. The sensor 300 may be an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, or the like. As an example, the sensor 300 is installed at the joint of a glove worn by the worker, but the installation location of the sensor 300 is not limited to this example. For example, the sensor 300 may be installed at each key point of a suit that covers the entire body of the worker.

[0034] The camera 400 is an imaging device that captures images. The camera 400 captures a video of an actual work. The camera 400 may be capable of capturing infrared light reflected by a marker attached to the body of a worker.

[0035] 2 is a diagram showing an example of the data structure of the candidate action list 121. The candidate action list 121 includes a candidate action ID (identification) 121a, a candidate action name 121b, and action data 121c. The candidate action ID 121a is identification information that identifies each record in the candidate action list 121. The candidate action name 121b is a character string that indicates the name of the candidate action.

[0036] The action data 121c is action data when a virtual skeleton model executes the candidate action. For example, the action data 121c is position information and rotation information of each skeleton. Note that the action data 121c is not limited to this example, and may be any information that can identify the candidate action. For example, the action data 121c may be the sensor data itself of the worker who executed the candidate action.

[0037] 3 is a diagram showing an example of the data structure of the photographing action list 122. The photographing action list 122 includes a photographing action ID 122a, a start timing 122b, a photographing action name 122c, a photographing target action image 122d, and an action ID 122e.

[0038] The photographing action ID 122a is identification information for identifying the action to be photographed. The start timing 122b is information indicating the timing at which the action starts. For example, the start timing 122b is information indicating the start timing of the action to be photographed, which is the timing at which the sensor 300 starts detecting the action of the worker, i.e., the timing at which acquisition of primary action data starts.

[0039] The photographed action name 122c is a character string indicating the name of the action to be photographed. The photographed action image 122d is an image of the action to be photographed. For example, the photographed action image 122d is a video of a candidate action that has been photographed in advance in association with each record in the candidate action list 121. Also, for example, the photographed action image 122d is a video that has been photographed after the photographed action list 122 has been generated and before the actual work image has been photographed. The action ID 122e is identification information that identifies the action of the worker. The action ID 122e corresponds to the candidate action ID 121a in the candidate action list 121.

[0040] 4 is a diagram showing an example of the hardware configuration of the skill transfer assistance device 100. The skill transfer assistance device 100 includes a calculation device 11, a memory 12, an external storage device 13, an input IF (Interface) 14, an output IF 15, and a communication IF 16, and each component is connected by a bus.

[0041] The arithmetic device 11 is a arithmetic device such as a CPU (Central Processing Unit), and executes processing according to a program recorded in the memory 12 or the external storage device 13. In the skill transfer assistance device 100, processing is performed by the arithmetic device 11 that operates according to a program read onto the memory 12 or the external storage device 13. The processing unit 110 realizes each function by the arithmetic device 11 executing the program.

[0042] The memory 12 is a storage device such as RAM (Random Access Memory) or flash memory, and functions as a storage area from which programs and data are temporarily read. The external storage device 13 is a writable and readable storage medium and storage media drive, such as an HDD (Hard Disk Drive), CD-R (Compact Disc-Recordable), or DVD-RAM (Digital Versatile Disk-Random Access Memory). The functions of the storage unit 120 are realized by the memory 12 or the external storage device 13. Note that the functions of the storage unit 120 may also be realized by a storage device connected via the communication IF 16.

[0043] The input IF14 is an interface for receiving input operations from an operator, and is connected to input devices such as a touch panel, keyboard, mouse, microphone, etc. in addition to the sensor 300 and camera 400. The output IF15 is an interface for outputting information to an output device such as an LCD (Liquid Crystal Display) display built into the skill transfer assistance device 100.

[0044] The communication IF 16 is an interface for connecting the skill transfer assistance device 100 to a network N, and is connected to a communication device such as a LAN (Local Area Network) card. The skill transfer assistance device 100 may also have a storage medium drive device (not shown) for inputting and outputting information from portable media such as a CD (Compact Disk) or a DVD (Digital Versatile Disk).

[0045] The processing of each component of the skill transfer support device 100 may be executed by one piece of hardware or by multiple pieces of hardware. Furthermore, the processing of each component of the skill transfer support device 100 may be realized by one program or by multiple programs.

[0046] 5 is a flowchart showing an example of the shooting action list generation process, which is started when the skill transfer support device 100 receives an input operation for instructing to start the shooting action list generation process.

[0047] First, the action extraction unit 111 acquires data for generating an action list (step S11). Specifically, the action extraction unit 111 acquires data obtained using the sensor 300 as the data for generating an action list.

[0048] Next, the movement extraction unit 111 extracts primary movement data from the movement list generation data (step S12). Specifically, the movement extraction unit 111 extracts primary movement data by replacing the movement list generation data acquired in step S11 with the movement of a skeletal model of the worker. As an example, the movement extraction unit 111 uses information such as acceleration and angular velocity obtained from the inertial sensor 300 to extract positional information and rotational information of each skeleton when a previously generated skeletal model reproduces the same movement as the primary movement data. The movement extraction unit 111 handles the extracted positional information and rotational information of each skeleton as primary movement data.

[0049] Next, the action extraction unit 111 refers to the candidate action list 121 and extracts secondary action data (step S13). Specifically, the action extraction unit 111 refers to the primary action data extracted in step S12 and extracts a portion that matches the action data 121c of the candidate action list 121 as secondary action data. Note that the portion extracted as secondary action data does not have to match the action data 121c of the candidate action list 121 completely. The action extraction unit 111 is configured to extract, as secondary action data, actions that are similar enough to be assumed to be the candidate action with some degree of certainty.

[0050] As a result of the processing of this step, one or more secondary action data are extracted from the primary action data.

[0051] Next, the action list generation unit 112 generates a photographing action list 122 in which names are associated with the extracted actions (step S14). Specifically, the action list generation unit 112 generates a record of the photographing action list 122 in which the candidate action ID of the candidate action list 121 used to extract the secondary action data in step S13 is set as action ID 122e and the candidate action name 121b is set as photographing action name 122c.

[0052] Furthermore, the action list generation unit 112 specifies the start timing of the extracted secondary action data relative to the start time of the primary action data, and includes it as start timing 122b in the shooting action list 122. Note that a demonstration video of the candidate action that has been shot in advance is stored in an area not shown in the figure of the storage unit 120. The action list generation unit 112 extracts from the storage unit 120 a demonstration video of the action corresponding to the extracted secondary action data, and includes it as a shooting target action image 122d in the shooting action list 122. Thereafter, the processing unit 110 ends the processing of this flowchart.

[0053] In step S13, the action extraction unit 111 may extract a portion where a predetermined preparatory movement was performed from the primary action data, and extract an action performed following that portion as secondary action data. In this case, the action extraction unit 111 may extract, as secondary action data, an action performed following the preparatory movement if the action is included in the candidate action list 121, or may extract, as secondary action data, action data of a predetermined length from among the actions performed following the preparatory movement. In this case, the action list generation unit 112 can set any name as the shooting action name.

[0054] Furthermore, the action list generation unit 112 may output a display requesting an image of the secondary action data instead of extracting the image of the secondary action data from the storage unit 120 and setting it as the photographed action image 122d. In this case, the action list generation unit 112 displays, for example, the photographed action name 122c, and displays a display prompting the user to photograph the photographed action, and sets the video photographed based on the display as the photographed action image 122d.

[0055] 6 is a flowchart showing an example of an actual work video determination process. This flowchart starts when the skill transfer support device 100 receives an instruction to start shooting an actual work video. Note that the work video determination process in this embodiment is executed in parallel with shooting the actual work video.

[0056] First, the video acquisition unit 113 acquires an actual work video (step S21). Specifically, the video acquisition unit 113 identifies an actual work video that is continuously being shot using the camera 400, and notifies the actual work video determination unit 114 of the storage location of the actual work video in order to immediately perform processing in step S22.

[0057] Next, the actual work video determination unit 114 extracts primary movement data indicating the movement of the worker (step S22). Specifically, the actual work video determination unit 114 extracts a skeletal model from the actual work video acquired in step S21, and extracts primary movement data from the movement of the skeletal model. For example, the actual work video determination unit 114 extracts a skeletal model from the silhouette of the worker's hand captured in the actual work video, and extracts position information and rotation information of each skeleton when the skeletal model reproduces the same movement as the worker. The actual work video determination unit 114 treats the extracted position information and rotation information of each skeleton as primary movement data.

[0058] As described above, the method by which the actual work video determining unit 114 extracts the primary action data is not limited to this example. This process can be executed in parallel with the acquisition of the actual work video in step S21.

[0059] Next, the actual work video determining unit 114 determines whether or not the primary action data includes a candidate action (step S23). Specifically, the actual work video determining unit 114 refers to the primary action data extracted in step S22 and determines whether or not any of the plurality of action data 121c included in the candidate action list 121 is included in the primary action data.

[0060] If the actual work video determining unit 114 determines that the primary action data does not include the candidate action ("NO" in step S23), the actual work video determining unit 114 executes the process of this step again (step S23). That is, the process of this step is repeated until it determines that the primary action data includes the candidate action.

[0061] When the actual work video determination unit 114 determines that the primary action data includes a candidate action ("YES" in step S23), the actual work video determination unit 114 identifies the action ID of the corresponding candidate action (step S24). Specifically, the actual work video determination unit 114 identifies the candidate action ID 121a of the candidate action included in the primary action data as the action ID of the corresponding secondary action data. Note that in this embodiment, the actual work video determination unit 114 sequentially monitors the primary action data to extract secondary action data, and therefore, only one secondary action data (for one action) is extracted in this step.

[0062] Next, the actual work animation determining unit 114 determines whether or not the action ID identified in step S24 is included in the shooting action list 122 (step S25). Specifically, the actual work animation determining unit 114 determines whether or not the action ID identified in step S24 is included in the action ID 122e of a record that has not yet been shot, among the records in the shooting action list 122.

[0063] If the actual work motion image determining unit 114 determines that the action ID identified in step S24 is not included in the photographed action list 122 ("NO" in step S25), the processing unit 110 shifts the process to step S23.

[0064] When the actual work animation determining unit 114 determines that the action ID identified in step S24 is included in the photographing action list 122 (in the case of "YES" in step S25), the actual work animation determining unit 114 associates information indicating that photographing has been performed with the record having the earliest start timing among the records in the corresponding photographing action list 122 (step S26). Specifically, the actual work animation determining unit 114 associates information indicating that photographing has been performed with the record that has not yet been photographed and has the earliest start timing 122b among the records in the photographing action list 122 that include the action ID identified in step S24.

[0065] Next, the actual work video determining unit 114 determines whether or not there is an unfilmed action between the previously filmed filmed action to be filmed (step S27). Specifically, the actual work video determining unit 114 determines whether or not there is an unfilmed action to be filmed between the record in the filming action list 122 associated with information indicating that filming was performed in step S26 and the record in the filming action list 122 associated with information indicating that filming was performed before this step.

[0066] When the actual work video determination unit 114 determines that there is no unfilmed action between the previously filmed action to be filmed ("NO" in step S27), the actual work video determination unit 114 determines whether the actual work video has ended (step S28). Specifically, the actual work video determination unit 114 determines that the actual work video has ended when filming of the actual work video acquired in step S21 has ended and extraction of secondary action data from primary action data included in the actual work video has been completed.

[0067] If the actual work video determination unit 114 determines that the actual work video has ended (YES in step S28), the processing unit 110 ends the processing of this flowchart. If the actual work video determination unit 114 determines that the actual work video has not ended (NO in step S28), the processing unit 110 shifts the processing of this flowchart to step S23.

[0068] When the actual work video determining unit 114 determines that there is an unfilmed action between the previously filmed action to be filmed (YES in step S27), the actual work video determining unit 114 identifies the action name of the unfilmed action (step S29). Specifically, the actual work video determining unit 114 refers to the filmed action list 122 and identifies the filmed action name 122c of one or more filmed action to be filmed that have been determined not to be filmed in step S27.

[0069] Next, the actual work animation determining unit 114 outputs an alert indicating the action name (step S30). Specifically, the actual work animation determining unit 114 displays a character string indicating the shooting action name 122c identified in step S29 on the shooting action list display screen 160 that displays the shooting action list 122.

[0070] 7 is a diagram showing an example of a photographing action list display screen 160. The photographing action list display screen includes an actual work image display area 161, a photographing action list display area 162, and an alert display area 163.

[0071] The actual work image display area 161 is an area in which actual work images are displayed in a reproducible manner. In this embodiment, the actual work image display area 161 displays the actions during work that are being continuously photographed by the camera 400. The photographing action list display area 162 is an area in which the photographing action list 122 is displayed. The photographing action list 122 included in the photographing action list display area 162 shown in this figure includes, in addition to the columns of the photographing action list 122 shown in FIG. 3, a photographing presence / absence column 1621 for displaying information indicating whether photographing has been performed.

[0072] The alert display area 163 is an area for displaying names of one or more unfilmed target actions included in the shooting action list 122 when the action is moved to the next action without being filmed. That is, in step S30 of FIG. 6 , the actual work video determination unit 114 controls to display names of the unfilmed target actions in the alert display area 163.

[0073] As described above, in this embodiment, the actual work video determination unit 114 extracts secondary action data contained in the actual work video while the video acquisition unit 113 acquires the actual work video. Therefore, on the shooting action list display screen 160 in this embodiment, the actual work video being shot is displayed in the actual work image display area 161, while information indicating actions that were not captured is displayed in the alert display area 163. In other words, since the user can be made aware of actions that were not captured while shooting the actual work video, the user can decide, midway through shooting the actual work video, whether to stop shooting and shoot again, or to continue shooting. Therefore, an actual work video that captures the necessary work can be shot more efficiently.

[0074] Furthermore, by displaying images of the actions to be filmed in the filming action list display area 162, it is possible to easily grasp the content of the actions to be filmed that were missing when filming the actual work video. Furthermore, since an image of each action to be filmed is displayed in the list of actions to be filmed included in the actual work video, it is possible to easily grasp what actions the actual work consists of.

[0075] In addition, in this embodiment, the action extraction unit 111 extracts the actions of the worker using data from the sensor 300 that acquires the action data of the worker, and generates the captured action list 122. Furthermore, the actual work video determination unit 114 analyzes the image of the worker included in the actual work video to determine whether or not the actions in the captured action list 122 are included in the actual work video. In this way, even if the algorithm used by the action extraction unit 111 to acquire the action data of the worker to generate the captured action list 122 is different from the algorithm used by the actual work video determination unit 114 to acquire the action data in the actual work video, the actual work video can be created efficiently.

[0076] <First Modification> 8 is a flowchart showing an example of an actual work video determination process in the first modified example. In the above-described embodiment, it is determined immediately whether or not the action to be captured has been captured for the actual work image being captured. In the actual work video determination process in the first modified example, after the capture of the actual work video has ended, it is determined whether or not the action in the captured action list 122 has been captured. Below, differences from the above-described embodiment will be described.

[0077] First, the video acquisition unit 113 acquires an actual work video (step S31). The video acquisition unit 113 acquires the actual work video whose filming has been completed.

[0078] Next, the actual work video determination unit 114 extracts primary action data indicating the actions of the worker (step S32). Specifically, the actual work video determination unit 114 extracts primary action data from the actual work video, similar to the process performed in step S22 of Fig. 6. Note that in this modified example, unlike step S22 of Fig. 6, extraction of primary action data is not performed in parallel with acquisition of the actual work video.

[0079] Next, the actual work video determining unit 114 determines whether or not the primary action data includes a candidate action (step S33). Specifically, the actual work video determining unit 114 refers to the primary action data extracted in step S32, and determines whether or not the primary action data includes any of the plurality of action data 121c included in the candidate action list 121. Note that if the primary action data includes at least one candidate action, the actual work video determining unit 114 proceeds to step S34.

[0080] If the actual work video determining unit 114 determines that the primary action data does not include the candidate action (NO in step S33), the actual work video determining unit 114 moves the process to step S36.

[0081] When the actual work video determination unit 114 determines that a candidate action is included in the primary action data ("YES" in step S33), the actual work video determination unit 114 extracts an action ID of the corresponding candidate action (step S34). Specifically, the actual work video determination unit 114 extracts a candidate action ID 121a of the candidate action included in the primary action data as the action ID of the corresponding secondary action data. Note that in this modification, the actual work video determination unit 114 extracts candidate action IDs 121a from one or more candidate action lists 121. Furthermore, when extracting multiple candidate action IDs 121a, the actual work video determination unit 114 associates information indicating the order of the actions with the candidate action IDs 121a so that the order of the actions of the extracted candidate action IDs 121a can be determined.

[0082] Next, the actual work video determining unit 114 refers to the shooting action list 122, and assigns the extracted shooting target actions to the shooting action list 122 so that the degree of agreement between the action order and the number of actions is highest (step S35). Specifically, the actual work video determining unit 114 assigns the shooting target actions to each record of the shooting action list 122 so that the combination of one or more candidate action IDs 121a and action order extracted in step S34 is highest in agreement with the combination of the action order and action ID 122e specified by the start timing 122b of the shooting action list 122. Information indicating that shooting has ended is associated with each record of the shooting action list 122 to which the shooting target action is assigned.

[0083] In addition, if there is an action to be photographed extracted in step S34 that cannot be assigned to the photographing action list 122 in step S35, the action not in the photographing action list 122 is treated as having been photographed as an actual work image.

[0084] Next, the actual work video determination unit 114 determines whether there are any excesses or deficiencies in the actions in the shooting action list 122 (step S36). Specifically, the actual work video determination unit 114 refers to the shooting action list 122 and determines whether there are any actions in the shooting action list 122 that were not shot in the actual work video, and whether there are any actions that were not in the shooting action list 122 but were shot in the actual work video. If the actual work video determination unit 114 determines that there are no excesses or deficiencies in the actions in the shooting action list 122 ("NO" in step S36), the processing unit 110 ends the processing of this flowchart.

[0085] When the actual work video determination unit 114 determines that there is a surplus or deficiency in the actions in the shooting action list 122 ("YES" in step S36), the actual work video determination unit 114 identifies the action names of the missing actions and the action names of the unnecessary actions (step S37). Specifically, the actual work video determination unit 114 refers to the shooting action list 122, and identifies the shooting action name 122c of a record that is not associated with information indicating that shooting has ended as the action name of the missing action. In addition, the actual work video determination unit 114 identifies the name of the action that is the shooting target action extracted in step S34 and that is not assigned to the shooting target action list as the action name of the unnecessary action.

[0086] Next, the actual work video determination unit 114 outputs an alert indicating the action name (step S38). Specifically, the actual work video determination unit 114 displays the shooting action list display screen 160, and displays the action name of the action identified in step S37 in the alert display area 163. Thereafter, the processing unit 110 ends the processing of this flowchart.

[0087] As described above, according to this embodiment, it is possible to recognize missing actions and extra actions in an actual work video, and to create a more accurate actual work video.

[0088] <Second Modification> 9 is a diagram showing an example of functional blocks of the skill transfer support system 1 in the second modified example. In the above-described embodiment, the action extraction unit 111 acquires data for generating an action list using the sensor 300. In this modified example, the action extraction unit 111 acquires primary action data from an image captured using the camera 400. In other words, the skill transfer support system 1 in this modified example does not have the sensor 300 for acquiring data for generating an action list. Below, differences from the above-described embodiment and modified example will be described.

[0089] Similar to the processing by the actual work video determining unit 114 in the above-described embodiment, the action extraction unit 111 in this modified example extracts a skeletal model of a worker from a video captured by the camera 400 and acquires the movement of the skeletal model as primary action data. Note that similar to the processing by the actual work video determining unit 114, the action extraction unit 111 may also acquire, as primary action data, information on a marker that reflects infrared light captured by the camera 400.

[0090] <Third Modification> 10 is a diagram showing an example of functional blocks of the skill transfer support system 1 in the third modified example. In the above-described embodiment and modified examples, the skill transfer support device 100 generates the shooting action list 122 and executes the actual work video judgment process. In the skill transfer support system 1 in this modified example, the skill transfer support device 100 generates the shooting action list 122, while the actual work video judgment device 200 executes the actual work video judgment process. Below, differences from the above-described embodiment and modified examples will be described.

[0091] The processing unit 110 of the skill transfer support device 100 in this modification includes a motion extraction unit 111 and a motion list generation unit 112, but does not include a video acquisition unit 113 or an actual work video determination unit 114. Furthermore, the skill transfer support device 100 is not connected to the camera 400. In addition, the motion extraction unit 111 of the skill transfer support device 100 may acquire data for generating a motion list from a camera (not shown) that is connected instead of the sensor 300.

[0092] The actual work video judging device 200 includes a processing unit 210, a storage unit 220, an input unit 230, an output unit 240, and a communication unit 250. The processing unit 210 comprehensively controls the entire actual work video judging device 200. The storage unit 220 stores information necessary for processing by the processing unit 210. The input unit 230 accepts information input to the actual work video judging device 200 from an input device (not shown). The output unit 240 outputs information stored in the actual work video judging device 200 from an output device (not shown). The communication unit 250 controls transmission and reception of information to and from other information processing devices communicably connected via a network.

[0093] The processing unit 210 includes a video acquisition unit 211 and an actual work video determination unit 212. The video acquisition unit 211 is similar to the video acquisition unit 113 in the above-described embodiment, and the actual work video determination unit 212 is similar to the actual work video determination unit 114 in the above-described embodiment, so description thereof will be omitted. In this modified example, the actual work video determination process shown in Fig. 6 etc. is executed by the actual work video determination device 200.

[0094] In this modification, the video acquisition unit 211 acquires actual work videos captured using one camera 400. However, the video acquisition unit 211 may acquire actual work videos captured using a plurality of cameras 400.

[0095] The storage unit 220 stores a shooting action list 221. The shooting action list 221 is the same as the shooting action list 122 stored in the storage unit 120 of the skill transfer support device 100. The actual work video determination unit 212 acquires the shooting action list 221 from the skill transfer support device 100 prior to the actual work video determination process.

[0096] As described above, according to this modified example, the skill transfer support system 1 can be configured more flexibly, for example, by distinguishing between the business entity that operates the skill transfer support device 100 and the business entity that operates the actual work video assessment device 200.

[0097] <Second embodiment> 11 is a diagram showing an example of functional blocks of the skill transfer support system 1 in the second embodiment. The skill transfer support system 1 in this embodiment generates a list of subjects to be filmed in an actual work video, and determines whether or not any of the subjects in the list are filmed in the actual work video. Differences from the above-described embodiment will be described below.

[0098] The processing unit 110 of the skill transfer support device 100 includes an action extraction unit 111, an action list generation unit 112, a video acquisition unit 113, an actual work video judgment unit 114, as well as a character string extraction unit 115, a subject list generation unit 116, and a material image management unit 117.

[0099] The character string extraction unit 115 extracts a character string indicating the subject from the audio data acquired together with the image used to extract the action. The character string extraction unit 115 extracts the character string indicating the subject by performing voice recognition using a known method on the explanatory audio data included in the explanatory video filmed prior to filming the actual work video. In this embodiment, the character string extraction unit 115 extracts the character string by performing voice recognition on the explanatory audio using term information 124 (described below) including the name of the candidate subject.

[0100] The subject list generation unit 116 generates a subject list 123 (described later) that includes the character string extracted by the character string extraction unit 115. The subject list 123 is displayed on a shooting list display screen 170 (described later).

[0101] The material image management unit 117 acquires a material image corresponding to the character string of the subject and associates it with the character string. For example, the material image management unit 117 requests a material image using a notification screen (not shown) that displays the character string of the subject. The material image management unit 117 includes the material image captured in response to the request in the subject list 123.

[0102] In this embodiment, the actual work video determination unit 114 determines whether or not a subject is included in the actual work video by using the material images. The actual work video determination unit 114 determines whether or not a subject related to the material images is included in the actual work video by, for example, performing pattern matching between the actual work video and the material images. The actual work video determination unit 114 may determine whether or not a subject is included in the actual work video by a method other than the method described in this example.

[0103] The storage unit 120 stores a candidate action list 121, a shooting action list 122, a subject list 123, and term information 124. The subject list 123 is a list of subjects to be included in the actual work image, which is generated by a subject list generation process described below. The term information 124 is dictionary information used to identify the subject.

[0104] 12 is a diagram showing an example of the data structure of the subject list 123. The subject list 123 is generated in a subject list generation process described below and stored in the storage unit 120. The subject list 123 includes a target subject ID 123a, a start timing 123b, a description 123c, a subject name 123d, and a material image 123e.

[0105] The target subject ID 123a is identification information for identifying a subject to be captured in an actual work image. The start timing 123b is information indicating the timing at which the subject identified by the target subject 123a will be displayed in the actual work video. For example, the start timing 123b is information indicating the timing at which the subject will speak, starting from the start timing of capturing the explanatory video or audio.

[0106] Description 123c is information obtained by converting audio into text, including the part of the audio of the explanatory video where the character string of the subject identified by target subject ID 123a was detected, as well as the parts before and after that part. Subject name 123d is a character string indicating the subject to be filmed. In the example shown in FIG. 12, the first record from the top of subject list 123 has subject name 123d as "desktop cover," and the description of that subject 123c includes the character string "to remove this desktop cover...." This indicates that the subject "desktop cover" was detected from the audio part of "to remove this desktop cover..." in the subject list generation data. Material image 123e is an image of the subject to be filmed.

[0107] 13 is a flowchart showing an example of the subject list generation process. This process is started, for example, when the skill transfer support apparatus 100 receives an input operation for instructing to start the subject list generation process.

[0108] First, the subject list generation unit 116 acquires data for generating a subject list (step S41). Specifically, the subject list generation unit 116 acquires explanatory videos and audio data captured by the camera 400 as data for generating a subject list. Note that the subject list generation unit 116 may acquire audio data without acquiring explanatory videos.

[0109] Next, the character string extraction unit 115 extracts character strings indicating the subjects by speech recognition (step S42). Specifically, the character string extraction unit 115 performs speech recognition on the subject list generation data acquired in step S41 using the term information 124, and extracts one or more character strings indicating the subjects.

[0110] Next, the subject list generation unit 116 generates a subject list 123 including the extracted character string (step S43). Specifically, the subject list generation unit 116 generates a new record in the subject list 123 by setting the character string extracted in step S42 as subject name 123d, the character string extracted from a predetermined length of audio before and after the character string as description 123c, and the time from the start timing of the explanatory video or audio data to the utterance of the character string as start timing 123b, and assigning a new target subject ID 123a.

[0111] Next, the material image management unit 117 acquires the material image of the subject and associates it with the subject list 123 (step S44). Specifically, the material image management unit 117 displays a material image request screen (not shown) including the subject name 123d of the subject list 123 generated in step S43. The material image management unit 117 acquires the material image input in response to the request and includes it in the subject list 123 as material image 123e. Thereafter, the processing unit 110 ends the processing of this flowchart.

[0112] Fig. 14 is a flowchart showing an example of subject determination processing in the second embodiment. Similar to the actual work video determination processing shown in Fig. 6, the processing of this flowchart starts when the skill transfer support device 100 receives an instruction to start shooting an actual work video. Note that this processing is performed in parallel with the above-mentioned actual work video determination processing. The processing performed in step S51 is the same as the processing performed in step S21 in Fig. 5, and therefore description thereof will be omitted.

[0113] Next, the actual work video determination unit 114 determines whether or not a subject corresponding to the material image has been detected (step S52). Specifically, the actual work video determination unit 114 performs pattern matching between the actual work video acquired in step S51 and the material image 123e in the subject list 123, thereby determining whether or not a subject corresponding to the material image has been detected from the actual work video (step S52).

[0114] This process is executed by sequentially monitoring the actual work video being shot, similar to the process of step S24 in Fig. 6. Therefore, one subject is extracted in this step. If the actual work video determination unit 114 determines that a subject corresponding to the material image has not been detected ("NO" in step S52), the processing unit 110 repeats the process of step S52.

[0115] When the actual work animation determination unit 114 determines that a subject corresponding to the material image has been detected (in the case of "YES" in step S52), the actual work animation determination unit 114 associates information indicating that the image has been captured with the record having the earliest start time among the records in the corresponding subject list 123 (step S53). Specifically, the actual work animation determination unit 114 associates information indicating that the image has been captured with the record that has not yet been captured and has the earliest start time 123b among the records in the subject list 123 that include the subject detected in step S52.

[0116] Next, the actual work video determining unit 114 determines whether or not there is a subject that has not been photographed between the previously photographed subject (step S54). Specifically, the actual work video determining unit 114 determines whether or not there is a subject that has not been photographed between the record in the subject list 123 associated with information indicating that the subject was photographed in step S53 and the record in the subject list 123 associated with information indicating that the subject was photographed before this step.

[0117] When the actual work video determination unit 114 determines that there is no unfilmed subject between the current work video and the previously filmed subject ("NO" in step S54), the actual work video determination unit 114 determines whether the actual work video has ended (step S55). Specifically, the actual work video determination unit 114 determines that the actual work video has ended when filming of the actual work video acquired in step S51 has ended and extraction of the subjects included in the actual work video has been completed.

[0118] If the actual work video determination unit 114 determines that the actual work video has ended (YES in step S55), the processing unit 110 ends the processing of this flowchart. If the actual work video determination unit 114 determines that the actual work video has not ended (NO in step S55), the processing unit 110 shifts the processing of this flowchart to step S52.

[0119] When the actual work video determination unit 114 determines that there is an unphotographed subject between the previously photographed subject (YES in step S54), the actual work video determination unit 114 identifies the name of the unphotographed subject (step S56). Specifically, the actual work video determination unit 114 refers to the subject list 123 and identifies subject names 123d of one or more subjects determined in step S54 as not yet being photographed.

[0120] Next, the actual work animation determining unit 114 outputs an alert indicating the name of the subject (step S57). Specifically, the actual work animation determining unit 114 displays the name of the subject identified in step S56 on the shooting list display screen 170, and prompts the user to shoot a material image of the subject.

[0121] 15 is a diagram showing an example of an imaging list display screen 170. The imaging list display screen 170 includes an actual work image display area 171, a subject list display area 172, and an alert display area 173. The imaging list display screen 170 may have a display list selection tab 174.

[0122] The actual work image display area 171 is an area where actual work images are displayed in a reproducible manner. In this embodiment, the actual work image display area 171 displays the actions during work captured by the camera 400. The subject list display area 172 is an area where the subject list 123 is displayed. The subject list 123 included in the subject list display area 172 shown in this figure includes, in addition to the columns of the subject list 123 shown in FIG. 12, a photographing presence / absence column 1721 for displaying information indicating whether photographing has been performed.

[0123] The alert display area 173 is an area for displaying the names of one or more subjects that were not photographed when a next subject is detected among the subjects included in the subject list 123 without being photographed. That is, in step S57 of FIG. 14 , the actual work moving image determination unit 114 controls to display the names of the subjects that were not photographed in the alert display area 173.

[0124] The display list selection tab 174 accepts the selection of a list to be displayed on the shooting list display screen 170. The shooting list display screen 170 shown in Fig. 15 shows a state in which the subject list 123 has been selected as the list to be displayed. When the shooting action list has been selected as the list to be displayed in the display list selection tab 174, the subject list display area 172 becomes an area that displays the shooting action list 122, similar to the shooting action list display area 162 shown on the shooting action list display screen 160 in Fig. 7.

[0125] 7 accepts a selection by a display list selection tab 174 as to whether to display the shooting action list 122 or the subject list 123. However, the shooting list display screen 170 may also display the subject actions to be photographed and the subjects in the same list.

[0126] In this embodiment, the actual work video determination unit 114 extracts subjects included in the actual work video in parallel with the acquisition of the actual work video by the video acquisition unit 113. Therefore, in the subject list display area 172 of this embodiment, the actual work video being filmed is displayed in the actual work image display area 171, while subjects that were not captured can be displayed in the alert display area 173. In other words, since the user can be made aware of subjects that were not captured while filming the actual work video, the user can determine whether to stop filming and film again, or to continue filming, midway through filming the actual work video. In other words, an actual work video that captures the necessary work can be filmed more efficiently.

[0127] Furthermore, by displaying material images in the subject list display area 172, it is possible to easily grasp the materials that are lacking in the actual work video.

[0128] In addition, the skill transfer support system 1 in this embodiment may determine whether or not a subject included in the subject list 123 is captured in the actual work video after filming of the actual work video is completed, similar to the first variant example in the first embodiment.

[0129] <Modification of the second embodiment> 16 is a flowchart showing an example of a subject list generation process in a modified example of the second embodiment. In this modified example, the skill transfer support system 1 determines whether or not predetermined subject designation information was detected when shooting the explanatory video in order to extract material images to be associated with the subject list 123. The following describes the differences from the above-described embodiment.

[0130] The processing performed in steps S61 to S63 is the same as the processing performed in steps S41 to S43 in FIG. 13, and therefore a description thereof will be omitted.

[0131] Next, the subject list generation unit 116 determines whether or not subject designation information has been detected (step S64). Specifically, the subject list generation unit 116 refers to the explanatory moving image acquired in step S61 and determines whether or not predetermined subject designation information has been detected in the pixel information constituting the explanatory moving image.

[0132] For example, the subject list generation unit 116 detects, as subject designation information, color information of a designated location that is a location pointed to by a laser pointer operated by the worker or another person. Alternatively, the subject list generation unit 116 detects, as subject designation information, color information of a designated location that corresponds to the tip of a pointer wand operated by the worker or another person. The subject list generation unit 116 stores the detection timing of the subject designation information.

[0133] Alternatively, the action extraction unit 111 extracts a skeletal model of the worker from the explanatory video and acquires the movement of the skeletal model. The action extraction unit 111 determines whether the movement of the skeletal model includes a pointing movement. If the pointing movement is included, the subject list generation unit 116 stores the timing when the pointing movement was performed, assuming that subject designation information has been detected.

[0134] Note that instead of acquiring the movement of the skeletal model from the explanatory video, the movement extraction unit 111 may determine whether or not a pointing movement has been performed from primary movement data acquired using the sensor 300, or may determine whether or not a pointing movement has been performed using information obtained by capturing an image of a marker that reflects infrared light using the camera 400. Furthermore, the movement extracted by the movement extraction unit 111 is not limited to a pointing movement, but may be any movement that can estimate to some extent the direction in which the subject is present and that is preset so that the movement can be extracted.

[0135] If the subject list generating unit 116 determines that subject designation information has not been detected ("NO" in step S64), the processing unit 110 moves the processing of this flowchart to step S62.

[0136] When the subject list generation unit 116 determines that subject designation information has been detected ("YES" in step S64), the material image management unit 117 acquires a portion corresponding to the subject designation information as a material image (step S65). Specifically, the material image management unit 117 acquires a still image at the timing when the subject designation information is detected from the explanatory video. The material image management unit 117 identifies an image of a portion corresponding to the subject designation information from the acquired still image and stores it as a material image.

[0137] The image of the portion corresponding to the subject designation information is, for example, an image having a radius of a predetermined number of pixels centered on the pointing location of a laser pointer or a wand. Alternatively, the image of the portion corresponding to the subject designation information is an image having a radius of a predetermined number of pixels centered on a point that is a predetermined number of pixels away from the fingertip in the pointing motion. In this case, the central point is a portion that starts from the fingertip and is spaced in the same direction as the direction from the base of the finger toward the fingertip.

[0138] Next, the material image management unit 117 associates the material image with a record in the subject list 123 that was generated at a timing close to the detection timing of the subject designation information (step S66). Specifically, the material image management unit 117 refers to the start timing 123b of the subject list 123 generated in step S63, and identifies a record that was generated at a timing close to the detection timing of the subject designation information stored in step S64. The material image management unit 117 associates the material image acquired in step S65 with the material image 123e of the identified record. Thereafter, the processing unit 110 ends the processing of this flowchart.

[0139] After the processing of step S66 is completed, the material image management unit 117 may refer to the subject list 123, and if there is a record that does not have a material image 123e, it may display a material request screen including the subject name 123d of the record, as in step S44 of Figure 13, and obtain the material image.

[0140] Furthermore, in this embodiment, the timing of detection of the subject designation information is used to associate the material image with the subject list 123. However, the material image management unit 117 may use subject dictionary data (not shown) including image information of candidate subjects to identify the name of the material image corresponding to the specified subject, and associate the material image with a record in which the name corresponds to the subject name 123d.

[0141] As described above, according to this embodiment, the actions and subjects to be filmed in the actual work video are listed, and it is determined whether or not they are filmed in the actual work video, so that the information necessary for skill transfer is included in the actual work video, thereby making it possible to more reliably transfer skills. Furthermore, there is no need to visually determine whether or not the necessary actions and subjects are filmed, and actual work videos for skill transfer can be more reliably filmed.

[0142] Although the above describes each embodiment of the present invention, the present invention is not limited to the above-described exemplary embodiment and includes various modifications. For example, the above-described exemplary embodiment has been described in detail to facilitate understanding of the present invention, and the present invention is not limited to an embodiment including all of the components described herein. Furthermore, part of the components of one exemplary embodiment can be replaced with the components of another exemplary embodiment. Furthermore, the components of another exemplary embodiment can be added to the components of one exemplary embodiment. Furthermore, part of the components of each exemplary embodiment can be added, deleted, or replaced with other components. Furthermore, some or all of the above-described components, functions, processing units, processing means, etc. may be implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the control lines and information lines in the figures are only those considered necessary for explanation, and not necessarily all of them are shown. It can be assumed that almost all components are interconnected.

[0143] Furthermore, the functional configurations of the skill transfer support device 100 and the actual work video judgment device 200 are classified according to the main processing content for ease of understanding. The classification method and names of the components do not limit the present invention. As described above, the configurations of the skill transfer support device 100 and the actual work video judgment device 200 can be classified into more components according to the processing content. Furthermore, classification can also be performed so that one component performs more processes. [Explanation of symbols]

[0144] 1: Skill transfer support system, 11: arithmetic unit, 12: memory, 13: external storage device, 14: input IF, 15: output IF, 16: communication IF, 100: skill transfer support device, 110-210: processing unit, 111: action extraction unit, 112: action list generation unit, 113-211: video acquisition unit, 114-212: actual work video judgment unit, 115: character string extraction unit, 116: subject list generation unit, 117: material image management unit, 120-220: storage unit, 121: candidate action list, 122-221: shooting action list , 123: subject list, 124: terminology information, 130-230: input section, 140-240: output section, 150-250: communication section, 160: shooting action list display screen, 161-171: actual work image display area, 162: shooting action list display area, 163-173: alert display area, 170: shooting list display screen, 172: subject list display area, 174: display list selection tab, 200: actual work video judgment device, 300: sensor, 400: camera, 1621-1721: shooting presence / absence column

Claims

1. an action extraction unit that extracts one or more actions of a worker; an action list generation unit that generates a shooting action list including the actions extracted by the action extraction unit; an actual work video acquisition unit that acquires an actual work video; an actual work video determination unit that determines whether or not an action included in the captured action list is captured in the actual work video.

2. The skill transfer support system according to claim 1, the motion extraction unit extracts the motion using a sensor that acquires motion data of the worker; The skill transfer support system is characterized in that the actual work video judgment unit analyzes an image of the worker included in the actual work video to detect the actions included in the captured action list.

3. The skill transfer support system according to claim 1, a storage unit that stores a candidate action list including candidate actions of the worker and names of the candidate actions; the action extraction unit extracts the action when the action of the worker is included in the candidate action list; The skill transfer support system is characterized in that the action list generation unit includes in the shooting action list the action extracted by the action extraction unit and the name of the candidate action related in the candidate action list.

4. The skill transfer support system according to claim 1, The skill transfer support system is characterized in that, when the movement extraction unit detects a predetermined preparatory movement, it extracts the movement that follows the preparatory movement.

5. The skill transfer support system according to claim 1, a character string extraction unit that extracts a character string indicating a subject from audio data acquired together with the image used by the action extraction unit to extract the action; a subject list generation unit that generates a subject list including the character string extracted by the character string extraction unit; a material image management unit that acquires a material image corresponding to the character string of the subject and associates the material image with the character string included in the subject list, The skill transfer support system is characterized in that the actual work video determination unit determines whether the subject is captured in the actual work video using the material image.

6. The skill transfer support system according to claim 5, A skill transfer support system characterized in that, when the action extraction unit detects subject instruction information, the material image management unit acquires a portion of the image corresponding to the subject instruction information as the material image.

7. A skill transfer support method using a skill transfer system, a motion extraction step of extracting one or more motions of a worker; an action list generation step of generating an action list including the actions extracted in the action extraction step; A procedure for acquiring actual work videos; an actual work video determination step of determining whether or not the actions included in the action list are captured in the actual work video.

8. A program that causes a processing unit of a computer to execute a skill transfer support method, a motion extraction step of extracting one or more motions of a worker; an action list generation step of generating an action list including the actions extracted in the action extraction step; A procedure for acquiring actual work videos; an actual work video determination procedure for determining whether or not a movement included in the movement list has been captured in the actual work video.

Images