Work analysis device, work analysis method, and work analysis program

Abstract

To provide a work analysis device capable of performing segmentation processing at a proper position, when segmenting a work moving image for each work element.SOLUTION: A work analysis device 1 acquires a work moving image from an imaging device and performs work analysis on the basis of the work moving image. The work analysis device 1 includes a first analysis processing part for performing the segmentation processing, for each work element, to a first work moving image in prescribed work associated with the imaging device and recorded by the imaging device and a second analysis processing part for performing the segmentation processing, for each work element, to a second work moving image in the prescribed work associated with the imaging device and recorded by the imaging device. The work analysis device 1 includes a segmentation position correction part for performing correction of a segmentation position on the basis of the segmentation position of the segmented first work moving image and the segmentation position of the segmented second work moving image and an analysis result output part for outputting the result of analysis processing including the information of the corrected segmentation position after correction.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] The present invention relates to an activity analysis device, an activity analysis method, and an activity analysis program, and more particularly to an activity analysis device, an activity analysis method, and an activity analysis program that perform activity analysis based on an activity video obtained by capturing an activity. [Background technology]

[0002] Conventionally, there has been known a work analysis device that can display a video of a work being performed and analyze the movements of the work, with the aim of "visualizing" the movements and time of workers, machines, and objects, and thereby shortening work time, saving labor, and reducing costs at production sites (see, for example, Patent Document 1). Among the above-mentioned work analysis devices, a mobile work analysis device that can be carried around to capture images of work in each work process in line work consisting of a plurality of work processes is known (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-163293 A [Patent Document 2] JP 2022-184111 A Summary of the Invention [Problem to be solved by the invention]

[0004] The work analysis device in Patent Document 1 described above is disclosed to extract skeletal information of a worker from a work video, determine from the extraction result whether the worker's movement is a prescribed movement or not, and perform a segmentation process to segment the work video into work elements according to the extraction timing when it is determined that the movement is the prescribed movement. In a work analysis device capable of performing segmentation processing based on analysis elements related to work (e.g., the actions of a worker), there was a demand for technology that could perform segmentation processing at a more appropriate position and output the results of the segmentation processing.

[0005] An object of the present invention is to provide an activity analysis device, an activity analysis method, and an activity analysis program that are capable of performing segmentation processing at appropriate positions when segmenting an activity video into activity elements. [Means for solving the problem]

[0006] The above problem is solved by the work analysis device of the present invention, which acquires work videos from an imaging device that records the work videos and performs work analysis based on the work videos, and which comprises a first analysis processing unit that is associated with the imaging device and performs segmentation processing for each work element of a first work video of a specified work recorded by the imaging device, a second analysis processing unit that is associated with the imaging device and performs segmentation processing for each work element of a second work video of the specified work recorded by the imaging device, a segment position correction unit that corrects segment positions based on the segment positions of the first work video segmented by the first analysis processing unit and the segment positions of the second work video segmented by the second analysis processing unit, and an analysis result output unit that outputs results of analysis processing including information on the segment positions corrected by the segment position correction unit.

[0007] As described above, the work analysis device includes a segment position correction unit that corrects segment positions based on the segment positions of the first work video segmented by the first analysis processing unit and the segment positions of the second work video segmented by the second analysis processing unit, and an analysis result output unit that outputs the results of the analysis process including information on the segment positions corrected by the segment position correction unit. Therefore, when performing segmentation processing for each work element on the work video, it is possible to perform the segmentation processing at a more appropriate position.

[0008] Furthermore, the above problem can also be solved by the work analysis method of the present invention, which is executed by a computer that acquires work videos from an imaging device that records the work videos and performs work analysis based on the work videos, in which the computer performs a first analysis process for each work element on a first work video of a specified work that is associated with the imaging device and recorded by the imaging device, performs a second analysis process for each work element on a second work video of the specified work that is associated with the imaging device and recorded by the imaging device, corrects segmentation positions based on the segmentation positions of the first work video segmented by the first analysis process and the segmentation positions of the second work video segmented by the second analysis process, and outputs a result of the analysis process including information on the corrected segmentation positions.

[0009] Furthermore, according to the work analysis program of the present invention, the above problem can also be solved by having a computer as a work analysis device that acquires work videos from an imaging device that records the work videos and performs work analysis based on the work videos execute the following processes: a process that is associated with the imaging device and performs a first analysis process for each work element of a first work video of a specified work recorded by the imaging device; a process that is associated with the imaging device and performs a second analysis process for each work element of a second work video of the specified work recorded by the imaging device; a process that corrects segmentation positions based on the segmentation positions of the first work video segmented by the first analysis process and the segmentation positions of the second work video segmented by the second analysis process; and a process that outputs results of the analysis process including information on the corrected segmentation positions. Effect of the Invention

[0010] According to the task analysis device, task analysis method, and task analysis program of the present invention, when segmenting a task video for each task element, it is possible to perform the segmentation process at an appropriate position. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a configuration diagram of an entire work analysis system. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of the work analysis system. [Diagram 3] FIG. 2 is a diagram illustrating functions of the work analysis system. [Figure 4] FIG. 13 is a diagram showing a menu screen. [Diagram 5] FIG. 13 is a diagram showing a work analysis screen. [Figure 6] FIG. 13 is a diagram showing "work element list data." [Figure 7A] FIG. 13 is a diagram showing “analysis feature data.” [Figure 7B] FIG. 13 is a diagram showing “analysis feature data.” [Figure 8] FIG. 11 is a diagram showing “second analysis feature data.” [Figure 9] FIG. 13 is a diagram showing “imaging device type data.” [Figure 10] FIG. 13 is a diagram showing a work video (a part of the work video) recorded by an imaging device. [Figure 11] FIG. 13 is a diagram showing the analysis results by the "task segmentation process." [Figure 12] FIG. 13 is a diagram showing the analysis results by "ergonomics processing." [Figure 13] FIG. 13 is a diagram showing "ergonomics evaluation data." [Figure 14] FIG. 13 shows the analysis results of the "two-hand / one-hand analysis process." [Figure 15] 13A to 13C are diagrams illustrating the segment position correction process and the merged analysis results. [Figure 16] FIG. 1 is a process flow diagram showing a work analysis method (1). [Figure 17] FIG. 13 is a process flow diagram showing the work analysis method (2). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Overall configuration of the work analysis system> Hereinafter, an embodiment of the present invention will be described with reference to FIGS. As shown in FIG. 1, the work analysis system S includes a work analysis device 1 that performs work analysis based on a work video that captures the work, and imaging devices 100, 200, and 300 that are each communicatively connected to the work analysis device 1, capture images of the work, record the work video, and transmit the work video (work video data) to the work analysis device 1. In this embodiment, a "task" refers to a line job or cell job that is made up of a plurality of work steps and in which each work step is performed in a linked manner. In addition to work performed by workers, there is also work performed by machines such as robots. "Line work" refers to a collection of a series of work processes, and is work carried out in a system known as the line production system, in which the work processes are arranged in a specific line shape. "Cell work" refers to a collection of a series of work processes, and is work carried out in a system known as the cell production system, in which the work processes are arranged in a special shape, such as a U-shape or a D-shape, among the specific line shapes in line work. An example of a task is an assembly line where products or assemblies are assembled through each process of processing, assembly, joining, painting, inspection, picking, etc. An "operation process (tasks in a work process)" refers to a collection of a series of work elements, and examples of such processes include assembling an engine, attaching a bumper, and mounting tires. An "operation element" refers to a collection of a series of actions, such as preparing vehicle parts, checking the setting positions of the vehicle parts, setting the vehicle parts, assembling the vehicle parts, etc. An "action" refers to the smallest unit into which a task is broken down.

[0013] 1 and 2, the work analysis device 1 is a computer used by an analyst who performs work analysis, and communicates with imaging devices 100, 200, and 300, and stores work videos (work video data) acquired from these imaging devices. Then, in order to improve the work, the work analysis is performed based on the work videos, such as work analysis that mainly involves user operations (also referred to as manual work analysis) and work analysis that does not mainly involve user operations (also referred to as automatic work analysis or AI work analysis). The work analysis device 1 then creates work standard data based on the results of the analysis process (analysis results), and outputs and displays the work standard data on a display screen, and can also output and print the work standard data on paper media. The work analysis device 1 can perform various work analysis processes (also referred to as work segmentation processes) described below as the above-mentioned automatic work analysis (AI work analysis) through the menu screen shown in FIG. 4 and the work analysis screen shown in FIG.

[0014] 1 and 2, the imaging devices 100, 200, and 300 are different types of imaging devices (e.g., imaging cameras) that capture a series of tasks and record work videos. Then, they create work video data indicative of the work videos and transmit the work video data to the work analysis device 1. The imaging devices 100, 200, and 300 are installed at different positions (also referred to as installation locations or installation places) and image different subjects. Differences in "installation positions" include the ceiling position, wall position (side position), floor position, entrance, etc. in the work space (work room), as well as the position of the work analyst's hands and the body position of the worker (e.g. the worker's head). Differences in "image capture targets" include target objects such as workers, machines, parts (products), etc., as well as the work content of the target objects (movement, hand movements, arm movements, etc.). Differences in "image capture targets" may further include the image capture content of the entire work space, a part of the work space (specific area), the worker's hands, etc.

[0015] The first imaging device 100 is a handheld camera whose "installation position" is the position of the analyst's hands, and which "image capture targets" all target objects and all work contents. The "image capture targets" mainly include the movement of the worker and the movement of the worker's hands. It is also possible to use a part of the work space as the "image capture target." The second imaging device 200 is a wearable camera worn on the worker's head, with the worker's body position (head position) as its "installation position," and its "imaging subjects" are primarily the movements of the worker's hands and the movements of the machine's arm. The third imaging device 300 is a ceiling camera whose "installation position" is the ceiling position of the work space, and whose "imaging target" is mainly the movement of the worker. In addition, a part or the whole of the work space can also be the "imaging target". As a specific example, FIG. 10 shows a work video (a part of the work video) captured by each of the imaging devices 100, 200, and 300.

[0016] Types of imaging devices include handheld cameras, wearable cameras, ceiling cameras, as well as fixed cameras (wall-mounted cameras), tracking cameras, 360-degree cameras, etc. More specifically, the "imaging device type data" shown in FIG. 9 shows type information of the imaging device, specifically type information related to "type", "installation position" and "imaging subject" in association with each other. In this way, depending on the "type," "installation location," and "image capture target" of the imaging device that captures a specific task, the captured work content will be recorded differently even for the same task, and different work video data will be created.

[0017] The work analysis system S is not limited to a configuration including the work analysis device 1 and the imaging devices 100, 200, 300, and may further include imaging devices of different types or the same type. The work analysis system S may also be configured to include the work analysis device 1 and one imaging device. Alternatively, the work analysis device 1 may also include an imaging unit (imaging device). Specifically, the work analysis device 1 may be a mobile work analysis device used to analyze work in real time while imaging work and recording a work video. In that case, the work analysis system S may be configured to include only the work analysis device 1 (mobile work analysis device).

[0018] <<Manual work analysis>> The work analysis device 1 displays a work video on a display screen, accepts a user operation, performs processing to segment the work video into work elements, and outputs the results of the analysis processing (analysis result data). For example, it can perform processing to segment each work video that has been captured in multiple cycles of a specific work, and perform cycle analysis to check the variation in the work (work elements) of each cycle.

[0019] Specifically, the work analysis device 1 accepts user input, sets a "rating rate (Rat)" for each work element, and calculates a "work standard time (standard time)" as shown in Figures 5 and 6. In addition, it can set an "action type (type)" for each work element. Note that Fig. 5 shows a work analysis screen for "automatic work analysis" described below, but the work analysis screen for "manual work analysis" has a similar screen layout. The "work element list data" shown in Fig. 6 is data showing a list of detailed content for each work element, and the above-mentioned item content is input and updated by accepting input of a user operation.

[0020] The "rating rate" is a numerical value for classifying or quantifying an object based on a predetermined standard. For example, it is an evaluation value for evaluating whether a worker shown in a work video is mature or accustomed to the work. "Standard work time" is the time required for a mature worker who is suitable for a job to complete the job at a normal work pace with the necessary leeway under specified working conditions. Standard work time is generally calculated by multiplying the rating rate by the effective operation time. "Action type" is an item in which, for example, one of the categories of "valid action," "invalid action," and "wasteful action" is selected and input for each work element. "Valid action" indicates that a valid action has been performed in the work, while "invalid action" indicates that an invalid action has been performed in the work. "Wasteful action" is an action that is valid as a work, but has been determined to be "unnecessary" as a result of analysis (it is different from invalid actions). Additionally, although not shown, the work analysis device 1 can set a "category" for each work element. A "category" is an item that is input by selecting one of the categories "operating," "semi-operating," and "non-operating" for each work element. "Operating" indicates a state in which a worker is working, "semi-operating" indicates a state in which a machine is operating, and "non-operating" indicates a waiting state in which neither a worker nor a machine is operating.

[0021] <<Automatic Work Analysis>> The work analysis device 1 performs a plurality of analysis processes (also referred to as analysis AI) based on analysis elements related to work (for example, worker, work content, type of imaging device, etc.) for each work element. The work analysis device 1 accepts the selection of a predetermined analysis element by a user operation as a selection condition, and selects an appropriate analysis process from among multiple analysis processes based on the selection condition by the user. Then, from among the selected analysis processes, it executes (automatically executes) the analysis process determined by the user operation, and outputs the result of the analysis process (analysis result data). Specifically, the method is as follows (although it is not limited to the following method).

[0022] The work analysis device 1 first accepts (manually) a user operation, performs an analysis process (segmentation process) for each work element on one cycle of work video, and learns the analysis result data as "teaching data". Then, using a three-dimensional object recognition technology, it estimates the posture, posture, and form of the target object (worker, machine, etc.) in three-dimensional space from the "analysis result data" that becomes the "teaching data". For example, it acquires the change in posture of the worker's upper body as a feature amount, and groups similar feature amounts to classify them into multiple unit actions. When the features of the change in the worker's posture are similar, it is considered that the same action has been performed. In this way, it is possible to recognize the action of the target object in response to differences in the worker's work position, differences in the installation position of the imaging device, etc. The work analysis device 1 then uses the "analysis result data" in which the boundaries of work elements are set, which serves as "teaching data," to associate the work elements in a task with the sequence of unit actions into which the work elements are subdivided. The work analysis device 1 also estimates the change pattern of the combination of unit actions that make up the work elements, and generates an "AI learning model (machine learning model)." This makes it possible to recognize differences in actions that differ from worker to worker, and subtle differences in actions made by the same worker, even for the same task. Furthermore, because the "AI learning model" recognizes work elements based on the sequence of unit actions, the work analysis device 1 can detect work elements by taking into account the sequence of unit actions in a task, even if there are multiple identical work elements, such as "tightening a screw part," in a series of tasks. By using the above method, the work analysis device 1 can recognize (automatically recognize) the action of a target object (worker, machine, etc.) and perform various analyses of the work (work content) performed by the target object for each work element.

[0023] <<Analysis processing (analysis AI)>> The work analysis device 1 selects one or more appropriate analysis processes from among a plurality of analysis processes in response to a user's request (accepts the selection of an analysis element by a user operation as a selection condition), and presents the selected analysis processes on the display screen. The selected analysis process group (also called analysis AI group or work improvement AI group) is as follows. Note that the analysis process is not limited to the following, and it is also possible to update new analysis processes as appropriate.

[0024] The analysis processes selected and executed by the work analysis device 1 include work segmentation processing (work segmentation AI), hand analysis processing (hand analysis AI), component position tracking processing (component position tracking AI), machine movement tracking processing (machine movement tracking AI), ergonomics processing (ergonomics AI), two-hand / one-hand analysis processing (two-hand / one-hand analysis AI), image anomaly detection processing (image anomaly detection AI), servric analysis processing (servric analysis AI), motion economics principles processing (motion economics principles AI), etc.

[0025] As shown in FIG. 11, "task segmentation processing" is a process that automatically segments the work content of a worker into work elements for a work video recorded by an imaging device, with the worker as the analysis target (also called the target object). Specific work content includes upper body work, work involving walking, collaborative work with other workers, collaborative work with machines, etc. It is also possible to analyze manual work. The analysis target may be a machine, a part, or a product. The "hands analysis process" is a process that analyzes the worker in the work video and automatically segments the detailed work content of the worker's hands into work elements. Specific work content includes hand work, upper body work, etc. On the other hand, it can be said that this process is relatively unsuitable for work that involves walking, collaborative work, etc., which are performed in the "work segmentation process."

[0026] "Component position change tracking processing" is a process that automatically segments the position, orientation, and changes (state changes) of work objects, such as parts and products, for each work element in the work video. Specifically, it is an analysis process used when tracking the position, orientation, and changes of work objects in line work, machining work, etc. "Machine movement tracking processing" is a process that automatically segments the machine movements into work elements by analyzing the machine in the work video. Specifically, it is an analysis process used to track the position and orientation of the machine movements in line work, machining work, etc.

[0027] "Ergonomics processing" is a process in which a work video is analyzed to determine whether the worker's movements are based on "predetermined movement standards," and if it is determined that the worker's movements for a part of the work video are not based on the "predetermined movement standards," "recorded information related to the movement standards" is added to that part of the work video. Specifically, the ergonomics process judges whether the movements of the worker in the work video are "ergonomic movements." In other words, even if the worker is performing the work appropriately, if the worker is performing the work in an unnatural position or with a strain on the body, the ergonomics process detects the movements of the worker. Ergonomics processing can segment work videos into tasks performed by workers in awkward positions or under great strain on the body.

[0028] In the ergonomics process, when it is determined that the worker's movements are not based on the "predetermined movement standards," a "record mark related to the movement standards" is added to a part of the work video, as shown in Fig. 12. Note that this is not limited to a record mark, and may be record information such as a record memo. Specifically, the ergonomics processing refers to the "ergonomics evaluation data" shown in FIG. 13 and assigns a recording mark that is determined based on the "load due to the weight of the work object" and the "load due to the worker's posture." According to the "Ergonomics Evaluation Data," the "load caused by the weight of the work object" is indicated by the size of the recording mark (e.g., large mark, medium mark, small mark). Also, the "load caused by the worker's posture" is indicated by the different colors of the recording mark (e.g., black, gray, white). For example, in the video of the third task element from the left in Figure 12, the ergonomics processing determines that the worker's "action of picking up the part" is not an "ergonomic action," and places a "large black mark" on the worker's lower back and a "small white mark" on the worker's upper body (elbow).

[0029] The "two-handed / one-handed analysis process" is a process that automatically detects the type of hand of a worker (for example, one hand (right hand, left hand), both hands, none) for a video of the worker being analyzed, and segments the hand type into each task element. In more detail, the two-handed / one-handed analysis process automatically detects whether a worker is performing one-handed tasks, two-handed tasks, or no manual tasks. For example, according to the segmentation process of the work video shown in FIG. 14, it can be seen that the two-handed / one-handed analysis process detects one-handed work and two-handed work from the work video, and performs task segmentation. As shown in Fig. 14, videos showing a worker performing one-handed tasks and videos showing a worker performing two-handed tasks can be distinguished by highlighting them in different colors. Alternatively, the characters "one hand" may be added to videos showing one-handed tasks and the characters "both hands" may be added to videos showing two-handed tasks.

[0030] The "image anomaly detection process" is a process in which a work video is analyzed to automatically detect any unauthorized behavior by the worker, and if unauthorized behavior by a worker is detected in part of the work video, "recorded information indicating unauthorized behavior" is added to that part of the work video. In detail, ergonomics processing detects "prohibited actions, unnecessary actions (clearly unnecessary actions)," "actions using prohibited tools, materials, parts, and products, actions using unauthorized tools," etc. while a worker is performing work.

[0031] The "image anomaly detection process" detects "prohibited actions" and "actions using prohibited tools" (first fraud detection), and can also detect "clearly unnecessary actions" and "actions using illegal tools" (second fraud detection). The first fraud detection has a higher importance (importance of the fraud) than the second fraud detection. For example, the importance may be expressed in order of increasing importance of the fraud as "high," "medium," or "low." In this way, the image anomaly detection process can be divided into tasks while setting first fraud detection, second fraud detection, etc. according to the detection level of fraud detection. In the image anomaly detection process, recording information according to the detection level is added to a part of the work video. The recording information at this time may be a recording mark, a recording memo, a symbol, or the like that differs depending on the detection level. When the first fraud detection is detected, the image anomaly detection process may be configured to issue an emergency alert.

[0032] "Servric analysis processing" is also called micro-motion analysis processing, and is a process that segments the micro-motions of the worker's hands, mainly those of both hands, into work elements for a video of the worker. This process analyzes the sequence and method of the worker's hand movements in relation to 18 types of basic movements (also called Servric movements). For example, this process is suitable for analyzing the detailed hand work of a worker.

[0033] The "principle of motion economy processing" analyzes a worker in a work video and judges whether the worker's movements are based on the "principle of motion economy." If it is judged that the worker's movements for a part of the work video are not based on the "principle of motion economy," "record information related to the principle of motion economy" is added to the part of the work video. The principle of motion economy is a principle that seeks to realize the best work motions so that workers can achieve maximum results with minimum fatigue. Specifically, the motion economy process judges whether or not motions based on the four basic principles of (1) reducing the number of motions, (2) performing motions simultaneously, (3) shortening the distance of motions, and (4) making motions easier are realized. The principle of motion economy processing adds recording information to a part of a work video when it is determined that the worker's motion is not based on the "principle of motion economy." The recording information at this time may be a recording mark, a recording memo, a symbol, etc. that differs depending on the detection level of the motion economy.

[0034] <<Selection of analysis process>> The work analysis device 1 stores "analysis feature data" including a plurality of analysis processes related to the work, a plurality of analysis elements related to the work, and evaluation information (score) indicating the weight of the analysis process for each analysis element (see FIGS. 7A and 7B). When the work analysis device 1 accepts the selection of a specific analysis element by user operation as a selection condition, it refers to the "analysis feature data" and selects one or more analysis processes (analysis AI) from the above-mentioned multiple analysis processes according to the weight of the analysis process based on the selection condition by the user (first selection process). Alternatively, the work analysis device 1 may acquire "information on type of imaging device" used to image the work, and select one or more analysis processes (analysis AI) from the above-mentioned multiple analysis processes by referring to the "analysis feature data" shown in Figures 7A and B and the "second analysis feature data" shown in Figure 8 (second selection process).

[0035] Details will be described later, but in the former "first selection process", the work analysis device 1 selects an appropriate analysis process based on the "analysis feature data" in accordance with the user's requests, regardless of the content of the work video recorded by the imaging device. For example, when a user wishes to analyze the operation of each target object, an analysis process group that excels in analyzing target objects is selected. As the latter "second selection process", the work analysis device 1 obtains type information of the imaging device (e.g., type, installation position, installation angle, etc.) by taking an overview of the entire content of the work video, and then selects an appropriate analysis process based on the "analysis feature data" and the "second analysis feature data" in accordance with the user's request. For example, when analyzing the assembly work of a box, first, the entire content of the work video is overviewed (the image composition is understood based on the entire content) to understand that it was recorded by an imaging device installed on the ceiling, that it is a video of work performed by only workers, only machines, or both workers and machines, etc. If the user then wants to analyze the actions of each target object, an analysis process group that is good at analyzing target objects is selected while taking into consideration the type of imaging device and the type of target object.

[0036] In other words, in the "first selection process", the work analysis device 1 selects an appropriate analysis process from all analysis process groups (analysis AI groups). In the "second selection process", the work analysis device 1 excludes incompatible analysis processes from the selection candidates based on the "contents of the work video", and then selects an appropriate analysis process from the remaining analysis process groups. Although there are differences in the selection process between the "first selection process" and the "second selection process," the same analysis process (analysis AI) may ultimately be selected. On the other hand, when there are many analysis processes that are candidates for selection, a more accurate selection can be made by adopting the "second selection process." Specifically, it is possible to roughly select (preliminary selection) analysis processes that can be selection candidates from a large number of analysis processes, and then select (main selection) an analysis process that meets the user's needs from the selection candidates.

[0037] <<Association of analysis processing and imaging device>> When performing a specified analysis process on a specified work video, the work analysis device 1 matches the imaging device with an analysis process suitable for the imaging device based on the type (type, installation location, imaging target, etc.) of the imaging device that records the work video and the characteristics of the analysis process (analysis elements 1, 2, 3, etc.). More specifically, the work analysis device 1 associates an analysis process suitable for each imaging device based on the "analysis feature data" shown in FIGS. 7A and 7B and the "imaging device type data" shown in FIG. When the work analysis apparatus 1 performs one or more work analyses on work videos recorded by one imaging device, there is no need to reassociate the imaging device with the analysis process. Even when the work analysis apparatus 1 performs one work analysis on work videos recorded by multiple imaging devices, there is no need to reassociate the imaging device with the analysis process.

[0038] As an example, it is assumed that the task analysis device 1 performs "task segmentation processing," "ergonomics processing," and "two-handed / one-handed analysis processing" for the task shown in FIG. 1 after selecting the above-mentioned analysis processing and having the user decide on the analysis processing. In this case, based on the "analysis feature data" shown in Figures 7A and B and the "imaging device type data" shown in Figure 9, the work analysis device 1 associates "task segmentation processing," "ergonomics processing," and "two-hand / one-hand analysis processing" with the first imaging device 100 (handheld camera), associates "task segmentation processing" and "two-hand / one-hand analysis processing" with the second imaging device 200 (wearable camera), and associates "task segmentation processing" with the third imaging device 300 (ceiling camera).

[0039] The work analysis device 1 may previously specify a correspondence between an image capture device that records the work video and an analysis process suitable for the image capture device. In other words, an arbitrary analysis process may be set as a default for each image capture device. In that case, an analysis process suitable for each image capture device may be set based on the type of image capture device (type, installation location, image capture target) and the characteristics of the analysis process (analysis elements 1, 2, 3). If a correspondence has been established in advance, it is needless to say that the work analysis device 1 does not need to select an appropriate analysis process from among multiple analysis processes. The work analysis apparatus 1 may associate an analysis process with each imaging device based on an arbitrary decision made by a user operation. Even in this case, the work analysis apparatus 1 does not need to select an analysis process.

[0040] <<Executing analysis processing>> The work analysis device 1 performs "one or more analysis processes" determined by a user operation from among the selected analysis processes on a work video that records the work, and outputs the results of the analysis processes (analysis result data). More specifically, the work analysis device 1 performs "one or more analysis processes" associated with an image capture device on a work video recorded by the image capture device, and outputs the results of the analysis processes. When performing multiple analysis processes on a predetermined task (task video), the work analysis device 1 outputs the result of the analysis process (also called a merged analysis result) that combines (combines) the results of the multiple analysis processes. Specifically, it outputs a merged analysis result (merged analysis result data) that combines (merges) work videos for the predetermined task and combines the respective analysis results. The work analysis device 1 may output each individual analysis result, an analysis result in which only the work videos have been merged, or a result in which only the analysis results have been merged without merging the work videos.

[0041] As a specific example, as shown in Figures 1 and 10 to 14, it is assumed that a work analysis device 1 is communicatively connected to a first imaging device 100, a second imaging device 200, and a third imaging device 300, and performs "work segmentation processing," "ergonomics processing," and "two-handed / one-handed analysis processing" on a part assembly work (work video). The work analysis device 1 performs "work segmentation processing," "ergonomics processing," and "hands analysis processing" on work videos recorded by the first imaging device 100 (handheld camera). Then, it performs "work segmentation processing" and "hands analysis processing" on work videos recorded by the second imaging device 200 (wearable camera). Then, it performs "work segmentation processing" on work videos recorded by the third imaging device 300 (ceiling camera). Thereafter, the work analysis device 1 outputs a merged analysis result obtained by merging the work videos and the analysis process results, as shown in FIG. The combined analysis results in Figure 15 show that (1) segmentation processing is performed for each task element, (2) a record mark is added to tasks (task elements) in which the worker does not perform "ergonomic movements," and (3) the worker's one-handed tasks and two-handed tasks are highlighted in different colors for each task element.

[0042] The work analysis device 1 can output the merged analysis results, which are obtained by merging work videos and merging the respective analysis results for a given work, to a display screen (work analysis screen) for each cycle, as shown in Fig. 15. Specifically, the merged work videos can be played back through the work analysis screen shown in Fig. 5, and each analysis result can be displayed in a list for each work element.

[0043] According to the layout of the work analysis screen 40 shown in FIG. 5, a work video 41 (merged work video) is displayed in the display area in the upper left portion. Analysis results 42 (combined analysis results) are displayed segmented by work element in the display area at the bottom of the work analysis screen 40. The analysis results 42 display information on the segmentation position of the work video and recording information recorded by the analysis process, as well as a seek bar 42a indicating the playback time of the work video and the analysis process (name of the analysis process) 42b. A work element list 43 is displayed in the display area in the upper right part of the work analysis screen 40. The work element list 43 displays information such as the identification number (No.), work element, ergonomics evaluation (ergo evaluation), both-hand / one-hand evaluation (both-hand / one-hand), type, element time, effective time, ineffective time, rating (Rat), and standard time, as well as the total work time (total), cycle information, etc. Note that display marks indicating the work times of the best lap and worst lap for the entire cycle may also be displayed. The above "ergonomics evaluation" indicates the recorded information recorded for the work element as a result of ergonomics processing. "Two-handed / one-handed evaluation" indicates the one-handed work information (two-handed work information) recorded for the work element as a result of two-handed / one-handed analysis processing. The screen layout of the work analysis screen 40 can be appropriately changed by using the layout change button 44. Also, the cycle change button 45 can be used to change the cycle display.

[0044] <<Correction of segment position by analysis processing (segment processing)>> The work analysis device 1 performs multiple segmentation processes on each work video recorded by multiple imaging devices of a specific work, performs "segment position correction" based on each segmented segment position, and outputs the analysis process result including the corrected "information on the corrected segment position". In detail, the work analysis device 1 extracts a predetermined segmentation process from among multiple segmentation processes based on the "type of analysis subject and / or work content", and performs "segment position correction" based on the segmentation positions segmented by each of the extracted segmentation processes. More specifically, since each of the extracted segmentation processes has its own strengths and weaknesses depending on the type of analysis subject and work content (there is compatibility depending on the performance of the segmentation process), a "weighting" is given to each type of analysis subject and work content. Therefore, the work analysis device 1 extracts a predetermined segmentation process from among multiple segmentation processes based on the type of analysis target and / or work content, applies a ``predetermined weighting'' to each of the extracted segmentation processes, and performs ``segment position correction'' based on the ``segment position'' segmented by each extracted segmentation process and the ``weighting'' applied to each segmentation process. Information on the weight of the segmentation process for each analysis target (target object) and type of work content is included in the "analysis feature data" shown in FIGS. 7A and 7B.

[0045] As a specific example, as shown in Figures 1 and 10 to 14, it is assumed that a work analysis device 1 is communicatively connected to a first imaging device 100, a second imaging device 200 and a third imaging device 300, and performs "work segmentation processing," "ergonomics processing," and "hand analysis processing" on a part assembly work (work video). In this case, the work analysis device 1 corrects the segment position based on the segment positions segmented by the "work segmentation processing", the "ergonomics processing", and the "hand analysis processing", as shown in Figure 15, and outputs the results of the analysis processing including "information on the corrected segment position". More specifically, it is as follows.

[0046] As a first correction pattern, the work analysis device 1 performs correction based on a segmentation position A1 at which a "work segmentation process" is performed on a work video recorded by a first imaging device 100 (handheld camera) and a segmentation position A2 at which a "work segmentation process" is performed on a work video recorded by a third imaging device 300 (ceiling camera), and sets a corrected segmentation position A. In this way, when the same analysis process is performed on different work videos recorded by imaging devices 100, 300 that are installed in different positions (imaging different subjects), the segment positions can be corrected based on the respective segment positions.

[0047] As a second correction pattern, the work analysis device 1 performs correction based on a segment position B1 where a "work segmentation process" is performed on a work video recorded by a first imaging device 100 (handheld camera) and a segment position B2 where a "hand analysis process" is performed on a work video recorded by a second imaging device 200 (wearable camera), and sets a corrected segment position B. In this way, when different analysis processes are performed on different work videos recorded by imaging devices 100 and 200 that are installed in different positions (imaging different subjects), the segment positions can be corrected based on the respective segment positions.

[0048] In addition, although not shown, the work analysis device 1 may perform corrections on work videos recorded for each cycle by the same imaging device (e.g., a handheld camera) based on the segment positions where the ``work segmentation processing'' was performed in the first cycle and the segment positions where the ``work segmentation processing'' was performed in the second cycle, and set the corrected segment positions. Alternatively, the work analysis device 1 may perform correction on work videos recorded for each cycle by the same imaging device (e.g., a handheld camera) based on the segment positions where the ``work segmentation processing'' was performed in the first cycle and the segment positions where the ``two-hand / one-hand analysis processing'' was performed in the second cycle, and set the corrected segment positions.

[0049] As a result, the work analysis device 1 can segment the work video recorded by the imaging device for each work element by the analysis process associated with the imaging device, and can further correct the segmentation position where the segmentation was performed. Therefore, more appropriate segmentation processing can be performed on the work video. In particular, it is possible to take into account different segment positions in work videos recorded by imaging devices with different installation positions (different imaging targets) and correct the segment positions while taking into account the characteristics of the work video (work content).

[0050] <Hardware configuration of the work analysis system> As shown in FIG. 2, the work analysis device 1 is a computer equipped with a CPU 2 as a data calculation and control processing device, a storage device 3 having a ROM, RAM, and HDD (SSD), a communication interface 4 for sending and receiving information data via a network, a display unit 5 for displaying text or image information, an input unit 6 that is operated by the user when inputting specified commands to the CPU, and an output unit 7 for outputting the text or image information. The work analysis device 1 stores work videos (work video data) recorded by the imaging devices, separated by imaging device and separated by cycle. It can also store the data separated by work process. In addition to a main program that performs the functions required of a computer, a work analysis program is stored in the storage device 3 (memory) of the work analysis device 1, and the program is executed by a CPU (processor) to achieve the functions of the work analysis device 1. Alternatively, the program may be executed by a semiconductor integrated circuit or a field-programmable gate array (FPGA) that implements a CPU. The imaging devices 100, 200, and 300 are also computers having similar hardware configurations.

[0051] In addition, when the work analysis device 1 is a mobile work analysis device, it is a mobile computer equipped with a CPU 2, a storage device 3, a communication interface 4, a display unit 5, an input unit 6, an output unit 7, and an imaging unit (imaging device). In this case, the mobile work analysis device 1 communicates with an external analysis device (not shown) and transmits analysis data related to the work analysis. The external analysis device communicates with the mobile work analysis device 1, receives the analysis data, performs a detailed analysis based on the analysis data, and transmits the detailed analysis data.

[0052] <Functions of the work analysis system> As shown in Figure 3, from a functional perspective, the work analysis device 1 has as its main components a memory unit 10 that stores various programs and various data such as "work video data," "work element list data," "analysis feature data," "second analysis feature data," "imaging device type data," and "analysis data (combined analysis data)," a screen display unit 11, an analysis processing unit 12, an analysis processing selection unit 13, a type information acquisition unit 14, an analysis processing presentation unit 15, a matching unit 16, an analysis execution unit 17, an analysis result output unit 18, and a segment position correction unit 19. These are composed of a CPU (processor), ROM, RAM, HDD, communication interface, and various programs.

[0053] Describing the imaging devices 100, 200, 300 from a functional standpoint, their main components are memory units 110, 210, 310 that store various programs and various data, communication units 111, 211, 311 that transmit and receive various data between the work analysis device 1, and operation execution units 112, 212, 312 that accept user operation input and execute operation processing.

[0054] Below, a detailed description is given of the functions (particularly the automatic analysis process) of the work analysis device 1. Note that in this embodiment, the work analysis device 1 is configured to include a storage unit 10, but this is merely an example, and the storage unit 10 may also be realized in an external storage device provided outside the work analysis device 1. In this case, it is preferable that the work analysis device 1 and the external storage device are connected by a communication path.

[0055] <<1. Display of analysis screen>> The screen display unit 11, for example, accepts input of a user operation, executes software installed in the work analysis apparatus 1, and displays a "menu screen" which is an initial screen when the user logs in. This menu screen is a menu screen for performing automatic analysis processing. Specifically, as shown in FIG. 4, the screen display unit 11 displays a list screen as a menu screen 30 for allowing the user to select a desired selection item from a plurality of selection items. When the screen display unit 11 receives a user selection of the selection item 31 "Model registration" on the menu screen 30, it transitions to a video registration screen (not shown). On this screen, in order to perform automatic task analysis, analysis result data obtained by analyzing one cycle of task video as described above is registered as "model data (teaching data)". The analysis processing unit 12 learns the analysis result data as "teaching data".

[0056] When the screen display unit 11 receives a user selection of the selection item 32 “automatic analysis” on the menu screen 30, it transitions to a task analysis screen 40 shown in Fig. 5. The task analysis screen 40 is a display screen used to register a task video and perform a task analysis. On the work analysis screen 40, while displaying the work video after shooting, it is possible to segment the work into work elements for each cycle, calculate the work element time, set the type, etc. It is also possible to check the variation in the work (work elements) of each cycle, rearrange the work elements, and organize the work process (recombination, replacement, rearrangement).

[0057] The screen display unit 11 has, as specific functional units, a moving image display unit 11a, an analysis result display unit 11b, and an operation element list display unit 11c. The video display unit 11a displays a work video of a recorded work on a display screen, and specifically displays a work video 41 on a work analysis screen .

[0058] The analysis result display unit 11b displays the analysis result obtained by performing automatic analysis processing in response to a user operation input. Specifically, the analysis result 42 is displayed on the task analysis screen 40. A seek bar 42a and an analysis process (the name of the analysis process) 42b performed on the work video are displayed in the analysis result 42. Also displayed are information on the segmentation position performed by the analysis process and the added record information (marks and highlights). Looking at the analysis result 42 shown in Fig. 5, it can be seen that it is a combined analysis result in which the results of multiple analysis processes are combined (combining task videos and analysis results). Also, looking at the analysis process 42b, it can be seen that "task segmentation process", "ergonomics process", and "two-handed / one-handed analysis process" were performed.

[0059] The task element list display unit 11c displays a task element list 43 on a display screen, the task element list 43 including information on task time for each segmented task element and information on the total time obtained by tallying up the task times. Specifically, the work element list 43 includes an identification number (No.) for identifying a work element, a work element name, an ergonomics evaluation (ergo evaluation), a two-handed / one-handed evaluation (two-handed / one-handed), a type, information on the work element time, and information on the total time of all the work elements, and is stored in the memory unit 10. The work element list data also includes information on the average work time, maximum work time, minimum work time, and work time for each cycle in the entire cycle. In addition, along with updating the information of the work element list 43 via the work analysis screen 40, the information of the “work element list data” shown in FIG.

[0060] Looking at the task element list 43 shown in Figure 5, we can see that for the identification number "3" and task element name "Pick up parts", the ergonomic evaluation is "●○", both hands / one hand is "one hand", type is "effective", element time for cycle 1 is "5.00 (seconds)", rating (Rat) is "100 (%)", and standard time is "5.00 (seconds)". It can also be seen that the total standard time for the task elements is "45.00 (seconds)". It should be noted that cycle 1 refers to the first cycle.

[0061] <<2. Selection and presentation of analysis processing>> The analysis processing unit 12 performs analysis processing for each work element based on analysis elements related to the work, on a work video in which a series of work is captured by an imaging device. The analysis processing unit 12 is also called an analysis processing group, and includes a plurality of analysis processing units according to the analysis elements as described above. Specifically, there are a work segmentation processing unit, a hand analysis processing unit, a component position tracking processing unit, a machine operation tracking processing unit, an ergonomics processing unit, a two-handed / one-handed analysis processing unit, an image anomaly detection processing unit, a service analysis processing unit, a principle of motion economy processing unit, and the like. In the present embodiment, the work analysis device 1 is described below as performing analysis processing using a first analysis processing unit 12a that performs "task segmentation processing", a second analysis processing unit 12b that performs "ergonomics processing", and a third analysis processing unit 12c that performs "two-handed / one-handed analysis processing" (of course, the work analysis device 1 may also perform "image anomaly detection processing").

[0062] (First selection process) The analysis processing selection unit 13 selects a predetermined analysis processing (analysis processing unit) from among a plurality of analysis processing (analysis processing units) based on an analysis element related to the work. In detail, the analysis process selection unit 13 accepts the selection of a specific analysis element by user operation as a selection condition, and selects one or more analysis processes from among multiple analysis processes by referring to the "analysis feature data" shown in Figures 7A and B according to the weight of the analysis process based on the selection condition set by the user. The analysis process presentation unit 15 presents the analysis process selected by the analysis process selection unit 13 on a display screen. The first selection process will now be described in detail.

[0063] The memory unit 10 stores "analysis feature data" including a plurality of analysis processes related to the work, a plurality of analysis elements related to the work, and evaluation information indicating the weight of the analysis process for each analysis element, as shown in Figures 7A and 7B. The "analysis feature data" shown in Figures 7A and 7B is analysis process selection condition data that enables the selection of analysis processes (analysis AI) based on analysis elements related to the work. Specifically, "evaluation information (score)" indicating the weight of each analysis process for each "analysis element" is shown.

[0064] The "analysis elements" include analysis element 1 "target object (subject of analysis)", analysis element 2 "work content", analysis element 3 "imaging device", analysis element 4 "average processing speed", analysis element 5 "usage fee", and analysis element 6 "actual data". These analysis elements are classified into analysis elements 1, 2, and 3, which belong to the first group and are scored according to the weight of the analysis process as described above, and analysis elements 4, 5, and 6, which belong to the second group and are not scored according to the weight of the analysis process. For the analysis elements 1, 2, and 3 belonging to the first group, the suitability is judged based on the score and an analysis process is selected. The score is information indicating the degree to which each analysis process is good or bad at the analysis it performs (information indicating the degree of suitability). The analysis process selection unit 13 selects a suitable analysis process from among multiple analysis processes based on the score for each of the analysis elements 1, 2, and 3.

[0065] The analysis element 1 "target object" indicates a tangible object such as a worker (person), machine, part, or product as the analysis target. For example, if an analyst user wishes to perform analysis processing of a work video with "worker" as the analysis target, the user selects "worker" in analysis element 1. Based on the selection conditions, the analysis processing selection unit 13 selects as selection candidates (preferentially selects) analysis processing with a high evaluation (score) for analysis element 1 "target object: worker." In other words, the analysis processing selection unit 13 selects as selection candidates analysis processing that is good at identifying a worker as the analysis target for a work video and segmenting the work of the worker into work elements.

[0066] Analysis element 2 "Work content" indicates the analysis content, such as manual work, upper body work, work involving walking, collaborative work with other workers, and collaborative work with machines. For example, when a user wishes to analyze a work video of "worker's manual tasks," the user selects "manual tasks" in analysis element 2. Based on the selection conditions, the analysis process selection unit 13 selects as selection candidates those analysis processes with high evaluations (scores) for analysis element 2 "task content: manual tasks." In other words, the analysis process selection unit 13 identifies "worker (both hands, one hand)" as the analysis target for the work video, and selects as selection candidates those analysis processes that are good at segmenting manual tasks into task elements. More specifically, the analysis process that is a candidate for selection in analysis element 2 is a process that enables identifying (specifying) specific work (hand work, upper body work, etc.) and performing segmentation processing from a combination of target objects (workers, machines, tools, materials, parts, products, etc.) contained in the work video, the position of the worker, the layout and configuration of the machines, information on the work location, information on the work time, etc.

[0067] Analysis element 3 "imaging device" indicates the type of imaging device (type, installation location, etc.) such as ceiling camera, fixed camera, wearable camera, handheld camera, etc. For example, if a user wants to perform analysis processing on a work video captured using a "wearable camera," the user selects "wearable camera" in analysis element 3. Based on the selection conditions, analysis processing selection unit 13 selects analysis processing with a high evaluation (score) for analysis element 3 "imaging device: wearable camera" as a selection candidate. According to Figures 7A and B, it is expected that the analysis process selection unit 13 will prioritize selecting the work segmentation process (score: 7 points), the hand analysis process (score: 8 points), and the component position tracking process (score: 9 points) as analysis processes suitable for the "wearable camera."

[0068] Analysis element 4 "average processing speed" is expressed as a percentage of the playback time of the work video. For example, when the average processing speed is 125%, it takes about 1.25 hours to perform analysis processing for a work video with a playback time of 1 hour. Analysis element 5 "usage fee" indicates the usage fee for analysis processing according to the playback time of the work video. For example, if the usage fee is 1 dollar / hour (h), a fee of approximately 1 dollar will be charged when performing analysis processing on a work video with a playback time of 1 hour (1h). Analysis element 6 "performance data" indicates past usage performance. For example, if the performance time is 10 million / h, it indicates that the corresponding analysis process has been used for 10 million hours in the past.

[0069] In the above, the analysis process selection unit 13 selects one or more analysis processes according to the weight (score) of the analysis process based on the selection conditions by the user when it is based on the analysis elements 1, 2, and 3 belonging to the first group included in the "analysis feature data" shown in Figures 7A and B. Also, when it is based on the analysis elements belonging to the second group included in the "analysis feature data", it selects one or more analysis processes based on the selection conditions by the user, not on the weight (score) of the analysis process. Then, the analytical process presentation unit 15 presents on the display screen the analytical process selected based on the analytical elements belonging to the first group and / or the analytical elements belonging to the second group.

[0070] That is, when the analysis process selection unit 13 selects an analysis process using the analysis elements 1, 2, and 3 in the first group, it does so based on the score of each analysis element. On the other hand, when the analysis process selection unit 13 selects an analysis process using the analysis elements 4, 5, and 6 in the second group, it selects an analysis process that directly matches the user's selection conditions. In other words, in the latter case, the analysis process is selected according to the relative selection criteria set by the user. In addition, the analysis process selection unit 13 may select one or more analysis processes based on a single analysis element as a result of a selection made by a user operation, or may select one or more analysis processes based on multiple analysis elements.

[0071] More specifically, when the analysis process selection unit 13 selects an analysis process by combining multiple analysis elements (e.g., analysis element 1 x analysis element 2), it selects the analysis process based on a predetermined calculation result using each score. The predetermined calculation method includes, for example, a method of calculating an average value, a method of calculating an average value using values ​​weighted according to the priority of the analysis elements, and the like. 7A and 7B, the user specifies the selection conditions by user operation (there are a method of specifying the conditions one by one, a method of selecting a combination of pre-specified conditions, etc.). When the user specifies the selection conditions, for example, it is necessary to select at least one analysis element belonging to the first group, and the analysis element belonging to the second group can be arbitrary. The analysis process selection unit 13 multiplies the conditions selected by the user and selects the analysis process with the highest adaptability by priority. The analysis process presentation unit 15 then presents the analysis processes in descending order of adaptability. The user selects an analysis process from the presented analysis processes.

[0072] (Second selection process) The analysis process selection unit 13 may select a predetermined analysis process from among a plurality of analysis processes by a "second selection process" different from the "first selection process." Specifically, this is as follows. The memory unit 10 stores the "analysis feature data" shown in Figures 7A and B, and also stores "second analysis feature data" as shown in Figure 8, which associates the type of imaging device (work content), the analysis target, and the analysis processing (performance of the analysis processing). When analyzing an operation, the type information acquisition unit 14 acquires "type information of the imaging device" used to image the operation. The analysis process selection unit 13 obtains the "imaging device type information" and selects one or more analysis processes (analysis AI) from the above-mentioned multiple analysis processes by referring to the "analysis feature data" shown in Figures 7A and B and the "second analysis feature data" shown in Figure 8. More specifically, the analysis process selection unit 13 obtains "information on the type of imaging device (e.g., type, installation position, installation angle, etc.)" by overviewing the entire content of the work video, and preliminarily selects analysis processes that can be candidates for selection based on the "analysis feature data" and "second analysis feature data" (excluding analysis processes that are not candidates for selection). Then, upon receiving a request from the user, the analysis process selection unit 13 finally selects an appropriate analysis process based on the "analysis feature data" and "second analysis feature data".

[0073] In the "second analysis feature data" shown in Figure 8, the "type of imaging device" is set to handheld camera, fixed camera, wearable camera, hand camera, ceiling camera, sensor, etc., the "analysis target" is set to task, ergonomics, fingertip, entire object, etc., and the "analysis processing performance" is set to analysis processes 1-1, 1-2, 1-3, etc. In addition, the "second analysis characteristic data" indicates evaluation information (selection priority) of the "analysis processing performance" based on the "type of imaging device" and the "analysis subject" using the symbols "◎, ◯, △, ×". The performance of an analysis process indicates the difference in the analysis method of the analysis elements performed by the analysis process (also called feature information). For example, analysis processes 1-1, 1-2, and 1-3 perform task segmentation processing, and analysis process 1-1 performs task segmentation processing by "a method of analyzing the worker's movements using bone data." Analysis process 1-2 performs task segmentation processing by "a method of analyzing the worker's movements using human shape." For example, in the case of a "handheld camera" type of imaging device and an "operation" subject of analysis, analysis process 1-1 (rating: ◎), which has a "method of analyzing the worker's movements using bone data," has a higher rating (larger feature value) than analysis process 1-2 (rating: △), which has a "method of analyzing the worker's movements using human shape," indicating that analysis process 1-1 should be selected as a priority.

[0074] As a specific example, let us consider a case where "upper body work of a worker" is analyzed and processed using a work video captured by an imaging device "handheld camera." The type information acquisition unit 14 acquires "type information of the imaging device (that it is a handheld camera)" by taking an overview of the entire contents of the work video recorded by the imaging device. After obtaining the "image capture device type information", the analysis process selection unit 13 performs a preliminary selection based on the "second analysis feature data" shown in FIG. 8, and excludes analysis processes that are unsuitable for a "handheld camera". The analysis process selection unit 13 accepts the selection of analysis elements by a user operation as selection conditions. For example, analysis element 1 "target object: worker" and analysis element 2 "operation content: upper body operation" are accepted as selection conditions. The analysis process selection unit 13 makes a final selection based on the "analysis feature data" shown in Figures 7A and 7B and the "second analysis feature data" shown in Figure 8 from the user's selection conditions, and selects a "task segmentation process", in particular, a "task segmentation process having a method of analyzing the movements of a worker using bone data."

[0075] According to the above, it is possible to perform a more accurate selection than the first selection process. Specifically, it is possible to roughly select (preliminary selection) analysis processes that can be selection candidates from a large number of analysis process groups, and then select (main selection) an analysis process that meets the user's request from the selection candidates. In particular, it is possible to select an analysis process according to its performance.

[0076] <<3. Matching of analysis processing and imaging device>> When performing a specified analysis process on a specified work video, the matching unit 16 matches the imaging device with an analysis process suitable for the imaging device based on the type (type, installation location, imaging target) of the imaging device that records the work video and the characteristics of the analysis process (analysis elements 1, 2, 3). More specifically, the association unit 16 associates analysis processing suitable for each imaging device based on the "analysis feature data" shown in FIGS. 7A and 7B and the "imaging device type data" shown in FIG. When the analysis process selection unit 13 selects a predetermined analysis process from among a plurality of analysis processes for a predetermined task (task video), the analysis process may be associated with each task video (each imaging device). In this case, the association unit 16 does not need to newly associate the imaging device with the analysis process.

[0077] According to the "imaging device type data" shown in FIG. 9, it is shown that the "target object" and "work content" to be imaged vary depending on the "type" and "installation position" of the imaging device. For example, an imaging device of type "Fixed Camera 1" and installed at the "Entrance" can detect a "Worker" as an imaging target, but cannot detect a "Machine." In other words, in the case of "Fixed Camera 1," the work analysis device 1 can segment the work (movement, hand movements) performed by a "Worker" into work elements, but cannot segment the work performed by a "Machine." Also, in the case of the type "ceiling camera 1", it is shown that the work analysis device 1 can detect both the target objects "worker" and "machine" and segment the work.

[0078] As a specific example, assume that the work analysis device 1 performs "task segmentation processing," "ergonomics processing," and "two-handed / one-handed analysis processing" on a work video recorded by the imaging devices 100, 200, and 300 that captured the "parts assembly work" shown in Figure 1, after selecting the above-mentioned analysis processing and having the user decide which analysis processing to use. In this case, based on the "analysis feature data" shown in Figures 7A and B and the "imaging device type data" shown in Figure 9, the association unit 16 associates "task segmentation processing," "ergonomics processing," and "two-hand / one-hand analysis processing" with the first imaging device 100 (handheld camera), associates "task segmentation processing" and "two-hand / one-hand analysis processing" with the second imaging device 200 (wearable camera), and associates "task segmentation processing" with the third imaging device 300 (ceiling camera).

[0079] According to the above specific example, the "task segmentation process" is associated with the image capture devices 100, 200, and 300, and performs analysis process for each task element on each of the task videos captured by these image capture devices. The "ergonomics process" is associated only with the first imaging device 100, and performs an analysis process on the work video captured by the first imaging device 100 for each work element. The "task segmentation process" is associated with the first imaging device 100 and the second imaging device 200, and performs an analysis process for each task element on the task video captured by these imaging devices.

[0080] <<4. Executing analysis processing>> Analysis execution unit 17 performs an analysis process determined by a user operation from among the selected analysis processes on the work video, and analysis result output unit 18 outputs the results of the analysis process (analysis result data). In detail, the analysis execution unit 17 performs "one or more analysis processes" associated with each imaging device on the work video recorded by each imaging device, and the analysis result output unit 18 outputs the results of the analysis processes. 10 to 12 and 14, when multiple work videos are recorded for a given work and multiple analysis processes are executed, the analysis result output unit 18 outputs the result of the analysis process (merged analysis result) that combines (merges) the results of the multiple analysis processes. Specifically, the analysis result output unit 18 merges each work video for the given work and outputs the merged analysis result (merged analysis result data) that merges each analysis result.

[0081] As a specific example, in Fig. 11, the analysis execution unit 17 performs "task segmentation processing" on a work video. Here, the analysis execution unit 17 performs "task segmentation processing" on a work video obtained by merging multiple work videos (work videos captured by the imaging devices 100, 200, 300) shown in Fig. 10, but of course the "task segmentation processing" may be performed for each work video. Also, according to Fig. 12, the analysis execution unit 17 performs "ergonomics processing" on the work video. According to Fig. 14, the analysis execution unit 17 performs "two-handed / one-handed analysis processing" on the work video.

[0082] The analysis result output unit 18 can output the merged analysis results, which are obtained by merging the work videos for a given work and merging the respective analysis processing results, to a display screen (work analysis screen) for each cycle, as shown in Fig. 15. Specifically, the merged work videos can be played back through the work analysis screen shown in Fig. 5, and each analysis result can be displayed in a list for each work element.

[0083] <<5. Correction of segment position by analysis processing (segment processing)>> The segment position correction unit 19 performs multiple analysis processes on each work video recorded by multiple imaging devices, and performs "segment position correction" based on each segmented segment position. The analysis result output unit 18 then outputs the analysis process result including the corrected "segment position information after correction." In detail, the segment position correction unit 19 extracts a specific analysis process from the multiple analysis processes based on the "type of analysis subject and / or work content", performs "specified weighting" on each of the extracted analysis processes, and performs "segment position correction" based on the "segment positions" segmented by each extracted analysis process and the "weighting" assigned to each analysis process. The "score (weight)" included in the "analysis feature data" shown in Figures 7A and 7B is used as "segmentation process weight information" for each analysis target (worker, machine) and type of work content.

[0084] In addition to extracting the predetermined analysis process based on the "type of analysis target and / or work content", the segment position correction unit 19 may extract the predetermined analysis process based on the "type of imaging device, installation position, imaging target (see FIG. 9)". It may also extract the predetermined analysis process based on a combination of these. Alternatively, the segment position correction unit 19 may calculate an average value of the segmentation results based on the results of all the analysis processes without weighting the analysis processes, and set the segment position resulting from this average value as the corrected segment position.

[0085] As a specific example, assume that the segment position correction unit 19 corrects the segment position based on the segment positions segmented by the "task segmentation processing", the "ergonomics processing", and the "hand analysis processing", as shown in Figure 15.

[0086] As a "first correction pattern", the segment position correction unit 19 performs correction based on a segment position A1 obtained by performing "work segmentation processing" on a work video recorded by the first imaging device 100 (handheld camera) and a segment position A2 obtained by performing "work segmentation processing" on a work video recorded by the third imaging device 300 (ceiling camera), and sets a corrected segment position A. In this way, when the "same analysis process" is performed on different work videos recorded by imaging devices 100, 300 having different installation positions (different imaging targets), the segment positions can be corrected based on each segment position. In addition, the first imaging device 100 and the third imaging device 300 are set with a "score (weight)" included in the "analysis feature data" shown in Figures 7A and B, and the corrected segmentation position is set taking into account the "score (weight)."

[0087] As a "second correction pattern", the segment position correction unit 19 performs correction based on a segment position B1 obtained by performing a "work segmentation process" on a work video recorded by the first imaging device 100 (handheld camera) and a segment position B2 obtained by performing a "hands-side analysis process" on a work video recorded by the second imaging device 200 (wearable camera), and sets a corrected segment position B. In this way, when "different analysis processes" are performed on different work videos recorded by imaging devices 100 and 200 having different installation positions (different imaging targets), the segment positions can be corrected based on each segment position.

[0088] By using the above-described work analysis device 1, it is possible to select and propose an appropriate analysis process from among multiple analysis processes according to the needs of the user performing the work analysis, and to execute the analysis process determined by the user. Furthermore, by using the work analysis device 1, it becomes possible to perform segmentation processing at more appropriate positions when performing segmentation processing on a work video for each work element.

[0089] <Work analysis method> Next, an example of the processing of the task analysis program (task analysis method) executed by the task analysis system S will be described with reference to FIGS. The above-mentioned program in this embodiment is a program for realizing the above-mentioned screen display unit 11, analysis processing unit 12, analysis processing selection unit 13, type information acquisition unit 14, analysis processing presentation unit 15, matching unit 16, analysis execution unit 17, analysis result output unit 18 and segment position correction unit 19 as functional components of a work analysis device 1 equipped with a memory unit 10, and the CPU (processor) of the work analysis device 1 executes this driving control program. The above program is executed upon receiving an operation instruction from a user (specifically, a task analyst).

[0090] The flow shown in Figure 16 shows a part of the processing flow of the "work analysis method." Specifically, it shows the processing flow of a method of proposing an analysis process (analysis AI) based on analysis elements related to the work and performing the analysis process on a work video. The flow shown in FIG. 17 shows a processing flow of a method for executing a plurality of analysis processes on a work video and correcting the segmentation positions segmented by each analysis process.

[0091] The work analysis flow (1) shown in FIG. 16 begins with step (S1) in which the screen display unit 11 accepts a user selection of the option 32 “automatic analysis” on the menu screen 30 shown in FIG. 4 and displays the work analysis screen 40 (analysis screen) shown in FIG. 5. The screen display unit 11, for example, accepts input of a user operation, executes software installed in the work analysis apparatus 1, and displays a menu screen 30, which is an initial screen, when the user logs in.

[0092] In step 2, the work analysis device 1 (also referred to as a work video acquisition unit) acquires a work video captured by an imaging device. Specifically, the work analysis device 1 accepts a selection of a user operation and imports a work video to be subjected to work analysis. The work analysis device 1 may acquire a work video before displaying the work analysis screen 40.

[0093] In step 3, the analysis process selection unit 13 accepts the selection of a specific analysis element by user operation as a selection condition, and selects one or more analysis processes from among the multiple analysis processes based on the weight of the analysis process and the user's selection condition, by referring to the "analysis feature data" shown in Figures 7A and B (first selection process). As described above, the analysis process selection unit 13 may select a predetermined analysis process by a "second selection process" different from the "first selection process." In the case of the "second selection process," the analysis process selection unit 13 obtains "information on the type of imaging device (e.g., type, installation position, installation angle, etc.)" by overviewing the entire content of the work video, and excludes analysis processes that are not candidates for selection based on the "analysis feature data" shown in Figs. 7A and 7B and the "second analysis feature data" shown in Fig. 8. Then, the analysis process selection unit 13 accepts the selection conditions of the user and finally selects an appropriate analysis process based on the "analysis feature data" and the "second analysis feature data."

[0094] In step 4, the analysis process presentation unit 15 presents the analysis process selected by the analysis process selection unit 13 on the display screen. Specifically, the analytical process presentation unit 15 presents the selected "candidate analytical processes" and the "score of the analytical process (supplementary information for determining the analytical process)" weighted for each analytical element on the display screen. In other words, the analytical process presentation unit 15 presents the analytical processes in order of suitability.

[0095] In step 5, the work analysis device 1 (also referred to as an analysis process determination unit) accepts a selection made by a user operation and determines an analysis process from among the multiple analysis processes presented. Specifically, the user selects and determines an analysis process from among the analysis processes presented on the display screen while referring to the "candidate analysis processes" and the "score of the analysis process" presented on the display screen.

[0096] In step 6, the matching unit 16 matches the "imaging device" with an "analysis process" suitable for the imaging device based on the type (type, installation location, imaging target) of the imaging device that records the work video and the characteristics of the analysis process (analysis elements 1, 2, 3, etc.). Specifically, the association unit 16 associates an analysis process suitable for each imaging device based on the "imaging device type data" shown in FIG. 9 and the "analysis feature data" shown in FIGS. 7A and 7B. When the analysis process selection unit 13 selects a predetermined analysis process from among a plurality of analysis processes for a predetermined task (task video), the analysis process may be associated with each task video (each imaging device). In this case, the association unit 16 does not need to reassociate the imaging device with the analysis process in step 6.

[0097] In step 7, analysis execution unit 17 performs analysis processing determined by a user operation on the work video. More specifically, the analysis execution unit 17 performs "one or more analysis processes" associated with each imaging device on the work video recorded by the imaging device. Then, in step 8, the analysis result output unit 18 outputs the results of the analysis process (analysis result data). In detail, when multiple work videos are recorded and multiple analysis processes are executed, the analysis result output unit 18 outputs a "combined analysis result" that combines the results of the multiple analysis processes. Specifically, the analysis result output unit 18 combines each work video and outputs combined analysis result data that combines each analysis result. The process of FIG. 16 is completed through steps 1 to 8 above.

[0098] Next, as shown in FIG. 17, a process flow of a method for executing a plurality of analysis processes and correcting the segment positions segmented by each analysis process will be described. The work analysis flow (2) shown in FIG. 17 starts with a step (S101) in which the work analysis device 1 (work video acquisition unit) acquires work videos from a plurality of imaging devices.

[0099] In step 102, the matching unit 16 matches the "imaging device" with an "analysis process" suitable for the imaging device based on the type (type, installation location, imaging target) of the imaging device that records the work video and the characteristics (analysis elements) of the analysis process. Specifically, the association unit 16 associates an analysis process suitable for each imaging device based on the "imaging device type data" shown in FIG. 9 and the "analysis feature data" shown in FIGS. 7A and 7B. The analysis process selection unit 13 may select a predetermined analysis process from among a plurality of analysis processes for a predetermined task (task video), and the association unit 16 may associate the analysis process with each task video (each imaging device).

[0100] In step 103, the analysis execution unit 17 performs a plurality of analysis processes associated with each imaging device on the work video recorded by the imaging device.

[0101] In step 104, the segment position correction unit 19 recognizes the segment positions segmented by each analysis process. Specifically, the segmentation position correction unit 19 extracts a predetermined segmentation process from among multiple segmentation processes based on the "type of analysis subject and / or work content", and recognizes the segmentation position segmented by the extracted segmentation process. Then, in step 105, the segment position correction unit 19 performs "segment position correction" based on the recognized segment position. Specifically, the segment position correction unit 19 performs "predetermined weighting" for each of the extracted analysis processes, and performs "segment position correction" based on the recognized "segment position" and the "weight" assigned to each analysis process. The "score (weight)" included in the "analysis feature data" shown in Figures 7A and B is used as "segment process weight information."

[0102] In step 106, the analysis result output unit 18 outputs the results of the analysis process (analysis result data) including information on the segment positions corrected by the segment position correction unit 19. The process of FIG. 17 is completed through steps 101 to 106.

[0103] <Other embodiments> In the above embodiment, as shown in FIG. 5, the work analysis device 1 displays the work analysis screen 40 and executes the work analysis, but it may also display an analysis screen other than the work analysis screen 40 and execute various work analyses. For example, the work analysis device 1 may display a "cycle analysis screen" that displays a list of work elements and performs a detailed analysis for each cycle, or a "piling graph screen" that displays a pile graph to check the variation of work elements in each cycle and organize the work elements. Alternatively, it may display a "work comparison screen" that simultaneously displays multiple work videos and perform an analysis to compare work videos of different cycles or workers.

[0104] In the above embodiment, a work analysis program is stored in a recording medium readable by the work analysis device 1, and processing is performed by the work analysis device 1 reading and executing the program. Here, the recording medium readable by the work analysis device 1 refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Alternatively, a terminal (mobile terminal) serving as the work analysis device 1 may be used to start up dedicated software, and the work analysis program may be executed on a web browser.

[0105] In the above embodiment, the task analysis device, the task analysis method, and the task analysis control program according to the present invention have been mainly described. However, the above embodiment is merely an example for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be modified or improved without departing from the spirit of the present invention, and the present invention naturally includes equivalents thereof. [Explanation of symbols]

[0106] S Work Analysis System 1 Work analysis device, mobile work analysis device 2 CPU 3 Storage device 4 Communication Interface 5 Display section 6 Input section 7 Output section 10 Storage section 11 Screen display section 11a Video display section 11b Analysis result display section 11c Work element list display section 12 Analysis processing section 12a First analysis processing section (first segmentation processing section) 12b Second analysis processing section (second segmentation processing section) 12c Third analysis processing section (third segmentation processing section) 13 Analysis and Processing Selection Department 14 Type information acquisition unit 15 Analysis processing presentation section 16 Mapping section 17 Analysis Execution Department 18 Analysis result output section 19 Segment position correction section 30 Menu screen 31, 32 Selection items 40 Work analysis screen 41 Work video 42 Analysis results 42a Seek bar 42b Analytical processing 43 Work Element List 44 Layout switching button 45 Cycle Switch Button 100 First imaging device (handheld camera) 110 Storage section 111 Communications Department 112 Operation Execution Unit 200 Second imaging device (wearable camera) 210 Storage section 211 Communications Department 212 Operation Execution Unit 300 3rd imaging device (ceiling camera) 310 Storage section 311 Communications Department 312 Operation Execution Unit A, A1, A2 segment position B, B1, B2 segment position

Claims

A work analysis apparatus that performs work analysis based on a work video obtained by imaging work, an analysis selection process that selects one or more analysis processes according to the evaluation information, based on a selection of a predetermined analysis element by a user, from among a plurality of analysis processes, with reference to analysis feature data including evaluation information indicating the weight of the analysis process for each analysis element related to the work; an analysis execution process that executes the analysis process selected by the analysis selection process on the work video, and a work analysis apparatus that performs the analysis execution process. The work analysis apparatus according to claim 1, further comprising a type information acquisition process that acquires type information of an imaging apparatus that images work, wherein in the analysis selection process, with reference to the analysis feature data including first evaluation information indicating the weight of the analysis process for each analysis element related to the work and second evaluation information indicating the priority of the analysis process for each type of the imaging apparatus, select an analysis process according to the first evaluation information and the second evaluation information, based on the selection of the analysis element by the user and the type of the imaging apparatus obtained by the type information acquisition process. The work analysis apparatus according to claim 1, further comprising an analysis presentation process that presents the analysis process selected by the analysis selection process, wherein in the analysis presentation process, the analysis process selected by the analysis selection process and the evaluation information of the analysis process weighted for each analysis element are presented. The analysis element according to claim 1 includes an analysis target, work content, and type of imaging apparatus, wherein in the analysis selection process, with reference to the analysis feature data including the evaluation information of each of the analysis target, the work content, and the type of imaging apparatus, select one or more analysis processes according to the evaluation information from among a plurality of analysis processes including work segmentation processing, hand analysis processing, ergonomics processing, and both hands / single hand analysis processing. A work analysis method executed by a computer that performs work analysis based on a work video obtained by imaging work, wherein the computer selects one or more analysis processes according to the evaluation information, based on a selection of a predetermined analysis element by a user, from among a plurality of analysis processes, with reference to analysis feature data including evaluation information indicating the weight of the analysis process for each analysis element related to the work; and executes the selected analysis process on the work video. A computer as a work analysis device that performs work analysis based on a work video obtained by imaging work, referring to analysis feature data including evaluation information indicating the weights of analysis processes for each analysis element related to the work, selecting, from among a plurality of analysis processes, one or more analysis processes according to the evaluation information based on the selection of a predetermined analysis element by the user, and executing a process of executing the selected analysis process on the work video. A work analysis program.

Images