Work analysis device, work analysis method, and work analysis program

Abstract

To provide a work analysis device capable of performing segmentation processing at a proper position, when segmenting a work moving image for each work element.SOLUTION: A work analysis device 1 acquires a work moving image from an imaging device and performs work analysis on the basis of the work moving image. The work analysis device 1 includes a first analysis processing part for performing the segmentation processing, for each work element, to a first work moving image in prescribed work associated with the imaging device and recorded by the imaging device and a second analysis processing part for performing the segmentation processing, for each work element, to a second work moving image in the prescribed work associated with the imaging device and recorded by the imaging device. The work analysis device 1 includes a segmentation position correction part for performing correction of a segmentation position on the basis of the segmentation position of the segmented first work moving image and the segmentation position of the segmented second work moving image and an analysis result output part for outputting the result of analysis processing including the information of the corrected segmentation position after correction.SELECTED DRAWING: Figure 15

Description

TECHNICAL FIELD

[0001] The present invention relates to a work analysis device, a work analysis method, and a work analysis program, and more particularly to a work analysis device, a work analysis method, and a work analysis program that perform work analysis based on a work video obtained by imaging a work operation. BACKGROUND ART

[0002] Conventionally, for the purpose of visualizing the movements and time of workers, machines and objects to achieve reduction of work time, labor saving and cost reduction at production sites, work analysis devices capable of displaying a work video obtained by imaging a work and analyzing the motions of the work have been known (see, for example, Patent Document 1). Further, among the above work analysis devices, a portable mobile work analysis device that can be carried to image the work in each work process in line work consisting of a plurality of work processes is known (see, for example, Patent Document 2). PRIOR ART DOCUMENTS PATENT DOCUMENTS

[0003] PATENT DOCUMENT 1 Japanese Unexamined Patent Application Publication No. 2021-163293 PATENT DOCUMENT 2 Japanese Unexamined Patent Application Publication No. 2022-184111 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] In the above, it is disclosed that the work analysis device of Patent Document 1 extracts skeleton information of a worker from a work video, determines whether the motion of the worker is a prescribed motion based on the extraction result, and performs segmentation processing to segment the work video for each work element according to the extraction timing when the motion is determined to be the prescribed motion. In a work analysis device capable of performing segmentation processing based on analytical elements related to work (e.g., worker movements), there was a need for a technology that could perform segmentation processing at a more appropriate location and output the results of the segmentation processing.

[0005] The objective of this invention is to create a video of the work process. analysis In doing so, appropriately Analysis The objective is to provide a work analysis device, a work analysis method, and a work analysis program capable of performing processing. [Means for solving the problem]

[0006] The aforementioned problem can be solved by the work analysis device of the present invention. The work was filmed. A work analysis device that performs work analysis based on work videos, The following steps are performed: an analysis selection process that selects one or more analysis processes from among multiple analysis processes based on the evaluation information, using analysis feature data that includes evaluation information indicating the weight of each analysis process for each analysis element related to the aforementioned work, and based on the user's selection of predetermined analysis elements; and an analysis execution process that executes the analysis processes selected by the analysis selection process on the work video. This will resolve the issue.

[0008] Furthermore, according to the work analysis method of the present invention, the above problem can be solved. The work was filmed. A work analysis method performed by a computer that performs work analysis based on work videos, wherein the computer, The system refers to analysis feature data, which includes evaluation information indicating the weight of each analysis process for each analysis element related to the aforementioned work, and selects one or more analysis processes from among multiple analysis processes, based on the user's selection of predetermined analysis elements, according to the evaluation information, and then executes the selected analysis processes on the work video. It can also be solved by doing and executing.

[0009] Furthermore, according to the work analysis program of the present invention, the above problem can be solved. The work was filmed. A computer used as a work analysis device that performs work analysis based on work videos, The process involves referring to analysis feature data that includes evaluation information indicating the weight of analysis processing for each analysis element related to the aforementioned work, and selecting one or more analysis processing from among multiple analysis processing methods based on the user's selection of predetermined analysis elements, in accordance with the evaluation information; and executing the selected analysis processing on the work video. This can also be solved by having the execution of the command. [Effects of the Invention]

[0010] According to the work analysis apparatus, work analysis method, and work analysis program of the present invention, when segmenting a work video for each work element, it becomes possible to perform segmentation processing at appropriate positions. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram illustrating the overall structure of the work analysis system. [Figure 2]This is a diagram illustrating the hardware configuration of a work analysis system. [Figure 3] This is a diagram illustrating the functions of the work analysis system. [Figure 4] This is a diagram showing a menu screen. [Figure 5] This is a diagram showing a work analysis screen. [Figure 6] This is a diagram showing "work element list data". [Figure 7A] This is a diagram showing "analysis feature data". [Figure 7B] This is a diagram showing "analysis feature data". [Figure 8] This is a diagram showing "second analysis feature data". [Figure 9] This is a diagram showing "imaging device type data". [Figure 10] This is a diagram showing a work video (part of the work video) recorded by an imaging device. [Figure 11] This is a diagram showing the analysis result by "work segmentation processing". [Figure 12] This is a diagram showing the analysis result by "ergonomics processing". [Figure 13] This is a diagram showing "ergonomics evaluation data". [Figure 14] This is a diagram showing the analysis result by "both-hands / one-hand analysis processing". [Figure 15] This is a diagram illustrating segmentation position correction processing and merged analysis results. [Figure 16] This is a processing flow diagram showing work analysis method (1). [Figure 17] This is a processing flow diagram showing work analysis method (2). [Mode for Carrying Out the Invention]

[0012] <Overall Configuration of Work Analysis System> Hereinafter, embodiments of the present invention will be described with reference to Figure 1 to Figure 17. As shown in Figure 1, the work analysis system S comprises a work analysis device 1 that performs work analysis based on work videos captured during work, and imaging devices 100, 200, and 300 that are respectively connected to the work analysis device 1 in a communication manner, capture images of the work, record work videos, and transmit the work videos (work video data) to the work analysis device 1. In this embodiment, "work" refers to line work or cell work consisting of multiple work processes, where each work process is performed in a linked manner. This includes work performed by workers as well as work performed by machines such as robots. "Line work" refers to a collection of a series of work processes, and is performed in a method called line production, where the work processes are arranged in a predetermined line shape. "Cell work" also refers to a collection of a series of work processes, and is performed in a method called cell production, where the work processes are arranged in a special shape, such as a U-shape or a V-shape, within the predetermined line shape of line work. Examples of work include assembly lines where products or assembled items are put together through various work processes such as processing, assembly, joining, painting, inspection, and picking. A "work process (work within a work process)" refers to a collection of a series of work elements, such as engine assembly, bumper installation, and tire installation. A "work element" refers to a set of actions, such as preparing vehicle parts, confirming the setting position of vehicle parts, setting vehicle parts, and assembling vehicle parts. An "action" refers to the smallest unit when a task is subdivided.

[0013] As shown in Figures 1 and 2, the work analysis device 1 is a computer used by an analyst performing work analysis. It communicates with imaging devices 100, 200, and 300 and stores work videos (work video data) acquired from these imaging devices. Based on the work videos, it performs work analysis primarily involving user operation (also called manual work analysis), work analysis not primarily involving user operation (also called automated work analysis or AI work analysis), etc., in order to improve work processes. The work analysis device 1 then creates work standard data based on the results of the analysis process (analysis results), and outputs the work standard data to the display screen, as well as printing it on paper. The work analysis device 1 can perform various work analysis processes (also referred to as work segmentation processes) described later, as automated work analysis (AI work analysis), through the menu screen shown in Figure 4 and the work analysis screen shown in Figure 5.

[0014] As shown in Figures 1 and 2, the imaging devices 100, 200, and 300 are different types of imaging devices (e.g., imaging cameras) that capture a series of operations and record operation videos. They then create operation video data showing the operation videos and transmit the operation video data to the operation analysis device 1. The imaging devices 100, 200, and 300 each have different installation locations (also referred to as installation points or locations) and image different targets. Differences in "installation location" include the ceiling position, wall position (side position), floor position, and entrance position within the workspace (workroom), as well as the position of the work analyst's hands and the position of the worker's body (e.g., the worker's head). Differences in "imaging target" include the target object, such as the worker, machine, or part (product), as well as the work performed on the target object (movement, hand movements, arm movements, etc.). Furthermore, differences in "imaging target" may also include imaging content such as the entire workspace, a part of the workspace (specific area), or the worker's hands.

[0015] The first imaging device 100 is a handheld camera whose "installation position" is the analyst's hand position, and whose "imaging targets" are all target objects and all work activities. Primarily, it "imaging targets" are the worker's movement and the worker's hand movements. However, it is also possible to "imaging targets" a portion of the workspace. The second imaging device 200 is a wearable camera that is worn on the worker's head, with the worker's body position (head position) as its "installation position," and primarily captures the movements of the worker's hands and the movements of the machine's arm as its "imaging targets." The third imaging device 300 is a ceiling camera whose "installation position" is the ceiling of the workspace, and whose "imaging target" is primarily the movement of the worker. It can also be used to "imaging target" a part or the entire workspace. As a concrete example, Figure 10 shows work videos (parts of work videos) captured by each imaging device 100, 200, and 300.

[0016] Types of imaging devices include handheld cameras, wearable cameras, ceiling cameras, as well as fixed cameras (wall-mounted cameras), tracking cameras, and 360-degree cameras. For more details, Figure 9 shows the "Imaging Device Type Data," which provides type information for imaging devices, specifically related to "Type," "Installation Location," and "Imaging Target." Thus, depending on the "type," "installation location," and "target of imaging" of the imaging device used to image the prescribed work, the content of the work being imaged will differ even for the same work, resulting in the creation of different work video data.

[0017] Furthermore, the work analysis system S is not limited to a configuration comprising the work analysis device 1 and imaging devices 100, 200, and 300, but may also include imaging devices of different types or imaging devices of the same type. The work analysis system S may also be configured to include the work analysis device 1 and one imaging device. Alternatively, the work analysis device 1 may also incorporate an imaging unit (imaging device). Specifically, the work analysis device 1 may be a mobile work analysis device used to perform real-time work analysis while imaging the work and recording work video. In that case, the work analysis system S may consist only of the work analysis device 1 (mobile work analysis device).

[0018] <<Manual work analysis>> The work analysis device 1 displays work videos on a display screen, accepts user input, performs a process to segment the work video into work elements, and outputs the results of the analysis (analysis result data). For example, it can perform a cycle analysis to check the variation in work (work elements) in each cycle by performing a segmentation process on each work video captured multiple times for a predetermined work.

[0019] Specifically, as shown in Figures 5 and 6, the work analysis device 1 accepts user input, sets a "rating rate (Rat)" for each work element, and calculates a "standard work time (standard time)". In addition, it is possible to set the "operation type (type)" for each work element. Figure 5 shows the work analysis screen in "Automated Work Analysis," which will be described later, but the work analysis screen in "Manual Work Analysis" has a similar screen layout. The "Work Element List Data" shown in Figure 6 is data that shows a list of detailed contents for each work element, and the contents of the items described above are entered and updated in response to user input.

[0020] A "rating rate" is a numerical value used to classify or quantify an object based on predetermined criteria. For example, it is an evaluation value used to assess whether a worker shown in a work video is mature or accustomed to the task. "Standard work time" refers to the time required for a skilled and experienced worker to perform a task under specified working conditions, with sufficient buffer time and at a normal work pace. Standard work time is generally calculated by multiplying the effective work time by a rating rate. "Action Type" is an item where, for example, for each work element, one of the following classifications is selected and entered: "Effective Action," "Ineffective Action," and "Wasteful Action." "Effective Action" indicates that an effective action was performed in the work, while "Ineffective Action" indicates that an invalid action was performed in the work. "Wasteful Action" is an action that is effective in terms of the work, but has been determined to be "unnecessary" as a result of the analysis (this is different from an invalid action). In addition, although not shown in the diagram, the work analysis device 1 can set a "category" for each work element. A "category" is an item in which one of the following classifications is selected and entered for each work element: "operating," "partially operating," and "non-operating." "Operating" indicates that the worker is performing the work, "partially operating" indicates that the machine is running, and "non-operating" indicates that neither the worker nor the machine is operating; it is in a waiting state.

[0021] <<Automated Work Analysis>> The work analysis device 1 performs multiple analysis processes (also referred to as analysis AI) for each work element based on analysis elements related to the work (e.g., worker, work content, type of imaging device, etc.). The analysis device 1 accepts the selection of predetermined analysis elements by the user as selection criteria, and selects an appropriate analysis process from among multiple analysis processes based on the user's selection criteria. Then, it executes the analysis process determined by the user from among the selected analysis processes (automatically) and outputs the results of the analysis process (analysis result data). Specifically, it is as follows (of course, it is not limited to the following methods).

[0022] The work analysis device 1 first accepts user input (manually) and performs analysis processing (segmentation processing) on ​​each work element of a work video for one cycle, learning the analysis result data as "training data". Then, using 3D object recognition technology, it estimates the posture, position, and form of the target object (worker, machine, etc.) in 3D space from the "analysis result data" which serves as "training data". For example, it acquires changes in the posture of the worker's upper body as a feature, groups similar features, and classifies them into multiple unit movements. If the features of the worker's posture changes are similar, it is considered that the same movement was performed. In this way, it can recognize the movement of the target object in response to differences in the worker's work position, differences in the installation position of the imaging device, etc. The work analysis device 1 then uses "analysis result data," which serves as "training data" and has defined boundaries for work elements, to associate work elements with sequences of unit actions that subdivide the work elements. It also estimates the change patterns of the combinations of unit actions that constitute the work elements and generates an "AI learning model (machine learning model)." In this way, even for the same task, it can recognize differences in actions between workers and subtle differences in actions that occur even among the same worker. Furthermore, because the "AI learning model" recognizes work elements based on the sequence of unit actions, even if there are multiple identical work elements in a series of tasks, such as "tightening screw parts," the work analysis device 1 can detect the work elements by considering the sequence of unit actions in the task. Using the above method, the work analysis device 1 can recognize (automatically recognize) the movements of target objects (workers, machines, etc.) and analyze the work performed by the target objects (work content) in various ways for each work element.

[0023] <<Analysis Processing (Analysis AI)>> The work analysis device 1, in response to the user's request (accepting the selection of analysis elements by user operation as a selection condition), selects one or more appropriate analysis processes from among multiple analysis processes and displays the selected analysis processes on the display screen. The selected analytical processing group (also referred to as the analytical AI group or work improvement AI group) is as follows. Note that this is not limited to the analytical processing groups listed below, and it is possible to update with new analytical processing groups as needed.

[0024] Examples of analysis processes selected and executed by the work analysis device 1 include work segmentation processing (work segmentation AI), hand analysis processing (hand analysis AI), component position tracking processing (component position tracking AI), machine motion tracking processing (machine motion tracking AI), ergonomics processing (ergonomics AI), two-handed / one-handed analysis processing (two-handed / one-handed analysis AI), image anomaly detection processing (image anomaly detection AI), Therblic analysis processing (Therblic analysis AI), and motion economy principle processing (motion economy principle AI).

[0025] "Work segmentation processing," as shown in Figure 11, is a process that automatically segments the work content of a worker into work elements, treating the worker as the analysis target (also referred to as the target object) in a work video recorded by an imaging device. Specific work content includes upper body work, work involving walking, collaborative work with other workers, and collaborative work with machines. It is also possible to analyze work performed by the hands. The analysis target may also be a machine, part, or product. "Hand analysis processing" is a process that analyzes work videos, focusing on the worker and automatically segmenting the detailed work content of the worker's hands into individual work elements. Specific examples of work content include hand work and upper body work. On the other hand, it is relatively unsuitable for tasks involving walking or collaborative work, which are handled by "task segmentation processing."

[0026] "Component position change tracking processing" is a process that analyzes work videos, using parts, products, and other work objects as the target of analysis, and automatically segments the position, orientation, and changes (state changes) of the work objects for each work element. Specifically, it is an analysis process used when tracking the position, orientation, and changes of work objects in line work, machining work, etc. "Machine motion tracking processing" is a process that analyzes work videos, primarily by automatically segmenting the machine's movements into individual work elements. Specifically, it is an analysis process used to track the position and orientation of machinery during assembly line work, machining operations, and other similar processes.

[0027] "Ergonomic processing" is a process that analyzes work videos, determines whether the worker's movements conform to "predetermined movement standards," and, if it is determined that the worker's movements in a portion of the work video do not conform to the "predetermined movement standards," adds "record information related to movement standards" to that portion of the work video. Specifically, ergonomic processing determines whether the worker's movements in the work video are "ergonomically sound movements." In other words, even if the worker is performing the work correctly, if the worker is working in an awkward posture or in a position that puts strain on their body, the ergonomic processing will detect such movements. Ergonomic processing can segment work videos to show tasks performed by workers in awkward positions or under significant physical strain.

[0028] As shown in Figure 12, ergonomic processing assigns a "record mark related to the operation standard" to a portion of the work video if it determines that the worker's actions do not conform to the "predetermined operation standard." Note that this is not limited to a record mark; it may also include other recorded information such as record notes. Specifically, the ergonomics processing refers to the "ergonomics evaluation data" shown in Figure 13 and assigns record marks determined based on the "load due to the weight of the workpiece" and the "load due to the worker's posture." According to the "ergonomics evaluation data," the "load due to the weight of the workpiece" is indicated by the size of the recording marks (e.g., large, medium, small marks). In addition, the "load due to the worker's posture" is indicated by the difference in the color of the recording marks (e.g., black, gray, white). For example, as shown in the video of the third work element from the left in Figure 12, the ergonomics process determines that the worker's "action of picking up the part" is not an "ergonomically correct action," and places a "large black mark" on the worker's waist and a "small white mark" on the worker's upper body (elbow).

[0029] The "two-handed / one-handed analysis process" automatically detects the type of hand the worker is using (e.g., one hand (right hand, left hand), both hands, none) in a work video, and segments the hand type according to the work element. More specifically, the two-handed / one-handed analysis process automatically detects whether the worker is performing a one-handed task, a two-handed task, or not performing any manual tasks. For example, as shown in Figure 14, the segmentation process of the work video indicates that the two-handed / one-handed analysis process detects the worker's one-handed work and the worker's two-handed work in the work video and segments the work accordingly. As shown in Figure 14, videos showing a worker performing one-handed tasks and videos showing a worker performing two-handed tasks can be distinguished by highlighting them in different colors. Alternatively, the text "one-handed" can be added to videos showing one-handed tasks, and the text "two-handed" can be added to videos showing two-handed tasks for identification.

[0030] The "image anomaly detection process" automatically detects any improper actions by workers by analyzing work videos, and if improper actions are detected in a portion of the work video, it adds "record information indicating improper actions" to that portion of the work video. More specifically, ergonomic processing detects "prohibited actions, unnecessary actions (clearly unnecessary actions)," "actions using prohibited tools, materials, parts, or products, and actions using unauthorized tools" while a worker is performing a task.

[0031] The "image anomaly detection process" can detect "prohibited actions" and "actions using prohibited tools" (first-stage fraud detection), as well as "clearly unnecessary actions" and "actions using unauthorized tools" (second-stage fraud detection). First-stage fraud detection is of higher importance (severity of the fraud) than second-stage fraud detection. For example, importance can be expressed as "high," "medium," and "low" in order of increasing severity of the fraud. By doing so, the image anomaly detection process can perform work segmentation while setting the first fraud detection, second fraud detection, etc., according to the detection level of fraud detection. The image anomaly detection process adds recording information to a portion of the work video according to the detection level. This recording information should ideally consist of different recording marks, notes, symbols, etc., depending on the detection level. Furthermore, if the first instance of fraud is detected, the image anomaly detection process may also issue an emergency alert.

[0032] "Therblig analysis," also known as micro-motion analysis, is a process that analyzes work videos, focusing on the worker and segmenting the minute hand movements, primarily those of both hands, into individual work elements. This process analyzes the sequence and method of the worker's hand movements in relation to 18 basic movements (also called Therblig movements). For example, it is a process well-suited for analyzing the detailed hand movements of a worker.

[0033] The "Principle of Motion Economy Processing" analyzes work videos, focusing on the worker as the subject of analysis, to determine whether the worker's actions are based on the "Principle of Motion Economy." If it is determined that the worker's actions in a portion of the work video are not based on the "Principle of Motion Economy," then "Record Information Regarding the Principle of Motion Economy" is added to that portion of the work video. The "principles of motion economy" refer to the principles of achieving the best possible work movements so that workers can achieve maximum results with minimal fatigue. Specifically, processing based on the principles of motion economy determines whether movements are being performed based on four basic principles: (1) reducing the number of movements, (2) performing movements simultaneously, (3) shortening the distance of movements, and (4) making movements easier. The motion economy principle processing adds recording information to a portion of a work video when it is determined that the worker's actions do not conform to the "motion economy principle." This recording information may be different depending on the motion economy detection level, such as recording marks, notes, or symbols.

[0034] <<Selection of Analysis Process>> The work analysis device 1 stores "analysis feature data" which includes multiple analysis processes related to the work, multiple analysis elements related to the work, and evaluation information (scores) indicating the weight of the analysis process for each analysis element (see Figures 7A and 7B). When the work analysis device 1 receives a selection of predetermined analysis elements by user operation as a selection condition, it refers to the "analysis feature data" and selects one or more analysis processes (analysis AI) from the above-mentioned multiple analysis processes according to the weight of the analysis processes based on the user's selection condition (first selection process). Alternatively, the work analysis device 1 may acquire "type information of imaging device" used for imaging the work, and select one or more analysis processes (analysis AI) from the above multiple analysis processes by referring to the "analysis feature data" shown in Figures 7A and 7B and the "second analysis feature data" shown in Figure 8 (second selection process).

[0035] As will be explained in more detail later, in the former "first selection process," the work analysis device 1 selects an appropriate analysis process based on the "analysis feature data" according to the user's request, regardless of the content of the work video recorded by the imaging device. For example, if a user wants to analyze the behavior of each target object, they can select a group of analysis processes that are particularly skilled at analyzing the target object. In the latter "second selection process," the work analysis device 1 takes an overview of the entire content of the work video and acquires type information of the imaging device (e.g., type, installation location, installation angle, etc.). Based on the user's request, it then selects an appropriate analysis process based on the "analysis feature data" and the "second analysis feature data." For example, when analyzing the assembly process of a box, the first step is to get an overview of the entire work video (understanding the image composition based on the overall content) and to understand that it was recorded by an imaging device installed on the ceiling, and that the video shows the work performed by workers only, machines only, or both workers and machines. Then, if the user wants to analyze the actions of each target object, the analysis processing group that is best suited to analyzing the target object is selected, taking into account the type of imaging device and the type of target object.

[0036] In other words, in the "first selection process," the work analysis device 1 selects an appropriate analysis process from all analysis process groups (analysis AI groups). In the "second selection process," the work analysis device 1 excludes unsuitable analysis processes from the selection candidates based on the "content of the work video," and then selects an appropriate analysis process from the remaining analysis process groups. It should be noted that while there are differences in the selection process between the "first selection process" and the "second selection process," the same analysis process (analysis AI) may ultimately be selected. On the other hand, if there are many candidate analysis processes, adopting the "second selection process" allows for a more accurate selection. Specifically, a rough selection of potential candidate analysis processes can be made from a large group of analysis processes (preliminary selection), and then the analysis process that meets the user's requirements can be selected from among these candidates (final selection).

[0037] <<Mortalization of analytical processing and imaging equipment>> When performing a predetermined analysis on a predetermined work video, the work analysis device 1 associates the imaging device with an analysis process suitable for that imaging device, based on the type of imaging device that records the work video (type, installation location, target of imaging, etc.) and the characteristics of the analysis process (analysis elements 1, 2, 3, etc.). More specifically, the work analysis device 1 associates the "analysis feature data" shown in Figures 7A and 7B with the "imaging device type data" shown in Figure 9 to perform an analysis process appropriate for each imaging device. Furthermore, if the work analysis device 1 performs one or more work analyses on work videos recorded by one imaging device, it is not necessary to re-associate the imaging device with the analysis process. Also, even if the work analysis device 1 performs one work analysis on work videos recorded by multiple imaging devices, it is not necessary to re-associate them.

[0038] As an example, let's consider a case where the work analysis device 1 performs "work segmentation processing," "ergonomic processing," and "two-handed / one-handed analysis processing" for the work shown in Figure 1, after the selection of the analysis processing described above and the user's decision on the analysis processing. In this case, the work analysis device 1, based on the "analysis feature data" shown in Figures 7A and 7B and the "imaging device type data" shown in Figure 9, associates "work segmentation processing," "ergonomic processing," and "two-handed / one-handed analysis processing" with the first imaging device 100 (handheld camera), associates "work segmentation processing" and "two-handed / one-handed analysis processing" with the second imaging device 200 (wearable camera), and associates "work segmentation processing" with the third imaging device 300 (ceiling camera).

[0039] Furthermore, the work analysis device 1 may pre-specify the correspondence between the imaging device that records work videos and the analysis process suitable for the imaging device. In other words, it is possible for each imaging device to have an arbitrary analysis process set as the default. In that case, it is preferable that the analysis process suitable for each imaging device is set based on the type of imaging device (type, installation location, imaging target) and the characteristics of the analysis process (analysis elements 1, 2, and 3). If the correspondence is set in advance, the work analysis device 1 naturally does not need to select the appropriate analysis process from among multiple analysis processes. The working analysis device 1 may associate analysis processing with each imaging device based on arbitrary decisions made by the user. In this case as well, the working analysis device 1 does not need to select the analysis processing.

[0040] <<Execution of analysis process>> The work analysis device 1 performs "one or more analysis processes" determined by the user from among the selected analysis processes on the work video recorded, and outputs the results of the analysis process (analysis result data). More specifically, the work analysis device 1 performs "one or more analysis processes" associated with the imaging device on the work video recorded by the imaging device, and outputs the results of the analysis process. When the work analysis device 1 performs multiple analysis processes on a predetermined work (work video), it outputs the result of an analysis process that combines (merges) the results of the multiple analysis processes (also referred to as a merged analysis result). Specifically, it outputs a merged analysis result (merged analysis result data) obtained by merging (also referred to as merging) the work videos for the predetermined work and merging the results of each analysis process. The work analysis device 1 may output individual analysis results, analysis results obtained by merging only the work videos, or results obtained by merging only the analysis results without merging the work videos.

[0041] As a specific example, consider a case where, as shown in Figures 1, 10 to 14, the work analysis device 1 is connected to the first imaging device 100, the second imaging device 200, and the third imaging device 300 in a communicative manner, and performs "work segmentation processing," "ergonomic processing," and "two-handed / one-handed analysis processing" on a parts assembly operation (work video). The work analysis device 1 performs "work segmentation processing," "ergonomic processing," and "hand analysis processing" on work videos recorded by the first imaging device 100 (handheld camera). Then, it performs "work segmentation processing" and "hand analysis processing" on work videos recorded by the second imaging device 200 (wearable camera). Finally, it performs "work segmentation processing" on work videos recorded by the third imaging device 300 (ceiling camera). Subsequently, as shown in Figure 15, the work analysis device 1 outputs a combined analysis result obtained by combining the individual work videos and the individual analysis processing results. According to the combined analysis results in Figure 15, it can be seen that (1) segmentation processing is performed for each work element, (2) a record mark is assigned to tasks (work elements) in which the worker's movements are not "ergonomically correct," and (3) one-handed work and two-handed work are highlighted in different colors for each work element.

[0042] As shown in Figure 15, the work analysis device 1 can output the combined analysis results, obtained by combining work videos and their respective analysis results for a predetermined work, to a display screen (work analysis screen) for each cycle. Specifically, the combined work videos can be played back through the work analysis screen shown in Figure 5, and the analysis results for each work element can be displayed in a list.

[0043] According to the layout of the work analysis screen 40 shown in Figure 5, the work video 41 (merged work video) is displayed in the upper left display area. In the lower display area of ​​the work analysis screen 40, the analysis results 42 (combined analysis results) are displayed segmented by work element. The analysis results 42 display information on the segmentation position of the work video, recorded information recorded by the analysis process, a seek bar 42a indicating the playback point of the work video, and the analysis process (name of the analysis process) 42b. The upper right display area of ​​the work analysis screen 40 shows the work element list table 43. The work element list table 43 displays information such as identification number (No.), work element, ergonomic evaluation (ergo evaluation), two-handed / one-handed evaluation (two-handed / one-handed), type, element time, effective time, ineffective time, rating (Rat), and standard time, as well as the total work time (Total) and cycle information. In addition, indicator marks showing the work time for the best lap and worst lap may be displayed for all cycles. The "Ergonomics Evaluation" above refers to the recorded information for work elements as a result of ergonomic processing. The "Two-Handed / One-Handed Evaluation" refers to the information on one-handed work (or two-handed work) recorded for work elements as a result of two-handed / one-handed analysis processing. The screen layout of the work analysis screen 40 can be switched as needed using the layout switching button 44. Additionally, the cycle can be switched and displayed using the cycle switching button 45.

[0044] <<Correction of segmentation position through analysis processing (segmentation processing)>> The work analysis device 1 performs multiple segmentation processes on each work video recorded by multiple imaging devices for a predetermined work, and performs "segmentation position correction" based on the segmented position. Then, it outputs the results of the analysis process, which include the corrected "information on the corrected segmentation position". More specifically, the work analysis device 1 extracts a predetermined segmentation process from among multiple segmentation processes based on the "type of analysis target and / or work content," and performs "correction of segmentation position" based on the segmentation position created by each of the extracted segmentation processes. More specifically, each extracted segmentation process has its strengths and weaknesses depending on the type of analysis target and work content (there are compatibility issues depending on the performance of the segmentation process), and therefore, "weighting" is applied to each type of analysis target and work content. Therefore, the work analysis device 1 extracts a predetermined segmentation process from among multiple segmentation processes based on the type of analysis target and / or work content, assigns a predetermined weight to each of the extracted segmentation processes, and performs a correction of the segmentation position based on the segmentation position created by each of the extracted segmentation processes and the weight assigned to each segmentation process. The weights for segmentation processing for each type of analysis target (target object) and work content are included in the "Analysis Feature Data" shown in Figures 7A and 7B.

[0045] As a specific example, consider a case where the work analysis device 1 is connected to the first imaging device 100, the second imaging device 200, and the third imaging device 300 in a communicative manner, and performs "work segmentation processing," "ergonomic processing," and "hand analysis processing" on the assembly work of parts (work video). In this case, as shown in Figure 15, the work analysis device 1 corrects the segmented positions based on the segmented positions obtained by the "work segmentation processing," "ergonomic processing," and "hand analysis processing," and outputs the results of the analysis processing, which include "information on the corrected segmented positions." More specifically, it is as follows:

[0046] As a first correction pattern, the work analysis device 1 corrects the segmentation position A1 obtained by performing "work segmentation processing" on the work video recorded by the first imaging device 100 (handheld camera) and the segmentation position A2 obtained by performing "work segmentation processing" on the work video recorded by the third imaging device 300 (ceiling camera), and sets the corrected segmentation position A. In this way, when the same analysis process is performed on separate work videos recorded by imaging devices 100 and 300 that are installed in different locations (i.e., different objects to be captured), the segmentation positions can be corrected based on the respective segmentation positions.

[0047] As a second correction pattern, the work analysis device 1 corrects the segmentation position B1 obtained by performing "work segmentation processing" on the work video recorded by the first imaging device 100 (handheld camera) and the segmentation position B2 obtained by performing "hand analysis processing" on the work video recorded by the second imaging device 200 (wearable camera), and sets the corrected segmentation position B. In this way, when different analysis processes are performed on separate work videos recorded by imaging devices 100 and 200 that are installed in different locations (i.e., different objects to be captured), the segmentation positions can be corrected based on the respective segmentation positions.

[0048] In addition, although not shown in the diagram, the work analysis device 1 may also correct the work video recorded cycle by cycle by the same imaging device (e.g., a handheld camera) based on the segmentation position where "work segmentation processing" was performed in the first cycle and the segmentation position where "work segmentation processing" was performed in the second cycle, and set the corrected segmentation position. Alternatively, the work analysis device 1 may correct the work video recorded cycle by cycle by the same imaging device (e.g., a handheld camera) based on the segmentation position where "work segmentation processing" was performed in the first cycle and the segmentation position where "two-handed / one-handed analysis processing" was performed in the second cycle, and set the corrected segmentation position.

[0049] As described above, the work analysis device 1 can segment the work video recorded by the imaging device into work elements using analysis processing associated with the imaging device, and further correct the segmented positions. Therefore, it is possible to perform more appropriate segmentation processing on the work video. In particular, it is possible to correct the segmentation position while taking into account the characteristics of the work video (work content), considering the different segmentation positions due to work videos recorded by imaging devices with different installation locations (different objects being captured).

[0050] <Hardware configuration of the work analysis system> As shown in Figure 2, the work analysis device 1 is a computer comprising a CPU 2 as a data calculation and control processing device, a storage device 3 having ROM, RAM and HDD (SSD), a communication interface 4 for sending and receiving information data via a network, a display unit 5 for displaying character or image information, an input unit 6 operated by the user when inputting predetermined commands to the CPU, and an output unit 7 for outputting character or image information. The work analysis device 1 stores work videos (work video data) recorded by the imaging device, separated by imaging device and by cycle. It can also store the data separately by work process. The memory 3 of the work analysis device 1 stores a work analysis program in addition to the main program that performs the functions necessary for a computer. The work analysis device 1 functions when this program is executed by the CPU (processor). Alternatively, it may be executed by a semiconductor integrated circuit or FPGA (Field-Programmable Gate Array) that implements the CPU. Furthermore, imaging devices 100, 200, and 300 are computers equipped with the same hardware configuration.

[0051] Furthermore, if the work analysis device 1 is a mobile work analysis device, it will be a mobile computer equipped with a CPU 2, a storage device 3, a communication interface 4, a display unit 5, an input unit 6, an output unit 7, and an imaging unit (imaging device). In that case, the mobile work analysis device 1 communicates with an external analysis device (not shown) and transmits analysis data related to the work analysis. The external analysis device communicates with the mobile work analysis device 1, receives the analysis data, performs a detailed analysis based on the analysis data, and transmits the detailed analysis data.

[0052] <Functions of the work analysis system> As shown in Figure 3, the work analysis device 1, in terms of its functions, mainly comprises a storage unit 10 for storing various programs and data such as "work video data," "work element list data," "analysis feature data," "second analysis feature data," "imaging device type data," and "analysis data (combined analysis data)," a screen display unit 11, an analysis processing unit 12, an analysis processing selection unit 13, a type information acquisition unit 14, an analysis processing presentation unit 15, a correspondence unit 16, an analysis execution unit 17, an analysis result output unit 18, and a segmentation position correction unit 19. These consist of a CPU (processor), ROM, RAM, HDD, communication interface, and various programs.

[0053] To explain the imaging devices 100, 200, and 300 from a functional standpoint, their main components include storage units 110, 210, and 310 for storing various programs and data, communication units 111, 211, and 311 for sending and receiving various data with the work analysis device 1, and operation execution units 112, 212, and 312 for receiving user input and executing operation processing.

[0054] The functions of the work analysis device 1 (particularly the automatic analysis process) will be described in detail below. In this embodiment, the work analysis device 1 is configured to include a storage unit 10, but this is merely one example, and the storage unit 10 may be implemented in an external storage device located outside the work analysis device 1. In this case, it is preferable that the work analysis device 1 and the external storage device be connected by a communication channel.

[0055] <<1. Displaying the Analysis Screen>> The screen display unit 11, for example, receives user input and executes the software installed on the work analysis device 1. Once the user logs in, it displays the initial screen, the "menu screen." This menu screen is for performing automatic analysis processing. Specifically, as shown in Figure 4, the screen display unit 11 displays a list screen as a menu screen 30, allowing the user to select a predetermined item from a set of multiple selection items. When the screen display unit 11 receives a user selection of selection item 31 "Register Example" on the menu screen 30, it transitions to a video registration screen (not shown). On this screen, in order to perform automated work analysis, the analysis result data obtained by analyzing one cycle of work video as described above is registered as "example data (training data)". The analysis processing unit 12 learns from this analysis result data as "training data".

[0056] When the screen display unit 11 receives a user selection of item 32 "Automatic Analysis" on the menu screen 30, it transitions to the work analysis screen 40 shown in Figure 5. The work analysis screen 40 is a display screen used to register work videos and perform work analysis. The work analysis screen 40 displays the work video after imaging, segmenting it into work elements for each cycle, calculating the time for each work element, setting its type, etc. You can also check the variability of the work (work elements) in each cycle, rearrange the work elements, and organize the work process (rearrange, swap, rearrange).

[0057] The screen display unit 11 has, as specific functional units, a video display unit 11a, an analysis result display unit 11b, and a work element list display unit 11c. The video display unit 11a displays the recorded work video on the display screen, specifically displaying the work video 41 on the work analysis screen 40.

[0058] The analysis result display unit 11b displays the analysis results after receiving user input and performing an automated analysis process. Specifically, it displays the analysis results 42 on the work analysis screen 40. Analysis Result 42 displays the seek bar 42a and the analysis process (name of the analysis process) 42b that was performed on the work video. It also displays information on the segmentation positions made by the analysis process and the recorded information (marks and highlights) that was added. As can be seen from the analysis results 42 shown in Figure 5, it is clear that this is a merged analysis result obtained by merging the results of multiple analysis processes (merging of work videos and merging of analysis results). Furthermore, looking at analysis process 42b, it can be seen that "work segmentation processing," "ergonomic processing," and "two-handed / one-handed analysis processing" were performed.

[0059] The work element list display unit 11c displays a work element list table 43 on the display screen, which includes information on the work time for each segmented work element and information on the total time obtained by aggregating the work times. Specifically, the work element list 43 includes an identification number (No.) for identifying the work element, the work element name, the ergonomic evaluation (ergo evaluation), the two-handed / one-handed evaluation (two-handed / one-handed), the type, information on the work element time, and information on the total time for all work elements, and is stored in the storage unit 10. Furthermore, the above list of work elements includes information on the average work time, maximum work time, minimum work time, and work time for each cycle across the entire cycle. Furthermore, when the information in the work element list table 43 is updated via the work analysis screen 40, the information in the "work element list data" shown in Figure 6 is also updated and stored in the storage unit 10.

[0060] Looking at the work element list table 43 shown in Figure 5, we can see that for identification number "3" and work element name "Pick up part", the ergo evaluation is "●○", two-handed / one-handed is "one-handed", type is "effective", the element time for cycle 1 is "5.00 (seconds)", the rating (Rat) is "100 (%)", and the standard time is "5.00 (seconds)". We can also see that the total standard time for the work elements is "45.00 (seconds)". Note that Cycle 1 refers to the first cycle.

[0061] <<2. Selection and presentation of analysis processing>> The analysis processing unit 12 performs analysis processing on the work video captured by the imaging device, based on analysis elements related to the work, for each work element. The analysis processing unit 12, also known as the analysis processing group, includes multiple analysis processing units corresponding to the analysis elements, as described above. Specifically, these include work segmentation processing, hand analysis processing, component position tracking processing, machine motion tracking processing, ergonomics processing, two-handed / one-handed analysis processing, image anomaly detection processing, Therblic analysis processing, and motion economy principle processing. In this embodiment, the work analysis device 1 is described below as performing analysis processing by a first analysis processing unit 12a that performs "work segmentation processing", a second analysis processing unit 12b that performs "ergonomic processing", and a third analysis processing unit 12c that performs "two-handed / one-handed analysis processing" (of course, the work analysis device 1 may also perform "image anomaly detection processing").

[0062] (First selection process) The analysis processing selection unit 13 selects a predetermined analysis processing unit (analysis processing unit) from among multiple analysis processing units (analysis processing units) based on the analysis elements related to the work. More specifically, the analysis process selection unit 13 accepts the selection of predetermined analysis elements by the user as selection criteria, and refers to the "analysis feature data" shown in Figures 7A and 7B to select one or more analysis processes from among multiple analysis processes, according to the weight of the analysis processes based on the selection criteria made by the user. The analysis processing presentation unit 15 displays the analysis processing selected by the analysis processing selection unit 13 on the display screen. The first selection process will be explained in detail below.

[0063] The memory unit 10 stores "analysis feature data" which includes multiple analysis processes related to the work, multiple analysis elements related to the work, and evaluation information indicating the weight of the analysis process for each analysis element, as shown in Figures 7A and 7B. The "analysis feature data" shown in Figures 7A and 7B is the selection condition data for analysis processing, which enables the selection of analysis processing (analysis AI) based on the analysis elements related to the task. Specifically, "evaluation information (score)" indicating the weight of each analysis processing is shown for each "analysis element".

[0064] The "analysis elements" include: Analysis Element 1: Target Object (Analysis Subject), Analysis Element 2: Work Content, Analysis Element 3: Imaging Device, Analysis Element 4: Average Processing Speed, Analysis Element 5: Usage Fee, and Analysis Element 6: Actual Data. These analytical elements are classified into two groups: analytical elements 1, 2, and 3, which belong to the first group and are scored according to the weight of the analytical process, as described above; and analytical elements 4, 5, and 6, which belong to the second group and are not scored according to the weight of the analytical process. For analysis elements 1, 2, and 3 belonging to the first group, the selection of an analysis process is made based on the score to determine eligibility. This score is information indicating the degree to which each analysis process excels or fails at performing the analysis (information indicating the degree of eligibility). The analysis process selection unit 13 selects an appropriate analysis process from among multiple analysis processes based on the score for each analysis element 1, 2, and 3.

[0065] Analysis element 1, "Target object," refers to tangible objects such as workers (people), machines, parts, and products that are to be analyzed. For example, if a user analyst wants to analyze a work video with "worker" as the analysis target, the user selects "worker" in analysis element 1. Based on this selection condition, the analysis process selection unit 13 selects analysis processes with a high evaluation (score) for analysis element 1 "Target object: worker" as selection candidates (it selects them with priority). In other words, the analysis process selection unit 13 selects analysis processes that are good at identifying workers as the analysis target in work videos and segmenting the workers' work into work elements as selection candidates.

[0066] Analysis element 2, "Work content," refers to the content of the analysis, including manual work, upper body work, work involving walking, collaborative work with other workers, and collaborative work with machinery. For example, if a user wants to analyze a video of a worker's "hands-on work," the user selects "hands-on work" in analysis element 2. Based on this selection condition, the analysis process selection unit 13 selects analysis processes with a high evaluation (score) for analysis element 2 "Work content: hands-on work" as candidates. In other words, the analysis process selection unit 13 selects analysis processes that are good at identifying the "worker (both hands, one hand)" as the analysis target for the work video and segmenting the hands-on work into work elements as candidates. More specifically, the analysis process selected as a candidate in analysis element 2 is a process that enables the identification (specification) of a predetermined task (hand work, upper body work, etc.) from a combination of target objects (worker, machine, tool, material, part, product, etc.) included in the work video, the worker's position, the machine's layout, information about the work location, and information about the work time, and then performs segmentation processing.

[0067] Analysis element 3, "Imaging device," refers to the type of imaging device (type, installation location, etc.), such as ceiling cameras, fixed cameras, wearable cameras, handheld cameras, etc. For example, if a user wants to perform analysis processing on work videos captured using a "wearable camera," the user selects "wearable camera" in analysis element 3. Based on this selection condition, the analysis processing selection unit 13 selects analysis processing candidates that have a high evaluation (score) for analysis element 3 "imaging device: wearable camera." According to Figures 7A and 7B, the analysis processing selection unit 13 is expected to prioritize selecting the following analysis processes suitable for a "wearable camera": work segmentation processing (score: 7 points), hand analysis processing (score: 8 points), and component position tracking processing (score: 9 points).

[0068] Analysis element 4, "Average Processing Speed," is expressed as a percentage of the playback time of the work video. For example, if the average processing speed is 125%, then an analysis of a work video with a playback time of 1 hour can be performed in approximately 1.25 hours. Analysis element 5, "Usage Fee," indicates the usage fee for analysis processing based on the playback time of the work video. For example, if the usage fee is $1 / hour (h), an analysis processing fee of approximately $1 will be incurred for a work video with a playback time of 1 hour (1h). Analysis element 6, "Performance Data," indicates past usage history. For example, if the performance time is 10 million / h, it indicates that the corresponding analysis process has been used for 10 million hours in the past.

[0069] In the above, the analysis processing selection unit 13 selects one or more analysis processes based on the weight (score) of the analysis processes, according to the user's selection criteria, when based on analysis elements 1, 2, and 3 belonging to the first group included in the "analysis feature data" shown in Figures 7A and 7B. Furthermore, when based on analysis elements belonging to the second group included in the "analysis feature data," it selects one or more analysis processes based on the user's selection criteria, without relying on the weight (score) of the analysis processes. The analysis processing presentation unit 15 then presents the analysis processing selected based on the analysis elements belonging to the first group and / or the analysis elements belonging to the second group on the display screen.

[0070] In other words, when the analysis process selection unit 13 selects an analysis process using the analysis elements 1, 2, and 3 of the first group, it does so based on the score of each analysis element. On the other hand, when selecting an analysis process using the analysis elements 4, 5, and 6 of the second group, it selects an analysis process that directly matches the user's selection criteria. In other words, in the latter case, it selects an analysis process according to the user's relative selection criteria. The analysis process selection unit 13 may select one or more analysis processes based on a single analysis element, or it may select one or more analysis processes based on multiple analysis elements, as a result of the selection made by the user.

[0071] More specifically, when the analysis process selection unit 13 selects an analysis process by combining multiple analysis elements (for example, analysis element 1 × analysis element 2), it selects the analysis process based on a predetermined calculation result using each score. Examples of predetermined calculation methods include calculating the average value, or calculating the average value using weighted values ​​according to the priority of the analysis elements. With "analysis feature data" as shown in Figures 7A and 7B stored, the user specifies selection criteria through user operation (there are methods such as specifying each condition one by one, or selecting a combination of pre-specified conditions). When the user specifies selection criteria, for example, it is necessary to select at least one analysis element belonging to the first group, while the analysis elements belonging to the second group can be left to the user's discretion. The analysis process selection unit 13 prioritizes selecting analysis processes with high fitness by multiplying the conditions selected by the user. Then, the analysis process presentation unit 15 presents the analysis processes in order of fitness. The user then decides on an analysis process from the presented options.

[0072] (Second selection process) The analysis process selection unit 13 may select a predetermined analysis process from among multiple analysis processes by a "second selection process" that is different from the "first selection process". Specifically, this is as follows. In addition to storing the "analysis feature data" shown in Figures 7A and 7B, the memory unit 10 also stores "second analysis feature data" which associates the type of imaging device (work content), the object of analysis, and the analysis process (performance of the analysis process), as shown in Figure 8. The type information acquisition unit 14 acquires "type information of imaging device" for imaging the work in order to perform work analysis. The analysis processing selection unit 13 obtains "imaging device type information" and selects one or more analysis processing (analysis AI) from the above multiple analysis processing methods by referring to the "analysis feature data" shown in Figures 7A and 7B and the "second analysis feature data" shown in Figure 8. More specifically, the analysis processing selection unit 13 surveys the entire content of the work video to obtain "information on the type of imaging device (e.g., type, installation location, installation angle, etc.)" and makes a preliminary selection of analysis processing candidates based on "analysis feature data" and "second analysis feature data" (excluding analysis processing that is not a candidate). Then, it receives the user's request and makes a final selection of an appropriate analysis processing based on "analysis feature data" and "second analysis feature data".

[0073] In the "Second Analysis Feature Data" shown in Figure 8, the "Type of Imaging Device" is set to handheld camera, fixed camera, wearable camera, hand-held camera, ceiling camera, sensor, etc., the "Analysis Target" is set to work, ergonomics, fingertip, entire object, etc., and the "Performance of Analysis Processing" is set to analysis processing 1-1, 1-2, 1-3, etc. Furthermore, the "Second Analysis Feature Data" includes evaluation information (selection priority) of the "performance of the analysis processing" based on the "type of imaging device" and the "object of analysis," indicated by the symbols "◎, ○, △, ×". The performance of an analysis process refers to the differences in the analysis methods used for the analysis elements performed by the analysis process (also called feature information). For example, analysis processes 1-1, 1-2, and 1-3 perform task segmentation processing. Analysis process 1-1 performs task segmentation processing using a method that analyzes the worker's movements using bone data. Analysis process 1-2 performs task segmentation processing using a method that analyzes the worker's movements using human shape. For example, in the case of an imaging device type "handheld camera" and an analysis target "work," analysis process 1-1 (evaluation: ◎), which uses "a method of analyzing the worker's movements using bone data," receives a higher evaluation (larger feature set) than analysis process 1-2 (evaluation: △), which uses "a method of analyzing the worker's movements using human shape," indicating that it is better to prioritize selecting analysis process 1-1.

[0074] As a concrete example, let's consider a case where work videos captured by an imaging device, a "handheld camera," are used to analyze and process "work performed by a worker's upper body." The type information acquisition unit 14 acquires "type information of the imaging device (that it is a handheld camera)" by taking an overview of the entire content of the work video recorded by the imaging device. The analysis processing selection unit 13 obtains "imaging device type information" and then performs a preliminary selection based on the "second analysis feature data" shown in Figure 8, excluding analysis processing unsuitable for "handheld cameras". The analysis processing selection unit 13 accepts the selection of analysis elements by the user as selection conditions. For example, it accepts analysis element 1 "Target object: Worker" and analysis element 2 "Work content: Upper body work" as selection conditions. The analysis processing selection unit 13 performs the final selection based on the user's selection criteria, using the "analysis feature data" shown in Figures 7A and 7B and the "second analysis feature data" shown in Figure 8, and selects a "work segmentation process," in particular a "work segmentation process that has a method for analyzing the worker's movements using bone data."

[0075] As described above, a more accurate selection can be made than with the first selection process. Specifically, a rough selection of potential analysis processes can be made from a large group of analysis processes (preliminary selection), and then an analysis process that meets the user's requirements can be selected from among these candidates (final selection). In particular, it is possible to select an analysis process according to its performance.

[0076] <<3. Correspondence between analytical processing and imaging equipment>> The matching unit 16, in order to perform a predetermined analysis process on a predetermined work video, matches the imaging device with an analysis process suitable for that imaging device based on the type of imaging device that records the work video (type, installation location, target of imaging) and the characteristics of the analysis process (analysis elements 1, 2, and 3). More specifically, the matching unit 16 matches the analysis processing appropriate for each imaging device based on the "analysis feature data" shown in Figures 7A and 7B and the "imaging device type data" shown in Figure 9. Furthermore, when the analysis process selection unit 13 selects a predetermined analysis process from among multiple analysis processes for a predetermined task (work video), it is also possible to associate the analysis process with each work video (with each imaging device). In that case, the matching unit 16 does not need to re-associate the imaging device with the analysis process.

[0077] Figure 9, which shows "Imaging Device Type Data," indicates that the "Target Objects" and "Work Contents" to be imaged differ depending on the "Type" and "Installation Location" of the imaging device. For example, an imaging device of type "fixed camera 1" installed at the "entrance" indicates that it can detect "workers" as the target of imaging, but cannot detect "machines". In other words, in the case of "fixed camera 1", the work analysis device 1 can segment the work performed by "workers" (movement, hand movements) into individual work elements, but cannot segment the work performed by "machines". Furthermore, in the case of type "Ceiling Camera 1," it is shown that the work analysis device 1 can detect both the target object "worker" and "machine" and segment the work accordingly.

[0078] As a concrete example, consider a case where the work analysis device 1 performs "work segmentation processing," "ergonomic processing," and "two-handed / one-handed analysis processing" on work videos recorded by imaging devices 100, 200, and 300 that captured the "parts assembly work" shown in Figure 1, after selecting the analysis processing described above and the user deciding on the analysis processing. In this case, the mapping unit 16 maps "work segmentation processing," "ergonomic processing," and "two-handed / one-handed analysis processing" to the first imaging device 100 (handheld camera) based on the "analysis feature data" shown in Figures 7A and 7B and the "imaging device type data" shown in Figure 9, maps "work segmentation processing" and "two-handed / one-handed analysis processing" to the second imaging device 200 (wearable camera), and maps "work segmentation processing" to the third imaging device 300 (ceiling camera).

[0079] According to the above example, the "work segmentation processing" is associated with imaging devices 100, 200, and 300, and analysis processing is performed for each work element in the work video captured by these imaging devices. The "ergonomic processing" is associated only with the first imaging device 100 and performs analysis processing on each work element of the work video captured by the first imaging device 100. The "work segmentation processing" is associated with the first imaging device 100 and the second imaging device 200, and performs analysis processing on each work element of the work video captured by these imaging devices.

[0080] <<4. Execution of Analysis Process>> The analysis execution unit 17 performs an analysis process on the work video that is determined by user operation from among the selected analysis processes, and the analysis result output unit 18 outputs the results of the analysis process (analysis result data). More specifically, the analysis execution unit 17 performs "one or more analysis processes" associated with each imaging device on the work video recorded by each imaging device, and the analysis result output unit 18 outputs the results of the analysis process. As shown in Figures 10 to 12 and 14, the analysis result output unit 18 outputs a combined (merged) analysis result (merged analysis result) when multiple work videos are recorded for a predetermined task and multiple analysis processes are executed. Specifically, it merges each work video for a predetermined task and also merges each analysis result to output a merged analysis result (merged analysis result data).

[0081] As a specific example, as shown in Figure 11, the analysis execution unit 17 performs "work segmentation processing" on the work video. Here, the analysis execution unit 17 performs "work segmentation processing" on a work video that is a merged version of the multiple work videos shown in Figure 10 (work videos captured by imaging devices 100, 200, and 300), but of course, "work segmentation processing" can also be performed on each work video individually. Furthermore, as shown in Figure 12, the analysis execution unit 17 performs "ergonomic processing" on the work video. As shown in Figure 14, the analysis execution unit 17 performs "two-handed / one-handed analysis processing" on the work video.

[0082] As shown in Figure 15, the analysis result output unit 18 can output the combined analysis result, which is obtained by combining work videos and their respective analysis processing results for a predetermined task, to a display screen (work analysis screen) for each cycle. Specifically, the combined work videos can be played back through the work analysis screen shown in Figure 5, and the analysis results for each work element can be displayed in a list.

[0083] <<5. Correction of segmentation position through analysis processing (segmentation processing)>> The segmentation position correction unit 19 performs multiple analysis processes on each work video recorded by multiple imaging devices for a predetermined work, and performs "segmentation position correction" based on the segmented segmentation positions. The analysis result output unit 18 then outputs the results of the analysis process, including the corrected "information on the corrected segmentation position". More specifically, the segmentation position correction unit 19 extracts a predetermined analysis process from the above-mentioned multiple analysis processes based on the "type of analysis target and / or work content," assigns a "predetermined weight" to each of the extracted analysis processes, and performs a "correction of the segmentation position" based on the "segmentation position" segmented by each of the extracted analysis processes and the "weight" assigned to each of the analysis processes. The "scores (weights)" included in the "analysis feature data" shown in Figures 7A and 7B are used as "information on the weight of segmentation processing" for each type of analysis target (worker, machine) and work content.

[0084] Furthermore, the segmentation position correction unit 19 may extract predetermined analysis processes based on the "type of analysis target and / or work content," as well as based on the "type of imaging device, installation location, and imaging target (see Figure 9)." It may also extract predetermined analysis processes based on a combination of these. Alternatively, the segmentation position correction unit 19 may calculate an average value of the segmentation results based on the results of all analysis processes without weighting the analysis processes, and set the segmentation position that corresponds to this average value as the corrected segmentation position.

[0085] As a specific example, the segmentation position correction unit 19 is assumed to correct the segmentation position based on the segmentation positions determined by the "work segmentation processing," "ergonomic processing," and "hand analysis processing," as shown in Figure 15.

[0086] As the "first correction pattern," the segmentation position correction unit 19 corrects the segmentation position A by performing "work segmentation processing" on the work video recorded by the first imaging device 100 (handheld camera) and the segmentation position A2 by performing "work segmentation processing" on the work video recorded by the third imaging device 300 (ceiling camera), and sets the corrected segmentation position A. In this way, when the "same analysis process" is performed on separate work videos recorded by imaging devices 100 and 300 that are installed in different locations (i.e., different objects to be imaged), the segmentation position can be corrected based on the respective segmentation positions. Furthermore, the first imaging device 100 and the third imaging device 300 are configured with "scores (weights)" included in the "analysis feature data" shown in Figures 7A and 7B, and the corrected segmentation position is set taking these "scores (weights)" into consideration.

[0087] As a "second correction pattern," the segmentation position correction unit 19 corrects the segmentation position B based on the segmentation position B1 obtained by performing "work segmentation processing" on the work video recorded by the first imaging device 100 (handheld camera) and the segmentation position B2 obtained by performing "hand analysis processing" on the work video recorded by the second imaging device 200 (wearable camera), and sets the corrected segmentation position B. In this way, when different analysis processes are performed on separate work videos recorded by imaging devices 100 and 200 that are installed in different locations (i.e., different objects to be captured), the segmentation positions can be corrected based on the respective segmentation positions.

[0088] By using the work analysis device 1 described above, it becomes possible to select and propose an appropriate analysis process from among multiple analysis processes according to the user's request for work analysis, and to execute the analysis process determined by the user. Furthermore, by using the work analysis device 1, it becomes possible to perform segmentation processing on work videos for each work element at more appropriate locations.

[0089] <Work analysis method> Next, an example of the processing of the work analysis program (work analysis method) executed by the work analysis system S will be explained based on Figures 16 and 17. The program described above in this embodiment is a program for realizing the above-mentioned screen display unit 11, analysis processing unit 12, analysis processing selection unit 13, type information acquisition unit 14, analysis processing presentation unit 15, correspondence unit 16, analysis execution unit 17, analysis result output unit 18, and segmentation position correction unit 19 as functional components of the work analysis device 1 equipped with a storage unit 10, and the CPU (processor) of the work analysis device 1 executes this driving control program. The above program is executed in response to operational instructions from the user (specifically, the work analyst).

[0090] The flow shown in Figure 16 illustrates a part of the processing flow of the "work analysis method." Specifically, it shows the processing flow of a method that proposes an analysis process (analysis AI) based on analysis elements related to the work, and then performs the analysis on the work video. The flow shown in Figure 17 illustrates a processing flow for a method that performs multiple analysis processes on a work video and corrects the segmented positions determined by each analysis process.

[0091] In the work analysis flow (1) shown in Figure 16, the process begins with step (S1) in which the screen display unit 11 accepts the user's selection of selection item 32 "Automatic Analysis" on the menu screen 30 shown in Figure 4, and displays the work analysis screen 40 (analysis screen) shown in Figure 5. The screen display unit 11, for example, receives user input and executes the software installed on the work analysis device 1. Once the user logs in, it displays the initial screen, the menu screen 30.

[0092] In Step 2, the work analysis device 1 (also referred to as the work video acquisition unit) acquires the work video captured by the imaging device. Specifically, it accepts user input and captures the work video that is the target of the work analysis. The work analysis device 1 may also acquire a video of the work before displaying the work analysis screen 40.

[0093] In step 3, the analysis process selection unit 13 accepts the selection of predetermined analysis elements by the user as selection criteria, and refers to the "analysis feature data" shown in Figures 7A and 7B to select one or more analysis processes from among multiple analysis processes according to the weight of the analysis processes based on the selection criteria by the user (first selection process). Furthermore, as described above, the analysis processing selection unit 13 may select a predetermined analysis processing by a "second selection processing" that is different from the "first selection processing". In the case of the "second selection processing", the analysis processing selection unit 13 takes an overview of the entire content of the work video to obtain "information on the type of imaging device (e.g., type, installation position, installation angle, etc.)", and excludes analysis processing that is not a candidate for selection based on the "analysis feature data" shown in Figures 7A and 7B and the "second analysis feature data" shown in Figure 8. Then, it accepts the user's selection criteria and makes the final selection of an appropriate analysis processing based on the "analysis feature data" and the "second analysis feature data".

[0094] In step 4, the analysis processing presentation unit 15 displays the analysis processing selected by the analysis processing selection unit 13 on the display screen. Specifically, the analysis processing presentation unit 15 displays the selected "candidate analysis processing" and the "analysis processing score (supplementary information for determining the analysis processing)" weighted for each analysis element on the display screen. In other words, the analysis processing presentation unit 15 displays the analysis processing in descending order of fitness.

[0095] In step 5, the working analysis device 1 (also referred to as the analysis process determination unit) accepts the selection made by the user and determines the analysis process from among the multiple analysis processes presented. Specifically, the user selects and decides on an analysis process from those presented on the screen, referring to the "candidate analysis processes" and "analysis process scores" displayed on the screen.

[0096] In step 6, the matching unit 16 associates the "imaging device" with an "analysis process" suitable for the imaging device, based on the type of imaging device that records the work video (type, installation location, target of imaging) and the characteristics of the analysis process (analysis elements 1, 2, 3, etc.). Specifically, the matching unit 16 matches the appropriate analysis processing for each imaging device based on the "imaging device type data" shown in Figure 9 and the "analysis feature data" shown in Figures 7A and 7B. Furthermore, when the analysis processing selection unit 13 selects a predetermined analysis processing from among multiple analysis processing methods for a predetermined task (work video), it is also possible to associate the analysis processing with each work video (with each imaging device). In that case, the matching unit 16 does not need to perform the matching of imaging devices and analysis processing again in step 6.

[0097] In step 7, the analysis execution unit 17 performs the analysis process determined by the user's operation on the work video. More specifically, the analysis execution unit 17 performs "one or more analysis processes" associated with each imaging device on the work video recorded by that imaging device. Then, in step 8, the analysis result output unit 18 outputs the results of the analysis process (analysis result data). More specifically, when multiple work videos are recorded and multiple analysis processes are executed, the analysis result output unit 18 outputs a "combined analysis result" which is a merged version of the results of the multiple analysis processes. Specifically, it merges each work video and also merges each analysis result to output combined analysis result data. The process shown in Figure 16 is completed by following steps 1 through 8 above.

[0098] Next, as shown in Figure 17, the processing flow of a method that performs multiple analysis processes and corrects the segmented positions segmented by each analysis process is shown. In the work analysis flow (2) shown in Figure 17, the process begins with the work analysis device 1 (work video acquisition unit) acquiring work videos from multiple imaging devices (S101).

[0099] In step 102, the matching unit 16 associates the "imaging device" with an "analysis process" suitable for the imaging device, based on the type of imaging device that records the work video (type, installation location, and target of imaging) and the characteristics of the analysis process (analysis elements). Specifically, the matching unit 16 matches the appropriate analysis processing for each imaging device based on the "imaging device type data" shown in Figure 9 and the "analysis feature data" shown in Figures 7A and 7B. Alternatively, the analysis processing selection unit 13 may select a predetermined analysis processing from among multiple analysis processing methods for a predetermined task (work video), and the matching unit 16 may associate the analysis processing with each work video (each imaging device).

[0100] In step 103, the analysis execution unit 17 performs a plurality of analysis processes associated with each imaging device on the work video recorded by that imaging device.

[0101] In step 104, the segmentation position correction unit 19 recognizes the segmented positions determined by each analysis process. Specifically, the segmentation position correction unit 19 extracts a predetermined segmentation process from among multiple segmentation processes based on the "type of analysis target and / or work content," and recognizes the segmented position created by the extracted segmentation process. Then, in step 105, the segmentation position correction unit 19 performs "segmentation position correction" based on the recognized segmentation position. Specifically, the segmentation position correction unit 19 assigns a predetermined weight to each extracted analysis process and performs a segmentation position correction based on the recognized segmentation position and the weight assigned to each analysis process. The score (weight) included in the analysis feature data shown in Figures 7A and 7B is used as the weight information for the segmentation process.

[0102] In step 106, the analysis result output unit 18 outputs the results of the analysis process (analysis result data) which includes the corrected segment position information corrected by the segment position correction unit 19. The process shown in Figure 17 is completed by following steps 101 through 106 described above.

[0103] <Other Embodiments> In the above embodiment, as shown in Figure 5, the work analysis device 1 displays the work analysis screen 40 and performs work analysis. However, it is also possible to display analysis screens other than the work analysis screen 40 and perform various work analyses. For example, the work analysis device 1 may display a "cycle analysis screen" that shows a list of work elements and performs a detailed analysis for each cycle, or a "stack graph screen" that displays a stack graph to check the variability of work elements in each cycle and organize the work elements. Alternatively, it may display a "work comparison screen" that displays multiple work videos simultaneously and perform an analysis to compare work videos from different cycles or by different workers.

[0104] In the above embodiment, the work analysis program is stored on a recording medium that the work analysis device 1 can read, and processing is performed by the work analysis device 1 reading and executing the program. Here, the recording medium that the work analysis device 1 can read refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, the work analysis program may be executed on a web browser by launching dedicated software using a terminal (mobile device) that serves as the work analysis device 1.

[0105] In the above embodiments, the work analysis apparatus, work analysis method, and work analysis program according to the present invention were mainly described. However, the embodiments described above are merely examples to facilitate understanding of the present invention and do not limit it. The present invention can be modified and improved without departing from its spirit, and of course, equivalents thereof are included. [Explanation of Symbols]

[0106] S Work Analysis System 1. Work analysis device, mobile work analysis device 2 CPU 3 Storage device 4. Communication Interface 5 Display section 6 Input section 7 Output section 10 Storage section 11 Screen display section 11a Video display section 11b Analysis result display section 11c Work element list display section 12. Analytical Processing Unit 12a First Analysis Processing Unit (First Segmentation Processing Unit) 12b Second Analysis Processing Unit (Second Segmentation Processing Unit) 12c Third Analytical Processing Unit (Third Segmentation Processing Unit) 13. Analysis Processing Selection Department 14. Type Information Acquisition Unit 15. Analysis Processing Presentation Unit 16 Correspondence section 17. Analysis Execution Department 18 Analysis result output section 19 Segment position correction section 30 Menu screen 31, 32 Selection Items 40. Work Analysis Screen 41 Work Videos 42 Analysis results 42a Seek Bar 42b Analysis Processing 43 List of Work Elements 44 Layout switching button 45 Cycle Switch Button 100 First imaging device (handheld camera) 110 Storage section 111 Communications Department 112 Operation Execution Unit 200 Second imaging device (wearable camera) 210 Storage section 211 Communications Department 212 Operation Execution Unit 300 Third imaging device (ceiling camera) 310 Storage section 311 Communications Department 312 Operation Execution Unit A, A1, A2 segment position B, B1, B2 segment position

Claims

1. A work analysis device that performs work analysis based on work video images taken of work, An analysis selection process that, by referring to analysis feature data including evaluation information indicating the weight of analysis processes for each analysis element related to the aforementioned work, selects one or more analysis processes from among multiple analysis processes according to the evaluation information, based on the user's selection of predetermined analysis elements; A work analysis device that performs an analysis execution process on the work video, which is an analysis process selected by the analysis selection process.

2. Further type information acquisition processing is performed to acquire type information of an imaging device for imaging work, In the aforementioned analysis and selection process, The analysis feature data includes first evaluation information indicating the weight of the analysis process for each analysis element related to the aforementioned work, and second evaluation information indicating the priority of the analysis process for each type of imaging device, The work analysis apparatus according to claim 1, which selects an analysis process according to the first evaluation information and the second evaluation information based on the user's selection of analysis elements and the type of imaging device obtained by the type information acquisition process.

3. Further, an analysis presentation process is performed to present the analysis process selected by the analysis selection process, The work analysis apparatus according to claim 1, wherein the analysis presentation process presents the analysis process selected by the analysis selection process and evaluation information of the analysis process weighted according to each analysis element.

4. The analysis element includes the object to be analyzed, the work content, and the type of imaging device, The work analysis apparatus according to any one of claims 1 to 3, wherein the analysis selection process refers to the analysis feature data, which includes evaluation information for the target of analysis, the work content, and the type of imaging device, and selects one or more analysis processes from among a plurality of analysis processes, including work segmentation processing, hand analysis processing, ergonomics processing, and two-handed / one-handed analysis processing, according to the evaluation information.

5. A work analysis method performed by a computer that performs work analysis based on work video images of the work, The aforementioned computer, By referring to analysis feature data that includes evaluation information indicating the weight of the analysis process for each analysis element related to the aforementioned work, one or more analysis processes are selected from among multiple analysis processes according to the evaluation information, based on the user's selection of predetermined analysis elements. A work analysis method that performs the selected analysis process on the work video.

6. A computer as a work analysis device that performs work analysis based on work video images of the work, A process that, by referring to analysis feature data including evaluation information indicating the weight of analysis processing for each analysis element related to the aforementioned work, selects one or more analysis processing from among multiple analysis processing, based on the user's selection of predetermined analysis elements, according to the evaluation information. A work analysis program that performs a process to execute the selected analysis process on the work video.

Images