Information processing device, program, and information processing method

Abstract

A work analysis device (130) comprises: a sensor data acquisition unit (132) that acquires sensor data items each indicating a detection result detected, by a sensor with respect to the bodies, when one or more workers are performing work; a feature amount calculation unit (133) that calculates a plurality of feature amounts from the plurality of sensor data items, and generates feature amount data items each indicating the plurality of feature amounts in time series; a periodic feature extraction unit (134) that extracts, from the plurality of feature amount data items, a periodic feature amount that is a feature amount appearing periodically, identifies a range by using a predetermined rule so as to include the periodic feature amount, and generates periodic feature data indicating the range; and a periodic feature data storage unit (135) that stores the periodic feature data.

Description

Information processing device, program, and information processing method

[0001] The present disclosure relates to an information processing device, a program, and an information processing method.

[0002] Conventionally, at work sites where products are manufactured, the elemental tasks that make up the work have been identified, and the work of workers has been evaluated based on the elemental tasks. For example, a work data management system described in Patent Literature 1 classifies the work performed by workers at the work site into multiple chronologically-series detailed processes based on work data including video data of the work site, and displays the video data and the multiple chronologically-series detailed processes, allowing a manager to understand the work content at the work site.

[0003] Japanese Patent Application Laid-Open No. 2019-16226

[0004] However, with conventional technology, the administrator or the like must set in advance data indicating the positions through which the worker's body parts will pass, which places a heavy burden on the administrator or the like in terms of time and effort.

[0005] Therefore, one or more aspects of the present disclosure aim to enable a worker to easily understand the elemental tasks that make up the work he or she is performing.

[0006] An information processing device according to one aspect of the present disclosure is characterized by comprising: a sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by sensors on the bodies of each of one or a plurality of workers during work; a feature calculation unit that calculates a plurality of feature amounts from each of the plurality of sensor data and generates feature amount data indicating the plurality of feature amounts in a time series, thereby generating a plurality of feature amount data; and a periodic feature extraction unit that extracts periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature amount data, specifies a range according to a predetermined rule so that the periodic feature amounts are included, and generates periodic feature data indicating the range.

[0007] an information processing device according to one aspect of the present disclosure, comprising: a sensor data acquisition unit that acquires target sensor data indicating detection results detected by a sensor with respect to a worker to be analyzed; a feature calculation unit that calculates a plurality of feature amounts from the target sensor data and generates target feature data indicating the plurality of feature amounts identified from the target sensor data; and a classification unit that calculates a plurality of feature amounts from each of a plurality of sensor data indicating detection results detected by a sensor with respect to the bodies of one or more workers while working, and generates feature data indicating the plurality of feature amounts in time series, thereby generating a plurality of feature data, extracts periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature data, classifies elemental tasks that make up the work of the worker to be analyzed into intervals separated by time periods in which the plurality of feature amounts indicated by the target feature data are included within a range indicated by the periodic feature data indicating a range specified by a predetermined rule so as to include the periodic feature amounts, and generates classification data indicating the times of the intervals classified for each elemental task.

[0008] A program according to one aspect of the present disclosure causes a computer to function as: a sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by sensors on the bodies of one or a plurality of workers while they are working; a feature calculation unit that calculates a plurality of feature values ​​from each of the plurality of sensor data and generates feature data indicating the plurality of feature values ​​in a time series, thereby generating a plurality of feature data; and a periodic feature extraction unit that extracts periodic feature values, which are feature values ​​that appear periodically, from the plurality of feature data, specifies a range according to a predetermined rule so that the periodic feature values ​​are included, and generates periodic feature data indicating the range.

[0009] A program according to one aspect of the present disclosure causes a computer to function as: a sensor data acquisition unit that acquires target sensor data indicating detection results detected by a sensor with respect to a worker to be analyzed; a feature calculation unit that calculates a plurality of feature amounts from the target sensor data and generates target feature data indicating the plurality of feature amounts calculated from the target sensor data; and a classifier that calculates a plurality of feature amounts from each of a plurality of sensor data indicating detection results detected by a sensor with respect to the bodies of one or more workers while they are working, and generates feature data indicating the plurality of feature amounts in a time series, thereby generating a plurality of feature data, extracts periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature data, classifies elemental tasks that make up the work of the worker to be analyzed into intervals separated by time periods in which the plurality of feature amounts indicated by the target feature data are included within a range indicated by the periodic feature data, which indicates a range specified by a predetermined rule so that the periodic feature amount is included, and generates classification data indicating the duration of the intervals classified for each elemental task.

[0010] An information processing method according to one aspect of the present disclosure includes acquiring a plurality of sensor data indicating detection results detected by sensors on the bodies of one or a plurality of workers during work, calculating a plurality of feature amounts from the plurality of sensor data, and generating feature amount data indicating the plurality of feature amounts in a time series, thereby generating a plurality of feature amount data, extracting periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature amount data, specifying a range according to a predetermined rule so that the periodic feature amounts are included, and generating periodic feature data indicating the range.

[0011] An information processing method according to one aspect of the present disclosure includes: acquiring target sensor data indicating detection results detected by a sensor with respect to a worker to be analyzed; calculating a plurality of feature amounts from the target sensor data; generating target feature amount data indicating the plurality of feature amounts identified from the target sensor data; calculating a plurality of feature amounts from the plurality of sensor data indicating detection results detected by a sensor with respect to the bodies of one or more workers while they are working; generating feature amount data indicating the plurality of feature amounts in a time series, thereby generating a plurality of feature amount data; extracting periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature amount data; classifying elemental tasks that make up the work of the worker to be analyzed into intervals separated by time periods in which the plurality of feature amounts indicated by the target feature amount data are included within a range indicated by the periodic feature data, which indicates a range specified by a predetermined rule so that the periodic feature amount is included; and generating classification data indicating the duration of the intervals classified for each elemental task.

[0012] According to one or more aspects of the present disclosure, it is possible to easily grasp the elemental tasks that make up the work being performed by a worker.

[0013] FIG. 1 is a block diagram showing an outline of the configuration of a work analysis system according to embodiments 1 and 2. FIG. 2 is a block diagram showing an outline of the configuration of a work analysis device according to embodiments 1 and 2. FIG. 3 is a graph for explaining features that appear periodically in one-dimensional feature amount data. FIG. 4 is a block diagram showing an outline of the configuration of a PC. FIG. 5 is a flowchart for explaining the operation of the work analysis device in the learning phase. FIG. 6 is a flowchart for explaining the operation of the work analysis device in the analysis phase.

[0014] 1 is a block diagram showing a schematic configuration of an work analysis system 100 according to embodiment 1. The work analysis system 100 includes a sensor 110 and a work analysis device 130 as an information processing device. The sensor 110 and the work analysis device 130 are connected to a network 101 such as a local area network (LAN).

[0015] The sensor 110 generates sensor data indicating the detection results of the sensor on the worker's body while working. The sensor 110 then transmits the sensor data to the work analysis device 130 via the network 101. In this embodiment, the sensor 110 is a camera that captures images of the worker, and transmits image data representing the image of the worker while working to the work analysis device 130 as sensor data.

[0016] 2 is a block diagram showing a schematic configuration of the work analysis device 130. The work analysis device 130 includes a communication unit 131, a sensor data acquisition unit 132, a feature calculation unit 133, a periodic feature extraction unit 134, a periodic feature data storage unit 135 as a storage unit, a classification unit 136, an evaluation unit 137, and a display unit 138.

[0017] The communication unit 131 communicates with the sensor 110 via the network 101. For example, the communication unit 131 receives sensor data from the sensor 110 via the network 101.

[0018] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101. The acquired sensor data is provided to the feature amount calculation unit 133.

[0019] The feature amount calculation unit 133 calculates a plurality of feature amounts from the sensor data and generates feature amount data that indicates the plurality of feature amounts in a time series.

[0020] Here, the feature calculation unit 133 calculates the positions of the worker's body parts as features. Specifically, the feature calculation unit 133 can calculate the positions of the body parts as features by inputting sensor data into a trained deep learning model. Here, a model that infers the positions of body parts from video data may be used as the deep learning model. This allows classification based on the movement of body parts. For example, when the work analysis system 100 learns the work of workers 1 to W (W is an integer greater than or equal to 2), the feature data of the w worker included in 1 to W is expressed by the following formula (1):

[0021] (1)

[0022] where Tw is the number of time steps of the feature data of worker w, and ｗ The feature vector, which is the feature at time step t in

[0023] (2)

[0024] In formula (2), d in "t, d" is a number indicating the dimension of the feature vector, and is an integer satisfying 1≦d≦D (D is an integer equal to or greater than 1). Note that D is the number of dimensions.

[0025] As described above, the feature amount calculation unit 133 calculates the feature amount data x from the sensor data obtained by measuring the work performed by each different worker. １ , x ２ , ..., x Ｗ The calculated feature data x １ , x ２ , ..., x Ｗ In the learning phase for calculating the periodic feature data, the periodic feature data x is provided to the periodic feature extraction unit 134. In the analysis phase for analyzing the work of the workers, the periodic feature data x of one worker g to be analyzed is provided to the periodic feature extraction unit 134. ｇ is provided to the classification unit 136. Here, the work analysis device 130 first calculates the periodic feature data in the learning phase, and then analyzes the work of the worker in the analysis phase.

[0026] In the learning phase, the sensor 110 transmits sensor data for one or more workers, and the sensor data acquisition unit 132 acquires the plurality of sensor data. The feature calculation unit 133 identifies a plurality of feature amounts from the plurality of sensor data and generates feature data indicating the plurality of feature amounts in time series, thereby generating a plurality of feature data. The periodic feature extraction unit 134 extracts periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature data, specifies a range according to a predetermined rule so that the periodic feature amount is included, and generates periodic feature data indicating the range.

[0027] Here, a periodic feature is a centroid of a cluster whose evaluation value calculated for each of the plurality of clusters exceeds a threshold, such that when a plurality of feature quantities represented by a plurality of feature data are clustered into a plurality of clusters, the greater the number of feature quantities belonging to each of the plurality of clusters, the higher the evaluation. In other words, a periodic feature is a feature that aggregates a plurality of feature quantities represented by a plurality of feature data into a certain region and that appears repeatedly in a time series. This will be described in detail below.

[0028] For example, the period feature extraction unit 134 extracts a plurality of feature amount data x １ , x ２ , ..., x Ｗ In the above, a feature amount that appears periodically is identified, and periodic feature data that indicates the feature amount that appears periodically is calculated.

[0029] For example, the feature data x of worker w ｗ The state data s corresponding to ｗ is shown in the following equation (3).

[0030] (3)

[0031] Status Data ｗ is a state value indicating the number of the cluster into which the feature vector at time step t shown in the above formula (2) is classified. k is a positive integer satisfying 1≦k≦K, as shown in the following formula (4). Here, K is the total number of clusters.

[0032] (4)

[0033] Here, the state data s １ , s ２ , ..., s Ｗ is the feature data x １ , x ２ , ..., x Ｗ The feature value represented by can be calculated as the cluster number when classified by the k-means method.

[0034] Next, the periodic feature extraction unit 134 extracts the central vector μ of the feature vector, which is the feature corresponding to the cluster k, as shown in the following equation (5): ｋ Identify.

[0035] (5)

[0036] where μ ｋ，ｄ is μ ｋ is the dth value in μ ｋ，ｄ can be calculated as the centroid of cluster k obtained by the k-means method.

[0037] Next, the periodic feature extraction unit 134 extracts the evaluation value r of the cluster k. ｋ For example, the evaluation value r ｋ is the state data s １ , s ２ , ..., s Ｗ The reward value of cluster k can be calculated using the above equation.

[0038] Specifically, the evaluation value r ｋ is the state data s of worker w ｗ The reward value r of cluster k inferred by inverse reinforcement learning using ｋ ｗ Based on this, it can be calculated using the following equation (6).

[0039] (6)

[0040] Here, by using equation (6), a cluster that has a high reward value for all state data can be highly evaluated.

[0041] In addition, the evaluation value r ｋ is the state data s of worker w ｗ In this case, s ｔ ｗ = Number of times k occurred N ｋ ｗ Based on this, it can also be calculated using the following equation (7) or (8).

[0042] (7)

[0043] (8)

[0044] In the above formula (7), the state data s １ , s ２ , ..., s Ｗ In this case, s ｔ ｗ = k, and the evaluation value r ｋ is calculated, and in equation (8), the state data s １ , s ２ , ..., s Ｗ In this case, s ｔ ｗ = k, and the evaluation value r ｋ These formulas allow for a high evaluation of clusters that appear frequently in all state data.

[0045] From the above, the period feature extraction unit 134 extracts the period feature data p ｊ can be determined by the following equation (9).

[0046] (9)

[0047] In equation (9), j is a number for identifying the periodic feature data and is a positive integer that satisfies 1≦j≦J. J is an integer equal to or greater than 1 that represents the total number of periodic feature data. σ ｊ is the periodic feature data pｊ In other words, in the first embodiment, the center vector μ of cluster k is ｊ and its central vector μ ｊ Area σ centered on ｊ The range is specified by the following.

[0048] Here, the periodic feature data p ｊ The central vector μ of the feature vector in ｊ is the evaluation value r ｋ When exceeds a predetermined threshold, μ ｊ = μ ｋ It can be calculated as:

[0049] In addition, the periodic feature data p ｊ The area σ of the feature vector in ｊ is the evaluation value r ｋ When exceeds a predetermined threshold, it can be calculated by either (i) or (ii) below: (i) Calculated as three times the standard deviation in each dimension for the feature vector corresponding to cluster k, or (ii) Calculated as a 95% confidence ellipse including the feature vector corresponding to cluster k.

[0050] Note that the periodic feature extraction unit 134 may reduce the number of dimensions D when determining the range. For example, if the feature vector has two dimensions, the right hand position and the left hand position, the range may be determined by deleting the dimension of the left hand position. In other words, when determining the range, the periodic feature extraction unit 134 may delete dimensions that are not directly related to the worker's work from the number of dimensions D, and determine the range using only dimensions that are directly related to the worker's work. For example, the range may be determined using only dimensions related to the worker's body parts and hands.

[0051] The periodic feature data calculated as described above will now be described. FIG. 3 is a graph illustrating periodic features appearing in one-dimensional feature data. As shown in FIG. 3, it is assumed that feature A, feature B, and feature C are determined to be periodically repeated in this feature data. In this case, the periodic feature extraction unit 134 generates periodic feature data indicating a range corresponding to feature A, a range corresponding to feature B, and a range corresponding to feature C. In the example of FIG. 3, feature A, feature B, and feature C appear in the following order: feature C, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, and feature C. It is possible to recognize that a worker is performing a certain task based on the task elements separated by each of these features.

[0052] Returning to FIG. 1, the periodic feature data storage unit 135 stores the periodic feature data calculated as described above.

[0053] Next, in the analysis phase, the sensor 110 transmits sensor data regarding the worker to be analyzed, and the sensor data acquisition unit 132 acquires the sensor data of the worker as target sensor data. The feature calculation unit 133 then generates feature data as target feature data from the target sensor data. The classification unit 136 then classifies the elemental tasks that make up the work of the worker to be analyzed into intervals separated by time periods in which the multiple feature values ​​indicated by the target feature data are included in the range indicated by the periodic feature data, and generates classification data that indicates the duration of the intervals classified for each elemental task.

[0054] For example, in the analysis phase, the classification unit 136 determines whether the feature amount indicated by the feature amount data of the worker to be analyzed by the feature amount calculation unit 133 is included in the range indicated by the periodic feature data stored in the periodic feature data storage unit 135. Then, the classification unit 136 identifies sections by dividing the work into sections based on the time points of the feature amounts included in the range indicated by the periodic feature data, and classifies the elemental tasks that make up the work into the identified sections. The elemental tasks to be classified may be identified, for example, by a combination of features corresponding to the divisions.

[0055] The classification unit 136 then generates classification data indicating the duration of each section of each classified element work. The classification data is provided to the evaluation unit 137.

[0056] The evaluation unit 137 evaluates the time for each elemental task indicated by the classification data according to a predetermined standard. For example, an ideal work time for each elemental task is predetermined as a standard time, and the evaluation unit 137 performs the evaluation so that the closer the time for each elemental task indicated by the classification data is to the standard time, the higher the evaluation. Evaluation data indicating the results of the evaluation performed in this manner is provided to the display unit 138.

[0057] The display unit 138 displays at least one of the sensor data, the feature amount data, the periodic feature data, the classification data, and the evaluation data. For example, the display unit 138 may display the classification data superimposed on the sensor data, and in this case, the evaluation data may also be superimposed.

[0058] The work analysis device 130 described above can be realized by, for example, a computer such as the PC 10 shown in Fig. 4. The PC 10 includes a storage 11 such as a hard disk drive (HDD) or a solid state drive (SSD), a memory 12, a processor 13 such as a central processing unit (CPU), a communication interface (I / F) 14 such as a network interface card (NIC), an input interface 15 such as a keyboard and mouse, and a display 16.

[0059] For example, the periodic feature data storage unit 135 can be realized by the storage 11 or the memory 12. The sensor data acquisition unit 132, the feature amount calculation unit 133, the periodic feature extraction unit 134, the classification unit 136, and the evaluation unit 137 can be realized by the processor 13 loading a program stored in the storage 11 into the memory 12 and executing the program. The communication unit 131 can be realized by the communication I / F 14. The display unit 138 can be realized by the display 16.

[0060] The above programs may be downloaded to the storage 11 from a recording medium (not shown) via a reader / writer (not shown) or from the network 101 via the communication I / F 14, and then loaded onto the memory 12 and executed by the processor 13. Alternatively, the programs may be directly loaded onto the memory 12 from a recording medium via the reader / writer or from the network 101 via the communication I / F 14, and then executed by the processor 13. In other words, the programs may be provided by a program product such as a recording medium.

[0061] FIG. 5 is a flowchart for explaining the operation of the work analysis device 130 in the learning phase.

[0062] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101 (S10). In the learning phase, a plurality of sensor data corresponding to one or a plurality of workers is acquired.

[0063] The feature amount calculation unit 133 extracts a plurality of feature amounts from each of the plurality of sensor data, and generates feature amount data that indicates the plurality of feature amounts in a time series (S11). Here, a plurality of feature amount data is generated corresponding to the plurality of sensor data.

[0064] The periodic feature extraction unit 134 extracts periodic feature amounts, which are feature amounts that appear periodically, from the plurality of feature amount data, and generates periodic feature data indicating the periodic feature amounts (S12).

[0065] The periodic characteristic data storage unit 135 stores the periodic characteristic data calculated as described above (S13).

[0066] FIG. 6 is a flowchart for explaining the operation of the work analysis device 130 in the analysis phase.

[0067] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101 (S20). In the analysis phase, sensor data of one worker to be analyzed (also referred to as a target worker) is acquired.

[0068] The feature amount calculation unit 133 extracts feature amounts from the sensor data and generates feature amount data that indicates the feature amounts in a time series (S21). The feature amount data generated here is also referred to as target feature amount data.

[0069] The classification unit 136 identifies sections by dividing them by the time points at which the feature values ​​indicated by the feature data are included in the range indicated by the periodic feature data, and classifies the elemental tasks that make up the work into the identified sections (S22).The classification unit 136 then generates classification data that indicates the duration of the sections for each classified task element.

[0070] The evaluation unit 137 evaluates the time for each elemental work indicated by the classification data according to a predetermined standard (S23).

[0071] The display unit 138 displays at least one of the sensor data, the feature amount data, the period feature data, the classification data, and the evaluation data (S24).

[0072] As described above, according to the first embodiment, periodic feature data indicating feature amounts that appear periodically can be automatically generated from the feature amount data of a worker, which makes it possible to easily analyze the work of a worker based on the elemental work that constitutes the work of the worker.

[0073] In the first embodiment described above, the feature amount calculation unit 133 calculates the positions of the worker's body parts as feature amounts, but the first embodiment is not limited to this example. For example, the feature amount calculation unit 133 may calculate the positions of the worker's hands as feature amounts. In this case, a model that infers the positions of the worker's hands from sensor data may be used as the deep learning model.

[0074] Furthermore, the feature calculation unit 133 may calculate the type of object, such as a tool or part, as a feature from an image of the area around the worker's hand, rather than from an image of the worker's body part. Specifically, the feature calculation unit 133 inputs sensor data into a deep learning model to identify the position of the worker's hand, and identifies an image of a predetermined range including the identified hand position as a partial image. The feature calculation unit 133 then inputs the identified partial image into the deep learning model, thereby calculating the type of object included in the partial image as a feature. In this case, a model that infers the type of object included in an image from an image may be used as the deep learning model. This allows classification based on the presence or absence of an object, such as a tool or part, held in the hand, and the type of object, such as the tool or part, held in the hand.

[0075] In this case, the plurality of sensor data are a plurality of video data showing video images of one or more workers, and the target sensor data are video data showing video images of the worker to be analyzed. The feature amount calculation unit 133 then identifies, from each frame of the plurality of sensor data, an image of the periphery of a specific body part of one or more workers as a first partial image, and calculates a feature amount of an object included in the first partial image. The feature amount calculation unit 133 also identifies, from the frame of the target sensor data, an image of the periphery of a specific body part of the worker to be analyzed as a second partial image, and calculates a feature amount of an object included in the second partial image.

[0076] Second Embodiment In a second embodiment, it is assumed that a worker performs work using his or her hands, and periodic feature data is acquired for both cases where the worker's dominant hand is the right hand and where the dominant hand is the left hand, so that the work of the worker can be analyzed.

[0077] 1 , the work analysis system 200 according to the second embodiment includes a sensor 110 and a work analysis device 230. The sensor 110 of the work analysis system 200 according to the second embodiment is similar to the sensor 110 of the work analysis system 100 according to the first embodiment.

[0078] 2 is a block diagram showing a schematic configuration of a work analysis device 230 according to embodiment 2. The work analysis device 230 includes a communication unit 131, a sensor data acquisition unit 132, a feature calculation unit 233, a periodic feature extraction unit 234, a periodic feature data storage unit 235, a classification unit 236, an evaluation unit 237, and a display unit 238.

[0079] The communication unit 131 and the sensor data acquisition unit 132 of the work analysis apparatus 230 in the second embodiment are similar to the communication unit 131 and the sensor data acquisition unit 132 of the work analysis apparatus 130 in the first embodiment.

[0080] The feature amount calculation unit 233 calculates the feature amounts of the right hand and the left hand from the sensor data, and generates right hand feature amount data and left hand feature amount data that indicate the respective feature amounts in time series.

[0081] For example, in the learning phase, the feature calculation unit 233 calculates multiple feature amounts related to the right hand from multiple sensor data and generates right hand feature amount data that indicates the multiple feature amounts calculated for the right hand in time series. Also, the feature calculation unit 233 calculates multiple feature amounts related to the left hand from multiple sensor data and generates left hand feature amount data that indicates the multiple feature amounts calculated for the left hand in time series. Note that in the learning phase, multiple right hand feature amount data and multiple left hand feature amount data are generated corresponding to one or multiple workers.

[0082] On the other hand, in the analysis phase, the feature calculation unit 233 calculates multiple feature amounts related to the right hand from the target sensor data and generates target right hand feature amount data that indicates the multiple feature amounts calculated for the right hand in time series. Also, the feature calculation unit 233 calculates multiple feature amounts related to the left hand from the target sensor data and generates target left hand feature amount data that indicates the multiple feature amounts calculated for the left hand in time series. As described above, the target sensor data is sensor data detected for the worker to be analyzed.

[0083] As in the first embodiment, the calculated feature data is provided to the periodic feature extraction unit 234 in the learning phase in which periodic feature data is calculated, and is provided to the classification unit 236 in the analysis phase in which the work of the worker is analyzed.

[0084] The period feature extraction unit 234 extracts right-hand period feature amounts, which are periodic feature amounts, from the multiple right-hand feature data, identifies a right-hand range specified by a predetermined rule so that the right-hand period feature amount is included, and generates right-hand period feature data indicating the right-hand range. The period feature extraction unit 234 also extracts left-hand period feature amounts, which are periodic feature amounts, from each of the multiple left-hand feature data, identifies a left-hand range specified by a predetermined rule so that the left-hand period feature amount is included, and generates left-hand period feature data indicating the left-hand range. Furthermore, when the similarity between the right-hand period feature data and the left-hand period feature data exceeds a predetermined threshold, the period feature extraction unit 234 integrates the right-hand period feature data and the left-hand period feature data to generate integrated period feature data. This will be described in detail below.

[0085] For example, the periodic feature extraction unit 234 identifies periodic feature amounts in the right hand feature amount data and calculates right hand periodic feature data indicating the periodic feature amounts. The right hand periodic feature data is expressed by the following equation (10).

[0086] (10)

[0087] Here, jright is a positive integer that satisfies 1≦jright≦Jright, and is a number for identifying right-hand period feature data. Jright is the total number of right-hand period feature data.

[0088] Similarly, the periodic feature extraction unit 234 identifies periodic feature amounts in the left hand feature amount data and calculates left hand periodic feature data indicating the periodic feature amounts. The left hand periodic feature data is expressed by the following equation (11).

[0089] (11)

[0090] Here, jleft is a positive integer that satisfies 1≦jleft≦Jleft, and is a number for identifying left-hand period feature data. Jleft is the total number of left-hand period feature data.

[0091] The period feature extraction unit 234 also calculates the similarity between the right-hand period feature data and the left-hand period feature data. The similarity may be, for example, the inverse of the Euclidean distance between the center vector of the right-hand period feature data and the center vector of the left-hand period feature data, or the overlap rate of the ranges in a D-dimensional space between the right-hand period feature data and the left-hand period feature data.

[0092] Then, the period feature extraction unit 234 calculates integrated period feature data by integrating the right-hand period feature data and the left-hand period feature data that have a high degree of similarity. The integrated period feature data is expressed by the following equation (12).

[0093] (12)

[0094] Here, Jeither is a positive integer that satisfies 1≦Jeither≦Jeither, and is a number for identifying the integrated periodic feature data. Jeither is the total number of integrated periodic feature data.

[0095] For example, when the similarity between the right-hand period feature data and the left-hand period feature data exceeds a predetermined threshold, the period feature extraction unit 234 can calculate the integrated period feature data using the following equation (13).

[0096] (13)

[0097] The period feature data storage unit 235 stores the right-hand period feature data, left-hand period feature data, and integrated period feature data calculated as described above.

[0098] The classification unit 236 classifies the element tasks constituting the work of the worker to be analyzed into sections separated by time periods in which the multiple feature quantities included in the target right hand feature quantity data fall within the range indicated by the right hand cycle feature data, and generates right hand classification data indicating the duration of the sections classified for each element task for the right hand. The classification unit 236 also classifies the element tasks of the worker to be analyzed into sections separated by time periods in which the multiple feature quantities included in the target left hand feature quantity data fall within the range indicated by the left hand cycle feature data, and generates left hand classification data indicating the duration of the sections classified for each element task for the left hand. The classification unit 236 also classifies the element tasks of the worker to be analyzed into sections separated by time periods in which the multiple feature quantities included in the target right hand feature quantity data and the multiple feature quantities included in the target left hand feature quantity data fall within the range indicated by the integrated cycle feature data, and generates integrated classification data indicating the duration of the sections classified for each element task for the left and right hands.

[0099] For example, the classification unit 236 classifies the work of the worker into work elements from the right hand feature amount data depending on whether the right hand feature amount data is included in the range indicated by the right hand period feature amount data, and identifies the interval of the classified work elements. In this way, the classification unit 236 generates right hand classification data that indicates the interval of each work element identified from the right hand feature amount data.

[0100] Furthermore, the classification unit 236 classifies the work of the worker into work elements from the left hand feature amount data depending on whether the left hand feature amount data is included in the range indicated by the left hand period feature amount data, and identifies the interval of the classified work elements. In this way, the classification unit 236 generates left hand classification data that indicates the interval of each work element identified from the left hand feature amount data.

[0101] Furthermore, the classification unit 236 classifies the work of the worker into work elements from the right hand feature amount data and the left hand feature amount data depending on whether either the right hand feature amount data or the left hand feature amount data is included in the range indicated by the integrated periodic feature amount data, and specifies the intervals of the classified work elements. In this way, the classification unit 236 generates integrated classification data that indicates the intervals of each work element specified from the right hand feature amount data and the left hand feature amount data.

[0102] The evaluation unit 237 evaluates the time indicated by the right hand classification data, the time indicated by the left hand classification data, and the time indicated by the integrated classification data based on predetermined criteria, thereby generating right hand evaluation data, left hand evaluation data, and integrated evaluation data.

[0103] The display unit 238 displays at least one of the sensor data, right hand feature amount data, left hand feature amount data, right hand period feature data, left hand period feature data, integrated period feature data, right hand classification data, left hand classification data, integrated classification data, right hand evaluation data, left hand evaluation data, and integrated evaluation data.

[0104] As described above, according to the second embodiment, the work of a worker who works with his right hand and the work of a worker who works with his left hand can be analyzed easily and appropriately.

[0105] The work analysis devices 130, 230 described above perform both the learning phase and the analysis phase, but may perform only one of them. In this case, a device that performs only the analysis phase may store the periodic feature data generated by the learning phase, or may perform the analysis phase by downloading periodic feature data stored in another device. Furthermore, even during the learning phase, the periodic feature data may be stored in a memory unit of another device. In this case, the periodic feature data memory unit 135, 235 may not be provided.

[0106] 100, 200 Work analysis system, 110 Sensor, 130, 230 Work analysis device, 131 Communication unit, 132 Sensor data acquisition unit, 133, 233 Feature calculation unit, 134, 234 Periodic feature extraction unit, 135, 235 Periodic feature data storage unit, 136, 236 Classification unit, 137, 237 Evaluation unit, 138, 238 Display unit.

Claims

1. A sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by a sensor for each of one or more workers during work; a feature quantity calculation unit that calculates a plurality of feature quantities from each of the plurality of sensor data and generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in time series; and a periodic feature extraction unit that extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, specifies a range according to a predetermined rule so that the periodic feature quantity is included, and generates periodic feature data indicating the range. An information processing apparatus characterized by comprising the above.

2. The periodic feature quantity is the centroid of a cluster whose evaluation value calculated for each of the plurality of clusters exceeds a threshold value so that a higher evaluation is obtained as the number of feature quantities belonging to each of the plurality of clusters is larger when the plurality of feature quantities indicated by each of the plurality of feature quantity data are clustered into a plurality of clusters. The information processing apparatus according to claim 1, characterized by this.

3. The sensor data acquisition unit further acquires target sensor data indicating a detection result detected by a sensor for a worker to be analyzed; the feature quantity calculation unit calculates a plurality of feature quantities from the target sensor data and further generates target feature quantity data indicating the plurality of feature quantities calculated from the target sensor data; and a classification unit that classifies the element operations constituting the work of the worker to be analyzed into intervals separated by times when the plurality of feature quantities indicated by the target feature quantity data are included in the range indicated by the periodic feature data, and generates classification data indicating the time of the intervals classified for each element operation. The information processing apparatus according to claim 1 or 2, characterized by further comprising the above.

4. The information processing apparatus according to claim 3, further comprising an evaluation unit that evaluates the time for each element operation indicated by the classification data according to a predetermined criterion.

5. The information processing apparatus according to any one of claims 1 to 4, characterized in that the feature quantity calculation unit calculates the plurality of feature quantities indicating the positions of the parts of each worker's body.

6. The plurality of sensor data are a plurality of video data indicating videos of the one or more workers, the target sensor data is video data indicating a video of the worker to be analyzed, and the feature quantity calculation unit specifies, from each frame of the plurality of sensor data, an image around a specific part of the one or more workers as a first partial image, calculates a feature quantity of an object included in the first partial image, specifies, from a frame of the target sensor data, an image around a specific part of the worker to be analyzed as a second partial image, and calculates a feature quantity of an object included in the second partial image. The information processing apparatus according to claim 3 or 4, characterized in that.

7. The feature quantity calculation unit calculates a plurality of feature quantities related to the right hand from the plurality of sensor data, and generates right hand feature quantity data, which is the feature quantity data indicating the plurality of feature quantities calculated for the right hand in time series, and specifies a plurality of feature quantities related to the left hand from the plurality of sensor data, and generates left hand feature quantity data, which is the feature quantity data indicating the plurality of feature quantities calculated for the left hand in time series, thereby generating a plurality of right hand feature quantity data and a plurality of left hand feature quantity data. The periodic feature extraction unit extracts a right hand periodic feature quantity, which is the periodic feature quantity, from the plurality of right hand feature quantity data, specifies a right hand range, which is the range specified by the predetermined rule so as to include the right hand periodic feature quantity, and generates right hand periodic feature data, which is the periodic feature data indicating the right hand range. The periodic feature extraction unit extracts a left hand periodic feature quantity, which is the periodic feature quantity, from each of the plurality of left hand feature quantity data, specifies a left hand range, which is the range specified by the predetermined rule so as to include the left hand periodic feature quantity, and generates left hand periodic feature data, which is the periodic feature data indicating the left hand range. When the similarity between the right hand periodic feature data and the left hand periodic feature data exceeds a predetermined threshold value, integrated periodic feature data is generated by integrating the right hand periodic feature data and the left hand periodic feature data. The information processing apparatus according to claim 1, characterized in that.

8. The sensor data acquisition unit further acquires target sensor data which is the sensor data of the worker to be analyzed. The feature quantity calculation unit calculates a plurality of feature quantities related to the right hand from the target sensor data, and generates target right hand feature quantity data showing the plurality of feature quantities calculated for the right hand in time series, and calculates a plurality of feature quantities related to the left hand from the target sensor data, and generates target left hand feature quantity data showing the plurality of feature quantities calculated for the left hand in time series. The plurality of feature quantities included in the target right hand feature quantity data are used to classify the elemental operations constituting the operation of the worker to be analyzed into intervals delimited by times included in the range indicated by the right hand cycle feature data, and generate right hand classification data showing the times of the intervals classified for each elemental operation with respect to the right hand. The plurality of feature quantities included in the target left hand feature quantity data are used to classify the elemental operations of the worker to be analyzed into intervals delimited by times included in the range indicated by the left hand cycle feature data, and generate left hand classification data showing the times of the intervals classified for each elemental operation with respect to the left hand. The plurality of feature quantities included in the target right hand feature quantity data and the plurality of feature quantities included in the target left hand feature quantity data are used to classify the elemental operations of the worker to be analyzed into intervals delimited by times included in the range indicated by the integrated cycle feature data, and further include a classification unit that generates integrated classification data showing the times of the intervals classified for each elemental operation with respect to the left hand and the right hand. The information processing apparatus according to claim 7, characterized in that.

9. The information processing apparatus according to claim 8, further comprising an evaluation unit that evaluates each of the time indicated by the right hand classification data, the time indicated by the left hand classification data, and the time indicated by the integrated classification data according to a predetermined criterion.

10. A sensor data acquisition unit that acquires target sensor data indicating a detection result detected by a sensor for an operator to be analyzed; a feature quantity calculation unit that calculates a plurality of feature quantities from the target sensor data and generates target feature quantity data indicating the plurality of feature quantities calculated from the target sensor data; and a classification unit that calculates a plurality of feature quantities from each of a plurality of sensor data indicating detection results detected by a sensor for each body of one or more operators during work, generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in time series, extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, and classifies the element work constituting the work of the operator to be analyzed into intervals delimited by the times when the plurality of feature quantities indicated by the target feature quantity data are included in a range indicated by periodic feature data indicating a range specified by a predetermined rule so that the periodic feature quantity is included, and generates classification data indicating the time of the intervals classified for each element work. An information processing apparatus characterized by comprising:

11. A program characterized by causing a computer to function as a sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by a sensor for each body of one or more operators during work; a feature quantity calculation unit that calculates a plurality of feature quantities from each of the plurality of sensor data and generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in time series; and a periodic feature extraction unit that extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, specifies a range according to a predetermined rule so that the periodic feature quantity is included, and generates periodic feature data indicating the range.

12. A program characterized by causing a computer to function as a sensor data acquisition unit that acquires target sensor data indicating a detection result detected by a sensor regarding an operator to be analyzed, a feature amount calculation unit that calculates a plurality of feature amounts from the target sensor data and generates target feature amount data indicating the plurality of feature amounts specified from the target sensor data, and a classification unit that calculates a plurality of feature amounts from each of a plurality of sensor data indicating detection results detected by a sensor regarding each body of one or a plurality of operators during work, generates a plurality of feature amount data by generating feature amount data indicating the plurality of feature amounts in time series, extracts a periodic feature amount that is a periodically appearing feature amount from the plurality of feature amount data, classifies the element work constituting the work of the operator to be analyzed into sections delimited by times when the plurality of feature amounts indicated by the target feature amount data are included in a range indicated by periodic feature data indicating a range specified by a predetermined rule so as to include the periodic feature amount, and generates classification data indicating the time of the sections classified for each element work.

13. An information processing method characterized by acquiring a plurality of sensor data indicating detection results detected by a sensor regarding each body of one or a plurality of operators during work, calculating a plurality of feature amounts from the plurality of sensor data, generating a plurality of feature amount data by generating feature amount data indicating the plurality of feature amounts in time series, extracting a periodic feature amount that is a periodically appearing feature amount from the plurality of feature amount data, specifying a range by a predetermined rule so as to include the periodic feature amount, and generating periodic feature data indicating the range.

14. Obtain target sensor data indicating the detection results detected by sensors for the operator to be analyzed, calculate a plurality of feature amounts from the target sensor data, generate target feature amount data indicating the plurality of feature amounts specified from the target sensor data, calculate a plurality of feature amounts from a plurality of sensor data indicating the detection results detected by sensors for each body of one or more operators during work, generate feature amount data indicating the plurality of feature amounts in time series, thereby generating a plurality of feature amount data, extract a periodic feature amount which is a feature amount that appears periodically from the plurality of feature amount data, and classify the element operations constituting the work of the operator to be analyzed into intervals delimited by the times when the plurality of feature amounts indicated by the target feature amount data are included in the range indicated by the periodic feature data indicating the range specified by a predetermined rule so as to include the periodic feature amount, and generate classification data indicating the time of the intervals classified for each of the element operations. This is an information processing method characterized by the above.

Images