Information Processing Apparatus, Program, and Information Processing Method

Abstract

The work analysis device (130) includes a sensor data acquisition unit (132) that acquires sensor data indicating detection results detected by sensors regarding the body when one or more workers are performing work, a feature amount calculation unit (133) that calculates a plurality of feature amounts from the plurality of sensor data and generates feature amount data indicating the plurality of feature amounts in time series, a periodic feature extraction unit (134) that extracts a periodic feature amount, which is a feature amount that appears periodically, from the plurality of feature amount data, specifies a range according to a predetermined rule so that the periodic feature amount is included, and generates periodic feature data indicating the range, and a periodic feature data storage unit (135) that stores the periodic feature data.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a program, and an information processing method.

Background Art

[0002] Conventionally, at a work site where products are manufactured, the element operations that make up the work are grasped, and based on those element operations, the work of the worker is evaluated. For example, the work data management system described in Patent Document 1 classifies the work performed by a worker at a work site into a plurality of time-series detailed processes based on work data including video data of the work site, and by displaying the video data and the plurality of time-series detailed processes, enables a manager to grasp the work content at the work site.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, data indicating the positions through which the worker's body parts pass, etc. must be set in advance by a manager or the like, and the time and labor load on the manager or the like is large.

[0005] Therefore, one or more aspects of the present disclosure aim to make it easy to grasp the element operations that make up the work being performed by a worker.

Means for Solving the Problems

[0006] An information processing apparatus according to an aspect of the present disclosure includes: a sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by sensors for each body of one or more workers during work; a feature quantity calculation unit that calculates a plurality of feature quantities from each of the plurality of sensor data and generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in a time series; and a periodic feature extraction unit that extracts a periodic feature quantity, which is a feature quantity that appears periodically, from the plurality of feature quantity data, specifies a range according to a predetermined rule so that the periodic feature quantity is included, and generates periodic feature data indicating the range. , the sensor data acquisition unit further acquires target sensor data indicating a detection result detected by a sensor for an operator to be analyzed, the feature amount calculation unit calculates a plurality of feature amounts from the target sensor data, further generates target feature amount data indicating the plurality of feature amounts calculated from the target sensor data, and when the plurality of feature amounts indicated by the target feature amount data are included in the range indicated by the periodic feature data, the computer classifies the element operations constituting the operation of the operator to be analyzed into sections delimited by time, and further includes a classification unit that generates classification data indicating the time of the sections classified for each of the element operations It is characterized by the following.

[0007] An information processing apparatus according to an aspect of the present disclosure includes: a sensor data acquisition unit that acquires target sensor data indicating a detection result detected by a sensor for a worker to be analyzed; a feature quantity calculation unit that calculates a plurality of feature quantities from the target sensor data and generates target feature quantity data indicating the plurality of feature quantities specified from the target sensor data; and a classification unit that calculates a plurality of feature quantities from each of a plurality of sensor data indicating detection results detected by sensors for each body of one or more workers during work, generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in a time series, extracts a periodic feature quantity, which is a feature quantity that appears periodically, from the plurality of feature quantity data, classifies the element operations that constitute the work of the worker to be analyzed into sections delimited by the times when the plurality of feature quantities indicated by the target feature quantity data are included in the range indicated by the periodic feature data indicating the range specified according to a predetermined rule so that the periodic feature quantity is included, and generates classification data indicating the time of the sections classified for each element operation. It is characterized by including the above.

[0008] A program according to an aspect of the present disclosure causes a computer to function as a sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by a sensor for each body of one or more workers during work, a feature quantity calculation unit that calculates a plurality of feature quantities from each of the plurality of sensor data and generates feature quantity data indicating the plurality of feature quantities in a time series, thereby generating a plurality of feature quantity data, and a periodic feature extraction unit that extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, specifies a range according to a predetermined rule so that the periodic feature quantity is included, and generates periodic feature data indicating the range. , the sensor data acquisition unit further acquires target sensor data indicating a detection result detected by a sensor for an operator to be analyzed, the feature amount calculation unit calculates a plurality of feature amounts from the target sensor data, further generates target feature amount data indicating the plurality of feature amounts calculated from the target sensor data, and causes the computer to further function as a classification unit that classifies the element operations constituting the operation of the operator to be analyzed into sections delimited by time when the plurality of feature amounts indicated by the target feature amount data are included in the range indicated by the periodic feature data, and generates classification data indicating the time of the sections classified for each of the element operations It is characterized by the above.

[0009] A program according to an aspect of the present disclosure causes a computer to function as a sensor data acquisition unit that acquires target sensor data indicating detection results detected by a sensor for a worker to be analyzed, a feature quantity calculation unit that calculates a plurality of feature quantities from the target sensor data and generates target feature quantity data indicating the plurality of feature quantities calculated from the target sensor data, and a feature quantity calculation unit that calculates a plurality of feature quantities from each of a plurality of sensor data indicating detection results detected by a sensor for each body of one or more workers during work, generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in a time series, extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, and classifies the element operations constituting the work of the worker to be analyzed into sections divided by the times when the plurality of feature quantities indicated by the target feature quantity data are included in the range indicated by the periodic feature data indicating the range specified according to a predetermined rule so that the periodic feature quantity is included, and generates classification data indicating the time of the sections classified for each element operation.

[0010] An information processing method according to an aspect of the present disclosure The sensor data acquisition unit acquires a plurality of sensor data indicating detection results detected by a sensor for each body of one or more workers during work, The feature amount calculation unit Calculate a plurality of feature amounts from the plurality of sensor data, and generate feature amount data indicating the plurality of feature amounts in time series, thereby generating a plurality of feature amount data. The periodic feature extraction unit An information processing method for extracting a periodic feature amount, which is a periodically appearing feature amount, from the plurality of feature amount data, specifying a range according to a predetermined rule so that the periodic feature amount is included, and generating periodic feature data indicating the range. The sensor data acquisition unit Further obtain target sensor data indicating a detection result detected by a sensor for an operator to be analyzed. The feature amount calculation unit Calculate a plurality of feature amounts from the target sensor data, and further generate target feature amount data indicating the plurality of feature amounts calculated from the target sensor data. The classification unit Classify the element operations constituting the operation of the operator to be analyzed into intervals delimited by times when the plurality of feature amounts indicated by the target feature amount data are included in the range indicated by the periodic feature data, and further generate classification data indicating the time of the intervals classified for each element operation.

[0011] An information processing method according to one aspect of the present disclosure. The sensor data acquisition unit Obtain target sensor data indicating a detection result detected by a sensor for an operator to be analyzed. The feature amount calculation unit Calculate a plurality of feature amounts from the target sensor data, and generate target feature amount data indicating the plurality of feature amounts specified from the target sensor data. The classification unit Calculate a plurality of feature amounts from a plurality of sensor data indicating detection results detected by sensors for each body of one or more operators during work, and generate feature amount data indicating the plurality of feature amounts in time series, thereby generating a plurality of feature amount data. Extract a periodic feature amount, which is a periodically appearing feature amount, from the plurality of feature amount data, and classify the element operations constituting the operation of the operator to be analyzed into intervals delimited by times when the plurality of feature amounts indicated by the target feature amount data are included in the range indicated by the periodic feature data indicating the range specified according to a predetermined rule so that the periodic feature amount is included, and generate classification data indicating the time of the intervals classified for each element operation.

Effect of the Invention

[0012] According to one or more aspects of the present disclosure, it is possible to easily grasp the element operations that constitute the operation being performed by the operator.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0014] Embodiment 1. FIG. 1 is a block diagram schematically showing the configuration of a work analysis system 100 according to Embodiment 1. The work analysis system 100 includes a sensor 110 and a work analysis apparatus 130 as an information processing apparatus. The sensor 110 and the work analysis apparatus 130 are connected to a network 101 such as a LAN (Local Area Network).

[0015] The sensor 110 generates sensor data indicating a detection result detected by the sensor regarding the operator's body during work. Then, the sensor 110 transmits the sensor data to the work analysis apparatus 130 via the network 101. In this embodiment, the sensor 110 is a camera as an imaging device that captures an image of an operator, and transmits video data representing the image of the operator during work to the work analysis device 130 as sensor data.

[0016] FIG. 2 is a block diagram schematically showing the configuration of the work analysis device 130. The work analysis device 130 includes a communication unit 131, a sensor data acquisition unit 132, a feature amount calculation unit 133, a periodic feature extraction unit 134, a periodic feature data storage unit 135 as a storage unit, a classification unit 136, an evaluation unit 137, and a display unit 138.

[0017] The communication unit 131 communicates with the sensor 110 via the network 101. For example, the communication unit 131 receives sensor data from the sensor 110 via the network 101.

[0018] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101. The acquired sensor data is provided to the feature amount calculation unit 133.

[0019] The feature amount calculation unit 133 calculates a plurality of feature amounts from the sensor data and generates feature amount data showing the plurality of feature amounts in time series.

[0020] Here, the feature amount calculation unit 133 calculates the position of a body part of the operator as a feature amount. Specifically, the feature amount calculation unit 133 can calculate the position of the body part as a feature amount by inputting the sensor data into a learned deep learning model. Here, as the deep learning model, a model that infers the position of the body part from the video data may be used. Thereby, classification can be performed based on the movement of the body part. For example, when the work analysis system 100 learns the work of operators from 1 to W (W is an integer of 2 or more), the feature amount data of the operator w included in 1 to W is represented by the following formula (1).

[0021]

Number

[0022] Here, Tw is the number of time steps of the feature data of worker w, and the feature vector, which is the feature at time step t in the feature data x w is represented by the following formula (2).

[0023]

Number

[0024] In formula (2), d in “t, d” is the number indicating the dimension in the feature vector, and is an integer satisfying 1 ≦ d ≦ D (D is an integer of 1 or more). Note that D is the number of dimensions.

[0025] As described above, the feature calculation unit 133 calculates the feature data x 1 , x 2 , ···, x W from the sensor data measuring the work by different workers. The calculated feature data x 1 , x 2 , ···, x W is given to the periodic feature extraction unit 134 in the learning phase for calculating the periodic feature data. In the analysis phase for analyzing the work of the worker, the feature data x g of one worker g to be analyzed is given to the classification unit 136. Here, the work analysis device 130 first calculates the periodic feature data in the learning phase, and then analyzes the work of the worker in the analysis phase.

[0026] In the learning phase, the sensor 110 transmits sensor data for each of one or more workers, and the sensor data acquisition unit 132 acquires a plurality of sensor data. Then, the feature amount calculation unit 133 identifies a plurality of feature amounts from the plurality of sensor data, and generates a plurality of feature amount data by generating feature amount data indicating the plurality of feature amounts in time series. Then, the periodic feature extraction unit 134 extracts a periodic feature amount, which is a periodically appearing feature amount, from the plurality of feature amount data, specifies a range according to a predetermined rule so that the periodic feature amount is included, and generates periodic feature data indicating the range.

[0027] Here, the periodic feature amount is the centroid of the cluster whose evaluation value calculated for each of the plurality of clusters exceeds a threshold so that the higher the number of feature amounts belonging to each of the plurality of clusters when the plurality of feature amounts indicated by each of the plurality of feature amount data are clustered into a plurality of clusters, the higher the evaluation obtained. In other words, the periodic feature amount is a feature amount that aggregates in a certain area among the plurality of feature amounts indicated by the plurality of feature amount data and repeatedly appears in time series. This will be specifically described below.

[0028] For example, the periodic feature extraction unit 134 identifies a periodically appearing feature amount in a plurality of feature amount data x 1 , x 2 , ···, x W and calculates periodic feature data indicating the periodically appearing feature amount.

[0029] For example, the state data s w corresponding to the feature amount data x w of the worker w is expressed by the following formula (3).

[0030]

Equation

[0031] State data s wis a state value indicating the number of the cluster obtained by classifying the feature vector at time step t represented by the above equation (2). k is a positive integer satisfying 1 ≤ k ≤ K as shown in the following equation (4). Here, K is the total number of clusters.

[0032]

Equation

[0033] Here, the state data s 1 , s 2 , ···, s W can be calculated as the number of the cluster when classifying the features represented by the feature data x 1 , x 2 , ···, x W by the k-means method.

[0034] Next, the periodic feature extraction unit 134 specifies the center vector μ k of the feature vector, which is the feature corresponding to the cluster k, shown in the following equation (5).

[0035]

Equation

[0036] Here, μ k,d is the d-th value in μ k and is a real number. Note that μ k,d can be calculated as the centroid of the cluster k obtained by the k-means method.

[0037] Next, the periodic feature extraction unit 134 calculates the evaluation value r k of the cluster k. For example, the evaluation value r k is the state data s 1 , s 2 , ···, s WIt can be calculated as the reward value of cluster k inferred by inverse reinforcement learning using this.

[0038] Specifically, the evaluation value r k is the reward value r w of cluster k inferred by inverse reinforcement learning using the state data s k w of worker w, and can be obtained by the following formula (6).

[0039]

Equation

[0040] Here, according to formula (6), a cluster with a high reward value for all state data can be highly evaluated.

[0041] Also, the evaluation value r k is the state data s w of worker w. In s t w = k, based on the number of times N k w , it can also be obtained by the following formula (7) or (8).

[0042]

Equation

[0043]

Equation

[0044] In the above formula (7), in the state data s 1 , s 2 , ···, s W = k, the evaluation value r t w is calculated at the ratio of s k = k, and in formula (8), in the state data s 1 , s 2 , ···, s W = k, in st w The evaluation value r is the product of the ratios that result in = k k is calculated. With these equations, it is possible to highly evaluate the clusters that frequently appear in all state data.

[0045] From the above, the periodic feature extraction unit 134 can define the periodic feature data p j by the following equation (9).

[0046]

Equation

[0047] In equation (9), j is a number for identifying the periodic feature data and is a positive integer satisfying 1 ≤ j ≤ J. J is an integer of 1 or more representing the total number of the periodic feature data. σ j is the area of the feature vector in the periodic feature data p j In other words, in Embodiment 1, the center vector μ of the cluster k j and the area σ centered on the center vector μ j j specify the range.

[0048] Here, the center vector μ of the feature vector in the periodic feature data p j j can be calculated as μ k = μ j when the evaluation value r k exceeds a predetermined threshold value.

[0049] Also, the area σ of the feature vector in the periodic feature data p j j can be calculated by the following (i) or (ii) when the evaluation value r k exceeds a predetermined threshold value. (i) For the feature vector corresponding to the cluster k, it is calculated as three times the standard deviation in each dimension. ​​​​(ii) Calculate as a 95% confidence ellipse including the feature vector corresponding to cluster k.

[0050] Note that the periodic feature extraction unit 134 may reduce the number of dimensions D when determining the range. For example, when the feature vector has two dimensions of the position of the right hand and the position of the left hand, the dimension of the position of the left hand may be deleted to determine the range. In other words, when determining the range, the periodic feature extraction unit 134 may delete the dimensions that are not directly related to the operator's work from the number of dimensions D, and determine the range only with the dimensions that are directly related to the operator's work. For example, the range may be determined only with the dimensions related to the hands from the parts of the operator's body.

[0051] The periodic feature data calculated as described above will be explained. FIG. 3 is a graph for explaining the periodically appearing features in the one-dimensional feature data. As shown in FIG. 3, it is assumed that in this feature data, feature A, feature B, and feature C are determined to be periodically repeated. In this case, the periodic feature extraction unit 134 generates periodic feature data indicating the range corresponding to feature A, the range corresponding to feature B, and the range corresponding to feature C. In the case of the example of FIG. 3, features A, B, and C appear in the order of feature C, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, and feature C, and it can be recognized that the operator is performing a certain operation by the element operation using each of these features as a delimiter.

[0052] Returning to FIG. 1, the periodic feature data storage unit 135 stores the periodic feature data calculated as described above.

[0053] Next, in the analysis phase, the sensor 110 transmits sensor data regarding the worker to be analyzed, and the sensor data acquisition unit 132 acquires the sensor data of the worker as target sensor data. Then, the feature quantity calculation unit 133 generates feature quantity data as target feature data from the target sensor data. Then, the classification unit 136 classifies the elemental operations that make up the work of the worker to be analyzed into intervals delimited by the time when the plurality of feature quantities indicated by the target feature quantity data are included in the range indicated by the periodic feature data, and generates classification data indicating the time of the intervals classified for each elemental operation.

[0054] For example, in the analysis phase, the classification unit 136 determines whether the feature quantities indicated by the feature quantity data of the worker to be analyzed are included in the range indicated by the periodic feature data stored in the periodic feature data storage unit 135 by the feature quantity calculation unit 133. Then, the classification unit 136 identifies the intervals by delimiting with the time of the feature quantities included in the range indicated by the periodic feature data, and classifies the elemental operations that make up the work in the identified intervals. The elemental operations to be classified may be specified, for example, by a combination of features corresponding to the delimiters.

[0055] Then, the classification unit 136 generates classification data indicating the time of the intervals for each classified elemental operation. The classification data is given to the evaluation unit 137.

[0056] The evaluation unit 137 evaluates the time for each elemental operation indicated by the classification data according to a predetermined criterion. For example, for each elemental operation, an ideal working time is predetermined as a reference time, and the evaluation unit 137 performs an evaluation such that the closer the time for each elemental operation indicated by the classification data is to the reference time, the higher the evaluation. The evaluation data indicating the result of the evaluation performed in this way is given to the display unit 138.

[0057] The display unit 138 displays at least one of sensor data, feature amount data, periodic feature data, classification data, and evaluation data. For example, the display unit 138 may display the classification data superimposed on the sensor data, and in that case, the evaluation data may also be superimposed.

[0058] The work analysis device 130 described above can be realized by a computer such as the PC 10 shown in FIG. 4, for example. The PC 10 includes a storage 11 such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), a memory 12, a processor 13 such as a CPU (Central Processing Unit), a communication I / F (InterFace) 14 such as a NIC (Network Interface Card), an input I / F 15 such as a keyboard and a mouse, and a display 16.

[0059] For example, the periodic feature data storage unit 135 can be realized by the storage 11 or the memory 12. The sensor data acquisition unit 132, the feature amount calculation unit 133, the periodic feature extraction unit 134, the classification unit 136, and the evaluation unit 137 can be realized by the processor 13 loading the program stored in the storage 11 into the memory 12 and executing the program. The communication unit 131 can be realized by the communication I / F 14. The display unit 138 can be realized by the display 16.

[0060] The above program may be downloaded from a recording medium (not shown) via a reader / writer (not shown) or from a network 101 via the communication I / F 14 to the storage 11, and then loaded onto the memory 12 and executed by the processor 13. Alternatively, it may be directly loaded onto the memory 12 from a recording medium via a reader / writer or from a network 101 via the communication I / F 14 and executed by the processor 13. In other words, the program may be provided by a program product such as a recording medium.

[0061] FIG. 5 is a flowchart for explaining the operation of the work analysis device 130 in the learning phase.

[0062] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101 (S10). In the learning phase, a plurality of sensor data corresponding to one or more workers are acquired.

[0063] The feature quantity calculation unit 133 extracts a plurality of feature quantities from each of the plurality of sensor data, and generates feature quantity data indicating the plurality of feature quantities in time series (S11). Here, a plurality of feature quantity data are generated corresponding to the plurality of sensor data.

[0064] The periodic feature extraction unit 134 extracts a periodic feature quantity, which is a periodically appearing feature quantity, from the plurality of feature quantity data, and generates periodic feature data indicating the periodic feature quantity (S12).

[0065] The periodic feature data storage unit 135 stores the periodic feature data calculated as described above (S13).

[0066] FIG. 6 is a flowchart for explaining the operation of the work analysis device 130 in the analysis phase.

[0067] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101 (S20). In the analysis phase, sensor data of one worker to be analyzed (also referred to as the target worker) are acquired.

[0068] The feature quantity calculation unit 133 extracts a feature quantity from the sensor data, and generates feature quantity data indicating the feature quantity in time series (S21). The feature quantity data generated here is also referred to as target feature quantity data.

[0069] The classification unit 136 identifies intervals with the time when the feature amount indicated by the feature amount data is included in the range indicated by the periodic feature data as delimiters, and classifies the element operations that constitute the operation in the identified intervals (S22). Then, the classification unit 136 generates classification data indicating the time of the intervals for each classified operation element.

[0070] The evaluation unit 137 evaluates the time for each element operation indicated by the classification data according to a predetermined criterion (S23).

[0071] The display unit 138 displays at least one of the sensor data, the feature amount data, the periodic feature data, the classification data, and the evaluation data (S24).

[0072] As described above, according to the first embodiment, periodic feature data indicating a periodically appearing feature amount can be automatically generated from the feature amount data of the operator. Therefore, based on the element operations that constitute the operator's operation, the operator's operation can be easily analyzed.

[0073] In the first embodiment described above, the feature amount calculation unit 133 calculates the position of the part of the operator's body as the feature amount, but the first embodiment is not limited to such an example. For example, the feature amount calculation unit 133 may calculate the position of the operator's hand as the feature amount. In this case, a model that infers the position of the operator's hand from the sensor data may be used as the deep learning model.

[0074] Also, the feature amount calculation unit 133 may calculate the type of an object such as a tool or a part as the feature amount from an image around the operator's hand instead of the operator's body part. Specifically, the feature amount calculation unit 133 inputs the sensor data into the deep learning model to identify the position of the operator's hand, and identifies an image in a predetermined range including the identified hand position as a partial image. Then, the feature amount calculation unit 133 can calculate, as a feature amount, the type of object included in the specified partial image by inputting the specified partial image into the deep learning model. In this case, as the deep learning model, a model that infers the type of object included in the image from the image may be used. Thereby, classification can be performed based on the presence or absence of objects such as tools or parts held by hand, and the type of objects such as tools or parts held by hand.

[0075] In this case, the plurality of sensor data is a plurality of video data indicating videos of one or more workers captured, and the target sensor data is video data indicating a video of the worker to be analyzed captured. Then, the feature amount calculation unit 133 specifies, as a first partial image, an image around a specific part of one or more workers from each frame of the plurality of sensor data, and calculates the feature amount of the object included in the first partial image. In addition, the feature amount calculation unit 133 specifies, as a second partial image, an image around a specific part of the worker to be analyzed from the frame of the target sensor data, and calculates the feature amount of the object included in the second partial image.

[0076] Embodiment 2. In Embodiment 2, it is assumed that the worker performs work using their hands, and periodic feature data is acquired corresponding to both the case where the worker's dominant hand is the right hand and the case where it is the left hand, so that the work of the worker can be analyzed.

[0077] As shown in FIG. 1, the work analysis system 200 according to Embodiment 2 includes a sensor 110 and a work analysis device 230. The sensor 110 of the work analysis system 200 according to Embodiment 2 is the same as the sensor 110 of the work analysis system 100 according to Embodiment 1.

[0078] FIG. 2 is a block diagram schematically showing the configuration of the work analysis device 230 in Embodiment 2. The work analysis device 230 includes a communication unit 131, a sensor data acquisition unit 132, a feature quantity calculation unit 233, a periodic feature extraction unit 234, a periodic feature data storage unit 235, a classification unit 236, an evaluation unit 237, and a display unit 238.

[0079] The communication unit 131 and the sensor data acquisition unit 132 of the work analysis device 230 in Embodiment 2 are the same as those of the work analysis device 130 in Embodiment 1.

[0080] The feature quantity calculation unit 233 calculates the feature quantities of the right hand and the left hand from the sensor data, and generates right hand feature quantity data and left hand feature quantity data, which are feature quantity data showing the respective feature quantities in time series.

[0081] For example, in the learning phase, the feature quantity calculation unit 233 calculates a plurality of feature quantities related to the right hand from a plurality of sensor data, and generates right hand feature quantity data showing the plurality of feature quantities calculated for the right hand in time series. Further, the feature quantity calculation unit 233 calculates a plurality of feature quantities related to the left hand from a plurality of sensor data, and generates left hand feature quantity data showing the plurality of feature quantities calculated for the left hand in time series. In the learning phase, a plurality of right hand feature quantity data and a plurality of left hand feature quantity data are generated corresponding to one or a plurality of workers.

[0082] On the other hand, in the analysis phase, the feature quantity calculation unit 233 calculates a plurality of feature quantities related to the right hand from the target sensor data, and generates target right hand feature quantity data showing the plurality of feature quantities calculated for the right hand in time series. Further, the feature quantity calculation unit 233 calculates a plurality of feature quantities related to the left hand from the target sensor data, and generates target left hand feature quantity data showing the plurality of feature quantities calculated for the left hand in time series. The target sensor data is the sensor data detected for the worker to be analyzed as described above.

[0083] The calculated feature amount data is given to the periodic feature extraction unit 234 in the learning phase for calculating periodic feature data and to the classification unit 236 in the analysis phase for analyzing the operator's work, in the same manner as in the first embodiment.

[0084] The periodic feature extraction unit 234 extracts a right - hand periodic feature amount, which is a periodically appearing feature amount, from a plurality of right - hand feature amount data, specifies a right - hand range specified by a predetermined rule so as to include the right - hand periodic feature amount, and generates right - hand periodic feature data indicating the right - hand range. Further, the periodic feature extraction unit 234 extracts a left - hand periodic feature amount, which is a periodically appearing feature amount, from each of a plurality of left - hand feature amount data, specifies a left - hand range specified by a predetermined rule so as to include the left - hand periodic feature amount, and generates left - hand periodic feature data indicating the left - hand range. Furthermore, when the similarity between the right - hand periodic feature data and the left - hand periodic feature data exceeds a predetermined threshold value, the periodic feature extraction unit 234 generates integrated periodic feature data by integrating the right - hand periodic feature data and the left - hand periodic feature data. This will be specifically described below.

[0085] For example, the periodic feature extraction unit 234 specifies a periodically appearing feature amount in the right - hand feature amount data and calculates right - hand periodic feature data indicating the periodically appearing feature amount. The right - hand periodic feature data is represented by the following formula (10).

[0086]

Equation

[0087] Here, jright is a positive integer satisfying 1 ≦ jright ≦ Jright and is a number for identifying the right - hand periodic feature data. Jright is the total number of right - hand periodic feature data.

[0088] Similarly, the periodic feature extraction unit 234 specifies a periodically appearing feature amount in the left - hand feature amount data and calculates left - hand periodic feature data indicating the periodically appearing feature amount. The left - hand periodic feature data is represented by the following formula (11).

[0089]

Number

[0090] Here, jleft is a positive integer satisfying 1 ≤ jleft ≤ Jleft, and is a number for identifying left - hand periodic feature data. Jleft is the total number of left - hand periodic feature data.

[0091] In addition, the periodic feature extraction unit 234 calculates the similarity between the right - hand periodic feature data and the left - hand periodic feature data. The similarity may be, for example, the reciprocal of the Euclidean distance between the center vector of the right - hand periodic feature data and the center vector of the left - hand periodic feature data, or may be the overlapping rate of the ranges in the D - dimensional space by the right - hand periodic feature data and the left - hand periodic feature data.

[0092] Then, the periodic feature extraction unit 234 calculates integrated periodic feature data obtained by integrating those with high similarity between the right - hand periodic feature data and the left - hand periodic feature data. The integrated periodic feature data is represented by the following formula (12).

[0093]

Number

[0094] Here, jeither is a positive integer satisfying 1 ≤ jeither ≤ Jeither, and is a number for identifying the integrated periodic feature data. Jeither is the total number of the integrated periodic feature data.

[0095] For example, when the similarity between the right - hand periodic feature data and the left - hand periodic feature data exceeds a predetermined threshold value, the periodic feature extraction unit 234 can calculate the integrated periodic feature data by the following formula (13).

[0096]

Number

[0097] The cycle feature data storage unit 235 stores the right - hand cycle feature data, the left - hand cycle feature data, and the integrated cycle feature data calculated as described above.

[0098] The classification unit 236 classifies the elemental operations that make up the operation of the operator to be analyzed into intervals delimited by the times when the plurality of feature quantities included in the target right - hand feature quantity data are included in the range indicated by the right - hand cycle feature data, and generates right - hand classification data indicating the time of the intervals classified for each elemental operation with respect to the right hand. Also, the classification unit 236 classifies the elemental operations of the operator to be analyzed into intervals delimited by the times when the plurality of feature quantities included in the target left - hand feature quantity data are included in the range indicated by the left - hand cycle feature data, and generates left - hand classification data indicating the time of the intervals classified for each elemental operation with respect to the left hand. Further, the classification unit 236 classifies the elemental operations of the operator to be analyzed into intervals delimited by the times when the plurality of feature quantities included in the target right - hand feature quantity data and the plurality of feature quantities included in the target left - hand feature quantity data are included in the range indicated by the integrated cycle feature data, and generates integrated classification data indicating the time of the intervals classified for each elemental operation with respect to the left hand and the right hand.

[0099] For example, the classification unit 236 divides the operation of the operator from the right - hand feature quantity data into work elements for each work element according to whether the right - hand feature quantity data is included in the range indicated by the right - hand cycle feature data, and specifies the intervals of the divided work elements. Thereby, the classification unit 236 generates right - hand classification data indicating the intervals for each work element specified from the right - hand feature quantity data.

[0100] Also, the classification unit 236 divides the operation of the operator from the left - hand feature quantity data into work elements for each work element according to whether the left - hand feature quantity data is included in the range indicated by the left - hand cycle feature data, and specifies the intervals of the divided work elements. Thereby, the classification unit 236 generates left - hand classification data indicating the intervals for each work element specified from the left - hand feature quantity data.

[0101] Furthermore, the classification unit 236 classifies the operator's work from the right - hand feature amount data and the left - hand feature amount data for each work element based on whether either the right - hand feature amount data or the left - hand feature amount data is included in the range indicated by the integrated cycle feature data, and identifies the intervals of the classified work elements. Thereby, the classification unit 236 generates integrated classification data indicating the intervals for each work element specified from the right - hand feature amount data and the left - hand feature amount data.

[0102] The evaluation unit 237 evaluates each of the time indicated by the right - hand classification data, the time indicated by the left - hand classification data, and the time indicated by the integrated classification data according to a predetermined criterion. Thereby, the evaluation unit 237 generates right - hand evaluation data, left - hand evaluation data, and integrated evaluation data.

[0103] The display unit 238 displays at least one of the sensor data, the right - hand feature amount data, the left - hand feature amount data, the right - hand cycle feature data, the left - hand cycle feature data, the integrated cycle feature data, the right - hand classification data, the left - hand classification data, the integrated classification data, the right - hand evaluation data, the left - hand evaluation data, and the integrated evaluation data.

[0104] As described above, according to the second embodiment, the work of the operator working with the right hand and the work of the operator working with the left hand can be analyzed easily and appropriately.

[0105] The work analysis devices 130 and 230 described above perform both the processing in the learning phase and the processing in the analysis phase, but only one of them may be performed. In this case, in the device that performs only the analysis phase, it may store the cycle feature data generated by the processing in the learning phase, or may perform the processing in the analysis phase by downloading the cycle feature data stored in another device. Also, even in the learning phase, the cycle feature data may be stored in the storage unit of another device. In this case, the cycle feature data storage units 135 and 235 may not be provided.

Description of Reference Numerals

[0106] 100, 200 operation analysis system, 110 sensor, 130, 230 operation analysis device, 131 communication unit, 132 sensor data acquisition unit, 133, 233 feature quantity calculation unit, 134, 234 periodic feature extraction unit, 135, 235 periodic feature data storage unit, 136, 236 classification unit, 137, 237 evaluation unit, 138, 238 display unit.

Claims

1. A sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by a sensor for each body of one or more workers during work; A feature quantity calculation unit that calculates a plurality of feature quantities from each of the plurality of sensor data and generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in time series; A periodic feature extraction unit that extracts a periodic feature quantity, which is a periodically appearing feature quantity, from the plurality of feature quantity data, specifies a range according to a predetermined rule so that the periodic feature quantity is included, and generates periodic feature data indicating the range, and The sensor data acquisition unit further acquires target sensor data indicating a detection result detected by a sensor for a worker to be analyzed; The feature quantity calculation unit calculates a plurality of feature quantities from the target sensor data and further generates target feature quantity data indicating the plurality of feature quantities calculated from the target sensor data; A classification unit that classifies the element operations constituting the work of the worker to be analyzed into intervals delimited by times when the plurality of feature quantities indicated by the target feature quantity data are included in the range indicated by the periodic feature data, and further generates classification data indicating the time of the intervals classified for each element operation; An information processing apparatus characterized by the above.

2. The classification unit determines whether or not the target feature quantity data is included in the range indicated by the periodic feature data. The information processing apparatus according to claim 1, characterized by the above.

3. The periodic feature quantity is the centroid of a cluster in which the evaluation value calculated for each of the plurality of clusters exceeds a threshold so that a higher evaluation is obtained as the number of feature quantities belonging to each of the plurality of clusters is larger when the plurality of feature quantities indicated by each of the plurality of feature quantity data are clustered into a plurality of clusters. The information processing apparatus according to claim 1, characterized by the above.

4. The information processing apparatus according to claim 1, further comprising an evaluation unit that evaluates the time for each element operation indicated by the classification data according to a predetermined criterion. The information processing apparatus according to claim 1, characterized by the above.

5. The feature quantity calculation unit calculates the plurality of feature quantities indicating the positions of the parts of each body of the worker. The information processing apparatus according to any one of claims 1 to 4, characterized by the above.

6. The plurality of sensor data are a plurality of video data indicating videos of the one or more workers. The target sensor data is video data showing a video of the worker to be analyzed, The feature amount calculation unit identifies, as a first partial image, an image around a specific part of the one or more workers from each frame of the plurality of sensor data, calculates a feature amount of an object included in the first partial image, and identifies, as a second partial image, an image around a specific part of the worker to be analyzed from a frame of the target sensor data, and calculates a feature amount of an object included in the second partial image. The information processing apparatus according to any one of claims 1 to 4, characterized in that.

7. The feature amount calculation unit calculates a plurality of feature amounts related to the right hand from the plurality of sensor data, and right hand feature amount data which is the feature amount data showing the plurality of feature amounts calculated for the right hand in time series, and identifies a plurality of feature amounts related to the left hand from the plurality of sensor data, and generates left hand feature amount data which is the feature amount data showing the plurality of feature amounts calculated for the left hand in time series, thereby generating a plurality of right hand feature amount data and a plurality of left hand feature amount data. The periodic feature extraction unit extracts a right hand periodic feature amount which is the periodic feature amount from the plurality of right hand feature amount data, identifies a right hand range which is the range specified by the predetermined rule so that the right hand periodic feature amount is included, generates right hand periodic feature data which is the periodic feature data indicating the right hand range, extracts a left hand periodic feature amount which is the periodic feature amount from each of the plurality of left hand feature amount data, identifies a left hand range which is the range specified by the predetermined rule so that the left hand periodic feature amount is included, generates left hand periodic feature data which is the periodic feature data indicating the left hand range, and generates integrated periodic feature data by integrating the right hand periodic feature data and the left hand periodic feature data when the similarity between the right hand periodic feature data and the left hand periodic feature data exceeds a predetermined threshold. The information processing apparatus according to claim 1, characterized in that.

8. The sensor data acquisition unit further acquires target sensor data which is the sensor data of the worker to be analyzed. The feature quantity calculation unit calculates a plurality of feature quantities related to the right hand from the target sensor data, generates target right hand feature quantity data indicating the plurality of feature quantities calculated for the right hand in time series, and calculates a plurality of feature quantities related to the left hand from the target sensor data, and generates target left hand feature quantity data indicating the plurality of feature quantities calculated for the left hand in time series. The classification unit further includes: classifying the element operations constituting the operation of the operator to be analyzed into intervals separated by times when the plurality of feature quantities included in the target right hand feature quantity data are included in the range indicated by the right hand cycle feature data, and generating right hand classification data indicating the time of the intervals classified for each element operation with respect to the right hand; classifying the element operations constituting the operation of the operator to be analyzed into intervals separated by times when the plurality of feature quantities included in the target left hand feature quantity data are included in the range indicated by the left hand cycle feature data, and generating left hand classification data indicating the time of the intervals classified for each element operation with respect to the left hand; classifying the element operations constituting the operation of the operator to be analyzed into intervals separated by times when the plurality of feature quantities included in the target right hand feature quantity data and the plurality of feature quantities included in the target left hand feature quantity data are included in the range indicated by the integrated cycle feature data, and generating integrated classification data indicating the time of the intervals classified for each element operation with respect to the left hand and the right hand. The information processing apparatus according to claim 7, characterized in that.

9. The information processing apparatus further includes an evaluation unit that evaluates each of the time indicated by the right hand classification data, the time indicated by the left hand classification data, and the time indicated by the integrated classification data according to a predetermined criterion. The information processing apparatus according to claim 8, characterized in that.

10. A sensor data acquisition unit that acquires target sensor data indicating a detection result detected by a sensor for an operator to be analyzed; A feature quantity calculation unit that calculates a plurality of feature quantities from the target sensor data and generates target feature quantity data indicating the plurality of feature quantities calculated from the target sensor data. From each of a plurality of sensor data indicating detection results detected by a sensor for each body of one or more workers during work, a plurality of feature amounts are calculated, and by generating feature amount data indicating the plurality of feature amounts in time series, a plurality of feature amount data are generated. From the plurality of feature amount data, a periodic feature amount that is a periodically appearing feature amount is extracted, and the work of the worker to be analyzed is segmented into intervals at times when the plurality of feature amounts indicated by the target feature amount data are included in a range indicated by periodic feature data indicating a range specified by a predetermined rule so as to include the periodic feature amount, and a classification unit that generates classification data indicating the time of the intervals classified for each of the elemental operations is provided. An information processing apparatus characterized by the above.

11. A computer is configured to function as a sensor data acquisition unit that acquires a plurality of sensor data indicating detection results detected by a sensor for each body of one or more workers during work, a feature amount calculation unit that calculates a plurality of feature amounts from each of the plurality of sensor data and generates a plurality of feature amount data by generating feature amount data indicating the plurality of feature amounts in time series, and a periodic feature extraction unit that extracts a periodic feature amount that is a periodically appearing feature amount from the plurality of feature amount data, specifies a range according to a predetermined rule so as to include the periodic feature amount, and generates periodic feature data indicating the range. The sensor data acquisition unit further acquires target sensor data indicating a detection result detected by a sensor for a worker to be analyzed, the feature amount calculation unit calculates a plurality of feature amounts from the target sensor data and further generates target feature amount data indicating the plurality of feature amounts calculated from the target sensor data, and the computer is further configured to function as a classification unit that segments the work of the worker to be analyzed into intervals at times when the plurality of feature amounts indicated by the target feature amount data are included in the range indicated by the periodic feature data, and generates classification data indicating the time of the intervals classified for each of the elemental operations. A program characterized by the above.

12. A computer is configured to function as a sensor data acquisition unit that acquires target sensor data indicating a detection result detected by a sensor for a worker to be analyzed, A feature quantity calculation unit that calculates a plurality of feature quantities from the target sensor data and generates target feature quantity data indicating the plurality of feature quantities specified from the target sensor data, and By calculating a plurality of feature quantities from each of a plurality of sensor data indicating detection results detected by a sensor regarding the body of each of one or a plurality of workers during work, and generating feature quantity data indicating the plurality of feature quantities in a time series, a plurality of feature quantity data are generated. From the plurality of feature quantity data, a periodic feature quantity that is a periodically appearing feature quantity is extracted, and the work of the worker to be analyzed is divided into element operations that make up the work in an interval delimited by a time when the plurality of feature quantities indicated by the target feature quantity data are included in a range indicated by periodic feature data indicating a range specified by a predetermined rule, and functioning as a classification unit that generates classification data indicating the time of the interval classified for each element operation A program characterized by the above.

13. A sensor data acquisition unit acquires a plurality of sensor data indicating detection results detected by a sensor regarding the body of each of one or a plurality of workers during work, A feature quantity calculation unit calculates a plurality of feature quantities from the plurality of sensor data and generates a plurality of feature quantity data by generating feature quantity data indicating the plurality of feature quantities in a time series, A periodic feature extraction unit extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, specifies a range according to a predetermined rule so that the periodic feature quantity is included, and generates periodic feature data indicating the range. An information processing method, The sensor data acquisition unit further acquires target sensor data indicating a detection result detected by a sensor regarding a worker to be analyzed, The feature quantity calculation unit calculates a plurality of feature quantities from the target sensor data and further generates target feature quantity data indicating the plurality of feature quantities calculated from the target sensor data, The classification unit classifies the element operations that make up the work of the worker to be analyzed in an interval delimited by a time when the plurality of feature quantities indicated by the target feature quantity data are included in the range indicated by the periodic feature data, and further generates classification data indicating the time of the interval classified for each element operation An information processing method characterized by the above.

14. The sensor data acquisition unit acquires target sensor data indicating a detection result detected by a sensor for an operator to be analyzed, the feature quantity calculation unit calculates a plurality of feature quantities from the target sensor data, and generates target feature quantity data indicating the plurality of feature quantities specified from the target sensor data, the classification unit calculates a plurality of feature quantities from a plurality of sensor data indicating detection results detected by a sensor for each body of one or a plurality of operators during work, and generates feature quantity data indicating the plurality of feature quantities in time series, thereby generating a plurality of feature quantity data, extracts a periodic feature quantity that is a periodically appearing feature quantity from the plurality of feature quantity data, and divides the work of the operator to be analyzed into element operations that make up the work in intervals delimited by the times when the plurality of feature quantities indicated by the target feature quantity data are included in a range indicated by periodic feature data indicating a range specified by a predetermined rule so as to include the periodic feature quantity, and generates classification data indicating the time of the intervals classified for each of the element operations is an information processing method characterized by the above.

Images