Information processing apparatus, program, and information processing method

Abstract

The work analysis device (130) includes a sensor data acquisition unit (132) that acquires sensor data representing detection results detected by sensors with respect to a body of one or more workers while the workers are performing work; a feature quantity calculation unit (133) that calculates a plurality of feature quantities from a plurality of pieces of sensor data, generates feature quantity data representing the plurality of feature quantities in a time series; a periodic feature extraction unit (134) that extracts a periodic feature quantity that periodically appears from a plurality of pieces of feature quantity data, determines a range according to a predetermined rule in a manner that the periodic feature quantity is included, and generates periodic feature data representing the range; and a periodic feature data storage unit (135) that stores the periodic feature data.

Description

Technical Field

[0001] This disclosure relates to information processing apparatus, programs, and methods. Background Technology

[0002] In the past, on the manufacturing site, the key elements of the work were identified, and the workers' performance was evaluated based on these key elements.

[0003] For example, the work data management system described in Patent Document 1 classifies the work performed by the operator at the work site into multiple time series detailed procedures based on work data including image data of the work site, and displays the image data and multiple time series detailed procedures, thereby enabling the manager to grasp the work content at the work site.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-16226 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, in existing technologies, managers must pre-set data indicating the positions through which the operator's body parts pass, which places a heavy burden on managers in terms of both time and labor.

[0009] Therefore, the purpose of one or more methods of this disclosure is to enable easy access to the elements of the work that constitutes the work being performed by the operator.

[0010] Methods for solving problems

[0011] One aspect of the information processing apparatus disclosed herein is characterized in that the apparatus comprises: a sensor data acquisition unit that acquires a plurality of sensor data, the plurality of sensor data representing detection results detected by sensors regarding the bodies of one or more workers during operation; a feature quantity calculation unit that calculates a plurality of feature quantities based on each of the plurality of sensor data, generating feature quantity data representing the plurality of feature quantities in a time series, thereby generating a plurality of feature quantity data; and a periodic feature extraction unit that extracts periodically occurring feature quantities, i.e., periodic feature quantities, from the plurality of feature quantity data, determines a range according to a predetermined rule in a manner that includes the periodic feature quantities, and generates periodic feature data representing the range.

[0012] An information processing apparatus according to one aspect of this disclosure is characterized in that the information processing apparatus comprises: a sensor data acquisition unit that acquires object sensor data, which represents detection results detected by sensors regarding an operator who is the object of analysis; a feature quantity calculation unit that calculates a plurality of feature quantities based on the object sensor data and generates object feature quantity data representing the plurality of feature quantities determined based on the object sensor data; and a classification unit that calculates the plurality of feature quantities based on each of the multiple sensor data representing detection results detected by sensors regarding the bodies of one or more operators during work, generates feature quantity data representing the plurality of feature quantities in a time series, thereby generating the plurality of feature quantity data, extracting periodically occurring feature quantities, i.e., periodic feature quantities, from the plurality of feature quantity data, classifying the element work constituting the operator who is the object of analysis into intervals divided by time, the time being the time when the plurality of feature quantities represented by the object feature quantity data are contained in a range, the range being the range represented by the periodic feature data representing the range determined according to a predetermined rule in a manner that includes the periodic feature quantities, and the classification unit generating classification data representing the time of the intervals classified according to each element work.

[0013] One aspect of the program disclosed herein is characterized by enabling a computer to function as: a sensor data acquisition unit that acquires multiple sensor data representing detection results detected by sensors on the bodies of one or more workers during operation; a feature quantity calculation unit that calculates multiple feature quantities based on each of the multiple sensor data, generating feature quantity data representing the multiple feature quantities in a time series, thereby generating multiple feature quantity data; and a periodic feature extraction unit that extracts periodically occurring feature quantities, i.e., periodic feature quantities, from the multiple feature quantity data, determines a range according to predetermined rules in a manner that includes the periodic feature quantities, and generates periodic feature data representing the range.

[0014] One aspect of the program disclosed herein is characterized by a computer functioning as follows: a sensor data acquisition unit that acquires object sensor data representing detection results detected by sensors regarding an operator who is the object of analysis; a feature quantity calculation unit that calculates multiple feature quantities based on the object sensor data, generating object feature quantity data representing the multiple feature quantities calculated based on the object sensor data; and a classification unit that calculates multiple feature quantities based on individual sensor data from multiple sensor data representing detection results detected by sensors regarding the bodies of one or more operators during work, generating feature quantity data representing the multiple feature quantities in a time series, thereby generating multiple feature quantity data, extracting periodically occurring feature quantities, i.e., periodic feature quantities, from the multiple feature quantity data, classifying the element work constituting the operator who is the object of analysis into intervals divided by time, the time being when the multiple feature quantities represented by the object feature quantity data are contained within a range, the range being represented by the periodic feature data representing the range determined according to predetermined rules in a manner that includes the periodic feature quantities, the classification unit generating classification data representing the time intervals classified according to each element work.

[0015] One aspect of the information processing method disclosed herein is characterized by acquiring multiple sensor data representing detection results detected by sensors on the bodies of one or more workers during operation; calculating multiple feature quantities based on the multiple sensor data; generating feature quantity data representing the multiple feature quantities in a time series; extracting periodically occurring feature quantities, i.e., periodic feature quantities, from the multiple feature quantity data; determining a range according to predetermined rules in a manner that includes the periodic feature quantities; and generating periodic feature data representing the range.

[0016] One aspect of the information processing method disclosed herein is characterized by: acquiring object sensor data, which represents detection results detected by sensors regarding an operator who is the object of analysis; calculating multiple feature quantities based on the object sensor data; generating object feature quantity data representing the multiple feature quantities determined based on the object sensor data; calculating multiple feature quantities based on multiple sensor data representing detection results detected by sensors regarding the bodies of one or more operators during work; generating feature quantity data representing the multiple feature quantities in a time series; thereby generating multiple feature quantity data; extracting periodically occurring feature quantities, i.e., periodic feature quantities, from the multiple feature quantity data; classifying the element work constituting the operator who is the object of analysis into intervals divided by time, where the time is a range in which the multiple feature quantities represented by the object feature quantity data are contained, and the range represented by the periodic feature data representing the range determined according to predetermined rules in a manner that includes the periodic feature quantities; the classification unit generating classification data representing the time intervals classified according to each element work.

[0017] Invention Effects

[0018] According to one or more methods of this disclosure, it is possible to easily grasp the elements of the work that constitutes the work being performed by the operator. Attached Figure Description

[0019] Figure 1 This is a block diagram that roughly illustrates the structure of the job analysis system in embodiments 1 and 2.

[0020] Figure 2 This is a block diagram that schematically illustrates the structure of the job analysis device in embodiments 1 and 2.

[0021] Figure 3 It is a chart used to illustrate the periodicity of features in one-dimensional feature data.

[0022] Figure 4 It is a block diagram schematically showing the structure of a PC.

[0023] Figure 5 This is a flowchart illustrating the actions during the learning phase of the job analysis device.

[0024] Figure 6 It is a flowchart used to illustrate the actions of the job analysis device during the analysis phase. Detailed Implementation

[0025] Implementation method 1.

[0026] Figure 1This is a block diagram that schematically illustrates the structure of the job analysis system 100 of Embodiment 1.

[0027] The job analysis system 100 includes a sensor 110 and a job analysis device 130 as an information processing device.

[0028] Sensor 110 and job analysis device 130 are connected to network 101 such as LAN (Local Area Network).

[0029] Sensor 110 generates sensor data that represents the detection results of the sensor on the worker's body during the operation.

[0030] Then, sensor 110 sends the sensor data to job analysis device 130 via network 101.

[0031] In this embodiment, sensor 110 is a camera that acts as a camera device for capturing images of the worker, and sends image data representing the worker's image during the operation to the job analysis device 130 as sensor data.

[0032] Figure 2 This is a block diagram that roughly shows the structure of the job analysis device 130.

[0033] The job analysis device 130 includes a communication unit 131, a sensor data acquisition unit 132, a feature calculation unit 133, a periodic feature extraction unit 134, a periodic feature data storage unit 135 as a storage unit, a classification unit 136, an evaluation unit 137, and a display unit 138.

[0034] The communication unit 131 communicates with the sensor 110 via the network 101.

[0035] For example, the communication unit 131 receives sensor data from the sensor 110 via the network 101.

[0036] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101.

[0037] The acquired sensor data is provided to the feature calculation unit 133.

[0038] The feature calculation unit 133 calculates multiple feature quantities based on sensor data and generates feature quantity data that represents the multiple feature quantities in a time series.

[0039] Here, the feature quantity calculation unit 133 calculates the position of the operator's body parts as feature quantities.

[0040] Specifically, the feature calculation unit 133 can calculate the position of body parts as features by inputting sensor data into a learned deep learning model. Here, a model that infers the position of body parts based on image data can be used as the deep learning model. Thus, classification can be performed based on the movement of body parts.

[0041] For example, when the job analysis system 100 learns the jobs of workers from 1 to W (W is an integer greater than 2), the characteristic data of the w workers included in 1 to W are represented by the following equation (1).

[0042] [Mathematical Expression 1] (1)

[0044] Here, Tw is the time step of the feature data of operator w, and the feature data x w The feature quantity, i.e. the feature vector, in time step t is represented by the following equation (2).

[0045] [Mathematical Expression 2] (2)

[0047] (2) In the formula, d in “t, d” is the number of the dimension in the feature vector, which is an integer that satisfies 1≤d≤D (D is an integer greater than 1). In addition, D is the number of dimensions.

[0048] As described above, the feature calculation unit 133 calculates feature data x based on sensor data obtained from measuring the work of different operators. 1 x 2 ... x W .

[0049] Calculated feature data x 1 x 2 ... x W The periodic feature data is provided to the periodic feature extraction unit 134 during the learning phase of calculating periodic feature data.

[0050] In addition, during the analysis phase of the operator's work, the characteristic data x of an operator g will become the object of analysis. g Provided to the classification department 136.

[0051] Here, the job analysis device 130 first calculates the periodic characteristic data in the learning phase, and then analyzes the operator's job in the analysis phase.

[0052] During the learning phase, sensor 110 sends sensor data to each of one or more operators, thereby acquiring multiple sensor data by sensor data acquisition unit 132. Then, feature quantity calculation unit 133 determines multiple feature quantities from the multiple sensor data and generates feature quantity data that represents the multiple feature quantities in a time series, thereby generating multiple feature quantity data.

[0053] Then, the periodic feature extraction unit 134 extracts periodically occurring feature quantities, i.e., periodic feature quantities, from multiple feature quantity data, and generates periodic feature data representing the range by determining the range according to a predetermined rule in a manner that includes the periodic feature quantity.

[0054] Here, the periodic feature is the centroid of the cluster whose evaluation value exceeds a threshold. When clustering multiple features represented by multiple feature data into multiple clusters, the evaluation value is calculated separately for each cluster, with higher evaluation values ​​given the greater number of features belonging to each cluster. In other words, the periodic feature is a feature that aggregates within the region represented by multiple feature data and appears repeatedly in the time series. This will be explained in detail below.

[0055] For example, the periodic feature extraction unit 134 extracts multiple feature data x 1 x 2 ... x W In this process, we determine the periodically occurring characteristic quantities and calculate the periodic characteristic data representing these periodically occurring characteristic quantities.

[0056] For example, the feature data x of operator w w Corresponding state data s w Expressed as in equation (3) below.

[0057] [Mathematical Expression 3] (3)

[0059] State data s w It represents the state value of the cluster number after classifying the feature vectors in time step t shown in equation (2) above.

[0060] As shown in equation (4) below, k is a positive integer that satisfies 1 ≤ k ≤ K. Here, K is the total number of clusters.

[0061] [Mathematical Expression 4] (4)

[0063] Here, regarding the state data s1 s 2 、…、s W It can be used as a method to analyze feature data x using the k-means method. 1 x 2 ... x W The cluster numbers are used to calculate the clusters when classifying the features shown.

[0064] Next, the periodic feature extraction unit 134 determines the center vector μ of the feature quantity corresponding to cluster k, as shown in equation (5) below. k .

[0065] [Mathematical Expression 5] (5)

[0067] Here, μ k,d It is μ k The d-th value in the array is a real number.

[0068] In addition, μ k,d It can be used as the centroid of cluster k obtained by the k-means method for calculation.

[0069] Next, the periodic feature extraction unit 134 calculates the evaluation value r of cluster k. k .

[0070] For example, the evaluation value r k Can be used as usage status data s 1 s 2 、…、s W The reward value is calculated by inferring the cluster k from inverse reinforcement learning.

[0071] Specifically, the evaluation value r k Able to use the status data s of operator w w The reward value r of cluster k derived through inverse reinforcement learning k w It can be obtained by the following equation (6).

[0072] [Mathematical Expression 6] (6)

[0074] Here, through equation (6), it is possible to evaluate clusters that have obtained high reward values ​​in all state data.

[0075] In addition, regarding the evaluation value r k It can also be based on the state data s of the operator w. w In the middle becomes s tw = k times N k w It can be obtained by formula (7) or (8) below.

[0076] [Mathematical Expression 7] (7)

[0078] [Mathematical Expression 8] (8)

[0080] In equation (7) above, in the state data s 1 s 2 、…、s W In the middle, to become s t w = k proportion to calculate evaluation value r k In equation (8), the state data s 1 s 2 、…、s W In the middle, to become s t w The evaluation value r is calculated by multiplying the proportions of k. k These formulas can provide a high degree of evaluation for clusters that occur frequently in all state data.

[0081] Based on the above, the periodic feature extraction unit 134 can determine the periodic feature data p by the following equation (9). j .

[0082] [Mathematical Expression 9] (9)

[0084] In equation (9), j is the number used to identify periodic feature data, and is a positive integer satisfying 1≤j≤J. J is an integer greater than 1 representing the total number of periodic feature data.

[0085] σ j It is periodic characteristic data p j The region of the feature vectors in the text.

[0086] In other words, in implementation 1, the center vector μ of cluster k is used... j and with the central vector μ j The central region σ j To determine the scope.

[0087] Here, the periodic characteristic data p j The center vector μ of the feature vectors in j In the evaluation value rk When the predetermined threshold is exceeded, it can be used as μ. j =μ k And calculation.

[0088] In addition, periodic characteristic data p j The region σ of the feature vector in j In the evaluation value r k When the predetermined threshold is exceeded, it can be calculated using either (i) or (ii) below.

[0089] (i) The feature vector corresponding to cluster k is calculated as three times the standard deviation of each dimension.

[0090] (ii) Calculate the 95% confidence ellipse containing the feature vectors corresponding to cluster k.

[0091] In addition, the periodic feature extraction unit 134 can also reduce the number of dimensions D when determining the range.

[0092] For example, if the feature vector has two dimensions, the right-hand position and the left-hand position, the left-hand position dimension can be removed to determine the range.

[0093] In other words, the periodic feature extraction unit 134 can also remove dimensions that are not directly related to the operator's work from the dimension number D when determining the range, and determine the range only based on the dimensions that are directly related to the operator's work. For example, the range can be determined only based on the dimensions related to the hands among the operator's body parts.

[0094] The periodic characteristic data calculated as described above will be explained.

[0095] Figure 3 It is a chart used to illustrate the periodicity of features in one-dimensional feature data.

[0096] like Figure 3 As shown, in this feature data, it is assumed that features A, B, and C repeat periodically.

[0097] In this case, the periodic feature extraction unit 134 generates periodic feature data representing the range corresponding to feature A, the range corresponding to feature B, and the range corresponding to feature C.

[0098] exist Figure 3 In the example case, features A, B, and C appear in the order of feature C, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, feature A, feature B, and feature C. By using these features as the elements of the division, it is possible to identify that the operator is performing a certain task.

[0099] return Figure 1 The periodic feature data storage unit 135 stores the periodic feature data calculated as described above.

[0100] Next, during the analysis phase, sensor 110 sends sensor data to the operator who is the object of analysis, thereby the sensor data acquisition unit 132 acquires the operator's sensor data as object sensor data. Then, the feature calculation unit 133 generates feature data as object feature data based on the object sensor data.

[0101] Then, the classification unit 136 classifies the element operations of the operator's operations that constitute the analysis object into intervals divided by time, where the time is the time when multiple feature quantities represented by object feature quantity data are contained in the range represented by periodic feature data. The classification unit 136 generates classification data that represents the time interval of each element operation.

[0102] For example, during the analysis phase, the classification unit 136 determines, through the feature quantity calculation unit 133, whether the feature quantity data of the operator being analyzed is included within the range of the periodic feature data stored in the periodic feature data storage unit 135.

[0103] Then, the classification unit 136 determines the interval by dividing the time of the characteristic quantity contained in the range represented by the periodic characteristic data, and classifies the elements constituting the operation into the determined interval.

[0104] The elements to be classified can be determined, for example, by a combination of features corresponding to the classification.

[0105] Then, the classification department 136 generates classification data representing the time interval of each element operation being classified.

[0106] The classification data was provided to the evaluation department 137.

[0107] Evaluation Unit 137 evaluates the time required for each element of the operation, represented by the categorical data, based on a predetermined benchmark.

[0108] For example, for each element's task, an ideal task time is predetermined as a baseline time. The evaluation unit 137 evaluates the task in a way that the closer the task time represented by the classification data is to the baseline time, the higher the evaluation. Evaluation data representing the results of this evaluation is provided to the display unit 138.

[0109] Display unit 138 displays at least one of sensor data, feature data, periodic feature data, classification data, and evaluation data.

[0110] For example, the display unit 138 can overlay classification data on sensor data, and in this case, evaluation data can also be overlaid.

[0111] The work analysis device 130 described above, for example, can be used through... Figure 4 This can be achieved using a computer like the PC10 shown.

[0112] PC10 includes storage devices such as HDD (Hard Disk Drive) or SSD (Solid State Drive) 11, memory 12, processors such as CPU (Central Processing Unit) 13, communication I / F (Interface) such as NIC (Network Interface Card) 14, input I / F such as keyboard and mouse 15, and a display 16.

[0113] For example, the periodic feature data storage unit 135 can be implemented by the storage unit 11 or the memory unit 12.

[0114] The sensor data acquisition unit 132, feature calculation unit 133, periodic feature extraction unit 134, classification unit 136, and evaluation unit 137 can be implemented by the processor 13 loading the program stored in the memory 11 into the memory 12 and executing the program.

[0115] The communication unit 131 can be implemented via communication I / F14.

[0116] The display unit 138 can be implemented via the display 16.

[0117] The above program can also be downloaded to storage 11 via a reader / writer (not shown) or a recording medium (not shown), or via communication I / F 14 from network 101, and then loaded onto memory 12 for execution by processor 13. Alternatively, it can be loaded directly onto memory 12 via a reader / writer from a recording medium or via communication I / F 14 from network 101 for execution by processor 13.

[0118] In other words, programs can also be provided through program products such as recording media.

[0119] Figure 5 This is a flowchart illustrating the actions during the learning phase of the job analysis device 130.

[0120] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101 (S10). During the learning phase, multiple sensor data corresponding to one or more operators are acquired.

[0121] The feature calculation unit 133 extracts multiple features from each of the multiple sensor data and generates feature data representing the multiple features in a time series (S11). Here, multiple feature data are generated corresponding to the multiple sensor data.

[0122] The periodic feature extraction unit 134 extracts periodically occurring feature quantities, i.e. periodic feature quantities, from multiple feature quantity data and generates periodic feature data representing the periodic feature quantity (S12).

[0123] The periodic feature data storage unit 135 stores the periodic feature data calculated as described above (S13).

[0124] Figure 6 This is a flowchart illustrating the actions during the analysis phase of the job analysis device 130.

[0125] The sensor data acquisition unit 132 acquires sensor data from the sensor 110 via the communication unit 131 and the network 101 (S20). During the analysis phase, sensor data of an operator (also known as the target operator) that is the subject of the analysis is acquired.

[0126] The feature calculation unit 133 extracts feature quantities from the sensor data and generates feature quantity data that represents the feature quantity in a time series (S21). The feature quantity data generated here is also called object feature quantity data.

[0127] The classification unit 136 uses the time when the feature quantity represented by the feature quantity data is contained within the range represented by the periodic feature data as a division, determines the interval, and classifies the elements constituting the operation into the determined interval (S22). Then, the classification unit 136 generates classification data representing the time interval of each classified operation element.

[0128] Evaluation unit 137 evaluates the time of operation for each element represented by the classification data based on a predetermined benchmark (S23).

[0129] Display unit 138 displays at least one of sensor data, feature quantity data, periodic feature data, classification data, and evaluation data (S24).

[0130] As described above, according to Embodiment 1, periodic feature data representing periodically occurring feature quantities can be automatically generated based on the operator's feature quantity data. Therefore, the operator's work can be easily analyzed based on the element work that constitutes the operator's work.

[0131] In the above-described embodiment 1, the feature quantity calculation unit 133 calculates the position of the operator's body parts as a feature quantity, but embodiment 1 is not limited to such an example.

[0132] For example, the feature calculation unit 133 can also calculate the position of the worker's hand as a feature. In this case, as a deep learning model, a model that infers the position of the worker's hand based on sensor data can be used.

[0133] Furthermore, the feature quantity calculation unit 133 may also use the image around the operator's hand, or the type of object such as a tool or component, as a feature quantity, instead of the operator's body parts.

[0134] Specifically, the feature calculation unit 133 determines the position of the operator's hand by inputting sensor data into a deep learning model, and defines an image containing a predetermined range of the determined hand position as a partial image.

[0135] Then, the feature calculation unit 133 inputs the determined partial image into the deep learning model and is able to calculate the category of the object contained in that partial image as a feature. In this case, as the deep learning model, a model that infers the type of object contained in the image based on the image can be used. Thus, classification can be performed based on the presence or absence of objects such as handheld tools or parts, and the category of such objects.

[0136] In this case, multiple sensor data refers to multiple image data representing images of one or more workers, and object sensor data refers to image data representing images of workers who become the objects of analysis.

[0137] Then, the feature calculation unit 133 determines the image surrounding a specific part of one or more operators as a first part image based on the frames of data from multiple sensors, and calculates the feature quantities of the objects contained in the first part image.

[0138] Furthermore, based on the frame of the object sensor data, the feature quantity calculation unit 133 determines the image surrounding a specific part of the operator that will be the object of analysis as the second part image, and calculates the feature quantity of the object contained in the second part image.

[0139] Implementation method 2.

[0140] In Implementation 2, the operator is assumed to perform the work by hand, and periodic feature data is obtained corresponding to the cases where the operator's dominant hand is the right hand and the cases where the operator's dominant hand is the left hand, so that the operator's work can be analyzed.

[0141] like Figure 1 As shown, the job analysis system 200 of Embodiment 2 includes a sensor 110 and a job analysis device 230.

[0142] The sensor 110 of the job analysis system 200 in Embodiment 2 is the same as the sensor 110 of the job analysis system 100 in Embodiment 1.

[0143] Figure 2 This is a block diagram that schematically shows the structure of the job analysis device 230 in Embodiment 2.

[0144] The job analysis device 230 includes a communication unit 131, a sensor data acquisition unit 132, a feature quantity calculation unit 233, a periodic feature extraction unit 234, a periodic feature data storage unit 235, a classification unit 236, an evaluation unit 237, and a display unit 238.

[0145] The communication unit 131 and sensor data acquisition unit 132 of the job analysis device 230 in Embodiment 2 are the same as those of the job analysis device 130 in Embodiment 1.

[0146] The feature calculation unit 233 calculates the right-hand feature and the left-hand feature based on the sensor data, and generates right-hand feature data and left-hand feature data as feature data of each feature in time series.

[0147] For example, during the learning phase, the feature calculation unit 233 calculates multiple feature quantities for the right hand based on multiple sensor data, and generates right-hand feature quantity data representing the multiple feature quantities calculated for the right hand as a time series. Furthermore, the feature calculation unit 233 calculates multiple feature quantities for the left hand based on multiple sensor data, and generates left-hand feature quantity data representing the multiple feature quantities calculated for the left hand as a time series.

[0148] In addition, during the learning phase, multiple right-handed feature data and multiple left-handed feature data are generated for one or more operators.

[0149] On the other hand, during the analysis phase, the feature calculation unit 233 calculates multiple feature quantities related to the right hand based on the object sensor data, and generates object right-hand feature quantity data that represents the multiple feature quantities calculated for the right hand in a time series format. Furthermore, the feature calculation unit 233 calculates multiple feature quantities related to the left hand based on the object sensor data, and generates object left-hand feature quantity data that represents the multiple feature quantities calculated for the left hand in a time series format. As described above, the object sensor data is sensor data detected by the operator who is the object of analysis.

[0150] The calculated feature data is provided to the periodic feature extraction unit 234 during the learning phase of calculating periodic feature data, and to the classification unit 236 during the analysis phase of analyzing the operator's work, in the same manner as in Implementation 1.

[0151] The periodic feature extraction unit 234 extracts right-hand periodic features that are periodically occurring features from multiple right-handed feature data, determines a right-hand range defined by a predetermined rule in a manner that includes the right-hand periodic feature, and generates right-hand periodic feature data representing the right-hand range. Furthermore, the periodic feature extraction unit 234 extracts left-hand periodic features that are periodically occurring features from each of multiple left-handed feature data, determines a left-hand range defined by a predetermined rule in a manner that includes the left-hand periodic feature, and generates left-hand periodic feature data representing the left-hand range. Moreover, when the similarity between the right-hand periodic feature data and the left-hand periodic feature data exceeds a predetermined threshold, the periodic feature extraction unit 234 generates comprehensive periodic feature data by combining the right-hand periodic feature data and the left-hand periodic feature data. This will be explained in detail below.

[0152] For example, the periodic feature extraction unit 234 determines the periodically occurring feature quantity in the right-hand feature quantity data, and calculates the right-hand periodic feature data representing the periodically occurring feature quantity. The right-hand periodic feature data is represented by the following equation (10).

[0153] [Mathematical Expression 10] (10)

[0155] Here, jright is a positive integer satisfying 1 ≤ jright ≤ Jright, and is the number used to identify right-handed periodic feature data. Jright is the total number of right-handed periodic feature data.

[0156] Similarly, the periodic feature extraction unit 234 determines the periodically occurring feature quantity in the left-handed feature quantity data and calculates the left-handed periodic feature data representing the periodically occurring feature quantity. The left-handed periodic feature data is represented by the following equation (11).

[0157] [Mathematical Expression 11] (11)

[0159] Here, jleft is a positive integer satisfying 1 ≤ jleft ≤ Jleft, and is the number used to identify the left-handed periodic feature data. Jleft is the total number of left-handed periodic feature data.

[0160] In addition, the periodic feature extraction part 234 calculates the similarity between the right-handed periodic feature data and the left-handed periodic feature data.

[0161] Similarity can be, for example, the reciprocal of the Euclidean distance between the center vectors of the right-handed and left-handed periodic feature data, or the overlap rate of the ranges of the right-handed and left-handed periodic feature data in D-dimensional space.

[0162] Furthermore, the periodic feature extraction unit 234 calculates the comprehensive periodic feature data, which is a combination of the right-hand periodic feature data and the left-hand periodic feature data with high similarity. The comprehensive periodic feature data is represented by the following equation (12).

[0163] [Mathematical Expression 12] (12)

[0165] Here, *jeither* is a positive integer satisfying 1 ≤ *jeither* ≤ *jeither*, and is the number used to identify the composite periodic characteristic data. *Jeither* is the total number of composite periodic characteristic data.

[0166] For example, if the similarity between the right-hand periodic feature data and the left-hand periodic feature data exceeds a predetermined threshold, the periodic feature extraction unit 234 can calculate the comprehensive periodic feature data using the following formula (13).

[0167] [Mathematical Expression 13] (13)

[0169] The periodic feature data storage unit 235 stores the right-hand periodic feature data, left-hand periodic feature data, and comprehensive periodic feature data calculated as described above.

[0170] The classification unit 236 classifies the elemental operations of the operator constituting the analysis object into intervals defined by time when multiple features contained in the right-hand characteristic data of the object are included in the range represented by the right-hand periodic characteristic data. The classification unit 236 generates right-hand classification data, which represents the time interval for right-hand operation classification by each element. Furthermore, the classification unit 236 classifies the elemental operations of the operator constituting the analysis object into intervals defined by time when multiple features contained in the left-hand characteristic data of the object are included in the range represented by the left-hand periodic characteristic data. The classification unit 236 generates left-hand classification data, which represents the time interval for left-hand operation classification by each element. Finally, the classification unit 236 classifies the elemental operations of the operator constituting the analysis object into intervals defined by time when multiple features contained in both the right-hand and left-hand characteristic data of the object are included in the range represented by the comprehensive periodic characteristic data. The classification unit 236 generates comprehensive classification data, which represents the time interval for both left-hand and right-hand operation classification by each element.

[0171] For example, classification unit 236 divides the operator's work into each work element based on the right-hand characteristic data, determining the intervals of the divided work elements, based on whether the right-hand characteristic data is included in the range represented by the right-hand periodic characteristic data. Thus, classification unit 236 generates right-hand classification data representing the intervals of each work element, determined based on the right-hand characteristic data.

[0172] Furthermore, the classification unit 236 divides the operator's work into each work element based on the left-handed characteristic data, determining the intervals of the divided work elements, based on whether the left-handed characteristic data is included in the range represented by the left-handed periodic characteristic data. Thus, the classification unit 236 generates left-handed classification data representing the intervals of each work element, determined based on the left-handed characteristic data.

[0173] Furthermore, the classification unit 236 divides the operator's work into each work element based on whether either of the right-hand characteristic data or the left-hand characteristic data is included within the range represented by the comprehensive periodic characteristic data, and determines the intervals of the divided work elements. Thus, the classification unit 236 generates comprehensive classification data representing the intervals of each work element, determined based on the right-hand characteristic data and the left-hand characteristic data.

[0174] Evaluation unit 237 evaluates the time represented by right-hand classification data, the time represented by left-hand classification data, and the time represented by composite classification data according to a predetermined standard. Thus, evaluation unit 237 generates right-hand evaluation data, left-hand evaluation data, and composite evaluation data.

[0175] Display unit 238 displays at least one of the following: sensor data, right-hand feature data, left-hand feature data, right-hand periodic feature data, left-hand periodic feature data, comprehensive periodic feature data, right-hand classification data, left-hand classification data, comprehensive classification data, right-hand evaluation data, left-hand evaluation data, and comprehensive evaluation data.

[0176] As described above, according to Embodiment 2, it is possible to easily and appropriately analyze the work of right-handed workers and left-handed workers.

[0177] The job analysis devices 130 and 230 described above perform both the learning phase processing and the analysis phase processing, but they can also perform only one of them. In this case, in the device that only performs the analysis phase, the periodic feature data generated by the processing in the learning phase can be stored. Furthermore, the processing in the analysis phase can be performed by downloading periodic feature data stored in other devices. Additionally, during the learning phase, the periodic feature data can also be stored in the storage unit of another device. In this case, the periodic feature data storage units 135 and 235 may not be provided.

[0178] Label Explanation

[0179] 100, 200 Job analysis system; 110 Sensor; 130, 230 Job analysis device; 131 Communication unit; 132 Sensor data acquisition unit; 133, 233 Feature calculation unit; 134, 234 Periodic feature extraction unit; 135, 235 Periodic feature data storage unit; 136, 236 Classification unit; 137, 237 Evaluation unit; 138, 238 Display unit.

Claims

1. An information processing device, characterized in that, The information processing device includes: The sensor data acquisition unit acquires multiple sensor data, which represent the detection results detected by the sensors on the bodies of one or more workers during operation. A feature quantity calculation unit calculates multiple feature quantities based on each sensor data in the plurality of sensor data, generates feature quantity data representing the plurality of feature quantities in a time series, thereby generating multiple feature quantity data; and The periodic feature extraction unit extracts periodically occurring feature quantities, i.e., periodic feature quantities, from the plurality of feature quantity data, determines a range according to a predetermined rule in a manner that includes the periodic feature quantities, and generates periodic feature data representing the range.

2. The information processing device according to claim 1, characterized in that, The periodic feature is the centroid of the cluster whose evaluation value exceeds the threshold. When the multiple feature values ​​represented by the multiple feature value data are clustered into multiple clusters, the evaluation value is calculated for each of the multiple clusters in a way that the more feature values ​​belonging to each of the multiple clusters, the higher the evaluation value is obtained.

3. The information processing apparatus according to claim 1 or 2, characterized in that, The sensor data acquisition unit also acquires object sensor data, which represents the detection results detected by the sensors regarding the operator who is the object of analysis. The feature calculation unit further calculates multiple feature quantities based on the object sensor data, and generates object feature quantity data representing the multiple feature quantities calculated based on the object sensor data. The information processing device further includes a classification unit that classifies the elemental operations of the operator's operations that constitute the analysis object into intervals divided by time periods, where the time periods are the times when the plurality of feature quantities represented by the object feature quantity data are contained within the range represented by the periodic feature data. The classification unit generates classification data that represents the time periods of the intervals classified according to each elemental operation.

4. The information processing apparatus according to claim 3, characterized in that, The information processing device further includes an evaluation unit that evaluates the time of operation for each of the elements represented by the classification data based on a predetermined benchmark.

5. The information processing apparatus according to any one of claims 1 to 4, characterized in that, The feature calculation unit calculates the plurality of feature quantities representing the positions of the respective body parts of the workers.

6. The information processing apparatus according to claim 3 or 4, characterized in that, The multiple sensor data represent multiple image data that show images captured by the one or more operators. The object sensor data refers to image data representing images captured of the operator who becomes the object of analysis. The feature calculation unit determines the image surrounding a specific part of one or more workers from the respective frames of the multiple sensor data as a first part image, calculates the feature quantity of the object contained in the first part image, and determines the image surrounding a specific part of the worker who will become the object of analysis from the frame of the object sensor data as a second part image, and calculates the feature quantity of the object contained in the second part image.

7. The information processing apparatus according to claim 1, characterized in that, The feature calculation unit calculates multiple feature quantities related to the right hand based on the multiple sensor data, generating feature quantity data (i.e., right-hand feature quantity data) represented as a time series of the multiple feature quantities calculated for the right hand. It then determines multiple feature quantities related to the left hand from the multiple sensors, generating feature quantity data (i.e., left-hand feature quantity data) represented as a time series of the multiple feature quantities calculated for the left hand. This process generates multiple right-hand feature quantity data and multiple left-hand feature quantity data. The periodic feature extraction unit extracts right-hand periodic features from the plurality of right-handed feature data as periodic features, determines the right-hand range as the range determined according to the predetermined rule in a manner that includes the right-hand periodic features, and generates right-hand periodic feature data as periodic feature data representing the right-hand range. It also extracts left-hand periodic features from the plurality of left-handed feature data as periodic features, determines the left-hand range as the range determined according to the predetermined rule in a manner that includes the left-hand periodic features, and generates periodic feature data representing the left-hand range, i.e., left-hand periodic feature data. When the similarity between the right-hand periodic feature data and the left-hand periodic feature data exceeds a predetermined threshold, it generates comprehensive periodic feature data by combining the right-hand periodic feature data and the left-hand periodic feature data.

8. The information processing apparatus according to claim 7, characterized in that, The sensor data acquisition unit also acquires object sensor data, which is the sensor data of the operator that is the object of analysis. The feature calculation unit calculates multiple feature quantities related to the right hand based on the object sensor data, and generates object right-hand feature quantity data, which is represented as a time series of the multiple feature quantities calculated for the right hand. Similarly, it calculates multiple feature quantities related to the left hand based on the object sensor data, and generates object left-hand feature quantity data, which is represented as a time series of the multiple feature quantities calculated for the left hand. The information processing device further includes a classification unit that classifies the elemental tasks of the operator constituting the analysis object into intervals divided by time, where the time is the moment when the plurality of features contained in the right-hand feature data of the object are contained within the range represented by the right-hand periodic feature data. The classification unit generates right-hand classification data representing the time interval for classifying the right-hand task by each element. The classification unit then classifies the elemental tasks of the operator constituting the analysis object into intervals divided by time, where the time is the moment when the plurality of features contained in the left-hand feature data of the object are contained within the range represented by the left-hand periodic feature data. The classification department generates left-handed classification data, which represents the time interval for the left-handed classification of each element operation. The classification department will classify the element operations of the operator of the analysis object into intervals divided by time when the multiple features contained in the right-handed feature data and the multiple features contained in the left-handed feature data of the object are included in the range represented by the comprehensive periodic feature data. The classification department generates comprehensive classification data, which represents the time interval for the left-handed and right-handed classification of each element operation.

9. The information processing apparatus according to claim 8, characterized in that, The information processing device further includes an evaluation unit that evaluates the time represented by the right-hand classification data, the time represented by the left-hand classification data, and the time represented by the comprehensive classification data according to a predetermined benchmark.

10. An information processing device, characterized in that, The information processing device includes: The sensor data acquisition unit acquires object sensor data, which represents the detection results detected by the sensor regarding the operator who is the object of analysis. The feature calculation unit calculates multiple feature quantities based on the object sensor data and generates object feature quantity data representing the multiple feature quantities calculated based on the object sensor data. as well as The classification department calculates multiple feature quantities based on individual sensor data from multiple sensor data representing detection results of one or more workers' bodies detected by sensors during operation, generates feature quantity data representing the multiple feature quantities in a time series, extracts periodically occurring feature quantities, i.e., periodic feature quantities, from the multiple feature quantity data, classifies the element operations of the operation constituting the operation being analyzed into intervals divided by time, where the time is the time when the multiple feature quantities represented by the object feature quantity data are contained within a range, which is the range represented by the periodic feature data determined according to predetermined rules in a manner that includes the periodic feature quantities, and generates classification data representing the time intervals classified according to each element operation.

11. A program, characterized by, To enable a computer to function as the following: The sensor data acquisition unit acquires multiple sensor data, which represent the detection results detected by the sensors on the bodies of one or more workers during operation. A feature quantity calculation unit calculates multiple feature quantities based on each sensor data in the plurality of sensor data, generates feature quantity data representing the plurality of feature quantities in a time series, thereby generating multiple feature quantity data; and The periodic feature extraction unit extracts periodically occurring feature quantities, i.e., periodic feature quantities, from the plurality of feature quantity data, determines a range according to a predetermined rule in a manner that includes the periodic feature quantities, and generates periodic feature data representing the range.

12. A program, characterized by, To enable a computer to function as the following: The sensor data acquisition unit acquires object sensor data, which represents the detection results detected by the sensor regarding the operator who is the object of analysis. The feature calculation unit calculates multiple feature quantities based on the object sensor data and generates object feature quantity data representing the multiple feature quantities determined based on the object sensor data. as well as The classification department calculates multiple feature quantities based on individual sensor data from multiple sensor data representing detection results of one or more workers' bodies detected by sensors during operation, generates feature quantity data representing the multiple feature quantities in a time series, extracts periodically occurring feature quantities, i.e., periodic feature quantities, from the multiple feature quantity data, classifies the element operations of the operation constituting the operation being analyzed into intervals divided by time, where the time is the time when the multiple feature quantities represented by the object feature quantity data are contained within a range, which is the range represented by the periodic feature data determined according to predetermined rules in a manner that includes the periodic feature quantities, and generates classification data representing the time intervals classified according to each element operation.

13. An information processing method, characterized in that, Data from multiple sensors is acquired, representing the detection results obtained by sensors regarding the bodies of one or more workers during the operation. Multiple feature quantities are calculated based on the data from the multiple sensors, and feature quantity data representing these multiple feature quantities in a time series format is generated, thereby generating multiple feature quantity data. Periodic features, i.e., periodic features, are extracted from the multiple feature data. A range is determined according to a predetermined rule in a manner that includes the periodic features, and periodic feature data representing the range is generated.

14. An information processing method, characterized in that, Acquire object sensor data, which represents the detection results detected by sensors on the operator that is the object of analysis. Multiple feature quantities are calculated based on the object sensor data, and object feature quantity data representing the multiple feature quantities determined based on the object sensor data is generated. Multiple feature quantities are calculated based on multiple sensor data representing detection results of the bodies of one or more workers during operation. Feature quantity data representing the multiple feature quantities in time series is generated. From the multiple feature quantity data, periodic feature quantities, i.e., periodic feature quantities, are extracted. The element operations of the operation constituting the operation object are classified into intervals divided by time, which are the times when the multiple feature quantities represented by the object feature quantity data are contained in the range, which is the range represented by the periodic feature data that is determined according to predetermined rules in a manner that includes the periodic feature quantities. The classification unit generates classification data that represents the time intervals classified according to each element operation.

Images