Work delay detection device and work delay detection method

Abstract

To detect and indicate a delay of work even when a work order is different for each worker in a system that analyzes actions of workers based upon sensor data.SOLUTION: A work delay detection device is configured to hold a standard work information table which stores a label corresponding to each work, a standard work time, a standard work order for each process, and information on a feature quantity of sensor data of each work in association with each other. The work delay detection device is further configured to: compare a feature quantity of extracted time-series sensor data with the information on the feature quantity stored in the table to perform clustering, and impart the label corresponding to each work in the table to sensor data belonging to each cluster; refer to the standard work order for each process to rearrange the time-series sensor data imparted with the label so as to be matched to the standard work order; and compare a feature quantity of the rearranged sensor data with the corresponding feature quantity to detect a delay of work.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a work delay detection device and a work delay detection method, and more particularly to a work delay detection device and a work delay detection method that are suitable for detecting and indicating work delays in a system that analyzes the movements of workers based on sensor data at a work site such as a factory. [Background technology]

[0002] In recent years, as the elderly population increases and the number of young workers decreases, and social security costs such as pensions, medical care, and nursing care increase, labor shortages have become a serious issue at production and logistics sites, construction sites, etc. In particular, at production sites, productivity is declining due to a shortage of skilled personnel caused by the aging of skilled workers, and improving productivity has become an issue.

[0003] Therefore, manufacturing operations are being improved by proactively utilizing advanced DX (Digital Transformation). Based on the concept of OEE (Overall Equipment Effectiveness), which has been traditionally introduced at production sites, the site is visualized using various KPIs (Key Performance Indicators) such as downtime, throughput, first-run rate, and defect rate, and the causes of productivity decline are identified and improvement activities are carried out. This OEE defines three measurable perspectives, namely availability, performance, and quality, and is considered an index of production efficiency. It is important to evaluate the site from this perspective, identify loss factors, and eliminate them. These loss factors can be divided into four loss factors: huMan, Machine, Material, and Method (hereinafter, these factors will be referred to as "4M"). In order to realize labor-saving and automation at each production site without excess, shortage, or surplus, it is necessary to quickly detect loss factors (=defect factors) and quickly feed back improvement measures to the site. In recent years, production sites have been using digital technology in this way to analyze data related to 4M information and visualize the information in order to improve productivity.

[0004] One example is DX technology that monitors equipment, workers, and other objects based on data acquired by various sensors, and contributes to improving the resilience of the manufacturing industry. However, when it comes to monitoring workers at manufacturing sites, image-based sensing methods using cameras, etc. are often avoided from the perspective of protecting privacy, so there is a high demand for technology that indirectly estimates the work of workers using sensors that output time-series waveforms, such as highly confidential wearable sensors and vibration sensors.

[0005] A technology for automatically adjusting processes at a production site using a data processing device is disclosed in, for example, Patent Document 1.

[0006] In the process procedure automatic adjustment method of Patent Document 1, in a multi-product variable production line having a plurality of manufacturing processes, the contents of defects in the sequence of process procedures and processing conditions expressed by the combination of manufacturing processes, violation points according to the contents of the defects, and a defect resolution method for resolving the defects are set in advance. Then, among the process procedures, a process procedure to be the subject of inference is created, the contents of the defects are inferred for the process procedure, and based on the result, a total violation point, which is the sum of the violation points, is calculated. Next, according to the result of this inference, the defect resolution method is applied to correct the process procedure, and a series of processes of inferring the contents of the defects, calculating the total violation point, and correcting the process procedure are repeated for the corrected process procedure, and the process procedure with the smallest total violation point is selected from the process procedures obtained by the correction.

[0007] Furthermore, technologies for detecting anomalies based on time-series data measured during work are disclosed in, for example, Patent Documents 2 and 3.

[0008] The anomaly detection device in Patent Document 2 acquires target time series data, calculates the distance of each sample of the acquired target time series data based on the standard time series data, and calculates the influence degree by dividing the distance of each sample by the total distance.When the influence degree exceeds a predetermined threshold, a warning is displayed.

[0009] In particular, it is described that when using a working path to compare time series data, a DTW (Dynamic Time Warping) notation is used (Figure 5, paragraphs

[0035] to

[0039] ).

[0010] In the anomaly detection device of Patent Document 3, standard time series model numerical parameters obtained by state modeling of reference time series data and standard transition time model numerical parameters obtained by modeling the duration of each state are stored in a model storage unit. The anomaly detection device receives input of time series data to be subjected to anomaly detection, performs state estimation for the time series data to be subjected to anomaly detection using the standard time series model numerical parameters, estimates state variables of each data point, and calculates an anomaly degree of the time required for each data point to transition from a predetermined state to another state using the standard transition time model numerical parameters and the state variables of each data point estimated by the state estimation unit.

[0011] In particular, the anomaly detection device of Patent Document 3 suggests using the DTW technique when comparing the data value series of target time series data with the data value series of standard time series data (FIG. 13, paragraph

[0061] ). [Prior art documents] [Patent documents]

[0012] [Patent Document 1] Japanese Patent Application Publication No. 11-48101 [Patent Document 2] JP 2009-245228 A [Patent Document 3] JP 2020-149208 A Summary of the Invention [Problem to be solved by the invention]

[0013] In the above-mentioned Patent Document 1, the degree of difference from the prescribed process procedure and processing conditions is calculated as a total violation score, and the process procedure with the minimum total violation score is selected, while the above-mentioned Patent Documents 2 and 3 are technologies for detecting anomalies based on time-series data measured during work.This makes it possible to estimate to some extent the location, work procedure, and work process of a defect in the manufacturing process and take measures.

[0014] Patent Document 1 describes a method for estimating and calculating the violation points for each manufacturing process, assuming a situation where line rearrangements occur frequently in small-lot, high-mix manufacturing, and adjusting the processes so that the total violation points are minimized when the processes are rearranged. However, it does not disclose a method that can analyze the causes of specific defect locations and time periods in each process.

[0015] On the other hand, in Patent Documents 2 and 3, in order to detect anomalies in time series data from various sensors such as wearable sensors attached to workers, the system uses DTW to stretch and expand the time of standard time series data and target time series data to find a matching point between the two data, calculate and add up the distance to measure the degree of anomaly, and judge whether or not there is an anomaly, i.e., a work defect. This makes it possible to make a comparison while ignoring time lags, but in addition to the difficulty of detecting abnormalities in required time (delay in a certain operation, etc.), in the case of a defect search method based on the DTW method, it is presumed that the work order expressed by the standard time series data and the search target time series data is the same, so it does not take into account cases where the work order differs for each worker, and in this respect there is room for improvement.

[0016] The object of the present invention is to provide a work delay detection device and work delay detection method that can detect and indicate work delays when KPIs such as production line throughput deteriorate, particularly when the task order differs for each worker, in a system that analyzes worker actions based on sensor data at a work site such as a factory. [Means for solving the problem]

[0017] The configuration of the work delay detection device of the present invention is preferably a work delay detection device that detects work delays based on time-series sensor data acquired by a sensor installed at a work site, and holds a standard work information table for storing information on work in standard work, and the standard work information table stores a label corresponding to each work, a standard work time, a standard work order for each process, and information on feature amounts of the sensor data for each work, each in association with each other, extracts features from the time-series sensor data, compares the feature amount information stored in the standard work information table with the features of the extracted time-series sensor data, clusters the extracted time-series sensor data, assigns labels corresponding to each work in the standard work information table to the time-series sensor data belonging to each cluster, refers to the standard work order for each process in the standard work information table, rearranges the labeled time-series sensor data to match the standard work order, and compares the features of the rearranged time-series sensor data with the corresponding features in the standard work information table to detect work delays. Effect of the Invention

[0018] According to the present invention, it is possible to provide a work delay detection device and a work delay detection method that can detect and indicate work delays when KPIs such as the throughput of a production line deteriorate, in a system that analyzes the actions of workers based on sensor data at a work site such as a factory, particularly when the task order differs for each worker. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram showing a functional configuration of an operation delay detection device according to a first embodiment and a data flow between functional units. [Diagram 2] FIG. 1 is a hardware / software configuration diagram of an operation delay detection device according to a first embodiment. [Diagram 3] FIG. 1 is a diagram showing an example of a work process at a manufacturing site where parts are assembled and processed. [Figure 4] FIG. 13 is a diagram illustrating delay information related to a delay in a work process. [Diagram 5] FIG. 1 is a diagram showing clustering in an n-dimensional feature space. [Figure 6] FIG. 13 is a diagram showing how sensor data is labeled and sorted based on a standard operating process. [Figure 7] FIG. 13 is a diagram illustrating how a delay time is calculated from a feature amount of sensor data by the DTW method. [Figure 8] FIG. 13 is a diagram illustrating an example of a standard work information table. [Figure 9] 4 is a flowchart showing an operation delay detection process performed by the operation delay detection device according to the first embodiment. [Figure 10] 13 is a flowchart showing a time series averaging process. [Figure 11] FIG. 13 is a diagram showing an example of a work delay information result output screen. [Figure 12] 10 is a flowchart showing an operation delay detection process performed by the operation delay detection device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, each embodiment of the present invention will be described with reference to FIGS.

[0021] [Embodiment 1] Hereinafter, a first embodiment of the present invention will be described with reference to FIGS.

[0022] First, an overview of the functions and operations of the work delay detection device according to the first embodiment will be described with reference to FIG. FIG. 1 also illustrates sensors at a manufacturing site such as a manufacturing factory.

[0023] The sensor 20 at the manufacturing site is a sensor device attached to a worker, such as a wearable sensor or a vibration sensor, or a device that collects sounds at the work site, such as a microphone, and outputs time-series waveform data related to the worker's actions. The sensor 20 and the work delay detection device 100 communicate wirelessly, and the work delay detection device 100 acquires the time-series waveform data from the sensor 20. For example, Wi-Fi (IEEE 802.11) or Bluetooth (IEEE 802.15.1) can be used as the wireless standard. The sensor 20 may also include a sensor for measuring the start time / end time of each process. For example, a camera or a line sensor that detects the passage of a product can be used.

[0024] The work delay detection device 100 is a device that detects process delays in work and outputs related information based on time-series waveform data from the sensor 20. As shown in Fig. 1, the work delay detection device 100 is functionally configured with a production management unit 110, a data collection unit 120, a work delay detection processing unit 130, and a storage unit 140.

[0025] The storage unit 140 stores a sensor data DB 200, a standard work DB 210, and a production management information DB 220.

[0026] The data collection unit 120 is a functional unit that transmits various control information required to acquire data from the various sensors 20 to the various sensors 20. In the work delay detection device 100, the data collection unit 120 functions as a trigger for acquiring data from the various sensors 20 by acquiring data transmitted from the various sensors 20.

[0027] The sensor data captured by the data collection unit 120 is stored in a sensor data DB 210 in the storage unit 140 and is used for various data processing at a later stage. In addition, data such as sensors for measuring the start time / end time of each process, such as a camera or a line sensor for detecting the passage of a product, time cards for workers to record the start / end of work, and authentication information using RFID or the like can also be stored together with timestamps. Furthermore, data on manufacturing-related information acquired from each line at the manufacturing site may be stored together with timestamps.

[0028] The production management unit 110 is a part of the factory's core system (such as MES: Manufacturing Execution System) or a functional unit that acquires data on manufacturing information from the core system and manages production at the work site. The production management unit 110 may be implemented in an information processing device separate from the work delay detection processing unit 130, or may be a part of the function of the core system in the factory equipment. In addition, the production management unit 110 may be a function realized by a cloud that can be accessed from the work delay detection device 100.

[0029] The production management section 110 is made up of sub-functional sections: a KPI calculation section 111 , a KPI pass / fail determination section 112 , and a work delay information output section 113 .

[0030] The KPI calculation unit 111 is a functional unit that calculates KPIs related to manufacturing lines and processes from manufacturing-related information acquired from each line at the manufacturing site, and sends them to the KPI pass / fail judgment unit 112. The KPI is, for example, throughput [pieces / second] in a production line = number of products / unit time.

[0031] The KPI quality determination unit 112 is a functional unit that compares the calculated KPI with a specified threshold value, determines whether the KPI is good or bad, and when it determines that the KPI is bad, requests the operation delay detection processing unit 130 to start up. In this way, by determining the process for detecting delays according to the quality of the KPI, it is possible to detect operation delays only when the production situation is bad.

[0032] The work delay information output unit 113 is a functional unit that outputs information related to a process delay detected by the work delay detection processing unit 130. The work delay information output unit 113 may output the information to other application software, or may display the information to a production manager on a device such as a display.

[0033] The work delay detection processing unit 130 is a functional unit that detects work delays based on the sensor data DB 200 and information related to production management. The work delay detection processing unit 130 includes a feature amount calculation unit 131, a rough classification unit 132, a waveform adjustment unit 133, and a waveform analysis unit 134.

[0034] When the work delay detection processing unit 130 receives a start request from the KPI pass / fail judgment unit 112, it acquires sensor data from the sensor data DB 200 in the storage unit 140 and passes it to the feature calculation unit 131. Furthermore, for processing in the feature calculation unit 131, data such as sensors for measuring the start time / end time of each process, for example, cameras or line sensors for detecting the passage of products, time cards for workers to record the start / end of work, authentication information by RFID or the like, may be acquired and transferred together with timestamps, or manufacturing-related information acquired from each line at the manufacturing site may be transferred together with timestamps by utilizing a factory core system.

[0035] The feature amount calculation unit 131 is a functional unit that calculates feature amounts of the sensor data in the sensor data DB 200 and the sensor data in the standard work DB 210.

[0036] The rough classification unit 132 is a functional unit that refers to information in the production management information DB 220, maps the sensor data into a feature amount space based on the feature amount of the sensor data, and classifies the sensor data.

[0037] The waveform adjustment unit 133 is a functional unit that rearranges the sensor data classified by the rough classification unit 132 according to a standard work process.

[0038] The waveform analysis unit 134 is a functional unit that analyzes the sensor data rearranged into the standard work process sequence by the waveform adjustment unit 133, and obtains work delay information.

[0039] Next, the hardware and software configuration of the work delay detection device will be described with reference to FIG. The hardware configuration of the work delay detection device 100 is realized by, for example, a general information processing device such as a personal computer shown in FIG.

[0040] The operation delay detection device 100 includes a processor 302, a main memory device 304, a network I / F (InterFace) 306, a display I / F 308, an input / output I / F 310, and an auxiliary memory I / F 312, all of which are connected via a bus.

[0041] The processor 302 controls each part of the operation delay detection device 100, and loads and executes necessary programs in the main memory device 304. Examples of the processor 302 include a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), but other semiconductor devices may be used as long as they are capable of executing predetermined processes. The main memory device 304 is typically configured as a volatile memory such as a RAM (Random Access Memory), and stores the programs executed by the processor 302 and data referenced by the processor 302.

[0042] The network I / F 306 is an interface for connecting to the network 5. The network I / F 306 is, for example, a network board in the case of a wired connection, or a wireless module including an antenna in the case of a wireless connection.

[0043] The display I / F 308 is an interface for connecting a display device 320 such as an LCD (Liquid Crystal Display).

[0044] The input / output I / F 310 is an interface for connecting input / output devices. In the example of Fig. 2, a keyboard 330 and a mouse 332 which is a pointing device are connected.

[0045] The auxiliary storage I / F 312 is an interface for connecting an auxiliary storage device such as a hard disk drive (HDD) 350 or a solid state drive (SSD).

[0046] The HDD 350 has a large storage capacity and stores programs for executing this embodiment. A production management program 370, a data collection program 360, and a work delay detection processing program 380 are installed in the work delay detection device 100.

[0047] A production management program 370, a data collection program 360, and an operation delay detection processing program 380 are programs which execute the functions of the data collection unit 120, the production management unit 110, and the operation delay detection processing unit 130, respectively.

[0048] The HDD 350 also stores a sensor data DB 200, a standard work DB 210, and a production management information DB 220 as databases.

[0049] The work delay detection device 100 may be a tablet, smartphone, or the like carried directly by the worker at the work site, or the work delay information may be displayed on the worker's head-mounted display via a wireless network.

[0050] Next, an overview of the process in which the work delay detection device detects a work delay at a work site and basic concepts related thereto will be described with reference to Figs.

[0051] In this embodiment, a work process in a manufacturing site shown in FIG. 3 will be taken as an example for explanation. The work processes assumed at the manufacturing site are as follows: Process 1: Picking up the necessary parts from the warehouse (PI), Process 2: Machining using a cutting machine (PII), and Process 3: Assembling by combining with other parts (PIII).

[0052] Here, the components of these work processes include a part 21 , a worker 22 , a cutting control unit 23 , a cutting blade 24 , a cutting machine 25 , a cut part 26 , and other parts 27 .

[0053] In process 1 (PI), a worker 22 picks up a required part 21 from a warehouse.

[0054] Next, in step 2 (PII), worker 22 sets part 21 in cutting machine 25 and turns on the cutting operation. Under the control of cutting control unit 23, cutting machine 25 moves cutting blade 24 to cut part 21, and machined part 26 is formed.

[0055] Next, in step 3 (PIII), the worker 22 combines the machined part 26 formed by the cutting machine 25 with another part 27 to assemble them.

[0056] Next, the delay in the work process and the associated delay information will be described using the example of the manufacturing process shown in FIG.

[0057] 4, the horizontal axis represents time and the vertical axis represents the progress of the process, and shows that the assembly process PIII is performed after the machining process PII. In this example, for example, the machining process PII took place over a period from time T1 to time T2, and similarly, the assembly process PIII took place over a period from time T3 to time T4. In addition, in the machining process PII, operation a321, operation b323, and operation c324 were performed during the period from time T1 to time T2, and in the assembly process PIII, operation e331, operation d332, and operation f333 were similarly performed.

[0058] In this embodiment, it is assumed that a delay 322 has occurred in an operation in the processing step PII as shown in Fig. 4. The information on the delay is expressed as follows: the previous process is operation a, the next process is operation b, the period during which the delay 322 occurs Δt, and the time t0 at the center of the period during which the delay 322 occurs. In reality, this means that the processing step took the time equivalent to the period during which the delay 322 occurred Δt. Here, specific causes of the delay include factors other than those that are considered to be the work of the original manufacturing operation. For example, a worker may operate a smartphone or fall asleep.

[0059] Next, the rough classification process for the sensor data will be described with reference to FIG. The rough classification process is performed by the rough classification unit 132, and is a process of classifying sensor data, which is time-series waveform data output from the sensor 20 such as a wearable sensor, a vibration sensor, or a microphone, based on features.

[0060] In the example of Figure 5, the cluster of sensor data shows the distribution of features in n-dimensional feature space (in Figure 5, X1 and X n (Only the correspondence between the

[0061] The rough classification unit 132 maps the sensor data into a feature space according to the feature and performs clustering. There are various methods for extracting features. For example, in the case of data in which time-series data from a vibration sensor, a microphone, etc. contains phase information, so-called statistical moment quantities such as mean, variance, skewness, and kurtosis may be used as features using RMS (Root Mean Square) data or Fourier transformed spectrum data to which a sliding window has been applied, or an n-dimensional feature distribution output by an intermediate layer in a DNN (Deep Neural Network) that uses time-series waveform data as input values ​​may be used. In addition, a feature extraction method using a DNN can be adopted in the case of mutually independent time-series data that does not contain phase information.

[0062] When a machine learning technique is used, the rough classification unit 132 performs two operations: an operation during learning and an operation during inference.

[0063] First, during learning, standard work is performed multiple times in order to obtain learning data, and the obtained various sensor data 11 are stored as sensor data for the standard work in the standard work DB 210. The rough classification unit 132 obtains the sensor data from the standard work DB 210 and extracts the features as described above. Furthermore, by performing cluster classification using an unsupervised learning method such as the k-means method, it is possible to obtain clustering of feature distribution as shown in FIG. 5.

[0064] Here, since it should be possible to obtain information (=label) on which task was performed (task a, task b, etc.) when acquiring the learning data, the above-mentioned DNN may be trained so that the feature distribution after cluster classification matches the correct answer (label information). As a result, the feature distribution after cluster classification is output for the sensor data of the standard task, and is stored in the production management information DB 220 together with the management information. Specific examples of tables stored in the production management information DB 220 will be described later.

[0065] Next, the inference operation of the rough classification unit 132 as a trained model for sensor data in standard work will be described. During inference, various time-series sensor data 11 are input to the same feature extraction process as during inference, and where the output feature is located in the cluster by the trained model / clustered is determined by processing such as linking to the cluster with the shortest distance from the center of the cluster by the feature of the sensor data of the standard work shown in FIG. 5 while referring to the cluster information stored in the production management information DB 220. This makes it possible to assign one label for the number of standard works to each point (or small area) of the feature distribution, i.e., each point (or small area) on the corresponding time-series sensor data. Therefore, it becomes possible to infer how the label changes in the time direction, i.e., in what order the work was performed in the time series in the data used during inference.

[0066] Next, the waveform adjustment process for the sensor data will be described with reference to FIG. The waveform adjustment process for the sensor data is a process performed by the waveform adjustment unit 133, and is a process of rearranging the sensor data labeled by the rough classification process based on the standard operating process.

[0067] In the rough classification process, inference was performed based on the classification model obtained after learning, and the sensor data was clustered based on features.

[0068] That is, as shown in FIG. 6, when target time-series sensor data 711 is given, the tasks performed at each time can be labeled as waveforms 72, 73, 74, and 75. In the case of FIG. 6, waveform 72 is the waveform drawn by sensor data measured by task b, waveform 73 is the waveform drawn by task c, waveform 74 is the waveform drawn by task a, and waveform 75 is the waveform drawn by sensor data measured by task a'. If waveform 75 is the delayed task that should be detected, and if tasks according to the process control data for standard tasks are tasks a, b, and c, then in clustering by rough classification processing, waveform 75 will be classified as either task a, task b, or task c. In the example shown in FIG. 6, waveform 75 is classified as sensor data that should belong to task a. Here, the dash "'" is added to task a' to indicate that the delayed task that should be detected has been classified as standard task.

[0069] Therefore, it is clear from the labeling that the task order of the target time-series sensor data 711 is task b → task c → task a → task a(a'), and when the standard task is task a → task b → task c, this sensor data is replaced with converted time-series sensor data 722 that complies with the standard task, as shown in FIG. 6.

[0070] This is to detect delays in later processing by comparing the waveforms of the sensor data in the standard time task DB 210 through waveform analysis processing using the DTW method. That is, since it is difficult to compare the waveforms of the sensor data for task a → task b → task c with the order of task b → task c → task a → task a(a') in the target time-series sensor data 711, the purpose is to detect delays by swapping them and comparing the waveforms of task a → task a(a') → task b → task c in the converted time-series sensor data 722.

[0071] Next, the process of calculating the delay time from the feature amount of the sensor data by the DTW method will be described with reference to FIG. This is a process performed by the waveform analysis section 130.

[0072] The waveform analysis unit 130 compares the waveforms of the converted time-series sensor data 712 and the standard time-series sensor data 710 using the DTW method.

[0073] In the DTW method, the cost (distance) is calculated for each data point of two different time series waveforms by a brute force convolution process, and the pair with the smallest cost is associated to calculate the similarity of each waveform. Figure 7 shows the result of comparing the converted time series sensor data 712 with the standard time series sensor data 710 and associating (pairing). By finding a time period in which the paired data points of either the converted time series sensor data 712 or the standard time series sensor data 710 do not change and only the paired data points of the other time series change, a delay time can be detected.

[0074] For example, it is assumed that point γ of the standard time series sensor data 710 is paired with point α of the converted time series sensor data 712, and point γ of the standard time series sensor data 710 is paired with point β of the converted time series sensor data 712.

[0075] At this time, point α of the converted time-series sensor data 712 corresponds to the sensor data at the end time of task a, and point β of the converted time-series sensor data 712 corresponds to the sensor data at the start time of task b, and it is presumed that the data between these is sensor data at a delay time that does not appear in standard tasks.

[0076] In this way, by analyzing the waveform data, the delay period Δt occurring between operations a and b and the time t0 which is the center point of the delay period Δt can be obtained.

[0077] Next, the data structure handled by the operation delay detection device will be described with reference to FIG.

[0078] The standard work information table 400 is a table stored in the production management information DB 220, and is a table that holds information related to standard works.

[0079] As shown in FIG. 8, the standard work information table 400 includes fields for a process 400a, a work name 400b, a standard work time 400c, a standard work sequence 400d, and cluster information 400e.

[0080] In the process 400a, an identifier or character string representing each process is stored. In the example of FIG. 8, "machining" and "assembly" are stored. In the work name 400b, an identifier or character string representing the work in each process is stored. In the standard work time 400c, the standard time for each work is stored, for example, in seconds. In the standard work order 400d, the work order defined by the standard is stored. In the cluster information 400e, a value indicating a data structure expressing a cluster of each feature amount based on the sensor data of the standard work DB 210 is stored. For example, it may be a pointer to data representing a cluster of the feature amount, or it may be a file name.

[0081] Next, the work delay detection process performed by the work delay detection device will be described with reference to Figs.

[0082] The work delay detection process performed by the work delay detection device is a process in which, after model learning using the sensor data in the standard work DB210, inference is made on specific sensor data, and by comparing it with the standard work, it is detected where and how much of a delay has occurred.

[0083] First, the work delay detection device 100 acquires sensor data from various sensors 20 that output time-series waveform data as measurement data, such as a wearable sensor, a vibration sensor, or a microphone (S101).

[0084] Next, the work delay detection device 100 calculates a feature distribution from the sensor data (S102).

[0085] As already explained, the features are RMS, Fourier transform spectrum calculations, statistical moment quantities such as mean / variance / skewness, and kurtosis, or features output from an intermediate layer in a DNN that uses time-series waveform data as input values.

[0086] Next, the operation delay detection device 100 performs an initialization process of the moving average amount (S103). That is, when the moving average amount is M, an initial value M0 is substituted.

[0087] Next, the operation delay detection device 100 performs a time series averaging process (S104). Note that the details of the time series averaging process will be described later with reference to FIG.

[0088] In the case of time series waveform data from vibration sensors, microphones, etc., if the sampling frequency is too large compared to the time constant of the process work, the inference results may change depending on the time constant of the sampling frequency, so time series averaging is used to analyze gradual changes. For example, if the work is assumed to be done by a person (such as tightening screws on a workpiece), if the time series period is too short, the human work cannot be analyzed well.

[0089] Next, the work delay detection device 100 refers to the work management data in the standard work information table 400 shown in FIG. 8, and performs a rough classification process based on the feature amounts averaged by the time-series averaging process (S105).

[0090] As already explained, the rough classification process (at the time of inference) is carried out by referring to the cluster information 400e in the standard work information table 400 and linking the standard work to the cluster that is the shortest distance from the center of the cluster based on the feature of the sensor data of the standard work shown in FIG. 5.

[0091] Next, the work delay detection device 100 assigns a label to each piece of time-series sensor data belonging to the classified cluster based on the character string of the work name 400b in the standard work information table 400 (S106). This involves assigning a label such as the work name 400b in the standard work information table 400 (such as "work name a", "work name b", "work name c") to each point (or small area) of the feature distribution, i.e., each point (or small area) on the corresponding time-series sensor data.

[0092] Next, the work delay detection device 100 detects the label (L i ) is the threshold value for each i ), and each duration of all the clustered time series sensor data is determined to be within a threshold Th(L i ) or more (S107). As described above, when the sampling frequency for the time series waveform data is too large compared to the time constant of the process operation, the amount of moving average is increased to avoid a situation in which the inference result may change due to the time constant of the sampling frequency. i ) is, for example, Th(L i )=(the value of the standard work time 400c in the standard work information table 400)+α, where α is an appropriate constant.

[0093] Each duration of all the clustered time series sensor data is determined to be within a threshold Th(L i) (S107: YES), the process proceeds to S108, and the duration of a certain clustered time-series sensor data is equal to or greater than the threshold Th(L i ) (S107: NO), the process returns to S104.

[0094] Next, the operation delay detection device 100 cuts out the target time-series sensor data for each label (S108).

[0095] Next, the work delay detection device 100 detects the work sequence from the time-series sensor data clustered by extracting each label (S109), and compares the detected work sequence with the standard work sequence 400c in the standard work information table 400 stored in the production management information DB 220 to determine whether or not there is a need to rearrange the work sequence to conform to the standard work (S110).

[0096] When it is determined that the work order needs to be changed to conform to the standard work (S110: YES), the work delay detection device 100 performs waveform adjustment processing (S111) and changes the time-series sensor data as described with reference to FIG. 6.

[0097] Next, the work delay detection device 100 performs waveform analysis on the time-series sensor data by the DTW method shown in FIG. 7 (S112) to obtain information on the work delay.

[0098] Next, the work delay detection device 100 outputs information about the work delay to a display device or the like in response to a request from the user (S113). A specific example of outputting information about the work delay will be described later.

[0099] Next, the time series averaging process will be described with reference to FIG. This process corresponds to S104 in FIG.

[0100] First, the work delay detection device 100 increments the moving average amount M (S201). That is, if the increment value of the moving average amount is ΔM, M←M+ΔM.

[0101] Next, the work delay detection device 100 performs moving average processing on the time-series sensor data based on the moving average amount M (S202).

[0102] That is, the features of M time series sensor data are expressed as (f1, f2...f M )=f, the moving average ma for the features of these sensor data is ma(f;M)=(f1+f2+...+f M ) / M.

[0103] Next, a user interface through which the work delay detection device outputs work delay information will be described with reference to FIG.

[0104] The work delay detection device 100 displays information related to the work delay on, for example, a work delay information result output screen 500 as shown in FIG.

[0105] The work delay detection device 100 comprises a work process information display area 510 , a feature amount information display area 520 , a time-series sensor data graph display area 530 , and a work delay information display area 540 .

[0106] The work process information display area 510 is an area that displays work process information for process management. The feature amount information display area 520 is an area that displays information related to the feature amount of sensor data. The time series sensor data graph display area 530 is an area that displays a feature amount graph of time series sensor data. The work delay information display area 540 is an area that displays information related to work delays.

[0107] The work delay information display area 540 includes, for example, a delayed process display column 541 , a previous work display column 542 , a subsequent work display column 543 , a delay time display column 544 , and a delay period display column 545 .

[0108] In the delayed process display field 541, the process of the delayed work is displayed, in the example of FIG. 4, "Machining process". The previous process display field 542 and the next process display field 543 display the names of the previous and next processes in the delayed period, respectively, "Task a" and "Task b" in the example of Fig. 4. The delay time display field 544 displays the time that is the center point of the delayed period (time t0 in the example of Fig. 4) in the format yyyy / mm / dd hh:mm:ss. The delay period display field 545 displays the delay period (time Δt in the example of Fig. 4) in seconds, for example.

[0109] As described above, the work delay detection device of this embodiment is a system that analyzes the actions of workers based on sensor data at a work site such as a factory, and can detect and point out work delays when KPIs such as the throughput of a production line deteriorate, especially when the work order differs for each worker.

[0110] In particular, sensors include sensor equipment attached to workers, such as wearable sensors and vibration sensors, and devices that collect sounds at the work site, such as microphones, and the features of time-series waveform data are analyzed to detect work delays.This makes it possible to detect work delays without violating the privacy of workers, compared to methods that use image recognition with cameras.

[0111] [Embodiment 2] Hereinafter, the second embodiment of the present invention will be described with reference to FIG.

[0112] In the first embodiment, a label is assigned to each point of the time-series sensor data, and the moving average number is adjusted until the duration of the time-series sensor data to which the label is assigned becomes greater than a certain threshold, as shown in FIG. 12.

[0113] In contrast, the work delay detection device of this embodiment adjusts the features (statistical moment quantities) of the sensor data until the duration of the labeled time-series sensor data becomes greater than a certain threshold value.

[0114] The following description will focus on the differences from the first embodiment.

[0115] Next, the work delay detection process performed by the work delay detection device will be described with reference to FIG. First, the work delay detection device 100 acquires sensor data from various sensors 20 that output time-series waveform data as measurement data, such as a wearable sensor, a vibration sensor, a microphone, etc. (S101). This is the same as in the first embodiment.

[0116] Next, the work delay detection device 100 calculates a spectrogram from the sensor data (S121). A spectrogram is a graph represented by time, frequency, and the strength (power) of a signal component.

[0117] In the statistical processing of the present embodiment, the statistical quantity of the power in the time direction is used as a feature quantity for classification. Here, a window W=[-T, T] in the time direction of the spectrum is taken, and the time t i Let P(t i ) and take N sample points.

[0118] Next, the work delay detection device 100 initializes the order r of the statistical moment amount (r←1) (S122).

[0119] Next, the statistical moment amount about the origin of the window W expressed by (Equation 1) when r=1 is calculated (S123). E[P 1 ]=(Σ i=1 N P(t i )) / N=μ …(Formula 1) μ is the so-called average value.

[0120] Next, r is incremented (r←r+1) (S124), and the statistical moment amount around the mean value at r is calculated (S125).

[0121] The statistical moment around the mean value when r=2 is expressed by the following (Equation 2). E[(P-μ) 2 ]=(Σ i=1 N (P(t i )-μ) 2 ) / N=σ 2 …(Formula 2) σ 2 is the so-called variance.

[0122] The statistical moment around the mean value for a general r is expressed as follows (Equation 3). E[(P-μ) r ]=(Σ i=1 N (P(t i )-μ) r ) / N …(Formula 3)

[0123] Next, the work delay detection device 100 moves the window W in the time direction, and performs a rough classification process using the statistical moment amount around the mean at r as the feature amount (S105).

[0124] Next, the work delay detection device 100 assigns a label to each piece of time-series sensor data belonging to the classified cluster based on the character string of the work name 400b in the standard work information table 400 or the like (S106).

[0125] S105 and S106 are the same as in the first embodiment.

[0126] Next, the work delay detection device 100 sets a threshold value for each label (Li) as Th(Li) and determines whether each duration of all the clustered time-series sensor data is equal to or greater than the threshold value Th(Li) (S107).

[0127] If the duration of each of the clustered time series sensor data is equal to or greater than the threshold Th(Li) (S107: YES), proceed to S108, and if the duration of any of the clustered time series sensor data is less than the threshold Th(Li) (S107: NO), return to S124.

[0128] The subsequent processes from S108 to S113 are the same as those in the first embodiment.

[0129] In the second embodiment, the order of the statistical moment amount is manipulated to ensure the length of the period during which time-series data to which the same label is assigned continues. However, after S125, a process may be added in which a moving average in the time direction is calculated again for the statistical moment amount around the average value at r.

[0130] In this embodiment, the rth order statistical moment quantity is used as a feature quantity, and the order of the statistical moment quantity is manipulated, thereby making it possible to ensure the length of the period during which time-series data with the same label continues, without taking a moving average as in the first embodiment. [Explanation of symbols]

[0131] 20...sensor, 100... work delay detection device, 110... production management unit, 120... data collection unit, 130... work delay detection processing unit, 140... storage unit, 200...sensor data DB, 210...standard work DB, 220...production management information DB, 400...Standard work information table

Claims

1. A work delay detection device that detects a work delay based on time-series sensor data acquired by a sensor installed at a work site, A standard work information table is maintained to store information on standard work. The standard work information table stores a label corresponding to each work, a standard work time, a standard work order for each process, and information on feature quantities of sensor data for each work, in association with each other; Extracting features from the time-series sensor data; comparing information on the feature amount stored in the standard work information table with the feature amount of the extracted time-series sensor data, clustering the extracted time-series sensor data, and assigning a label corresponding to each task in the standard work information table to the time-series sensor data belonging to each cluster; referring to a standard work order for each process in the standard work information table, rearranging the time-series sensor data to which the label has been assigned so as to conform to the standard work order; A work delay detection device that detects work delays by comparing the feature amounts of the rearranged time-series sensor data with the corresponding feature amounts of the standard work information table.

2. 2. The operation delay detection device according to claim 1, A work delay detection device that outputs, as work delay information, information on the time of the delay, the delay period, and work before and after the delay.

3. 2. The operation delay detection device according to claim 1, A work delay detection device characterized in that DTW (Dynamic Time Warping) is used when comparing the features of the rearranged time series sensor data with the corresponding features of the standard work information table.

4. 2. The operation delay detection device according to claim 1, Taking a moving average of the feature values ​​from the time series of sensor data, a work delay detection device which compares information on the feature quantities stored in the standard work information table with moving average values ​​of the feature quantities from the time-series sensor data, clusters the extracted time-series sensor data, and assigns a label corresponding to each work in the standard work information table to the time-series sensor data belonging to each cluster.

5. In the work delay detection device according to claim 4, When the duration of the clustered time-series sensor data does not meet a predetermined threshold across all clusters, a moving average of features from the time-series sensor data is increased.

6. 2. The operation delay detection device according to claim 1, Calculating statistical moments of features from the time series sensor data; a work delay detection device which compares information on feature quantities stored in the standard work information table with values ​​of statistical moment quantities of feature quantities from the time-series sensor data, clusters the extracted time-series sensor data, and assigns a label corresponding to each work in the standard work information table to the time-series sensor data belonging to each cluster.

7. In the work delay detection device according to claim 4, When the duration of the clustered time-series sensor data does not meet a predetermined threshold across all clusters, a moment order of a statistical moment amount of a feature from the time-series sensor data is increased.

8. 2. The operation delay detection device according to claim 1, A work delay detection device characterized by acquiring a production index and determining whether to execute a process for detecting a work delay based on the quality of the index.

9. A work delay detection method using a work delay detection device that detects work delays based on time-series sensor data acquired by a sensor installed at a work site, comprising: the work delay detection device holds a standard work information table for storing information on work in standard work, The standard work information table stores a label corresponding to each work, a standard work time, a standard work order for each process, and information on feature quantities of sensor data for each work, in association with each other; A step in which a processor of the operation delay detection device extracts a feature amount from the time-series sensor data; a processor of the operation delay detection device comparing information on the feature amount stored in the standard work information table with the feature amount of the extracted time-series sensor data, clustering the extracted time-series sensor data, and assigning a label corresponding to each operation in the standard work information table to the time-series sensor data belonging to each cluster; A processor of the work delay detection device refers to a standard work order for each process in the standard work information table, and rearranges the time-series sensor data to which the label has been added so as to conform to the standard work order; and detecting a delay in work by comparing the feature amounts of the rearranged time-series sensor data with the corresponding feature amounts of the standard work information table.

10. The method for detecting a delay in work according to claim 9, A method for detecting work delays, comprising the steps of: outputting, as work delay information, the time of the delay, the period of the delay, and information on work before and after the delay.

11. The method for detecting a delay in work according to claim 9, A work delay detection method, characterized in that in the step of comparing the features of the rearranged time series sensor data with the corresponding features of the standard work information table, DTW (Dynamic Time Warping) is used.

Images