Work assisting device and work assisting method

Abstract

An operation assisting device (1) assists an operation constituted by a series of a plurality of actions performed by an operation subject, and has: a sensor data input device (10) that receives a sensor data row arranged each time sensor data measured for the operation of the operation subject is acquired; a category data calculator (20) that compares the sensor data row with a set, i.e., a template, set for each type, i.e., a category, of a temporal change of the sensor data and constituted by a probability distribution of each time of the sensor data, thereby calculating a category data row indicating intervals into which the sensor data row is divided; a reliability calculator (40) that calculates, for each interval, a reliability with respect to the interval based on information of the interval indicated by the category data row; a reliability determiner (65) that determines whether the reliability satisfies a criterion, and generates information of an action included in an interval in which the reliability satisfies the criterion as first assistance information; and a notifier (60) that notifies of the first assistance information.

Description

TECHNICAL FIELD

[0001] The present application relates to a work assisting apparatus and a work assisting method that assist a work of a work subject. BACKGROUND

[0002] In a work performed at a manufacturing site of a product or the like, an error such as forgetting installation of a component or taking a wrong component can occur. In contrast, a technique is known in which a rework of a work is instructed by detecting an error of the work, or a video related to a correct work is prompted for understanding of the work.

[0003] For example, the work assisting apparatus of Patent Literature 1 sets a monitoring area indicating a range in which a work is monitored, in a photographing range of a photographing section, in correspondence with a position at which each work is performed. The work assisting apparatus of Patent Literature 1 compares a portion of the monitoring area of a measured image and a portion of the monitoring area in another image measured at a time point before the image, and thereby performs dynamic detection of the monitoring area as a target. The work assisting apparatus of Patent Literature 1 notifies of “normal” when dynamic detection is possible for a certain time, and notifies of “abnormal” when dynamic detection is not possible.

[0004] Patent Literature 1: Japanese Patent Application Laid-Open No. 2018-156279 SUMMARY

[0005] However, in the technique of Patent Literature 1 described above, it is possible that a false detection is made at the time of dynamic detection. For example, a hand of a work subject is set as a target of dynamic detection by the work assisting apparatus of Patent Literature 1. In a case where the hand is stretched toward a wrong position in a path in which the work subject passes through the monitoring area, the work assisting apparatus of Patent Literature 1 performs dynamic detection of the hand of the work subject, and thus notifies of “normal” even though it is a wrong position. As described above, in the work assisting apparatus of Patent Literature 1, there is a problem that a false work assistance is performed.

[0006] The present application has been made in view of the above-described circumstances, and an object thereof is to obtain a work assisting apparatus capable of performing accurate work assistance.

[0007] To solve the aforementioned problems and achieve the objectives, the present invention provides a work assisting device that assists with a work consisting of a series of multiple actions performed by a work subject. The device includes a sensor data input device that receives a sequence of sensor data obtained each time sensor data is acquired from measuring the work performed by the work subject. Furthermore, the work assisting device includes: a category data calculator that compares the sensor data sequence with a template, a set of probability distributions of the sensor data at each moment, defined for each type of temporal change in the sensor data, thereby calculating a category data sequence representing the intervals into which the sensor data sequence is segmented; and a reliability calculator that calculates the reliability of each interval based on the information about the intervals indicated by the category data sequence. Furthermore, the work assisting device includes: a reliability determiner that determines whether the reliability satisfies a criterion and generates, as first assistance information, information about the actions included in the intervals for which the reliability satisfies the criterion; and a notification device that notifies the user of the first assistance information.

[0008] Effects of the Invention

[0009] The work assisting device according to the present invention has the effect of being able to perform accurate work assisting. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a diagram showing an example of the configuration of the work assisting device according to the first embodiment.

[0011] Figure 2 This is a diagram for explaining a usage example of the work assisting device according to the first embodiment.

[0012] Figure 3 This is a diagram showing an example of sensor data output by the sensor according to the first embodiment.

[0013] Figure 4 This is a diagram for explaining the data structure of a sensor data sequence stored in the sensor data storage device included in the work assisting device according to the first embodiment.

[0014] Figure 5 It will Figure 4 A diagram showing a sequence of sensor data represented by a time series graph.

[0015] Figure 6 This is a diagram for explaining the data structure of standard data stored in the work information storage device included in the work assisting device according to the first embodiment.

[0016] Figure 7This is a diagram for explaining the data structure of the action name data stored in the work information storage device included in the work assisting device according to the first embodiment.

[0017] Figure 8 This is a diagram for explaining the data structure of the first reference image data stored in the work information storage device included in the work assisting device according to the first embodiment.

[0018] Figure 9 This is a diagram for explaining the data structure of the first template stored in the work information storage device included in the work assisting device according to the first embodiment.

[0019] Figure 10 It will Figure 9 The first template shown is represented by a timing graph.

[0020] Figure 11 This is a diagram for explaining the data structure of the first category transition probability stored in the work information storage device included in the work assisting device according to the first embodiment.

[0021] Figure 12 This is a diagram for explaining the data structure of a category data sequence generated by the category data calculator included in the work support device according to the first embodiment.

[0022] Figure 13 It will Figure 12 The diagram shows a time series graph representing a column of categorical data.

[0023] Figure 14 This is a diagram for explaining the data structure of sequential data values ​​generated by the standard data determiner included in the work assisting device according to the first embodiment.

[0024] Figure 15 This is a diagram for explaining the data structure of the interval evaluation value calculated by the interval evaluator included in the work assisting device according to the first embodiment.

[0025] Figure 16 This is a diagram for explaining the data structure of the reliability calculated by the reliability calculator included in the work support device according to the first embodiment.

[0026] Figure 17 This is a diagram for explaining the data structure of the reliability determination result determined by the reliability determination unit included in the work assisting device according to the first embodiment.

[0027] Figure 18This is a diagram showing, in a time series graph, sensor data sequences before and after the sensor data are removed by the reliability determination unit included in the work support device according to the first embodiment.

[0028] Figure 19 This is a diagram for explaining the data structure of a performance sensor data sequence stored in the work performance storage device included in the work assisting device according to the first embodiment.

[0029] Figure 20 This is a diagram for explaining the data structure of performance data stored in the work performance storage device included in the work support device according to the first embodiment.

[0030] Figure 21 This is a diagram schematically showing how the first assistance information and the second assistance information are displayed on the display device by the work assistance device according to the first embodiment.

[0031] Figure 22 This is a flowchart showing the processing procedure of the process executed by the work assisting device according to the first embodiment.

[0032] Figure 23 This is a diagram showing a first example of the hardware configuration of the work assisting device according to the first embodiment.

[0033] Figure 24 This is a diagram showing a second example of the hardware configuration of the work assisting device according to the first embodiment.

[0034] Figure 25 This is a diagram showing an example of the configuration of a work assisting device according to the second embodiment.

[0035] Figure 26 This is a diagram for explaining the data structure of performance data stored in the work performance storage device included in the work support device according to the second embodiment.

[0036] Figure 27 This is a diagram for explaining the data structure of detection result data detected by the cycle evaluator included in the work assisting device according to the second embodiment.

[0037] Figure 28 This is a diagram schematically showing an aspect of the first assistance information displayed on the display device by the work assistance device according to the second embodiment.

[0038] Figure 29 This is a diagram showing an example of the configuration of a work assisting device according to the third embodiment.

[0039] Figure 30is a view for explaining a use example of the work assistance device according to Embodiment 3. DETAILED DESCRIPTION

[0040] Hereinafter, the work assistance device and the work assistance method according to the embodiments of the present application will be described in detail based on the drawings.

[0041] Embodiment 1

[0042] Figure 1 is a view showing one example of the structure of the work assistance device according to Embodiment 1. The work assistance device 1 is a computer that analyzes a work performed by a specific work subject and executes work assistance.

[0043] In Embodiment 1, the work subject can be one person or a plurality of persons, one machine or a plurality of machines, or a combination thereof. In Embodiment 1, a case where the work subject is one person will be described. The work assistance device 1 is mounted in an arbitrary terminal device in a factory or the like, for example, together with a sensor 2, a display device 3, and an input device 4.

[0044] The work assistance device 1 is connected with the sensor 2, the display device 3, and the input device 4. The work assistance device 1 evaluates a work using data measured by the sensor 2 (hereinafter, referred to as sensor data), generates assistance information indicating work assistance based on an evaluation value, and notifies the assistance information.

[0045] The work assistance device 1 has a sensor data input device 10, a sensor data storage device 31, a work information storage device 32, a category data calculator 20, a standard data determiner 30, a reliability calculator 40, a work performance storage device 34, an interval evaluator 50, a reliability determiner 65, and a notifier 60.

[0046] The sensor data input device 10 acquires time-series sensor data output from the sensor 2 as a sensor data series x and stores it in the sensor data storage device 31. The sensor data storage device 31 stores the sensor data series x arranged each time the sensor data is acquired.

[0047] The work information storage device 32 stores standard data designed in advance with respect to a series of a plurality of actions included in a cycle work. The standard data will be described later. In addition, the work information storage device 32 stores the names of actions (action name data described later) included in the cycle work and images of the actions (first reference image data described later) included in the cycle work.

[0048] Furthermore, the operation information storage device 32 stores pre-designed templates for each type of action, or category. A category is the type of temporal variation in sensor data acquired for each of the multiple actions included in a cyclic operation. The templates contain the mean and variance of the Gaussian distribution of the sensor data. In other words, a template is set for each type of temporal variation in sensor data, or category, and consists of a set of probability distributions at each moment in the sensor data. Details of the templates will be described later.

[0049] Furthermore, the task information storage device 32 stores a pre-designed first category transition probability. The first category transition probability indicates the probability that, after the task subject performs an action corresponding to the first category, it will subsequently perform an action corresponding to the second category. In other words, the first category transition probability indicates the probability of transitioning from a specific category to a specific category.

[0050] The category data calculator 20 calculates the category data sequence s based on the sensor data sequence x stored in the sensor data storage device 31 and the template stored in the work information storage device 32. The category data sequence s is a data sequence representing the intervals and category numbers of the sensor data sequence x. The category data calculator 20 divides the sensor data sequence x into multiple intervals and classifies the time-series sensor data in each of the divided intervals into one of multiple categories, thereby calculating the category data sequence s. The category data calculator 20 sends the category data sequence s to the standard data determiner 30, the reliability calculator 40, and the interval evaluator 50. Furthermore, the category data calculator 20 generates a time-series graph representing the category data sequence s in time series and stores it in the work performance storage device 34. The time-series graph will be described later.

[0051] The standard data determiner 30 generates sequential data values, representing the standard data corresponding to each segment of the sensor data sequence x, that is, each segment represented by the category data sequence s. Specifically, the standard data determiner 30 sets the standard data for each segment, thereby generating sequential data values ​​for each segment. The standard data determiner 30 generates sequential data values ​​based on the standard data stored in the work information storage device 32 and the category data sequence s transmitted from the category data calculator 20. The standard data determiner 30 transmits the sequential data values ​​to the segment evaluator 50. Furthermore, the standard data determiner 30 stores the sequential data values ​​in the work performance storage device 34.

[0052] The interval evaluator 50 compares the category data sequence s with the standard data within the sequential data values ​​to calculate interval evaluation values ​​for each interval divided from the sensor data sequence x. The interval evaluator 50 stores the interval evaluation values ​​in the work performance storage device 34.

[0053] The reliability calculator 40 calculates the reliability R m for each interval shown by the category data column s calculated by the category data calculator 20. The reliability R m is the reliability of each interval and each category number. The reliability calculator 40 generates a performance sensor data column, which is arranged by each time the sensor data included in the interval satisfying the criterion for the reliability R m , and stores it in the job performance storage 34.

[0054] The job performance storage 34 stores the order data value, the interval evaluation value, and the performance sensor data column. In addition, the job performance storage 34 stores the performance category data column, the performance order data value, the performance interval evaluation value, and the like, which are performance data described later.

[0055] The reliability determiner 65 outputs information on the action (the first auxiliary information described later) included in the interval satisfying the criterion for the reliability R m and information on the action included only in the latest interval (the second auxiliary information described later) to the notifier 60. The reliability determiner 65 generates at least one of the first auxiliary information and the second auxiliary information using the action name data, the first reference video data, and the like stored in the job information storage 32, and outputs it to the notifier 60.

[0056] In addition, the reliability determiner 65, in the case where the reliability R m satisfies the criterion, removes the sensor data included in the interval satisfying the criterion for the reliability R m from the sensor data column x and updates the sensor data column x. In addition, the reliability determiner 65, with respect to the interval where the reliability R m satisfies the criterion, updates the performance data indicating the performance of the job. The reliability determiner 65 updates the sensor data column x with respect to the sensor data storage 31 and updates the performance data with respect to the job performance storage 34.

[0057] The notifier 60 outputs the first auxiliary information and the second auxiliary information generated by the reliability determiner 65 to the display device 3.

[0058] Figure 2 is a diagram illustrating an example of use of the job assistance device according to Embodiment 1. In Embodiment 1, the job subject 100 repeatedly performs a job, that is, a cycle job, which is composed of a series of a plurality of actions, over a plurality of times.

[0059] (Sensor 2)

[0060] The sensor 2 outputs sensor data detected by measuring the work to the work assistance device 1. The sensor 2 is, for example, a depth sensor configured to be able to take an image of the work performed by the left hand 101 and the right hand 102 of the work subject 100. The sensor 2 has, for example, a light source that emits infrared light in a specific pattern, and a photographing element that receives light of the infrared light reflected by the detection object, that is, the object such as the left hand 101 and the right hand 102, and the sensor 2 generates depth image data that represents a depth up to the object as a pixel value. Also, the sensor 2 detects the height positions of the left hand 101 and the right hand 102 of the work subject 100 based on the depth image data, and outputs the height positions as sensor data, for example, every 200 milliseconds. As a specific example of the depth sensor, an existing depth sensor such as Kinect (registered trademark) is given. The process of detecting the positions of the left hand 101 and the right hand 102 based on the depth image data can be implemented by an existing process used by the depth sensor.

[0061] The work assistance device 1 acquires the sensor data from the sensor 2, and generates assistance information that represents work assistance based on the sensor data. The work assistance device 1 causes the assistance information to be displayed on the display device 3, and thereby notifies the assistance information to the work subject 100.

[0062] Figure 3 is a graph that represents one example of the sensor data output by the sensor according to Embodiment 1. In Figure 3 , the horizontal axis represents the time at which the sensor data is acquired, and the vertical axis represents the value of the sensor data, that is, the sensor value, that is, the coordinate value of the height position of the left hand 101 and the right hand 102 represented by the sensor data. In Figure 3 , the coordinate value of the height position of the left hand 101 is represented by the coordinate value LH1, and the coordinate value of the height position of the right hand 102 is represented by the coordinate value RH2.

[0063] In Embodiment 1, the sensor data is the height positions of the left hand 101 and the right hand 102 of the work subject 100, and thus becomes a 2-dimensional value.

[0064] In addition, in embodiment 1, the case where a depth sensor is used as sensor 2 is described, but the use of the depth sensor is not limited to that. Any device can be used as long as it can measure the work of the working subject 100 and generate sensor data. As devices other than the depth sensor, for example, a camera, a three-dimensional acceleration sensor, a three-dimensional angular velocity sensor, etc. can be used. In addition, in embodiment 1, the positions of the left hand 101 and the right hand 102 of the working subject 100 are set as the detection objects of sensor 2, but the detection objects are not limited to these. Sensor 2 can also use the head position of the working subject 100, the angles of multiple joints in the body, or the biological information of the working subject 100 as detection objects. Examples of biological information of the working subject 100 are heartbeat and breathing.

[0065] (Operation of each structural element)

[0066] Next, the operation of each component of the work assist device 1 will be described.

[0067] (Sensor data input device 10)

[0068] The sensor data input device 10 obtains the time-series sensor data output from the sensor 2 and appends it to the end of the sensor data sequence x stored in the sensor data storage device 31. In the first embodiment, the sensor data input device 10 appends five sensor data items to the end of the sensor data sequence x stored in the sensor data storage device 31 each time the sensor 2 outputs five sensor data items. In the first embodiment, the sensor 2 outputs sensor data every 200 milliseconds, so the sensor data input device 10 appends five sensor data items to the end of the sensor data sequence x in units of one second.

[0069] (Sensor data storage device 31)

[0070] The sensor data storage device 31 stores a sensor data sequence x, which is arranged each time sensor data is acquired. In the first embodiment, the sensor data sequence x = {x(1), x(2), ..., x(N)}. Here, x(n) is the nth acquired sensor data in the sensor data sequence x. N is the number of sensor data items included in the sensor data sequence x. N is a positive integer, and n is a positive integer ranging from 1 to N.

[0071] Figure 4This diagram illustrates the data structure of the sensor data sequence stored in the sensor data storage device of the work assist device according to Embodiment 1. In Embodiment 1, sensor data is a two-dimensional value, so the sensor data storage device 31 stores the sensor data sequence x(n) as two values. Sensor data sequences SD1 to SD4 respectively represent the structure of the sensor data sequence x acquired 1, 2, 3, and 4 seconds after the start of the movement.

[0072] In the first embodiment, sensor 2 outputs sensor data every 200 milliseconds. Therefore, the number N of sensor data in sensor data sequences SD1 to SD4 is 5, 10, 15, and 20, respectively. That is, sensor data sequence SD1 includes sensor data sequences x(1) to x(5), and sensor data sequence SD2 includes sensor data sequences x(1) to x(10). Furthermore, sensor data sequence SD3 includes sensor data sequences x(1) to x(15), and sensor data sequence SD4 includes sensor data sequences x(1) to x(20).

[0073] Sensor data sequence SD2 is generated by appending sensor data sequences x(6) to x(10) acquired from 1 to 2 seconds after the start of sensor data sequence SD1. Sensor data sequence SD3 is generated by appending sensor data sequences x(11) to x(15) acquired from 2 to 3 seconds after the start of sensor data sequence SD2. Sensor data sequence SD4 is generated by appending sensor data sequences x(16) to x(20) acquired from 3 to 4 seconds after the start of sensor data sequence SD3.

[0074] Figure 5 It will Figure 4 The sensor data columns shown are represented by a time series graph. Figure 5 In the graph, the horizontal axis represents the number n of the time-series sensor data output in the sensor data sequence x, and the vertical axis represents the sensor value of the sensor data. Timing graph G1 corresponds to sensor data sequence SD1, timing graph G2 corresponds to sensor data sequence SD2, timing graph G3 corresponds to sensor data sequence SD3, and timing graph G4 corresponds to sensor data sequence SD4.

[0075] The sensor data input device 10 generates a timing pattern G1 based on the sensor data sequence SD1 and a timing pattern G2 based on the sensor data sequence SD2. Furthermore, the sensor data input device 10 generates a timing pattern G3 based on the sensor data sequence SD3 and a timing pattern G4 based on the sensor data sequence SD4. The sensor data input device 10 stores the timing patterns G1 to G4 in the sensor data storage device 31.

[0076] (Job Information Storage Device 32)

[0077] Next, the operation information storage device 32 according to the first embodiment will be described.

[0078] (Work Information Storage Device 32: Standard Data Column)

[0079] The operation information storage device 32 stores a pre-designed standard data sequence STD for each of the plurality of actions included in the cyclic operation. Standard data sequence STD = {STD(1), STD(2), ..., STD(F)}. STD(f) is the pre-designed standard data for the f-th action included in the cyclic operation. f is a number used to identify each of the plurality of standard data, and is a positive integer from 1 to F. f is the order of the plurality of actions included in the cyclic operation. F is the number of standard data included in the cyclic operation, which is the number of pre-designed actions.

[0080] In the first embodiment, the standard data sequence STD(f) is composed of the time length of the action designed in advance for the f-th action included in the cycle operation and the type of action, that is, the category.

[0081] STD(f) = {STDb(f), STDc(f)}. STDb(f) is a standard length indicating the length of time, pre-designed for the f-th action included in the cyclic operation. STDc(f) is a standard category number indicating the category, pre-designed for the f-th action included in the cyclic operation.

[0082] Figure 6 This is a diagram for explaining the data structure of standard data stored in the work information storage device included in the work assisting device according to the first embodiment. Figure 6 The example of the standard data sequence STD(f) shows a case where the cyclic operation includes F=10 actions. The standard data sequence STD(f) includes STDb(f) and STDc(f).

[0083] exist Figure 6 In the example, the standard category number, STDc(f), is always the same as f, but this does not necessarily have to be the case. Furthermore, the standard data sequence STD(f) can contain multiple pieces of standard data with the same standard category number. This means that the same action can be performed multiple times in a loop operation.

[0084] (Job Information Storage Device 32: Action Name Data)

[0085] The job information storage device 32 according to the first embodiment also stores the name of the f-th action included in the loop job, i.e., action name data LABEL(f). Here, f is also a number used to identify each of the plurality of standard data items, and is an integer from 1 to F. F is the number of the aforementioned standard data items.

[0086] Figure 7 This is a diagram for explaining the data structure of the action name data stored in the work information storage device of the work assisting device according to the first embodiment. Figure 7 In the example of action name data LABEL(f), Figure 6 Likewise, the case where the working subject 100 performs F=10 actions is shown.

[0087] (Job Information Storage Device 32: First Reference Image Data)

[0088] The operation information storage device 32 according to the first embodiment also stores first reference image data REF(f), which indicates an image indicating the f-th action included in the loop operation. Here, f is also a number used to identify a series of actions performed by the operator 100 and is an integer from 1 to F. F represents the number of standard data items.

[0089] Figure 8 This is a diagram for explaining the data structure of the first reference image data stored in the work information storage device included in the work assisting device according to the first embodiment. Figure 8 In the example of the first reference image data REF(f), Figure 6 Similarly, the case where the loop operation includes F=10 motions is shown. That is, the first reference image data REF(f) here includes image data for 10 motions.

[0090] (Job Information Storage Device 32: First Template)

[0091] The operation information storage device 32 also stores the first template g designed in advance for each of the above categories. j Here, j is a number used to identify multiple categories, and is a positive integer from 1 to J. J is the number of the above categories, that is, the first template g j The number of .

[0092] In the first embodiment, the first template g j is a set of Gaussian distributions of sensor data at each moment. In this case, the first template g j The first template g can be designed as the parameters of the Gaussian distribution of the sensor data obtained for the action corresponding to the category j.j ={g j (1), g j (2),…,g j (L)}. g j (i) is the parameter of the Gaussian distribution of the i-th (i is a positive integer from 1 to L) sensor data obtained for the action corresponding to category j, g j (i)={μ j (i), σ j 2 (i)}. Here, μ j (i) is the mean of the Gaussian distribution, σ j 2 (i) is the variance of the Gaussian distribution. In addition, L is the first template g j The length of , that is, the maximum value of the number of sensor data obtained for each action.

[0093] Regarding the first template g according to the first embodiment j , will be described in more detail. As mentioned above, μ j (i) is the average of the Gaussian distribution of the i-th sensor data obtained for the action corresponding to category j. j (i) is a two-dimensional value similar to sensor data. j 2 (i) is the variance of the Gaussian distribution of the i-th sensor data obtained for the action corresponding to category j. In the first embodiment, the variance of the Gaussian distribution of the sensor data is assumed to be the same in any dimension. Therefore, σ j 2 (i) is a 1-dimensional value.

[0094] Figure 9 This is a diagram for explaining the data structure of the first template stored in the work information storage device included in the work assisting device according to the first embodiment. j (i) Contains μ j (i) and σ j 2 (i). exist Figure 9 The first template g j In the example (i), the number of categories J = 10. As mentioned above, j is a number used to identify the category. Figure 9 In the example, the first template g j The length of (i) is L=20.

[0095] Figure 10 It will Figure 9The first template shown is represented by a timing graph. The timing graph 201 includes the first template g corresponding to each category j. j That is, the timing graph 201 includes the first template g corresponding to each class j. 1～10 .exist Figure 10 In the example, the first template g of category j is j It is represented by the timing graphs GTa(j) and GTb(j). The timing graph GTa(j) is μ j (i) is the timing graph, and the timing graph GTb(j) is σ j 2 In the timing pattern 201, the first template g of categories 1 to 10 is 1～10 , including the timing graphs GTa(1), GTb(1)~GTa(10), GTb(10).

[0096] In both the timing graphs GTa(j) and GTb(j), the horizontal axis represents the number i of the sensor data obtained for the action corresponding to each category j. The vertical axis of the timing graph GTa(j) represents the mean μ of the Gaussian distribution of the sensor data obtained for the action corresponding to each category j. j (i) The vertical axis of the time series graph GTb(j) represents the variance σ of the Gaussian distribution of the sensor data obtained for the action corresponding to each category j. j 2 (i).

[0097] As mentioned above, the mean of a Gaussian distribution is a two-dimensional value, similar to sensor data. Furthermore, the variance of a Gaussian distribution is assumed to be the same in any dimension. Therefore, the variance of a Gaussian distribution is a one-dimensional value.

[0098] (Job Information Storage Device 32: First Category Conversion Probability)

[0099] The operation information storage device 32 according to the first embodiment also stores a pre-designed first category transition probability P(j|j'). Here, j and j' are numbers used to identify multiple categories. P(j|j') represents the probability that, after the operator 100 performs an action corresponding to category j', it will subsequently perform an action corresponding to category j.

[0100] Figure 11 This is a diagram for explaining the data structure of the first category conversion probability stored in the work information storage device of the work assisting device according to Embodiment 1. As described above, j and j' are numbers for identifying the category. Figure 11 , the values ​​of the first class transition probabilities P(j|j') for the combinations of j=1 to 10 and j'=1 to 10 are shown.

[0101] (Category Data Calculator 20)

[0102] Next, the category data calculator 20 according to the first embodiment will be described. The category data calculator 20 calculates the category data based on the sensor data sequence x stored in the sensor data storage device 31 and the templates g stored in the operation information storage device 32. j , calculate the category data column s.

[0103] Specifically, the type data calculator 20 determines a plurality of sections into which the sensor data sequence x is temporally divided, and a type of each section indicating a type of temporal change in the sensor data included in each section.

[0104] Then, the category data calculator 20 generates a category data sequence s representing each section and each category number of the sensor data sequence x. Category data sequence s = {s1, s2, ..., s m ,…,s M M (M is a positive integer) is the number of intervals included in the category data sequence s, that is, the number of intervals into which the sensor data sequence x is divided.

[0105] m is a number for identifying each of the multiple intervals divided into the sensor data sequence x, and is an integer from 1 to M. M is a number indicating the latest interval among the intervals divided into the sensor data sequence x. m is the order of the multiple intervals in the sensor data sequence x.

[0106] s m is the element of the class data sequence s in the mth interval into which the sensor data sequence x is divided, s m ={a m , b m , c m}. a m is the starting number of the mth interval into which the sensor data column x is divided, b m is the length of the mth interval into which the sensor data column x is divided, c m is the category number for classifying the mth interval into which the sensor data sequence x is divided. The category data calculator 20 uses the category data sequence s m ={a m , b m , c m}, thereby, for example, the time series sensor data x included in the mth interval obtained by dividing the sensor data sequence x can be m Represented as x m ={x(a m ), x(a m+1 ),…,x(am +b m -1)}.

[0107] In the first embodiment, the work support device 1 uses FF-BS (Forward Filtering-Backward Sampling) to segment the sensor data sequence x into multiple intervals and classify the time-series sensor data in each segment into one of multiple categories. FF-BS consists of two steps: the probability calculation involved in the FF step and the segmentation and classification involved in the BS step.

[0108] (Category Data Calculator 20: FF Step)

[0109] First, the FF step is described. In the FF step, the category data calculator 20 uses the following formula (1) to convert the first template g corresponding to the nth sensor data column x(n) in the sensor data column x to the category j. j (i) is generated by the probability P(x(n)|X j , I j ) is calculated as Gaussian distributionNormal.

[0110] Furthermore, when the sensor data sequence x, which has already been segmented into the first to n-ith intervals, can be segmented into the nth interval, the category data calculator 20 calculates the probability α[n][i][j] that the nth interval is category j using the following equation (2). P(j|j') is the first category transition probability described above. Equation (2) is a loop equation, and the category data calculator 20 can sequentially calculate the probabilities α[n][i][j] from n=1 to n=N.

[0111] [Formula 1]

[0112]

[0113] [Formula 2]

[0114]

[0115] (Category Data Calculator 20: BS Step)

[0116] Next, the BS step will be described. In the BS step, the category data calculator 20 samples the category data using the following equation (3) for the intervals into which the sensor data column x is divided. In equation (3), the b in the first row is m’ and c m’ It is a random variable obtained from the probability distribution on the right. The second row is the variable a m’ The cycle formula.

[0117] [Formula 3]

[0118]

[0119] According to the formula (3), the category data calculator 20 can generate the category data s from m'=1 to m'=M in sequence. m’ ={a m’ , b m’ , c m’ Here, M is the number of intervals into which the sensor data sequence x is divided by equation (3). m’ is the category data in the m'th interval from the rear of the sensor data sequence x. In formula (3), the category data in the intervals from which the sensor data sequence x is divided are calculated sequentially from the rear of the sensor data sequence x. That is, the category data sequence s in the m'th interval from the front of the sensor data sequence x is m ={a m , b m , c m}={a M-m’+1 , b M-m’+1 , c M-m’+1}.

[0120] Figure 12 This is a diagram for explaining the data structure of the category data column generated by the category data calculator included in the work assisting device according to the first embodiment. Figure 12 In the figure, it is shown that Figure 4 The sensor data sequence x shown is based on multiple first templates g j The generated category data column 202. In addition, the number of categories J=10.

[0121] In the category data sequence 202, the category data sequence s corresponding to the sensor data sequences SD1 to SD4 is m The data includes category data CD1 to CD4. Category data CD1 and CD2 both include category data s1. Category data CD3 includes category data s1 and s2. Category data CD4 includes category data s1, s2, and s3.

[0122] Figure 13 It will Figure 12 The diagram shows a sequence of category data represented by a time series graph. The time series graph 203 includes time series graphs GA1 to GA4 corresponding to category data CD1 to CD4. Figure 13 In the middle, the horizontal axis and Figure 5 Similarly, it is the number n at which the time-series sensor data is output in the sensor data sequence x.

[0123] The category data calculator 20 generates a timing graph GA1 based on the category data CD1 and a timing graph GA2 based on the category data CD2. Furthermore, the category data calculator 20 generates a timing graph GA3 based on the category data CD3 and a timing graph GA4 based on the category data CD4. The category data calculator 20 stores the generated timing graphs GA1 to GA4 in the work record storage device 34.

[0124] Each rectangle including any of the numbers "1" to "10" represents each interval [a m , a m +b m -1]. In addition, the numbers "1" to "10" written in any section of each section represent the category number c after classifying each section. m For example, in the timing graph GA4, the interval where m=1 is a m =1, a m +b m -1=9, c m = 1. In addition, the interval where m = 2 is a m =10, a m +b m -1=17, c m =2, m=3 is a m =18, a m +b m -1=20, c m =3.

[0125] (Category Data Calculator 20: Basis of Quick Reporting)

[0126] Generally speaking, after the operation main body 100 starts each action of the cyclic operation included in a series of actions, the number of sensor data acquired for the action is small, so there is a m The tendency of the estimation-related accuracy to deteriorate.

[0127] On the other hand, the category data calculator 20 can also calculate the interval and category number c corresponding to each action immediately after the operation main body 100 starts each action included in the loop operation. m Highly accurate estimation is performed. That is, the category data calculator 20 probabilistically samples each interval divided from the sensor data sequence x and performs estimation, thereby being able to estimate that the action performed by the operator 100 has changed from the previous action to the current action. Furthermore, the number of sensor data that can be obtained for the current action is small, but the category data calculator 20 uses the first category transition probability to consider the category number c of the interval corresponding to the previous action of the current action. mThe category number c of the interval corresponding to the action m Make an inference.

[0128] (Standard Data Determiner 30)

[0129] Next, the standard data determiner 30 according to the first embodiment will be described. The standard data determiner 30 generates sequential data values ​​representing the standard data corresponding to the respective sections into which the sensor data sequence x is divided. m is a sequential data value generated in the m-th interval obtained by dividing the sensor data sequence x, and is a standard data sequence STD(l m ).

[0130] The standard data determiner 30 in the first embodiment is used to determine the category data column s calculated by the category data calculator 20. m Included category numbers c m The series {c1, c2, ..., c M} and standard data column STD(l m ) is a series of standard category numbers included in the series, and a flexible matching method such as multiple sequence arrangement is used to generate sequential data values ​​l m In the first embodiment, the standard data determiner 30 determines the series {STDc1, STDc2, ..., STDc F , STDc1, STDc2, …, STDc F}Use elastic matching method.

[0131] The cyclic operation performed by the operation main body 100 is repeatedly performed over a plurality of times, so the category number c included in the category data column s is m Sometimes the category number c corresponding to the last action of the loop operation is m and the category number c corresponding to the first action of the cycle operation m For the series as described above, the standard data determiner 30 also includes the standard data column STD (l m ) by connecting multiple series of standard category numbers included in the series, thereby easily generating sequential data values ​​l m When the standard data determiner 30 does not correspond to the m-th interval into which the sensor data sequence x is divided, it uses the sequence data value l as the standard data value l. m NAs are calculated.

[0132] Figure 14This is a diagram for explaining the data structure of the sequential data value generated by the standard data determiner included in the work assisting device according to the first embodiment. Figure 14 In the figure, it is shown that Figure 12 The category data columns shown are m and the sequential data value l generated by the standard data determiner 30 m The corresponding data group 204 is arranged in association. The corresponding data group 204 includes the sequential data value l corresponding to the category data CD1 to CD4. m .exist Figure 14 In the example, the category data CD1 and the sequence data value l m The associated data is represented by data DL1, which combines the category data CD2 and the sequence data value l m The associated data is represented by data DL2. In addition, the category data CD3 and the sequence data value l m The associated data is represented by data DL3, which combines the category data CD4 and the sequence data value l m The associated data is represented by data DL4 . The standard data determiner 30 stores the corresponding data group 204 in the work performance storage device 34 .

[0133] (Standard Data Determiner 30: Distinguishing Multiple Occurrences of the Same Action in a Cyclic Operation)

[0134] The standard data determiner 30 can determine the order corresponding to each action even when the same action is performed multiple times in a cycle operation. That is, the standard data determiner 30 can determine the order corresponding to each action. m And the series of standard data, namely the standard data series STD(l m ) are compared, so the cyclic operation includes m In the case of multiple types of data, it is also possible to represent the sequential data value l corresponding to each standard data m Make confirmation.

[0135] (Interval Evaluator 50)

[0136] Next, the interval evaluator 50 according to the first embodiment will be described. The interval evaluator 50 calculates an interval evaluation value, which is a value obtained by evaluating each interval divided into the sensor data sequence x. The interval evaluator 50 evaluates the category data sequence s m Compare with the corresponding standard data and calculate the interval evaluation value.

[0137] In the first embodiment, the interval evaluator 50 evaluates the interval evaluation value V for each interval m. m Perform calculations.m ={Vb m , Vc m}. Vb m is the length b of the mth interval into which the sensor data sequence x is divided m Does it exceed the standard length STDb(l m The interval evaluator 50 uses the following formula (4) to evaluate Vb m Here, len coef is a specific coefficient, which is set to len in the first embodiment. coef =1.5. In addition, Vc m is the category number c of the mth interval into which the sensor data sequence x is divided m Is it consistent with the standard category number STDc(l m ) is evaluated. The interval evaluator 50 uses the following formula (5) to evaluate Vc m Perform calculations.

[0138] [Formula 4]

[0139]

[0140] [Formula 5]

[0141]

[0142] Figure 15 This is a diagram for explaining the data structure of the interval evaluation value calculated by the interval evaluator included in the work assisting device according to the first embodiment. Figure 15 In the figure, it is shown that Figure 14 The category data columns shown are m , sequential data value l m and the interval evaluation value V calculated by the interval evaluator 50 m The corresponding data group 205 is arranged in association.

[0143] The corresponding data group 205 includes the interval evaluation values ​​V corresponding to the category data CD1 to CD4. m .exist Figure 15 In the example, the category data CD1 and the sequence data value l m and interval evaluation value V m The associated data is represented by data DV1, and the category data CD2 and the sequence data value l m and interval evaluation value V m The associated data is represented by data DV2. In addition, the category data CD3 and the sequence data value l m and interval evaluation value V mThe associated data is represented by data DV3, and the category data CD4 and the interval evaluation value V m and sequential data values ​​l m The associated data is represented by data DV4.

[0144] By using the sequential data value l m , so that the standard data corresponding to the mth interval divided from the sensor data column x can be expressed as STD(l m ).For example Figure 15 The sequential data value in the second interval of the data DV3 is the sequential data value l2=1, and the corresponding standard data is based on Figure 6 , is STD(1) = {STDb(1), STDc(1)} = {8, 1}. In addition, the category data s2 in the second interval of data DV3 = {a2, b2, c2} = {3, 13, 1}. That is, according to formula (4), b2 = 13 > STDb(1) × len coef =12, so Vb2 = "abnormal" is calculated.

[0145] The interval evaluator 50 calculates the value of each Vb m and each Vc m “Normal” or “Abnormal” is calculated and registered in the corresponding data group 205. The section evaluator 50 stores the corresponding data group 205 in the work record storage device 34.

[0146] (Reliability Calculator 40)

[0147] Next, the reliability calculator 40 according to the first embodiment will be described. The reliability calculator 40 calculates the type data sequence s calculated by the type data calculator 20. m Each interval shown has a reliability R m Perform calculations. m In the first embodiment, the reliability calculator 40 calculates the reliability R for each interval divided from the sensor data sequence x in the order from the newest interval. m .

[0148] Figure 16 This is a diagram for explaining the data structure of the reliability calculated by the reliability calculator included in the work assisting device according to the first embodiment. Figure 16 In the figure, it is shown that Figure 15 The category data columns shown are m and the reliability R calculated by the reliability calculator 40 m The corresponding data group 206 is arranged in association.

[0149] The corresponding data group 206 includes the reliability R corresponding to the category data CD1 to CD4. m .exist Figure 16 In the example, the category data CD1 and the reliability R m The associated data is represented by data DR1, and the category data CD2 and the reliability R m The associated data is represented by data DR2. In addition, the category data CD3 and the reliability R m The associated data is represented by data DR3, and the category data CD4 and the reliability R m The associated data is represented by data DR4.

[0150] (Reliability Calculator 40: Reliability R m Changes in

[0151] Furthermore, the reliability calculator 40 of the first embodiment has the reliability R m In the calculation of , the order from the end of each interval into which the sensor data sequence x is divided is used, but the present invention is not limited to this. m The intervals and categories shown are numbered c m The calculation is performed again each time new sensor data is obtained. As described above, when new sensor data is obtained, the reliability calculator 40 can use the previously calculated intervals and category numbers c m and the newly calculated interval and category numbers c m The consistency after comparison is taken as the reliability R m That is, the reliability calculator 40 may determine that the calculated intervals and category numbers c are higher if the above-mentioned consistency is higher. m The reliability R m The higher.

[0152] In addition, the reliability calculator 40 may use the time difference between the current time and the end time of each section into which the sensor data is divided as the reliability R m That is, the reliability calculator 40 may determine that the longer the time difference is, the more likely it is that the calculated intervals and category numbers c m The reliability R m The higher the reliability R m The smaller the number of sensor data obtained for the actions included in each interval divided into the sensor data sequence x, the higher the reliability R m The lower it becomes, the more sensor data there is, the lower the reliability R m Become higher.

[0153] In addition, the reliability calculator 40 can divide the sensor data sequence x into sections, and the sensor data included in each section and the category number c in each section. m The corresponding first template g j The compared consistency is taken as the reliability R m The reliability calculator 40 stores the corresponding data set 206 in the work performance storage device 34 .

[0154] (Work Performance Storage Device 34)

[0155] The work record storage device 34 stores the reliability R in each section obtained by dividing the sensor data sequence x each time. m The sensor data included in the section that meets the criteria is stored in the array of actual sensor data.

[0156] The work performance storage device 34 also stores each section and each category number c representing the performance sensor data sequence. m The performance category data column is stored.

[0157] The work performance storage device 34 also stores performance sequence data values ​​which are values ​​indicating standard data corresponding to each section into which the performance sensor data sequence is divided.

[0158] The work performance storage device 34 also stores the interval evaluation value V in each interval into which the performance sensor data sequence is divided. m That is, the performance interval evaluation value is stored.

[0159] The work performance storage device 34 stores the performance sensor data sequence, the performance category data sequence, the performance order data value, and the performance interval evaluation value in accordance with the instruction from the reliability determination unit 65 .

[0160] (Reliability Determiner 65)

[0161] Next, the reliability determination unit 65 according to the first embodiment will be described. The reliability determination unit 65 determines the reliability R in each interval calculated by the reliability calculator 40. m In the first embodiment, the reliability judgement unit 65 determines whether the reliability R m If the value exceeds 2, the reference is determined to be satisfied. m >2, the reliability R in each interval m Calculate the judgment result.

[0162] Figure 17 This is a diagram for explaining the data structure of the reliability determination result determined by the reliability determination unit included in the work assisting device according to the first embodiment. Figure 17 In the figure, it is shown that Figure 16 The category data columns shown are m and reliability R m and the reliability R determined by the reliability determiner 65 m Here, the reliability R m The benchmark is 2, and the reliability judgement device 65 is in R m When the value is greater than 2, it is determined to be the reliability R m The situation in which the benchmark is met is explained. Figure 17 In R m The character string "True" written in the column with >2 indicates the reliability R m Satisfies the benchmark, "False" indicates the confidence level R m Benchmark not met.

[0163] The corresponding data group 207 includes the reliability R corresponding to the category data CD1 to CD4. m The judgment result. Figure 17 In the above example, the category data CD1 and the reliability R m and reliability R m The data related to the judgment result is represented by data DJ1, and the category data CD2, the reliability R m and reliability R m The data related to the judgment result is represented by data DJ2. In addition, the category data CD3, the reliability R m and reliability R m The data related to the judgment result is represented by data DJ3, and the category data CD4, the reliability R m and reliability R m The data related to the determination result is represented by data DJ4. Figure 17 In the example of , only the reliability R1 of the first interval in the data DJ4 satisfies the criterion.

[0164] (Reliability Determination Device 65: Second Auxiliary Information)

[0165] The reliability determination unit 65 generates the second auxiliary information. The second auxiliary information in the first embodiment includes the interval evaluation value V in the Mth interval obtained by dividing the sensor data sequence x. M , action name data LABEL(l corresponding to the Mth interval M ) and the first reference image data REF(l corresponding to the Mth interval M As described above, M is the number indicating the latest interval among the intervals into which the sensor data sequence x is divided. Therefore, the second auxiliary information is the category data sequence s mThe reliability determination unit 65 is based on the interval evaluation value V M , generating the second auxiliary information, thereby enabling operation assistance based on the length of time spent on each action or the correctness of the type of each action.

[0166] (Reliability Determinator 65: Updates Sensor Data Sequence x)

[0167] The reliability determiner 65 also calculates the reliability R m The sensor data included in the section that satisfies the reference is removed from the sensor data sequence x, and the sensor data sequence x in the sensor data storage device 31 is updated.

[0168] Figure 18 This is a diagram showing, in a time series graph, sensor data sequences before and after the sensor data are removed by the reliability determination unit included in the work support device according to the first embodiment. Figure 18 The upper graph in the figure is the sensor data column x before the reliability determiner 65 removes the sensor data, showing the difference between the sensor data column x and the reliability determiner 65. Figure 5 The same graphic as the bottom graphic.

[0169] The reliability determination device 65 according to the first embodiment is Figure 17 The data DJ4 is determined to be reliable in the first interval R m Since the criterion is satisfied, the time series sensor data sequence 1={x(a1), x(a2), ..., x(a1+b1-1)}={x(1), x(2), ..., x(9)} included in the first interval divided from the sensor data sequence x is removed. Figure 18 The lower graph in FIG is the sensor data sequence x after the sensor data is removed by the reliability determiner 65. The sensor data sequence x is obtained by removing {x(1), x(2), ..., x(9)}, so that {x(10), x(11), ..., x(20)} before the removal becomes {x(1), x(2), ..., x(11)} after the removal.

[0170] (Reliability Determination Module 65: Adding Actual Sensor Data Column)

[0171] The reliability determination unit 65 also updates the actual sensor data sequence stored in the work record storage device 34. In the first embodiment, the reliability determination unit 65 updates the reliability R of each section obtained by dividing the sensor data sequence x. m The sensor data included in the section that meets the criteria is added to the actual sensor data column.

[0172] Figure 19This figure illustrates the data structure of the performance sensor data sequence stored by the work performance storage device of the work assist device according to Embodiment 1. In Embodiment 1, sensor data is a two-dimensional value, so the work performance storage device 34 stores the performance sensor data sequence Hx(n2) as two values. In Embodiment 1, the performance sensor data sequence Hx = {Hx(1), Hx(2), ..., Hx(N2)}. Here, Hx(n2) is the n2th acquired sensor data in the performance sensor data sequence Hx. Furthermore, N2 is the number of sensor data items included in the performance sensor data sequence Hx.

[0173] (Reliability Determination Module 65: Adding Performance Category Data Column)

[0174] The reliability determination unit 65 also updates the performance category data column Hs stored in the work performance storage device 34. Performance category data column Hs = {Hs1, Hs2, ..., Hs m2 ,…,Hs M2 Here, M2 is the number of intervals included in the performance category data sequence Hs, that is, the number of intervals into which the performance sensor data sequence Hx is divided. m2 is a number used to identify each of the multiple intervals divided into the performance sensor data sequence Hx, and is an integer from 1 to M2. m2 is the order of the multiple intervals in the performance sensor data sequence Hx. Hs m2 The elements of the performance category data sequence Hs in the m2th interval obtained by dividing the performance sensor data sequence Hx are shown in FIG. m2 ={Ha m2 , Hb m2 , Hc m2}. Ha m2 Hb is the starting number of the m2th interval into which the actual sensor data sequence Hx is divided. m2 Hc is the length of the m2th interval into which the actual sensor data sequence Hx is divided. m2 The m2th interval segmented from the actual sensor data sequence Hx is classified into a category number. m2 ={Ha m2 , Hb m2 , Hc m2}, for example, the time series sensor data included in the m2-th interval divided from the actual sensor data sequence Hx can be expressed as {Hx(Ha m2 ), Hx(Ha m2+1 ),…,Hx(Ha m2 +Hb m2 -1)}.

[0175] In the first embodiment, the reliability determination unit 65 determines the reliability R of each section obtained by dividing the sensor data sequence x. m The interval and category number that meet the criteria c m Append to the performance category data column Hs.

[0176] (Reliability Determination Module 65: Adding Performance Order Data Value)

[0177] The reliability determination unit 65 also adds the new performance order data value to the work performance storage device 34. In the first embodiment, the reliability determination unit 65 compares the reliability R in each section obtained by dividing the sensor data sequence x. m The order data value corresponding to the interval that meets the standard is added as a new performance order data value. The new performance order data value added by the reliability determiner 65 is the performance order data value H1 representing the standard data corresponding to the m2th interval divided from the performance sensor data sequence Hx. m2 .

[0178] (Reliability Determination Module 65: Adding Performance Interval Evaluation Value)

[0179] The reliability determination unit 65 also adds the new performance interval evaluation value to the work performance storage device 34. In the first embodiment, the reliability determination unit 65 calculates the reliability R of each interval obtained by dividing the sensor data sequence x. m The interval evaluation value Vm in the interval that meets the standard is added as a new performance interval evaluation value. The performance interval evaluation value in the m2th interval divided from the performance sensor data sequence Hx is the performance interval evaluation value HV m2 ={HVb m2 , HVc m2 Here, HVb m2 The length Hb of the m2th interval obtained by dividing the actual sensor data sequence Hx m2 Does it exceed the standard length STDb(Hl m2 ) is evaluated. m2 The category number Hc for the m2th interval into which the actual sensor data sequence Hx is divided m2 Is it consistent with the standard category number STDc(Hl m2 ) are evaluated consistently.

[0180] (Reliability Determination 65: Performance Data)

[0181] Figure 20This figure is used to explain the data structure of the performance data stored in the work performance storage device of the work assisting device according to the first embodiment. The performance data 208 stored in the work performance storage device 34 by the reliability determination unit 65 includes the performance category data column Hs m2 , performance order data value Hl m2 and performance interval evaluation value HV m2 .

[0182] exist Figure 20 In the example, the performance category data column Hs m2 The number of intervals included is M2=1, Figure 19 The actual sensor data sequence Hx shown is divided into 1 interval. Figure 20 The performance order data value H1 shown m2 , performance interval evaluation value HV m2 Add respectively indicated in Figure 17 The reliability R in the data DJ4 m The values ​​of the sequential data value l1 and the interval evaluation value V1 of the standard data corresponding to the first interval that meets the standard.

[0183] (Reliability Determination Device 65: First Auxiliary Information)

[0184] The reliability determination unit 65 generates the first auxiliary information. The first auxiliary information in the first embodiment is the actual section evaluation value HV in each section into which the actual sensor data sequence Hx is divided. m2 The performance interval evaluation value of the first auxiliary information is the performance interval evaluation value HV in the m2th interval divided by the performance sensor data sequence Hx. m2 The action name data of the first auxiliary information is the action name data LABEL (H1) corresponding to the m2th interval. m2 ). The reliability determination unit 65 is based on the performance interval evaluation value HV m2 By generating the first auxiliary information, it is possible to perform operation assistance based on the time taken for each cycle operation or the omission of operations in each cycle operation.

[0185] (Notifier 60)

[0186] Next, the notification device 60 according to the first embodiment will be described. The notification device 60 outputs the first auxiliary information and the second auxiliary information generated by the reliability determination device 65 to the display device 3. The first auxiliary information output by the notification device 60 is the reliability R in each section into which the sensor data sequence x is divided. mThe second auxiliary information output by the notification device 60 is information related to the action included in the latest interval among the intervals into which the sensor data sequence x is divided. In addition, the notification device 60 follows the instruction from the reliability determination device 65 to convert the first reference image data REF(l M ) to notify by repeating the playback.

[0187] (Display device 3)

[0188] Next, the display device 3 according to Embodiment 1 will be described. The display device 3 according to Embodiment 1 is an image forming device such as a liquid crystal display device. The display device 3 displays the first and second auxiliary information notified by the notification device 60 as images to the operator 100.

[0189] Figure 21 Schematically showing the first and second auxiliary information displayed on the display device by the work assisting device according to the first embodiment. Figure 21 In FIG. 1 , the display device 3 is shown as the first auxiliary information. Figure 20 The actual performance interval evaluation value HV1 = {HVb1, HVc1} = {"normal", "normal"} in the m2 = 1th interval shown and the corresponding action name data LABEL (Hl1) = LABEL (1) = "Take out component A" are displayed.

[0190] The first auxiliary information displayed on the display device 3 is the reliability R in each section into which the sensor data sequence x is divided as described above. m Information about the actions included in the interval that satisfies the criterion. That is, the first auxiliary information is the information with a higher reliability R than the second auxiliary information. m As mentioned above, the first auxiliary information is the information about the reliability R m By displaying the first auxiliary information, it is possible to perform a comparison based on the reliability R. m High-information work assistance can prevent erroneous work assistance from being performed.

[0191] exist Figure 21 In the display device 3, the second auxiliary information is also Figure 15 The interval evaluation value V3 = {"Normal", "Normal"} for the latest interval, i.e., the third interval, indicated by the data DV4, the action name data LABEL(l3) = LABEL(3) = "Remove Part B" corresponding to the third interval, and the first reference image data REF(l3) = REF(3) corresponding to the third interval are displayed. At this time, the first reference image data is repeatedly played as described above.

[0192] Furthermore, the second auxiliary information displayed on the display device 3 is information related to the action included in only the latest interval among the intervals into which the sensor data sequence x is divided. As described above, the work assisting device 1 can use the category data calculator 20 to calculate the interval and category number c corresponding to each action immediately after the work subject 100 starts each action included in the cycle work. m That is, the second auxiliary information can be used to provide operation assistance based on highly rapid information.

[0193] The display device 3 can be a head-mounted display or the like mounted on the body. In addition, a speaker can be used instead of the display device 3, or the speaker can notify the first auxiliary information and the second auxiliary information by sound. The case where the reference image data is displayed as a 2D video as the second auxiliary information is shown, but the second auxiliary information is not limited to this, and the reference image data can also be displayed as a still picture or as a 3D image. Alternatively, the display device 3 can also display the movement trajectory of both hands superimposed on the reference image data. In addition, the work assisting device 1 can also include a video indicating a series of actions that connect the first reference image data with the first auxiliary information.

[0194] (Input device 4)

[0195] Next, the input device 4 involved in embodiment 1 is described. The input device 4 is composed of a device that can input information from the outside, such as a mouse, a keyboard, a touch panel, and a memory card reading device. The input device 4 in embodiment 1 inputs the "action start" or "action end" signal received from the work body 100 to the work assist device 1. When the work assist device 1 receives the "action start" signal from the input device 4, it starts the action corresponding to the "action start" signal. On the other hand, when the work assist device 1 receives the "action end" signal from the input device 4, it ends the action corresponding to the "action end" signal. The work assist device 1 involved in embodiment 1 operates in the above manner.

[0196] (flow chart)

[0197] Then, the operation of the work assisting device 1 according to the first embodiment will be described with reference to a flowchart. Figure 22 This is a flowchart showing the processing procedure of the process executed by the work assisting device according to the first embodiment.

[0198] The sensor data input device 10 adds the sensor data newly acquired by the sensor 2 to the end of the sensor data column x stored in the sensor data storage device 31 (step S100 ).

[0199] Next, the category data calculator 20 calculates the first template g based on the sensor data sequence x stored in the sensor data storage device 31 and the first template g stored in the operation information storage device 32. j , calculate the category data column s (step S101).

[0200] Next, the standard data determiner 30 generates sequential data values ​​that are values ​​representing standard data corresponding to each section of the category data sequence s calculated by the category data calculator 20 (step S102 ).

[0201] Next, the interval evaluator 50 evaluates the interval evaluation value V for each interval indicated by the category data sequence s calculated by the category data calculator 20. m Calculation is performed (step S103).

[0202] Next, the reliability calculator 40 calculates the reliability R for each interval indicated by the category data column s calculated by the category data calculator 20. m Calculation is performed (step S104).

[0203] Next, the reliability determiner 65 generates second auxiliary information, and the notifier 60 notifies the second auxiliary information (step S105 ).

[0204] Next, the reliability determiner 65 substitutes 1 into the number m for identifying each section into which the sensor data sequence x is divided (step S106 ).

[0205] Next, the reliability determination unit 65 determines the reliability R in the m-th interval into which the sensor data sequence x is divided. m Whether the criterion is satisfied (step S107). m If the reference is satisfied (step S107, Yes), the operation of the work assisting device 1 proceeds to step S108. On the other hand, in step S107, if the reliability R m When the criterion is not satisfied (step S107 , No), the work assisting device 1 proceeds to step S110 .

[0206] In the reliability R m If the criteria are met, the reliability determiner 65 sets the reliability R m The sensor data included in the section that meets the criterion is removed from the sensor data sequence x, and the sensor data sequence x is updated (step S108 ).

[0207] Next, the reliability determiner 65 determines the reliability R mThe section satisfying the criterion updates the performance data 208 (step S109). Specifically, the reliability determiner 65 updates the performance sensor data column Hx, the performance category data column Hs, the performance order data value Hl, and the performance section evaluation value HV with respect to the reliability R m The section satisfying the criterion updates the performance sensor data column Hx, the performance category data column Hs, the performance order data value Hl, and the performance section evaluation value HV m2 with respect to the reliability R m2 .

[0208] Next, the reliability determiner 65 increments m by 1 (step S110). Also, the reliability determiner 65 determines whether m exceeds the number M of sections included in the category data column s, that is, whether m > M (step S111).

[0209] In the case where m does not exceed M (step S111, No), the operation of the work support device 1 returns to step S107. On the other hand, in the case where m exceeds M in step S111 (step S111, Yes), the operation of the work support device 1 proceeds to step S112.

[0210] In the case where m exceeds M, the reliability determiner 65 notifies the first support information to the display device 3 (step S112).

[0211] Next, the work support device 1 confirms the signal input from the input device 4, and determines whether the operation is ended based on the input signal (step S113). In the case where the signal of "operation end" is not input from the input device 4 (step S113, No), the operation of the work support device 1 returns to step S100. On the other hand, in the case where the signal of "operation end" is input from the input device 4 (step S113, Yes), the work support device 1 ends the operation.

[0212] (Hardware structure)

[0213] Next, the hardware structure of the work support device 1 according to the embodiment 1 will be described. Each function of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the section evaluator 50, the notifier 60, and the reliability determiner 65 in the work support device 1 can be realized by a processing circuit. The processing circuit can be a dedicated hardware device, or a general-purpose device such as a CPU (also referred to as Central Processing Unit, central processing device, processing device, arithmetic device, microprocessor, microcomputer, processor, DSP (Digital Signal Processor)) which executes a program stored in a memory. In addition, each function of the sensor data storage device 31, the work information storage device 32, and the work performance storage device 34 can be realized by a memory.

[0214] In a case where the processing circuit is a dedicated hardware device, the processing circuit can be, for example, a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65 can be implemented by the processing circuit individually, or the functions of each of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65 can be aggregated and implemented by the processing circuit.

[0215] In a case where the processing circuit is a CPU, each of the functions of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65 is implemented by software, firmware, or a combination of software and firmware. At least one of the software and the firmware is described as a program and stored in a memory. The processing circuit reads out the program stored in the memory and executes it, thereby implementing the functions of each of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65. These programs can cause a computer to execute the order or the method of the actions of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65. Here, the memory can be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a magnetic disk, a floppy disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), or the like.

[0216] Furthermore, as for each of the functions of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65, a part thereof can be implemented by a dedicated hardware device, and the other part thereof can be implemented by software or firmware. For example, as for the function of the sensor data input device 10, it can be implemented by the processing circuit as a dedicated hardware device, and as for each of the functions of the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65, it can be implemented by the processing circuit reading out and executing a program stored in the memory.

[0217] Figure 23 1 is a diagram showing a first example of the hardware configuration of the work assisting device according to the first embodiment. Figure 23 , an example of a hardware configuration is shown in which the processing circuit 1001 is a dedicated hardware device. Figure 23 In the example shown in FIG1 , the functions of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65 are implemented by a processing circuit 1001. Furthermore, the functions of the sensor data storage device 31, the work information storage device 32, and the work performance storage device 34 are implemented by a memory 1002. The processing circuit 1001 is connected to the memory 1002 via a data bus 1003.

[0218] Figure 24 : is a diagram showing a second example of the hardware configuration of the work assisting device according to the first embodiment. Figure 24 In FIG, a hardware configuration example is shown in which the processing circuit is a CPU. Figure 24 In the example of FIG, the functions of the sensor data input device 10, the category data calculator 20, the standard data determiner 30, the reliability calculator 40, the interval evaluator 50, the notifier 60, and the reliability determiner 65 are implemented by executing the program stored in the memory 1005 by the processor 1004. In addition, the functions of the sensor data storage device 31, the work information storage device 32, and the work performance storage device 34 are implemented by the memory 1002. The processor 1004 is connected to the memory 1002 and the memory 1005 via the data bus 1003. In addition, the work assisting devices according to the second and third embodiments can also be implemented by the same hardware structure as the work assisting device 1 according to the first embodiment.

[0219] As described above, according to the first embodiment, the work assisting device 1 calculates the reliability R based on the detection result of the work performed by the work subject 100. m Since the first assistance information is notified, it is possible to suppress erroneous work assistance and perform accurate work assistance.

[0220] In addition, the work assisting device 1 lists the category data s m Since information on the actions included in the latest section among the indicated sections is notified as the second assistance information, it is possible to provide work assistance based on the length of time taken for each action or the correctness of the type of each action.

[0221] In addition, even after the operation main body 100 has just started each action included in the cycle operation, the operation support device 1 can assign the section and category number c corresponding to each action. m Since estimation is performed, it is possible to provide work assistance based on highly rapid information.

[0222] Furthermore, even when the same action is performed multiple times in a cycle work, the work assisting device 1 can determine the order corresponding to each action, and thus can perform accurate work assisting.

[0223] Implementation method 2.

[0224] Next, use Figures 25 to 28 Embodiment 2 will be described. The shorter the length of the cycle performed by the operator 100, the higher the productivity; the longer the cycle, the lower the productivity. As described above, the correlation between cycle length and productivity is strong, making it important information for operation assistance. Furthermore, during the cycle performed by the operator 100, required operations may be omitted. Such omitted operations are related to production quality and therefore provide valuable information for operation assistance.

[0225] The work assisting device according to the second embodiment identifies the length of the cycle work performed by the operator 100 and any omissions in the work, thereby assisting in improving productivity and suppressing a decrease in work quality.

[0226] (Work assist device 1A)

[0227] Figure 25 1 is a diagram showing an example of the structure of a work assisting device according to Embodiment 2. Figure 25 Among the structural elements of Figure 1 In the work assist device 1 according to the illustrated embodiment 1, components having the same functions are denoted by the same reference numerals, and redundant descriptions are omitted.

[0228] The work assist device 1A according to the second embodiment differs from the work assist device 1 according to the first embodiment in that, for a multi-cycle work consisting of a series of multiple actions performed by the work subject 100 , the length of the cycle work and the omission of work can be determined.

[0229] The work assist device 1A includes a notifier 60A instead of the notifier 60. Compared to the work assist device 1 according to Embodiment 1, the work assist device 1A includes a new cycle evaluator 70, and the operation of the notifier 60A partially differs from that of the notifier 60. Through the above-described structure and operation, the work assist device 1A can easily determine the length of a cycle and the omission of work for a multi-cycle operation consisting of a series of multiple actions performed by the work body 100. The following description of the work assist device 1A focuses on the differences from the work assist device 1 according to Embodiment 1.

[0230] (Circulation Evaluator 70)

[0231] The cycle evaluator 70 according to the second embodiment will be described. The cycle evaluator 70 is based on the performance sequence data value H1 stored in the work performance storage device 34. m2 and the standard data sequence STD(1 stored in the operation information storage device 32 m ), and detects the length of the cycle work and the omission of work performed by the work main body 100. The cycle evaluator 70 stores the detection results in the work performance storage device 34.

[0232] (Cycle Evaluator 70: Length of Cycle Operation)

[0233] The cycle evaluator 70 uses the following judgment formulas (6) and (7) to perform the performance sequence data value Hl on all the intervals from the first to the M2th intervals obtained by dividing the performance sensor data sequence Hx. m2 judgment.

[0234] [Formula 6]

[0235] m2=1 or H1 m2 >Hl m2-1 …(6)

[0236] [Formula 7]

[0237] m2=M2 or H1 m2 >Hl m2+1 …(7)

[0238] M2 is the number of intervals into which the performance sensor data sequence Hx is divided. m2 is a number used to identify each of the multiple intervals in the performance sensor data sequence Hx and represents the order of the multiple intervals in the performance sensor data sequence Hx. If the determination formula shown in equation (6) is satisfied in the m2th interval, the loop evaluator 70 adds m2 to the starting interval CVhead of the loop operation.

[0239] In addition, when the judgment formula shown in formula (7) is satisfied in the m2-th interval, the cycle evaluator 70 adds m2 to the end interval CVtail of the cycle operation. Here, CVhead = {CVhead(1), CVhead(2), ..., CVhead(D)}. CVhead(d) represents the starting interval of the d-th cycle operation implemented by the operation body 100. CVtail = {CVtail(1), CVtail(2), ..., CVtail(D)}. CVtail(d) represents the end interval of the d-th cycle operation implemented by the operation body 100. d is a number for identifying multiple cycle operations implemented by the operation body 100, and is an integer from 1 to D. d is the order of multiple cycle operations implemented by the operation body 100. D is the number of cycle operations implemented by the operation body 100.

[0240] The cycle evaluator 70 also detects the length of the cycle operation, CVlen. CVlen = {CVlen(1), CVlen(2), ..., CVlen(D)}. CVlen(d) is the length of the dth cycle operation performed by the operator 100. The cycle evaluator 70 can calculate CVlen(d) using equation (8).

[0241] [Formula 8]

[0242] CVlen(d)=Ha CVtail(d) +Hb CVtail(d) -Ha CVhead(d) …(8)

[0243] (Cycle Evaluator 70: Omission of Work)

[0244] The cycle evaluator 70 then calculates the performance order data value H1 included in each cycle operation. m2 The number of the standard data not included in the series is detected as an omission of the operation. More specifically, the method of detecting the omission of the operation by the cycle evaluator 70 is different between the d-th cycle operation and the D-th cycle operation where d<D.

[0245] The series H1 of sequential data values ​​included in the d-th cycle operation performed by the operation main body 100 d ={Hl CVhead(d) , Hl CVhead(d)+1 ,…,Hl CVtail(d) The d-th cycle operation with d<D is the cycle operation after the operation body 100 is completed. That is, the d-th cycle operation with d<D needs to include the standard data sequence STD(l m) contains all the standard data from the 1st to the Fth. As above, F is the number of standard data. That is, the cycle evaluator 70 performs the following operations for H1 d , evaluates whether each value from 1 to F is included. If the value f is not included, the loop evaluator 70 detects a missing action corresponding to the f-th standard data. In this case, the loop evaluator 70 appends the action name data LABEL(f) to the missing action CVmiss(d) in the d-th loop operation performed by the operator 100.

[0246] In addition, the D-th cycle operation is the cycle operation currently being performed by the operation main body 100. That is, the D-th cycle operation requires the implementation of the standard data sequence STD (l m ) included in the 1st to H1th CVtail(D) Therefore, the cycle evaluator 70 is for H1 D , whether to include 1 to Hl CVtail(D) The values ​​up to and including the value f are evaluated. If there is a value f that is not included, the omission of the action corresponding to the f-th standard data is detected. At this time, the cycle evaluator 70 adds the action name data LABEL(f) to the omission CVmiss(D) of the action in the D-th cycle operation performed by the operation body 100.

[0247] Figure 26 This is a diagram for explaining the data structure of the performance data stored in the work performance storage device of the work assisting device according to the second embodiment. Figure 26 As shown, the performance data 209 stored in the work performance storage device 34 includes a performance category data column Hs m2 , performance order data value Hl m2 and performance interval evaluation value HV m2 .

[0248] Figure 27 This diagram illustrates the data structure of detection result data detected by the cycle evaluator included in the work assist device according to Embodiment 2. Detection result data 210 includes the starting interval of the cyclic work detected by the cycle evaluator 70, the ending interval of the cyclic work, the length of the cyclic work, and the type of missed work. Specifically, detection result data 210 associates the starting interval CVhead(d) of the cyclic work, the ending interval CVtail(d) of the cyclic work, the length CVlen(d) of the cyclic work, and the type of missed work CVmiss(d).

[0249] According to formula (6), the judgment formula is satisfied in the first and eleventh intervals, so the loop evaluator 70 detects the starting interval CVhead={1,11} of the loop operation. Figure 26 The performance data 209 shown, the performance category data column Hs m2 The number of included intervals M2 = 12. At this time, according to equation (7), the judgment formula is satisfied in the 10th and 12th intervals. Therefore, the loop evaluator 70 detects CVtail = {10, 12}. Furthermore, the loop evaluator 70 detects the length of the loop operation CVlen = {82, 13} using equation (8).

[0250] As described above, the number of standard data F = 10. In addition, the number of loop operations D = 2. In the d = 1st loop operation where d < D, the starting interval CVhead(1) of the loop operation is 1, and the ending interval CVtail(1) is 10. Thus, the loop evaluator 70 calculates the actual performance order data value Hl included in the first loop operation. m2 Series Hl 1 ={1, 2, 3, 4, 5, 6, 8, NA, 9, 10}, and whether the values ​​from 1 to 10 are included is evaluated, and it is detected that 7 is not included. In this case, the cycle evaluator 70 refers to Figure 7 , action name data LABEL(7) = "screw fastening component C". Thus, the cycle evaluator 70 detects the missing of the operation CVmiss(1) = {"screw fastening component C"} in the cycle operation 1.

[0251] In addition, the starting section CVhead(2) of the cycle operation in the D=2nd cycle operation is 11, and the ending section CVtail(2) is 12. Therefore, the cycle evaluator 70 calculates the Hl 2 ={Hl 11 , Hl 12}={2,3}, evaluate whether the values ​​from 1 to 3 are included, and detect that 1 is not included. Figure 7 , action name data LABEL(1) = "Take out component A". In this case, the cycle evaluator 70 detects the omission of the operation CVmiss(2) = {"Take out component A"} in the cycle operation 2.

[0252] The loop evaluator 70 stores the detected information in the work performance storage device 34. The information stored by the loop evaluator 70 in the work performance storage device 34 is detection result data 210. This detection result data 210 includes the loop work start interval CVhead, the loop work end interval CVtail, the loop work length CVlen, and the missing work CVmiss. The reliability determiner 65 generates first auxiliary information based on the detection result data 210 stored by the loop evaluator 70 in the work performance storage device 34.

[0253] (Notifier 60A)

[0254] Next, the notification device 60A in Embodiment 2 will be described. The notification device 60A outputs the first auxiliary information, including the loop operation start section CVhead, loop operation end section CVtail, loop operation length CVlen, and operation omission CVmiss, to the display device 3 in addition to the output of the notification device 60 .

[0255] Thereby, the work assisting device 1A can perform work assisting based on the time taken for each cycle work or the omission of work in each cycle work.

[0256] Figure 28 Schematically shows the form of the first auxiliary information displayed on the display device by the work assisting device according to the second embodiment. Figure 28 In the figure, the display device 3 is shown as the first auxiliary information, and Figure 21 The first auxiliary information shown is associated with a number d for identifying a cyclic operation, and displays the length of the cyclic operation as a value CVlen(d)[sec] expressed in seconds and a missing operation CVmiss(d) indicating the type of the missed operation.

[0257] In the second embodiment, the work assist device 1A acquires sensor data every 200 milliseconds. Therefore, the work assist device 1A multiplies the work length CVlen(d) by 0.2 seconds to calculate CVlen(d) [sec]. The work subject 100 can easily understand how many work cycles have been performed, how long each cycle takes, and whether any work has been omitted in each cycle by checking the first assist information displayed on the display device 3.

[0258] As described above, according to the second embodiment, the work assisting device 1A performs the operation based on the performance ranking data value H1 of each section. m2, the starting section CVhead and the ending section CVtail of the cyclic operation are determined, and thus the length of the cyclic operation performed by the operation main body 100 can be determined. In addition, the operation support device 1A calculates the actual performance sequence data value H1 included in the cyclic operation. m2 The number of the standard data not included in the series, that is, the actual performance sequence data value Hl m2 Thus, the work assisting device 1A can identify and notify the length of the cycle work performed by the operator 100 and the omission of work, thereby assisting productivity improvement and suppressing degradation of work quality.

[0259] Implementation method 3.

[0260] Next, use Figure 26 、 Figure 29 and Figure 30 Embodiment 3 will now be described. When the operator 100 is, for example, a person, they may improve their work while performing it, changing their hand movements or repositioning their tools. As described above, if the cyclical work performed by the operator 100 changes over time, it may not match the information stored in the work information storage device 32, degrading the performance of the work assistance.

[0261] Specifically, when the cyclic operation changes over time, for example, a series of multiple actions included in the cyclic operation performed by the operation body 100 are not consistent with the first template g. j Matching degrades the accuracy of the classification data calculation. Furthermore, if the cyclical operation changes over time, the content of the cyclical operation performed by the operator 100 may not match the content of the reference image data, making it difficult to serve as a reference for the cyclical operation. To address these issues, the information stored in the operation information storage device 32 must be updated to match the current cyclical operation. However, manual updating requires a significant amount of time.

[0262] The work assisting device according to the third embodiment can automatically update the information stored in the work information storage device 32 even when the work performed by the work subject 100 changes over time, thereby maintaining the performance of the work assisting device without manual intervention.

[0263] (Work assist device 1B)

[0264] Figure 29 : is a diagram showing an example of the structure of the work assisting device according to the third embodiment. Figure 29 Among the various structural elements of Figure 25The same functional structural elements of the work support device 1A of the illustrated embodiment 2 are denoted by the same reference numerals, and repeated description is omitted.

[0265] The work support device 1B according to the embodiment 3 differs from the work support device 1A according to the embodiment 2 in that, for a multiple-cycle work composed of a series of a plurality of actions performed by the work subject 100, the information stored by the work information storage device 32 is updated based on the information stored by the work performance storage device 34B.

[0266] The work support device 1B has the reliability determiner 65B instead of the reliability determiner 65, and has the work performance storage device 34B instead of the work performance storage device 34. The work support device 1B differs from the work support device 1A according to the embodiment 2 in that, in addition to newly having the updater 80, a part of the action of the reliability determiner 65B differs from the action of the reliability determiner 65. In addition, the work support device 1B differs from the work support device 1A in that a part of the data stored by the work performance storage device 34B differs from the data stored by the work performance storage device 34, and in that the camera 5 is newly connected. With the above-described structure and action, the work support device 1B can easily update the information stored by the work information storage device 32 for a multiple-cycle work composed of a series of a plurality of actions performed by the work subject 100. Hereinafter, the work support device 1B will be described focusing on the differences from the work support device 1A according to the embodiment 2.

[0267] The updater 80 is connected to the work information storage device 32 and the work performance storage device 34B. The updater 80 updates the information stored by the work information storage device 32. The details of the updater 80 will be described later.

[0268] Figure 30 This is a diagram illustrating an example of use of the work support device according to the embodiment 3. In the embodiment 3, the work subject 100 repeatedly performs a work composed of a series of a plurality of actions, i.e., a cycle work, over a plurality of times.

[0269] (Camera 5)

[0270] The camera 5 outputs an image acquired by photographing the cycle work performed by the work subject 100 to the work support device 1B.

[0271] (Reliability determiner 65B)

[0272] Next, the reliability determiner 65B possessed by the work support device 1B according to the embodiment 3 will be described. The reliability determiner 65B performs a process based on the process performed by the reliability determiner 65, and determines the reliability R of the work subject 100 based on the data stored by the work performance storage device 34B.m In the section satisfying the criterion, a series of images taken by the camera 5 is stored in the work performance storage device 34B as actual performance image data.

[0273] (Updater 80)

[0274] Next, the updater 80 according to Embodiment 3 will be described. The updater 80 updates the information stored in the work information storage device 32 based on the information of the performance of the action stored in the work performance storage device 34B.

[0275] (Updater 80: 2nd Template)

[0276] The updater 80 generates the 2nd template g2 for each category based on the performance sensor data column Hx and the performance category data column Hs stored in the work performance storage device 34B. j Here, j is a number for identifying a plurality of categories, and is an integer from 1 to J. J is the number of categories, that is, the number of the 2nd template g2 j .

[0277] In Embodiment 3, the updater 80 generates the 2nd template g2 j as a set of Gaussian distributions of sensor data at each time point by using Gaussian process regression. j = {g2 j (1), g2 j (2),..., g2 j (L)}. g2 j (i) is a parameter of the Gaussian distribution of the i-th sensor data in the section classified into the category j, g2 j (i) = {μ2 j (i), σ2 j 2 (i)}. Here, μ2 j (i) is the mean of the Gaussian distribution, σ2 j 2 (i) is the variance of the Gaussian distribution. In addition, L is the length of the 2nd template g2 j , that is, the maximum value showing the number of sensor data included in each section into which the performance sensor data column Hx is divided. The 1st template g j and the 2nd template g2 j are the same length.

[0278] The 2nd template g2 j according to Embodiment 3 will be described in more detail. As described above, μ2 j (i) is the mean of the Gaussian distribution of the i-th sensor data in the section classified into the category j. μ2 j(i) is a two-dimensional value similar to sensor data. j 2 (i) is the variance of the Gaussian distribution of the i-th sensor data in the interval classified as class j. In the third embodiment, the variance of the Gaussian distribution of sensor data is assumed to be the same in any dimension. Therefore, σ2 j 2 (i) is a 1-dimensional value.

[0279] Template 2 g2 j The set X of sensor data in the interval classified as category j by the performance category data column Hs is used. j and a set I of numbers of sensor data output in the interval classified into category j by the performance category data sequence Hs. j Here, X j ={X j (1), X j (2),…,X j (N3 j )},I j ={I j (1), I j (2),…,I j (N3 j )}. For example, X j (1) is the first interval in the interval classified as category j j (1) Output sensor data.

[0280] In addition, N3 j is the set X j and I j The number of elements included. That is, N3 j The sum of the number of sensor data included in the intervals classified with category j among the intervals into which the actual sensor data sequence Hx is divided. In the third embodiment, the updater 80 uses the following equations (9) and (10) to update the second template g2 j (i) = {μ2 j (i), σ2 j 2 (i)} make a presumption.

[0281] [Formula 9]

[0282] μ2 j (i) = v j，i T (K j +β -1 E) -1 X j …(9)

[0283] [Equation 10]

[0284]

[0285] Here, β indicates a specific parameter, and E indicates an identity matrix. Further, K j is a matrix calculated by the following equation (11), and v j,i is a vector calculated by the following equation (12). Further, k is a kernel function, and a Gaussian kernel shown by the following equation (13) can be used. θ0, θ1, θ2, θ3 are specific parameters in the kernel function k.

[0286] [Equation 11]

[0287]

[0288] [Equation 12]

[0289]

[0290] [Equation 13]

[0291]

[0292] (Updater 80: 2nd category transition probability)

[0293] The updater 80 related to Embodiment 3 also generates a probability of category transition, i.e., a 2nd category transition probability P2(j|j') based on the result category data column Hs stored by the result performance storage device 34B. The updater 80 calculates the 2nd category transition probability by the following equation (14). Here, N4 J’,j is the number of times that the category number in the m2th interval becomes the category number c2 m2 = j' and the category number in the m2+1th interval becomes the category number c2 m2+1 = j in the result category data column Hs. Further, N5 j’ is the number of times that the category number becomes j in the result category data column Hs. γ is a specific parameter.

[0294] [Equation 14]

[0295] P2(j|j') = (N4 j′，j + γ) / (N5 j + jγ)... (14)

[0296] (Updater 80: 2nd reference image data)

[0297] The updater 80 according to Embodiment 3 also generates an image indicating the f-th action included in the cyclic operation, namely, second reference image data REF2(f), based on the actual image data stored in the actual operation storage device 34B. As described above, f is a number used to identify the plurality of standard data, and is an integer from 1 to F. F is the number of standard data included in the cyclic operation, which is the number of pre-designed actions. The updater 80 that updates the second class transition probability is the first updater, and the updater 80 that updates the second reference image data is the second updater.

[0298] Here, use Figure 26 The update process performed by the updater 80 will be described in detail. Here, a method for generating the second reference image data REF2(2) with f=2 by the updater 80 will be described.

[0299] The updater 80 first updates the performance order data value H1. m2 =f, and HVb m2 = "Normal", and HVc m2 = "normal" interval m2 is calculated. Figure 26 For example, the updater 80 calculates the second interval and the 11th interval. Next, the updater 80 calculates the interval length Hb based on the calculated multiple intervals. m2 The smallest interval is calculated. Figure 26 For example, the length of the interval with m2=2 is Hb2=8, and the length of the interval with m2=11 is Hb 11 =7, so m2=11 is calculated. Next, the updater 80 calculates the start number Ha in the m2-th interval based on the calculation. m2 , the length of the interval Hb m2 , the second reference image data REF2(f) is generated based on the actual image data stored in the work record storage device 34B. Figure 26 For example, the starting number Ha in interval m2=11 11 =83, the length of the interval Hb 11 = 7. The performance category data sequence Hs is calculated for the sensor data sequence x acquired every 200 milliseconds. Therefore, the updater 80 generates a section of length 7×0.2=1.4 seconds from the beginning of the reference image data, which is 83×0.2=16.6 seconds, as the second reference image data REF2(2) with f=2.

[0300] (Updater 80: Update)

[0301] The second template g2 generated by the updater 80 j The data structure of the first template g stored in the operation information storage device 32j The data structure of the second class transition probability generated by the updater 80 is the same as the data structure of the first class transition probability stored in the operation information storage device 32 .

[0302] The updater 80 also uses the second template g2 j The first template g stored in the job information storage device 32 j The updater 80 also updates the first class transition probability stored in the operation information storage device 32 with the second class transition probability. The updater 80 also updates the first reference image data stored in the operation information storage device 32 with the second reference image data.

[0303] As described above, in the third embodiment, the work support device 1B generates a new template, namely, the second template g2, based on the data stored in the work record storage device 34B. j , the new category transition probability, i.e., the second category transition probability, and the new reference image data, i.e., the second reference image data. Thus, even when the cyclical work performed by the operator 100 changes over time, the data stored in the work information storage device 32 can be automatically updated using the generated data, thereby maintaining the performance of the work assisting device 1B without consuming manpower.

[0304] (Variation)

[0305] Furthermore, the above-described embodiments 1 to 3 can also be applied to situations other than when the operator 100 is a human. For example, when the operator 100 is a machine such as a robot or a machine tool, the operator analyzing the operation can grasp information and reliability R of the occurrence of an abnormality in the cycle operation immediately after the abnormality occurs. m , therefore, the application of embodiments 1 to 3 becomes effective.

[0306] In addition, the first auxiliary information according to the first to third embodiments may include the reliability R m The first auxiliary information according to the first to third embodiments may be the image data of the same performance sequence data value H1. m2 The information of multiple performance image data corresponding to each interval is played simultaneously.

[0307] In addition, the operation information storage device 32 generates a first template g as a set consisting of Gaussian distribution of sensor data at each time. j However, other appropriate probability distributions may be used instead of the Gaussian distribution.

[0308] Furthermore, the updater 80 generates a second template g2 as a set consisting of Gaussian distributions of sensor data at each time. j , but other appropriate probability distributions can be used instead of Gaussian distribution.

[0309] Alternatively, the sensor data input device 10 may include a detachable storage medium reader instead of being connected to the sensor 2. Thus, instead of acquiring the real-time sensor data sequence x detected by the sensor 2, the sensor data input device 10 can read the sensor data sequence x measured in the past from the storage medium.

[0310] In addition, the interval evaluator 50 may evaluate the sensor data sequence x as "abnormal" when the length of each interval is shorter than the corresponding standard length. In addition, the interval evaluator 50 compares the sensor data included in each interval divided into the sensor data sequence x with the corresponding first template g. j If the degree of similarity in the comparison is low, it can be evaluated as “abnormal.” In addition, the updater 80 described in the third embodiment can be applied to the work support device 1 of the first embodiment.

[0311] The configuration shown in the above embodiment is merely an example, and can be combined with other known technologies, and the embodiments can be combined with each other. Part of the configuration can also be omitted or changed without departing from the scope of the invention.

[0312] Description of the label

[0313] 1, 1A, 1B work assisting device, 2 sensor, 3 display device, 4 input device, 5 camera, 10 sensor data input device, 20 category data calculator, 30 standard data determiner, 31 sensor data storage device, 32 work information storage device, 34, 34B work performance storage device, 40 reliability calculator, 50 interval evaluator, 60, 60A notifier, 65, 65B reliability determiner, 70 cycle evaluator, 80 updater, 100 work main body, 101 left hand, 102 right hand, 201, 203 timing graph , 202 category data column, 204~207 corresponding data groups, 208, 209 performance data, 210 detection result data, 1001 processing circuit, 1002, 1005 memory, 1003 data bus, 1004 processor, CD1~CD4 category data, DJ1~DJ4, DL1~DL4, DR1~DR4, DV1~DV4 data, G1~G4, GA1~GA4, GTa(1)~GTa(10), GTb(1)~GTb(10) timing graph, SD1~SD4 sensor data column.

Claims

1. A work assisting device that assists a work consisting of a series of multiple actions performed by a work subject. The work assist device is characterized by having: a sensor data input device for receiving a sensor data column arranged each time sensor data obtained by measuring the work of the work subject is acquired; a category data calculator for comparing the sensor data sequence with a template set for each category, the template being a set of probability distributions of the sensor data at each moment set for each type of temporal change in the sensor data, thereby calculating a category data sequence representing intervals into which the sensor data sequence is segmented; a reliability calculator for calculating, for each of the intervals, a reliability with respect to the interval based on information about the intervals indicated by the category data column; a reliability determiner that determines whether the reliability satisfies a reference and generates information on actions included in a section of the section in which the reliability satisfies the reference as first auxiliary information; as well as A notifier is configured to notify the first auxiliary information.

2. The work assist device according to claim 1, wherein: The reliability determiner generates information on actions included in the latest interval among the intervals as second auxiliary information. The notifier notifies the second auxiliary information.

3. The work assist device according to claim 1 or 2, characterized in that: The category data calculator calculates the category data column using a category transition probability, which is the probability that the operation subject performs a first action corresponding to the first category among the categories and then performs a second action corresponding to the second category among the categories.

4. The work assist device according to claim 2, wherein: Also features: a standard data determiner that generates a sequential data value corresponding to the interval, the sequential data value indicating a value of standard data including a temporal length of the action; and an interval evaluator that compares the category data sequence with the standard data in the sequential data value to thereby calculate an interval evaluation value obtained by evaluating the interval; The reliability determiner generates the first auxiliary information and the second auxiliary information based on the section evaluation value.

5. The work assisting device according to claim 4, wherein: The apparatus further comprises a cycle evaluator for detecting as an omission of a job a job which is not included in the standard data in the sequential data value corresponding to the interval in which the reliability satisfies the reference in the standard data, The reliability determiner generates the first auxiliary information based on the omission of the operation.

6. The work assisting device according to claim 3, wherein: The system further includes a first updater that updates the template and the class transition probability based on the performance of the operation.

7. The work assisting device according to claim 4, wherein: The reliability determiner generates the second auxiliary information using reference image data, which is an image indicating the action.

8. The work assisting device according to claim 7, wherein: The system further includes a second updater configured to update the reference image data based on the performance of the operation.

9. The work assist device according to any one of claims 1 to 8, characterized in that: The reliability calculator uses the order of the intervals counted from the rear of the sensor data column for calculating the reliability.

10. A method for assisting a task comprising a series of multiple actions performed by a task subject. The work assisting method is characterized by: a receiving step of receiving a sensor data column arranged each time sensor data obtained by measuring the operation of the operation subject is acquired; an interval calculation step of comparing the sensor data sequence with a template set for each category, the template being a set of probability distributions of the sensor data at each moment set for each type of temporal change in the sensor data, thereby calculating a category data sequence representing the intervals into which the sensor data sequence is segmented; a reliability calculation step of calculating the reliability of each of the intervals based on the information of the intervals indicated by the category data column; a reliability determination step of determining whether the reliability satisfies a reference, and generating information on actions included in a section of the section in which the reliability satisfies the reference as first auxiliary information; as well as The notification step is to notify the first auxiliary information.

Images