Machine learning method and machine learning device for learning about work processes

Abstract

The present invention relates to a machine learning method and a machine learning device for learning about work processes. The machine learning device determines whether multiple unit operations of the same type are identical for all unit operations included in multiple different work processes. The machine learning device uses a common machine learning algorithm to generate a similar operation learning model based on a first unit operation and a second unit operation that are similar to each other as a similar operation set. Based on the similar operation learning model, the device learns about a first work process including the first unit operation and a second work process including the second unit operation.

Description

Technical Field

[0001] The present invention relates to a machine learning method and a machine learning device for learning about a work process. Background Art

[0002] When people are performing certain work processes (such as assembly, inspection, cleaning, packaging, or clerical work), they may wish to understand the characteristics of the work process (such as work content, operator, work time, and work date and time) for the purpose of work assistance or quality control. For example, based on the understood work process characteristics, the correct work sequence can be taught to the operator through video or audio, or the main causes of process defects can be analyzed as needed.

[0003] Japanese Patent Application Laid-Open No. 2014-155207 discloses a head-mounted display device that estimates the current order by performing image recognition on an external scene image and displays a character string indicating instruction content or an image indicating content that a user should perform.

[0004] Japanese Patent No. 6224873 discloses an information processing system that displays an instruction image related to correction of a job in a superimposed manner in real space when a difference between standard data in which a job is registered and acquired job data is equal to or greater than a threshold value.

[0005] International Publication No. 2010 / 143361 discloses a work recognition device that identifies a user's work using the output of a speed sensor and the output of an imaging unit.

[0006] Japanese Patent Application Laid-Open No. 2017-131335 discloses an endoscope cleaning management system that determines provided work sequence information based on the skill level of an operator. Summary of the Invention

[0007] By associating the characteristics of a work process (such as work content, operator, work time, and work date and time) with conditions required for work support or quality management (such as conditions for teaching work sequences or conditions for analyzing process failure factors), and collecting and accumulating these as known information, it is possible to efficiently set, modify, or change these conditions. In this case, it is desirable to collect and accumulate information quickly and accurately, and to improve the reliability of the accumulated information.

[0008] One embodiment of the present disclosure is a machine learning method that performs learning related to multiple different work processes, each of which includes at least one type of unit work, wherein all unit works included in the multiple different work processes are summed up, an interpretation is made as to whether there are multiple unit works of the same type, and a judgment is made as to whether the multiple unit works of the same type are similar to each other. The first unit work and the second unit work that are similar to each other are taken as a similar work set, and a similar work learning model is generated using the characteristics of the similar work set as input data through a common machine learning algorithm. Based on the similar work learning model, learning is performed related to a first work process including the first unit work and a second work process including the second unit work.

[0009] Another embodiment of the present disclosure is a machine learning device that performs learning related to multiple different work processes, each of which includes at least one type of unit work, and the machine learning device comprises: a work aggregation unit that aggregates all the unit works included in the multiple different work processes and interprets whether there are multiple unit works of the same type; a similarity judgment unit that judges whether multiple unit works of the same type are similar to each other; a first learning unit that takes the first unit work and the second unit work that are similar to each other as a similar work set, and generates a similar work learning model with the characteristics of the similar work set as input data through a common machine learning algorithm; a second learning unit that performs learning related to the first work process including the first unit work and the second work process including the second unit work based on the similar work learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a functional block diagram showing a machine learning device according to one embodiment.

[0011] Figure 2 This is a flowchart illustrating a machine learning method according to one embodiment.

[0012] Figure 3 It is conceptually expressed Figure 2 A diagram showing a specific example of a machine learning method.

[0013] Figure 4 Yes Figure 1 A functional block diagram of an example of a learning unit of a machine learning device.

[0014] Figure 5 Yes Figure 4 A flowchart of an example of a machine learning algorithm executed by the learning unit.

[0015] Figure 6 It means the equipment has Figure 4 A functional block diagram of a display system of a machine learning device of a learning unit.

[0016] Figure 7 Yes Figure 6 A flowchart of an example of display processing in a display system. DETAILED DESCRIPTION

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In all the drawings, corresponding components are denoted by common reference numerals.

[0018] Figure 1 The machine learning device 10 according to one embodiment is represented by functional blocks. Figure 2 A machine learning method according to one embodiment executed by the machine learning device 10 is shown.

[0019] The machine learning device 10 autonomously learns various items related to the characteristics of a work process (such as the work content, operator, work time, and work date and time) when a person completes a certain work process (such as an assembly process, an inspection process, a cleaning process, a packaging process, or a clerical process) through so-called machine learning. The device includes hardware such as a processor (such as a central processing unit (CPU)) of an electronic computer for executing software that defines various learning algorithms. For example, the machine learning device 10 can learn by associating the characteristics of a work process with conditions required for work support or quality control (such as conditions for teaching a work sequence or conditions for analyzing the causes of process defects). The machine learning device 10 uses a learning model generated through learning to effectively re-set, modify, or change these conditions.

[0020] like Figure 1As shown, the machine learning device 10 includes a processor 12 that performs learning related to a plurality of different work processes, each of which includes at least one type of unit work. The processor 12 includes: a work aggregation unit 14 that aggregates all unit works included in the plurality of different work processes and interprets whether there are multiple unit works of the same type; a similarity determination unit 16 that determines whether multiple unit works of the same type are similar to each other; a first learning unit 18 that uses a common machine learning algorithm to generate a similar work learning model using the characteristics of the similar work set as input data, using a first unit work and a second unit work as input data; and a second learning unit 20 that performs learning related to a first work process including the first unit work and a second work process including the second unit work based on the similar work learning model. Here, the similar operation set can be composed of n (n≥2) unit operations (i.e., the 1st to nth unit operations), the first learning unit 18 can generate a similar operation learning model for the 1st to nth unit operations through a common learning algorithm, and the second learning unit 20 can perform learning related to the 1st to nth operation processes respectively including the 1st to nth unit operations based on the similar operation learning model.

[0021] The processor 12 can have a structure such as a CPU, an ASIC (application specific integrated circuit), or an FPGA (field programmable gate array). The operation aggregation unit 14, the similarity judgment unit 16, the first learning unit 18, and the second learning unit 20 can each be configured as a functional block of the processor 12, or can be configured as a program or a recording medium (a non-transitory recording medium readable by a computer) for making the processor 12 function. In the case where the operation process is, for example, an assembly process of a device, the unit operations included in the operation process can be exemplified by type as cable installation operations, sheet metal installation operations, screw installation operations, substrate installation operations, unit installation operations, and clamping operations. In the case where the operation process is, for example, an inspection process of a device, the unit operations included in the operation process can be exemplified by type as appearance confirmation operations of the assembled device and fixed state confirmation operations of the assembled components.

[0022] like Figure 2As shown, the machine learning method executed by the machine learning device 10 performs learning related to multiple different work processes, each of which includes at least one type of unit work, and the machine learning method includes: a step of aggregating all the unit works included in the multiple different work processes and interpreting whether there are multiple unit works of the same type (steps S1 to S3: the action of the work aggregation unit 14); a step of judging whether multiple unit works of the same type are similar to each other (steps S4 to S6: the action of the similarity judgment unit 16); a step of treating the first unit work and the second unit work that are similar to each other as a similar work set, and generating a similar work learning model using the characteristics of the similar work set as input data through a common machine learning algorithm (step S7: the action of the first learning unit 18); a step of performing learning related to the first work process including the first unit work and the second work process including the second unit work based on the similar work learning model (step S8: the action of the second learning unit 20).

[0023] Figure 3 A specific example of the machine learning method executed by the machine learning device 10 is conceptually shown. Figure 1 、 Figure 2 as well as Figure 3 , the structure of the machine learning device 10 and the machine learning method will be described in detail based on a specific example. Figure 3 In the specific example, the work process is the assembly process of the equipment, and the learning objects of the machine learning device 10 are three different assembly processes A to C. Assembly process A includes three unit operations: cable installation A, sheet metal installation A, and screw installation A. Assembly process B includes three unit operations: cable installation B, sheet metal installation B, and substrate installation B. Assembly process C includes three unit operations: cable installation C, unit installation C, and clamping C.

[0024] like Figure 2 As shown, learning starts when any operator starts each assembly process A to C at any date and time in any work environment. First, the work totaling unit 14 extracts all the unit operations included in the three assembly processes A to C in step S1. As an example of an extraction method, a method of using a feature recognition device such as a camera or a barcode reader set in the work environment (i.e., the learning environment) to identify the various features of the assembly processes A to C can be adopted. For example, the initial unit operation of each assembly process is photographed by a camera, or the barcode of the initial work object of each assembly process is read by a barcode reader, thereby extracting a series of unit operations including the initial unit operation by comparing with a pre-prepared database.

[0025] Next, in step S2, the work aggregation unit 14 classifies all the extracted unit works by type and defines a plurality of work types. Figure 3 In the specific example, all unit operations are classified into: operation type 22 for installing cables A to C of the same type, operation type 24 for installing sheet metal A and B of the same type, operation type 26 for installing A with screws only, operation type 28 for installing B with substrates only, operation type 30 for installing C with units only, and operation type 32 for clamping C only.

[0026] Next, in step S3, the operation totaling unit 14 determines whether there are multiple unit operations of the same type among all the unit operations. Specifically, it determines whether there is an operation type including multiple unit operations among the multiple operation types 22 to 32 defined in step S2. Through this determination, it is interpreted whether there are unit operations of the same type. Figure 3 In the specific example, since there are job types 22 and 24 including a plurality of unit jobs, they are interpreted as having unit jobs of the same type.

[0027] When there are unit operations of the same type (if the judgment in step S3 is yes), the similarity judgment unit 16 judges whether these unit operations of the same type are similar to each other in steps S4 to S6. Figure 3 In the specific example, it is determined whether cable installations A to C belonging to the work type 22 are similar to each other, and whether sheet metal installations A and B belonging to the work type 24 are similar to each other.

[0028] In order to judge the similarity of unit operations, it is effective to interpret the commonality of the operation elements that constitute the unit operations. An operation element refers to each of the multiple setting items that a person needs to complete a unit operation. For example, when the unit operation is a cable installation operation, the installation position of the cable, the shape of the cable connector, and the length of the cable can be exemplified as operation elements. In addition, when the unit operation is a sheet metal installation operation, the installation position of the sheet metal material, the size and shape of the sheet metal material, the position of the assembly hole set in the sheet metal material, etc. can be exemplified as operation elements. Figure 3 In the specific example, in operation type 22, as the operation elements of cable installation A, the installation position 1, connector shape 1, and cable length 1 are shown; as the operation elements of cable installation B, the installation position 1, connector shape 1, and cable length 2 are shown; as the operation elements of cable installation C, the installation position 1, connector shape 3, and cable length 3 are shown (the numbers 1 to 3 represent the difference in set values).

[0029] As an example of the similarity judgment processing in steps S4 to S6, the similarity judgment unit 16 can judge whether multiple unit operations are similar to each other based on the number of common operation elements among the operation elements that constitute each of the multiple unit operations of the same type. First, in step S4, the similarity judgment unit 16 extracts all the operation elements that constitute each of the unit operations of the same type and calculates the number of common operation elements (number of repeated operation elements) between these unit operations. As an extraction method, for example, a method using a feature recognition device such as the aforementioned camera or barcode reader can be adopted. In Figure 3 In the specific example, for work type 22, the extracted items are installation position 1, connector shape 1, and cable length 1 for cable installation A; installation position 1, connector shape 1, and cable length 2 for cable installation B; and installation position 1, connector shape 3, and cable length 3 for cable installation C. Since installation position 1 and connector shape 1 are common between cable installation A and cable installation B, the number of repeated work elements for cable installation A and B is calculated as "2." Similarly, the number of repeated work elements for cable installation B and C is calculated as "1," and the number of repeated work elements for cable installation C and A is calculated as "1."

[0030] Next, in step S5, the similarity between unit operations of the same type is calculated based on the number of repeated operation elements, for example, according to the following formula 1. Formula 1 is a function for converting the number of repeated operation elements into a value (i.e., similarity) An that takes into account the weight of each operation element.

[0031] [Formula 1]

[0032] An=X·x+Y·y+Z·z+…

[0033] In Formula 1, X, Y, Z, and the variables after Z represent the work elements that constitute the unit work (regardless of the set value). For the work elements that are common between the same type of unit work, the same parameter representing "common" is assigned, and for the work elements that are not common, zero is assigned as a parameter. X, Y, Z, and the variables after Z represent the weights of the work elements, and are set, for example, according to the empirical rules of the designer of the machine learning device 10. Figure 3 In this specific example, X represents the installation position, Y represents the connector shape, and Z represents the cable length. The similarity between cable installations A and B, with a repetition count of 2, is calculated by adding the weights of installation position 1 and connector shape 1 to form An = X·x + Y·y. Similarly, the similarity between cable installations B and C, with a repetition count of 1, is calculated as An = X·x, and the similarity between cable installations C and A, with a repetition count of 1, is calculated as An = X·x.

[0034] Next, in step S6, it is determined whether the similarity An is greater than a pre-set threshold. The threshold is a value that serves as a criterion for determining whether unit operations of the same type are similar to each other, and is set based on the type of unit operation, for example, based on the empirical rules of the designer of the machine learning device 10. Figure 3 In the specific example, the threshold value of the similarity An of the cable installation tasks is set to a value greater than X·x and less than (X·x+Y·y). As a result, cable installation A and cable installation B are determined to be similar in task type 22. Similarly, sheet metal installation A and sheet metal installation B are determined to be similar in task type 24.

[0035] When multiple unit operations of the same type are similar to each other (if the judgment in step S6 is yes), the first learning unit 18 regards the similar unit operations as a similar operation set in step S7, and generates a similar operation learning model using the characteristics of the similar operation set (i.e., the characteristics of the multiple unit operations included in the similar operation set) as input data through a common machine learning algorithm. Figure 3 In this specific example, cable installation A and cable installation B are treated as a set of similar tasks, and a single similar task learning model (i.e., common to cable installation tasks A and B) is generated using the characteristics of cable installation A (task content, operator, task time, and task date and time, etc.) and cable installation B (task content, operator, task time, and task date and time, etc.) as input data. Similarly, a common similar task learning model is generated for sheet metal installation tasks A and B.

[0036] When there is a unit operation that is dissimilar to other unit operations in the same type of unit operation belonging to each of multiple types of operations, in the above-mentioned step S7, the first learning unit 18 executes a machine learning algorithm for the dissimilar unit operation separately to generate a dissimilar operation learning model using the characteristics of the dissimilar unit operation as input data. The machine learning algorithm used to generate the dissimilar operation learning model may be the same as or different from the machine learning algorithm used to generate the similar operation learning model. Figure 3 In the specific example, for work type 22, cable installation C is a unit work that is dissimilar to other unit works. Therefore, a dissimilar work learning model is generated using the characteristics of cable installation C (work content, worker, work time, work date and time, etc.) as input data. On the other hand, for work type 24, since there are no unit works that are dissimilar to other unit works, a dissimilar work learning model is not generated.

[0037] In step S7, a non-similar operation learning model is also generated for the operation type including only a single unit operation, similar to the above-mentioned process for the unit operation that is not similar to the other unit operations. Figure 3 In the specific example, a non-similar operation learning model is generated for each of screw installation A belonging to operation type 26, substrate installation B belonging to operation type 28, unit installation C belonging to operation type 30, and clamping C belonging to operation type 32.

[0038] Once similar or dissimilar work learning models have been generated for all unit operations extracted in step S4, the second learning unit 20 then performs learning related to multiple different work processes (assembly processes A to C) based on the similar or dissimilar work learning models (at least based on the similar work learning models) in step S8. Learning related to each work process is essentially the same as generating a similar or dissimilar work learning model for each unit operation included in that work process. However, the second learning unit 20 appropriately selects the similar or dissimilar work learning model (at least the similar work learning model) generated by the first learning unit 18 for each unit operation to perform learning for each work process. Consequently, when the similar or dissimilar work learning model for each unit operation (i.e., the accuracy of the input-output correlation) reaches the desired level of confidence, the second learning unit 20 concludes its learning related to each work process.

[0039] exist Figure 3 In the specific example, learning related to assembly process A is performed based on the following models: a similar work learning model generated for cable installation A, a similar work learning model generated for sheet metal installation A, and a dissimilar work learning model generated for screw installation A. Furthermore, learning related to assembly process B is performed based on the following models: a similar work learning model generated for cable installation B, a similar work learning model generated for sheet metal installation B, and a dissimilar work learning model generated for substrate installation B. Furthermore, learning related to assembly process C is performed based on the following models: a dissimilar work learning model generated for cable installation C, a dissimilar work learning model generated for unit installation C, and a dissimilar work learning model generated for clamping C.

[0040] When multiple unit operations of the same type are not similar to each other (when the judgment of step S6 is no), in step S9, the second learning unit 20 executes the machine learning algorithm separately for multiple different operation processes (assembly processes A to C) to generate a learning model with the characteristics of each operation process as input data. The machine learning algorithm used to generate the learning model of each operation process can be the same as or different from the machine learning algorithm used to generate the similar operation learning model. In addition, the learning model of each operation process can be a learning model generated by the entire operation process, and the first learning unit 18 can combine multiple non-similar operation learning models generated for each unit operation included in the operation process. The learning model for each operation process (i.e., the accuracy of the correlation between input and output) reaches the desired level of confidence, thereby ending the learning related to each operation process.

[0041] Furthermore, if all of the unit operations included in the multiple different work processes are of different types (if the judgment in step S3 is negative), in step S10, the second learning unit 20 executes a machine learning algorithm for each of the multiple different work processes (assembly processes A to C) to generate a learning model using the characteristics of each work process as input data. The machine learning algorithm used to generate the learning model for each work process can be the same as or different from the machine learning algorithm used to generate the learning model for similar operations. When the learning model for each work process (i.e., the accuracy of the correlation between input and output) reaches the desired level of confidence, the learning for each work process is terminated.

[0042] In this way, learning related to multiple different work processes (assembly processes A to C) is carried out, and when the reliability of the learning models related to each work process (similar work learning model and non-similar work learning model) is deepened to the desired level, the work process used for learning is completed.

[0043] As described above, in the machine learning device 10 and machine learning method according to one embodiment, learning related to multiple different work processes is performed by generating a common similar work learning model for the unit works that are similar to each other among the unit works included in each work process. Since the input data of the similar work learning model is data related to the characteristics of the similar work set (i.e., the characteristics of each of the multiple unit works included in the similar work set), the amount of data (i.e., the number of learning times) increases compared to a configuration in which learning is performed separately for multiple work processes (or multiple unit works). Therefore, the reliability of the similar work learning model for the similar unit works is improved, and with this, the precision and accuracy of learning related to each work process is improved.

[0044] For example, in an application where the characteristics of a work process (such as work content, operator, work time, and work date and time) are associated with conditions required for work support or quality management (such as conditions for teaching work sequences or conditions for analyzing process failure factors) and accumulated as known information, the machine learning device 10 and machine learning method according to one embodiment can not only quickly and accurately collect and accumulate such information to form a learning model, but also improve the reliability of the accumulated information, i.e., the learning model. As a result, it is possible to efficiently and accurately set, modify, or change the conditions required for work support or quality management.

[0045] In the machine learning device 10 and the machine learning method having the above-described structure, the learning method (machine learning algorithm) executed by the first learning unit 18 is not particularly limited. For example, known learning methods such as supervised learning, unsupervised learning, and reinforcement learning can be used. Supervised learning is a method in which a known learning data set is provided in advance, in which a large amount of input data and corresponding output data (generally referred to as teacher data or labels, etc.) are provided. Based on these known learning data sets, features that suggest the correlation between input data and output data are identified, thereby generating a correlation model (similar to a task learning model in the machine learning device 10) for inferring the required output corresponding to new input. In contrast, unsupervised learning and reinforcement learning start learning from a state where the relationship between input and output is unknown.

[0046] Reference Figure 4 as well as Figure 5 An example of a machine learning algorithm (that is, a process of generating a similar task learning model) executed by the first learning unit 18 in the machine learning device 10 will be described. Figure 4 The configuration of the first learning unit 18 is represented by functional blocks. Figure 5 It represents the machine learning algorithm executed by the first learning unit 18 .

[0047] The first learning unit 18 includes: a data acquisition unit 34, which acquires status data D1 and label data D2 as a learning data set DS, wherein the status data D1 includes content information of the unit operation and represents the characteristics of the unit operation, and the label data D2 represents whether the behavior of the unit operation corresponding to the status data D1 is abnormal; a threshold calculation unit 36, which calculates the threshold T for determining whether the behavior of the unit operation is prone to abnormality based on the learning data set DS; and a learning model generation unit 38, which uses the learning data set DS to generate a similar operation learning model M representing the correlation between the content information (status data D1) and the threshold T.

[0048] The machine learning algorithm executed by the first learning unit 18 includes: step U1 (action of the data acquisition unit 34) of acquiring state data D1 and label data D2 as a learning data set DS, which is equivalent to the above-mentioned supervised learning; step U2 (action of the threshold calculation unit 36) of calculating the threshold T based on the learning data set DS; and step U3 (action of the learning model generation unit 38) of generating a similar task learning model M using the learning data set DS.

[0049] Figure 6 As a Figure 4 The specific example of application of the machine learning device 10 to the first learning unit 18 shown in FIG. 1 is a diagram showing the structure of the display system 40 by means of functional blocks. Figure 6 , based on a specific example, the structure of the machine learning algorithm executed by the first learning unit 18 is described in detail. Figure 6 As shown, the display system 40 includes an environment 42 for performing the aforementioned work process, a work object 44 placed in the environment 42, and an operator 46 who performs a predetermined work process on the work object 44. The display system 40 has a function of displaying information to the operator 46 to assist with the work process.

[0050] Display system 40 includes a machine learning device 10 and a display device 48. Machine learning device 10 and display device 48 can communicate with each other via a wired or wireless network. Machine learning device 10 is configured as a PC (personal computer) or a server. Display device 48 is configured as a laptop, PDA (personal digital assistant), smartphone, or wearable device, etc., located in environment 42. Display system 40 may include multiple display devices 48, in which case, each display device 48 is located in the same environment or in different environments.

[0051] The display system 40 includes, in addition to the first learning unit 18 of the machine learning device 10 as a first processor, a second processor 50 for the display device 48. The second processor 50 includes a display control unit 52, which controls the timing for displaying normal operation information on the display unit 54 of the display device 48 based on the threshold value T calculated by the threshold value calculation unit 36 ​​of the first learning unit 18. The second processor 50 can also have functions for determining whether the workpiece 44 is a normal product, displaying defective parts of the workpiece 44 and instructing replacement, and determining whether a unit operation is normal.

[0052] "Normal work information" can be the normal work sequence related to a unit of work, or it can also be normal work results. The display unit 54 is configured as a commonly known display such as a liquid crystal display or a touch panel display. The display unit 54 displays work support information according to instructions from the display control unit 52, and in particular, displays normal work information according to the display timing controlled by the display control unit 52. The display unit 54 can be configured as an augmented reality device, in which case the normal work information can be displayed superimposed on an image in real space.

[0053] The data acquisition unit 34 of the first learning unit 18 acquires state data D1 and label data D2 from the environment 42 as a learning dataset DS. In addition to the unit task content information, the state data D1 may also include at least one additional piece of information: operator information, task time information, and task date and time information. In this case, the similar task learning model M generated by the learning model generation unit 38 of the first learning unit 18 represents the correlation between the content information and the additional information and the threshold value T.

[0054] "Content information" is defined as information indicating which unit operation the operator 46 performs on the operation object 44 in the environment 42. For example, the content information includes the operation category code, the difficulty of the operation, etc. The content information is determined using a feature recognition device such as a camera or a barcode reader. The feature recognition device can be provided on the display device 48, or can be provided separately. The content information can be determined based on the ID or image of the operation object 44, for example. For example, in the case of an assembly operation in which a plurality of parts are assembled to each other, the content information can be determined by comparing the ID of each part with pre-stored reference data. In addition, the content information can also be used to determine whether the operation is normal. Since the content of the unit operation has a direct correlation with the operation result or whether operation assistance is required, the status data D1 is set to include at least the content information.

[0055] "Operator information" is defined as information indicating the proficiency of operator 46. For example, operator information includes operator ID, age, years of service, and years of work experience. Operator information is determined, for example, based on access rights to environment 42 or the operator ID entered when verifying access rights on display device 48. There are cases where a correlation is found between operator 46's proficiency and work results or the need for work assistance.

[0056] "Work time information" is defined as information indicating the time taken from the start to the end of a unit work. The work time information is determined using a feature recognition device such as a camera or a barcode reader. Alternatively, the work time information is determined using a timing device such as an internal clock. The work start time may be the time when the ID of the work object 44 is acquired, the time when the power of the display device 48 is turned on, or the time when the operator 46 inputs the start of the work into the display device 48. The work end time may be the time when an image of a work end mark assigned to the work object 44 is acquired, the time when the power of the display device 48 is turned off, or the time when the operator 46 inputs the end of the work into the display device 48. There are cases where a correlation is found between the work time and the work result or whether work assistance is required.

[0057] "Job date and time information" is defined as information indicating the date, time, and day of the week when the operator 46 performed a unit job. For example, the job date and time information may be the job start date and time, the job end date and time, or a date and time intermediate between the job start date and time and the job end date and time. The job date and time information is determined using, for example, a feature recognition device such as a camera or a barcode reader. Alternatively, the job date and time information is determined using a timing device such as an internal clock. There are cases where a correlation is found between the job date and time and the job results or the need for job assistance, such as when the working hours or number of working days exceeds a predetermined value, thereby impairing the accuracy of the job.

[0058] Label data D2 is determined using, for example, a feature recognition device such as a camera or barcode reader. In this case, label data D2 can be determined based on, for example, the ID or image of the work object 44. For example, in the case of an assembly operation in which multiple components are assembled, label data D2 can be determined by comparing the IDs of each component or the image of the assembled work object 44 with pre-stored reference data. When using image data, any image of normal or abnormal operation can be used as the reference data. The determination of which unit operation is normal is made, for example, based on the rule of thumb of the designer of the machine learning device 10.

[0059] Next, an example of the operation of the threshold calculation unit 36 ​​and the learning model generation unit 38 will be described. Equation 2 below is an example of a function that models the relationship between the abnormal work flag F and the status data D1. The abnormal work flag F indicates whether the behavior of the unit work corresponding to the status data D1 is likely to be abnormal.

[0060] [Formula 2]

[0061] F=G·g0+H·h0+I·i0+J·j0

[0062] In Equation 2, G, H, I, and J represent content information, operator information, work time information, and work date and time information, respectively. g0, h0, i0, and j0 represent the weight of the content information, operator information, work time information, and work date and time information, respectively. The initial values ​​of these weights can be determined through regression analysis, etc., after a certain amount of state data D1 has been accumulated.

[0063] The following formula 3 is an example of a function that models the relationship between the weights l1 and l2 of the label data D2 and the weights g0, h0, i0, and j0 of the state data D1.

[0064] [Formula 3]

[0065] g1=g0·l1, h1=h0·l1, i1=i0·l1, j1=j0·l1

[0066] g2=g0·l2, h2=h0·l2, i2=i0·l2, j2=j0·l2

[0067] In Equation 3, weight l1 represents the weight of normal operation label data L1 within label data D2, and weight l2 represents the weight of abnormal operation label data L2 within label data D2. g1, h1, i1, and j1 represent the weights of each state data D1 when L1 is acquired, and g2, h2, i2, and j2 represent the weights of each state data D1 when L2 is acquired.

[0068] The following formula 4 is a function obtained by modeling the relationship between the abnormal operation flag F and the threshold value T when L1 is acquired and when L2 is acquired.

[0069] [Formula 4]

[0070] When L1 is acquired:

[0071] T=T′+(G·g1+H·h1+I·i1+J·j1)

[0072] When L2 is acquired:

[0073] T=T′-(G·g2+H·h2+I·i2+J·j2)

[0074] In Equation 4, T′ represents the threshold value when the tag data D2 was previously obtained. The initial value of T′ can be zero. As shown in Equation 4, when L1 is obtained, the threshold value T increases, and when L2 is obtained, the threshold value T decreases. Thus, each time the data acquisition unit 34 obtains the state data D1 and the tag data D2, the threshold value T is updated. Thus, the threshold calculation unit 36 calculates the threshold value T, and the learning model generation unit 38 generates a similar operation learning model M representing the correlation between the state data D1 and the threshold value T.

[0075] The following Equation 5 is a function for determining whether the behavior of a unit operation corresponding to the state data D1 is likely to be abnormal when performing display processing for operation assistance on-site using the learning result (operation process learning model) of the machine learning device 10.

[0076] 〔Equation 5〕

[0077] When F ≥ T: Display normal operation information before the operation

[0078] When F < T: Do not display normal operation information before the operation

[0079] As shown in Equation 5, when the abnormal operation flag F is above the threshold value T, since the unit operation is likely to be abnormal, "normal operation information" is displayed on the display unit 54 before the operator 46 completes the unit operation. On the other hand, when the abnormal operation flag F does not reach the threshold value T, since the unit operation is unlikely to be abnormal, "normal operation information" is displayed on the display unit 54 only when the unit operation actually completed by the operator 46 is abnormal.

[0080] Figure 7 This represents an example of the display processing flow for operation assistance performed on-site using the learning result (operation process learning model) of the machine learning device 10 in the display system 40 equipped with the machine learning device 10. First, in step P1, the data acquisition unit 34 acquires the state data D1. In the illustrated example, the state data D1 includes, in addition to the content information, operator information, operation time information, and operation date and time information.

[0081] Next, in step P2, for example, the second processor 50 determines whether the work object 44 is a normal product. Whether the work object 44 is a normal product can be determined by image recognition or the like. For example, in the case of an assembly operation in which a plurality of components are assembled to each other, it is possible to determine whether each component is a normal product by comparing the image of each component with the reference data. If the work object 44 is not a normal product (if the judgment in step P2 is yes), in step P3, the display control unit 52 causes the display unit 54 to display the defective part, instructs the replacement of the work object 44 as needed, and then returns to step P1. If the work object 44 is a normal product (if the judgment in step P2 is yes), proceed to step P4.

[0082] In step P4, for example, the second processor 50 determines whether the behavior of the unit operation corresponding to the acquired status data D1 is likely to be abnormal, that is, whether the abnormal operation flag F (Formula 2) is greater than or equal to the threshold value T (Formula 5). If the unit operation is unlikely to be abnormal (if the determination in step P4 is negative), for example, after the operator 46 completes the unit operation, the second processor 50 determines in step P5 whether the behavior of the unit operation is normal.

[0083] If the unit operation is not operating normally (if the determination in step P5 is negative), in step P6, the display control unit 52 causes the display unit 54 to display normal operation information and the defective part, and instructs the operator 44 to replace the workpiece as needed. This process allows, for example, a skilled operator 46 to confirm normal operation information only when an incorrect operation has actually been performed. Next, for example, the second processor 50 increments the abnormal operation flag F in step P7 and returns to step P1.

[0084] If the unit operation is normal (if the determination in step P5 is yes), for example, the second processor 50 decrements the abnormal operation flag F in step P8 and determines in step P9 whether the unit operation has been completed. If learning is performed during the on-site operation assistance display process, the threshold value T (Equation 4) is updated in steps P7 and P8.

[0085] If the behavior of the unit work corresponding to the acquired status data D1 is likely to be abnormal (if the judgment in step P4 is yes), in step P10, the display control unit 52 causes the display unit 54 to display normal work information before the operator 46 completes the unit work. In this way, even if the status data D1 indicates that the operator 46 is unskilled, the work content is complex, the work time is long, or the operator 46 works many hours or days, it is possible to prevent erroneous work or process defects before they occur.

[0086] After the operator 46 completes the unit operation under the operation assistance display, the second processor 50, for example, determines in step P11 whether the unit operation is behaving normally. If the unit operation is not behaving normally (if the determination in step P11 is negative), the display control unit 52 causes the display unit 54 to display normal operation information again in step P12, and also displays the defective part, and instructs the replacement of the work object 44 as needed. This process can reduce the frequency of erroneous operations or process defects even when normal operation information is displayed before the operation. Next, for example, the second processor 50 increases the abnormal operation flag F in step P13 and returns to step P1.

[0087] If the unit operation is normal (if the judgment in step P11 is yes), for example, the second processor 50 decrements the abnormal operation flag F in step P14 and determines whether the unit operation has been completed in step P9. If learning is performed during the on-site operation assistance display processing, the threshold value T (Equation 4) is updated in steps P13 and P14.

[0088] If the unit job has not been completed (if the judgment in step P9 is No), the process returns to step P1. If the unit job has been completed (if the judgment in step P9 is Yes), the process flow is terminated and displayed as a job completion.

[0089] The machine learning device and machine learning method disclosed in the present invention are not limited to the structures of the above-mentioned and illustrated embodiments. For example, the operating procedures and unit operation characteristics (i.e., input data) that are the objects of machine learning are not limited to the above-mentioned and illustrated embodiments, and various characteristics corresponding to the user's requirements can be used as input data. In addition, the machine learning algorithm is not limited to the above-mentioned and illustrated embodiments, and various algorithms corresponding to the user's requirements can be used. As an application of the machine learning device, in addition to the aforementioned display system, for example, a system for newly setting or changing the recorded items in the process sequence manual based on the learning model of the operating procedure generated by the machine learning device can also be exemplified.

[0090] According to a machine learning method according to one embodiment, a common similar operation learning model is generated for unit operations similar to each other among the unit operations included in each operation process, thereby performing learning related to multiple different operation processes. Therefore, compared with a structure in which learning is performed separately for multiple operation processes (or multiple unit operations), the amount of data (i.e., the number of learning times) increases, and the precision and accuracy of learning related to each operation process are improved. For example, not only can the characteristics of the operation process (operation content, operator, operation time, operation date and time, etc.) be associated with the conditions required for operation assistance or quality management (conditions for teaching the operation sequence, or conditions for analyzing the causes of process defects, etc.) to quickly and accurately perform learning modeling, but the reliability of the learning model can also be improved, thereby enabling the new setting, correction, or change of the conditions required for operation assistance or quality management to be performed efficiently and accurately.

[0091] According to a machine learning device according to another embodiment, an effect equivalent to the above-mentioned effect of the machine learning method is produced.

Claims

1. A machine learning method for performing learning on a plurality of different work processes, each of which includes at least one type of unit work, characterized in that: The machine learning method comprises: an operation summing step of summing up all the unit operations included in the plurality of different operation steps to determine whether there are a plurality of unit operations of the same type; a similarity determination step of determining whether the plurality of unit operations of the same type are similar to each other based on the number of common operation elements among the operation elements constituting each of the plurality of unit operations; The first learning step comprises: taking the first unit operation and the second unit operation that are similar to each other as a similar operation set, and generating a similar operation learning model using characteristics of the similar operation set as input data through a common machine learning algorithm; The second learning step is to perform learning related to the first operation step including the first unit operation and the second operation step including the second unit operation based on the similar operation learning model. The first learning step performs: a data acquisition step of acquiring state data and label data as a learning data set, wherein the state data includes content information of the unit operation and represents the characteristic, and the label data represents whether the behavior of the unit operation corresponding to the state data is abnormal; a threshold calculation step of calculating, based on the learning data set, a threshold for determining whether the behavior of the unit operation is likely to become abnormal; The learning model generating step generates the similar task learning model indicating the correlation between the content information and the threshold value using the learning data set.

2. The machine learning method according to claim 1, wherein: The work aggregation step classifies all the unit works included in the plurality of different work steps by type, defines a plurality of work types, and determines whether there is a work type including a plurality of the unit works.

3. The machine learning method according to claim 1 or 2, characterized in that The first learning step executes a machine learning algorithm for a unit task that is dissimilar to other unit tasks, thereby generating a dissimilar task learning model using the characteristics of the dissimilar unit task as input data. The second learning step performs the learning related to the plurality of different work steps based on the similar work learning model and the non-similar work learning model.

4. The machine learning method according to claim 3, wherein: When the first learning process or the second learning process determines that the multiple unit operations of the same type are not similar to each other, the machine learning algorithm is executed separately for the multiple different operation processes to generate the similar operation learning model or the non-similar operation learning model with the characteristics of each of the operation processes as input data.

5. The machine learning method according to claim 3, wherein: When the first learning process or the second learning process determines that all the unit operations included in the multiple different operation processes are of different types, the machine learning algorithm is executed separately for the multiple different operation processes to generate the similar operation learning model or the non-similar operation learning model with the characteristics of each of the operation processes as input data.

6. The machine learning method according to claim 1 or 2, characterized in that The status data further includes at least one additional information of worker information, work time information, and work date and time information of the unit work, and the similar work learning model represents correlation between the content information and the additional information and the threshold.

7. A machine learning device that performs learning related to a plurality of different work processes, each of which includes at least one type of unit work, characterized in that: The machine learning device comprises: a work aggregating unit for aggregating all the unit works included in the plurality of different work steps and determining whether there are a plurality of unit works of the same type; a similarity determination unit that determines whether the plurality of unit operations of the same type are similar to each other based on the number of common operation elements among the operation elements constituting each of the plurality of unit operations; a first learning unit that uses the first unit task and the second unit task that are similar to each other as a similar task set and generates a similar task learning model using characteristics of the similar task set as input data using a common machine learning algorithm; as well as a second learning unit that performs learning on the first operation process including the first unit operation and the second operation process including the second unit operation based on the similar operation learning model; The first learning unit includes: a data acquisition unit that acquires state data and label data as a learning data set, wherein the state data includes content information of the unit operation and represents the characteristic, and the label data represents whether the behavior of the unit operation corresponding to the state data is abnormal; a threshold value calculation unit that calculates a threshold value for determining whether the behavior of the unit operation is likely to become abnormal based on the learning data set; A learning model generating unit generates the similar task learning model indicating the correlation between the content information and the threshold value using the learning data set.

8. The machine learning device according to claim 7, wherein: The operation aggregation unit classifies all the unit operations included in the multiple different operation processes by type and defines multiple operation types, and determines whether there are multiple unit operations of the same type by judging whether there is an operation type that includes multiple unit operations.

9. The machine learning device according to claim 7 or 8, wherein: The first learning unit executes a machine learning algorithm for a unit task that is dissimilar to other unit tasks, and generates a dissimilar task learning model using characteristics of the dissimilar unit task as input data. The second learning unit performs the learning related to the plurality of different work steps based on the similar work learning model and the non-similar work learning model.

10. The machine learning device according to claim 9, wherein: The status data further includes at least one additional information of worker information, work time information, and work date and time information of the unit work, and the similar work learning model represents correlation between the content information and the additional information and the threshold.

Images