Machine learning system and machine learning model management method using machine learning system
The machine learning system addresses the challenge of model deterioration by associating sensor data with unique identifiers, tracking maintenance events, and managing model retraining to ensure accurate abnormality detection in industrial systems.
Patent Information
- Application Number
- JP2021082783
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-05-14
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2041-05-14
AI Technical Summary
Machine learning models in industrial systems fail to detect abnormalities accurately due to age-related deterioration of equipment, leading to discrepancies between learned sensor data and new input data, and maintenance events cause unpredictable changes in data trends, making it difficult to identify affected models and retrain them effectively.
A machine learning system that uses sensor data from multiple sensors to detect equipment abnormalities, identifies affected models by associating sensor data with unique identifiers, tracks maintenance events, calculates the degree of influence on sensor data trends, and manages model retraining based on these associations.
The system efficiently identifies models requiring retraining after maintenance events, reducing false positives and ensuring timely model updates to maintain accurate abnormality detection.
Smart Images

Figure 0007587472000002 
Figure 0007587472000003 
Figure 0007587472000004
Abstract
Description
[Technical field]
[0001] The present invention relates to a machine learning system and a machine learning model management method by a machine learning system. [Background technology]
[0002] The present invention preferably relates to a machine learning system that is equipped with a machine learning model created using machine learning or the like and performs inference using machine learning or the like, and a machine learning model management method using this machine learning system (hereinafter, the machine learning model is also referred to as a model).
[0003] There are machine learning systems that monitor the operating status of a factory or the like with sensors and detect abnormalities such as breakdowns using a machine learning model created using the sensor data as input. However, the machine learning system targeted by the present invention is not limited to being used for anomaly detection, and may be any system that is equipped with a machine learning model and executes inference, such as finding a target image using a machine learning model.
[0004] In such machine learning systems, one usage pattern is that a single machine learning system is equipped with multiple machine learning models and performs inference or diagnosis.
[0005] However, due to the aging of factory equipment, the sensor data learned by the model may diverge from the new input sensor data, making it impossible for the mole to properly detect anomalies. In some cases, the sensor data learned by the model is periodically updated to deal with deteriorating equipment. However, when planned maintenance is performed and the data trends change, some machine learning models may no longer be able to properly detect anomalies.
[0006] In Patent Document 1, a machine learning model and learning data that are input to the machine learning model are linked and managed. Information indicating that a new trained model has been generated using the learning data is displayed. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] International Publication No. 2019 / 150565 Summary of the Invention [Problem to be solved by the invention]
[0008] Conventionally, a machine learning model is specified and the specified machine learning model is retrained. However, in the case of maintenance, since various types of data for each maintenance type cause deviations, it is not known which model the maintenance will affect, and it is not possible to identify the machine learning model affected by the maintenance. Even if the machine learning model is identified, the model cannot be retrained until data showing changes in trends after maintenance is accumulated, and during that time the model cannot properly detect anomalies.
[0009] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a machine learning system and a machine learning model management method using a machine learning system that are capable of identifying models that are affected by maintenance or other events, change the trends in input data, and require re-learning when such events occur. [Means for solving the problem]
[0010] In order to solve the above problem, a machine learning system according to one aspect of the present invention has one or more machine learning models to which a machine learning algorithm is applied, and uses sensor data output by one or more sensors that detect the state of an equipment to be detected as input data, and the machine learning model detects anomalies in the equipment based on this input data.The machine learning system has a processor, and the processor manages a model identifier unique to the machine learning model that identifies the machine learning model, in association with a sensor identifier specific to the sensor that identifies the sensor that outputs the sensor data that becomes input data for the machine learning model, calculates an impact degree that represents a change in the trend of the sensor data before and after the maintenance work was performed for each maintenance event identifier unique to maintenance work that identifies maintenance work to be performed on the equipment, and manages this impact degree in association with each sensor identifier, and considers that sensors whose impact degree meets a predetermined condition have been affected by the maintenance work, and presents a model identifier associated with the sensor identifier of the sensor that meets the condition. Effect of the Invention
[0011] According to the present invention, it is possible to realize a machine learning system and a machine learning model management method using a machine learning system that can identify models that are affected by an event such as maintenance and have changed the trend of input data, and that require re-learning when the maintenance occurs. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram illustrating an overview of the operation of a machine learning system according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating a schematic configuration of a machine learning system according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a model-sensor association table of the machine learning system according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of an event-sensor association table of the machine learning system according to the first embodiment. [Diagram 5]1 is a flowchart illustrating an example of an operation of the machine learning system according to the first embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of update candidate model information according to the first embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of an event-sensor association table of the machine learning system according to the second embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of an event-sensor association table of the machine learning system according to the second embodiment. [Figure 9] 11 is a flowchart illustrating an example of an operation of the machine learning system according to the second embodiment. [Figure 10] 11 is a flowchart illustrating an example of an operation of the machine learning system according to the third embodiment. [Figure 11] FIG. 13 is a diagram showing an example of a screen displayed on a display unit of the machine learning system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.
[0014] In the drawings explaining the embodiments, the same reference numerals are used to designate parts having the same functions, and repeated explanations thereof will be omitted.
[0015] In the following description, an expression such as "xxx data" may be used as an example of information, but the information may have any data structure. In other words, to show that the information does not depend on the data structure, "xxx data" may be referred to as "xxx table." Furthermore, "xxx data" may simply be referred to as "xxx." In the following description, the structure of each piece of information is an example, and the information may be divided and stored, or combined and stored.
[0016] In the following explanation, the process may be described with a "program" as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or a communication interface device (e.g., a port) as appropriate, the subject of the process may also be the program. Processes explained with a program as the subject may also be processes performed by a processor or a computer having that processor.
[0017] 1 is a diagram for explaining an overview of the operation of a machine learning system according to an embodiment. The overview of the embodiment will be explained using this diagram, and then the details will be explained using subsequent diagrams.
[0018] Reference numeral 11 in the figure indicates the correspondence between models and the sensor data that the models are to learn from. Model A performs learning using the brightness sensor, temperature sensor, and water pressure sensor as inputs. Model B performs learning using the water pressure sensor as inputs. Model C performs learning using the temperature sensor and vibration sensor as inputs. Model D performs learning using the vibration sensor as inputs. Model E performs learning using the temperature sensor and water pressure sensor as inputs.
[0019] Reference numeral 12 in the figure indicates the correspondence of sensor data affected by a maintenance event (maintenance work) based on the sensor data. As shown in 121, when maintenance of rust prevention coating with maintenance ID: MaintenanceId1 is performed, the brightness sensor is affected by the maintenance event and the trend of the sensor data changes. After that, when brightness sensor data whose trend has changed due to the event specified by maintenance ID: MaintenanceId1 is accumulated for a certain period of time or more, model A, which is a model trained using the brightness sensor data as input, inputs the sensor data shown in the range of 1211 in the figure and performs re-learning. At this time, the type of sensor data used for re-learning and the period of time are recorded. However, model A may cause false detection until re-learning is completed.
[0020] As shown in 122 in the figure, when maintenance is performed to replace parts with Maintenance ID: MaintenanceID2, the brightness sensor, temperature sensor, and water pressure sensor are affected by the maintenance event, and the trend of the sensor data changes. After that, when the sensor data, etc. whose trend has changed due to the Maintenance ID: MaintenanceId2 event has been accumulated for a certain period of time or more, Model B and Model D, which are models trained using these sensor data as input, perform re-learning. Specifically, Model B inputs sensor data, etc. within the range shown in 1221 in the figure and performs re-learning. Model D inputs sensor data, etc. within the range shown in 1222 in the figure and performs re-learning. At this time, the type of sensor data used for re-learning and the period of time are recorded. However, Model B and Model D may cause false detection until re-learning is completed.
[0021] As shown in 123 in the figure, a case will be described where maintenance of anti-rust coating with maintenance ID: MaintenanceId1 is performed again. Assume that the brightness sensor is also affected by the maintenance event at this time, and the trend of the sensor data changes. As described above, the maintenance event with maintenance ID: MaintenanceId1 was performed at 121, and at that time, the type of sensor data used for re-learning the model and its period were recorded. Therefore, when the first maintenance event with maintenance ID: MaintenanceId1 is performed, model A re-learns using the sensor data at 1211 in the figure that was used for re-learning. At this time, model A can perform re-learning without waiting for a certain period of time to pass and for sensor data to be accumulated, so it is possible to shorten the period during which the model causes false positives.
[0022] As shown in 124 in the figure, a case where maintenance is performed again to replace parts with maintenance ID: MaintenanceId2 will be described. At this time, the brightness sensor, temperature sensor, and water pressure sensor are also affected by the maintenance event, and the trend of the sensor data changes. As described above, the maintenance event with maintenance ID: MaintenanceId2 was performed at 122, and the type of sensor data used for model re-learning and its period were recorded at that time. Therefore, when the first maintenance event with maintenance ID: MaintenanceId2 was performed, model B re-learns using the sensor data with 1221 in the figure that was used for re-learning. When the first maintenance event with maintenance ID: MaintenanceId2 was performed, model D re-learns using the sensor data with 1222 in the figure that was used for re-learning. At this time, models B and D can perform re-learning without waiting for a certain period of time to pass and for sensor data to be accumulated, so there is a possibility that the period during which the model causes false detection can be shortened.
[0023] The model indicated by 125 in the figure shows a model newly created during operation. As described above, model E takes a temperature sensor and a water pressure sensor as input. This is different from the sensors that are input to any of models A, B, C, and E. If, after model E is created, maintenance is performed to replace parts with maintenance ID: MaintenanceId2 indicated by 124 in the figure, this maintenance is the first maintenance for model E, but since maintenance with maintenance ID: MaintenanceId2 has been performed in the past and re-learning has been performed using the water pressure sensor that model E inputs, the data that model E can use for re-learning is data after the past maintenance to replace parts with maintenance ID: MaintenanceId2 was performed, and therefore re-learning is performed using sensor data in the range indicated by 1251 in the figure. This is data after the past maintenance to replace parts with maintenance ID: MaintenanceId2 was performed. EXAMPLES
[0024] FIG. 2 is a diagram illustrating an outline of a system configuration of the machine learning system according to the first embodiment.
[0025] The machine learning system of this embodiment shown in Figure 2 has a machine learning model 20, a model management unit 21, a model-sensor correspondence table 22, an event management unit 23, an event-sensor correspondence table 24, a correspondence management unit 25, update candidate model information 26, a correspondence processing unit 27, a calculation unit 28, and a display unit 29.
[0026] The machine learning system of this embodiment is a device capable of various information processing, for example, an information processing device such as a computer, etc. The machine learning system has a processor represented by a CPU and a memory, and further has a storage and an input device.
[0027] The processor is, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory and storage include, for example, a magnetic storage medium such as a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), or a semiconductor storage medium such as a solid state drive (SSD). In addition, a combination of an optical disk such as a digital versatile disk (DVD) and an optical disk drive is also used as the memory and storage. In addition, known storage media such as magnetic tape media are also used as the memory and storage.
[0028] The storage stores programs such as firmware. When the machine learning system starts to operate (for example, when the power is turned on), the programs such as firmware are read from the storage and executed in memory to control the entire machine learning system. In addition to the programs, the storage also stores data necessary for each process of the machine learning system.
[0029] The machine learning system of this embodiment may be configured as a so-called cloud in which multiple information processing devices are configured to be able to communicate with each other via a communication network.
[0030] The machine learning model 20 uses sensor data output from various sensors such as a brightness sensor not shown in Fig. 2 as input data, and performs learning based on this input data according to a known machine learning algorithm. As described in detail in Fig. 1, the machine learning system of this embodiment has multiple machine learning models 20, and preferably, the sensor data that is the input data for each machine learning model 20 can be different. Here, different sensor data means that when there is one or more input data for the machine learning model 20, even one of these input data is different.
[0031] The model management unit 21 manages a delta-sensor association table 22 in which associations between (machine learning) models 20 and the sensor data that the models 20 use as inputs to execute learning are described.
[0032] The model-sensor correspondence table 22 records information managed by the model management unit 21. This may be recorded as in the table of FIG.
[0033] The event management unit 23 manages an event-sensor association table 24 in which associations between events such as maintenance and sensor data affected by the events are described. In this embodiment, the event is exemplified by maintenance, but it does not have to be maintenance and may be, for example, a typhoon or snowfall. An event refers to an occurrence of which the trend of sensor data changes from normal.
[0034] Information managed by the event management unit 23 is recorded in the event-sensor association table 24. This may be recorded as shown in FIG.
[0035] The association management unit 25 associates the information managed by the model management unit 21 with the information managed by the event management unit 23, and when a maintenance event occurs, identifies the model that has been affected by this maintenance event.
[0036] The update candidate model information 26 records information managed by the association management unit 25. This may be recorded as in the table of FIG.
[0037] The association processing unit 27 refers to the association between the model 20 and the sensors and the association between the events and the sensors shown in the second embodiment, and presents the period of the sensor data used as the input of the model 20. Details are shown in Fig. 9 and Fig. 10. The calculation unit 28 calculates the degree of influence and the like to identify the sensor data affected by the event.
[0038] The display unit 29 displays a predetermined screen based on a display control signal sent from the calculation unit 28. The input device sends an input instruction signal to each component of the machine learning system based on an input instruction action from an operator (not shown) of the machine learning system.
[0039] 2, the model 20 and the model-sensor association table 22 are input by a data scientist or the like, or are prepared in advance. The event-sensor association table 24 may be input by the operator of the machine learning system.
[0040] FIG. 3 illustrates an example of a model-sensor association table 22 that manages associations between the model 20 and the sensor data that the model 20 uses as input to execute learning.
[0041] The model-sensor correspondence table 22 manages correspondence between model identifiers such as a model ID and a model name for identifying the model 20 and sensor identifiers such as a sensor ID and a sensor name for outputting sensor data that the model 20 uses as input for learning.
[0042] For example, model A takes a brightness sensor, a temperature sensor, and a water pressure sensor as inputs, but does not take a vibration sensor as input. Model B takes a water pressure sensor as input, but does not take a brightness sensor, a temperature sensor, or a vibration sensor as inputs. In this way, the sensor identifiers used as inputs for learning are managed for each model identifier. However, the correspondence between the model used in this embodiment and the sensor data that the model uses as input to perform learning does not necessarily have to be the same as that in FIG. 3. In other words, it is sufficient to know the relationship between the model and the information that serves as input to the model.
[0043] FIG. 4 illustrates an example of an event-sensor association table 24 that manages associations between events such as maintenance and sensor data affected by the events.
[0044] The event-sensor association table 24 manages associations between maintenance identifiers such as a maintenance ID and a maintenance name for identifying a maintenance event and the impact level for each sensor identifier such as a sensor ID and a sensor name for identifying a sensor.
[0045] The calculation unit 28 records the degree of impact for each sensor identifier in order to identify the maintenance event with the maintenance ID MaintenanceID1, i.e., the date and time when the rust-preventive coating of part 1 was performed, and the sensor affected by the maintenance event. The degree of impact is a value indicating the degree of change in the sensor data before and after the time when the maintenance event was performed. The degree of impact is calculated, for example, by the following formula.
number
[0046] In the above formula, the contribution degree is disclosed in Japanese Patent No. 4832609, and the anomaly degree is disclosed in Japanese Patent No. 5480440. Details are given in the reference publication, but to give an outline, the anomaly degree is calculated by obtaining a feature vector obtained by normalizing the sensor data, clustering the feature vector, and taking the absolute value of the feature vector with the greatest distance from the cluster center as the cluster radius, and dividing the distance d of the feature vector of the sensor data to be calculated for the anomaly degree to the cluster center by the cluster radius r. The difference in the anomaly degree is calculated as the difference in the anomaly degree before and after the time when the maintenance event was performed. The contribution degree indicates the degree of contribution of each sensor data constituting the multiple sensor data used to calculate the anomaly degree to the calculated anomaly degree, and is calculated as the difference between the sensor data and the representative value of the sensor data constituting the cluster divided by the above-mentioned distance d.
[0047] Naturally, the degree of influence does not necessarily have to be expressed by the above formula. It may be a value that indicates the degree of change in the sensor data before and after an event such as maintenance, or it may be expressed as high, medium, low, etc. If the information input to the model is an image, the change in the brightness of the image corresponds to the degree of influence. In other words, it is sufficient to know whether the information input to the model was affected by the event or not.
[0048] FIG. 5 is a flowchart for explaining the operation of the machine learning system of this embodiment, and is a flow diagram illustrating the process of identifying a model that should be re-learned because the sensor data has changed due to the influence of an event such as maintenance work.
[0049] When an event such as maintenance occurs, the event management unit 23 records a maintenance event identifier, such as a maintenance ID or a maintenance name, and a date and time in the event-sensor association table 24 (step 51).
[0050] Next, the event management unit 23 records the degree of influence calculated by the calculation unit 28 for each sensor identifier in the event-sensor association table 24 (step 52).
[0051] Then, if the degree of influence calculated in step S52 is higher than a preset threshold value (YES in step 53), for the sensor identifier having that degree of influence, the association management unit 25 refers to the model-sensor association table 22 and identifies a model whose input includes the sensor identifier (step 54). Note that it is recommended that the threshold value be set in advance by confirming with the data scientist who created the model any changes in the sensor data that require the model to be re-learned, but this is not a limitation.
[0052] Then, the association management unit 25 presents the model identified in step 54 via a display unit using a GUI or the like, as shown in, for example, FIG. 6 or FIG. 10 described later (step 55).
[0053] FIG. 6 is a table that associates event identifiers with model identifiers that may be affected by the event and require re-learning.
[0054] For example, in the illustrated event-sensor correspondence table 24, when maintenance was performed to apply anti-rust coating to part 1 with a maintenance ID of MaintenanceId1, the influence degree of the luminance sensor with a sensor ID of Sensor1 was 1.49 as shown in Fig. 4. In this case, if the threshold value is assumed to be 0.5, the influence degree of the luminance sensor of 1.49 is greater than the threshold value, so it can be determined that the maintenance event with the maintenance ID of MaintenanceId1 influenced the luminance sensor.
[0055] Therefore, the association management unit 25 identifies the model 20 in which the luminance sensor is included in the input by referring to the model-sensor association table 22. By referring to the model-sensor association table 22, it is found that the model 20 in which the luminance sensor is included in the input is model A.
[0056] Therefore, when a maintenance event for applying anti-rust coating to part 1 is carried out, model A is likely to need to be re-learned, so model A will be marked as "required to be updated," while the other models B, C, and D will be marked as "no need to be updated." This also occurs for other events, such as the replacement of part 1 or the connection of part 1, and for each event, a model that is likely to need to be re-learned is identified.
[0057] The above is the flow of processing in the machine learning system of Example 1. By managing and processing information in this manner, when an event such as maintenance occurs, it is possible to identify the model 20 that is likely to be affected by the event and should be re-learned. This makes it possible to reduce the possibility that the model 20 will cause a false positive. EXAMPLES
[0058] Second Embodiment Next, a machine learning system according to a second embodiment will be described.
[0059] Fig. 7 is a diagram showing a model-sensor association table 22 according to a second embodiment, which manages association between a model 20 and sensor data that the model 20 uses as an input to execute learning, similar to the model-sensor association table 22 in the first embodiment shown in Fig. 3. Therefore, the present figure will be described focusing on the differences from Fig. 3.
[0060] A model identifier such as a model ID or a model name is associated with the date and time when the model identifier was created, a tolerance determined for each model 20, and an approximate guide period of sensor data that the model 20 learns as an input. Furthermore, the association of sensor identifiers such as a sensor ID or a sensor name that outputs sensor data that the model learns as an input, as shown in Fig. 3, is managed.
[0061] The tolerance defined for each model 20 is a guideline value for when the model should be re-learned. For example, if the degree of abnormality of the aforementioned model before and after maintenance exceeds the tolerance in this table, it is recommended that the model having that model identifier be re-learned. This tolerance may differ for each model, so it may be defined for each model in this table.
[0062] The approximate guide period of the sensor data that the model 20 learns as input is the period of the sensor data used for learning by the model 20. For example, the model A learns the model 20 using approximately 30 days' worth of brightness sensor data, approximately 30 days' worth of temperature sensor data, and approximately 30 days' worth of water pressure sensor data as input. The model B learns the model 20 using approximately 15 days' worth of water pressure sensor data as input. By indicating the guide period of learning, it can be used as a reference when re-learning using past sensor data in the processing flows shown in Figs. 9 and 10 described below.
[0063] Fig. 8 is a diagram showing an event-sensor association table 24 according to a second embodiment, which manages associations between events such as maintenance and sensor data affected by the events, similar to the event-sensor association table 24 in the first embodiment shown in Fig. 4. Therefore, the explanation will be focused on the differences from Fig. 4.
[0064] For each maintenance identifier such as a maintenance ID or a maintenance name and a sensor identifier such as a sensor ID or a sensor name, if there is a change in the sensor data that is input to the model 20 before and after a maintenance event and re-learning of the model 20 is performed, the period during which the sensor data was used is recorded for each sensor identifier if it was sensor data used during the re-learning.
[0065] For example, assume that a maintenance event for applying anti-rust coating to part 1, identified by MaintenanceId1, is carried out, and the tendency of the luminance sensor data changes due to the influence of the maintenance event, similar to Example 1. After that, assume that re-learning of model A using the luminance data as input is carried out by the method shown in Example 1, based on the judgment of a data scientist or the like.
[0066] At this time, the period used for the re-learning is recorded in the brightness sensor data. That is, 2017 / 08 / 29-2017 / 09 / 27 is recorded as the period used for the re-learning in the brightness sensor data column in Fig. 8. By doing so, it can be understood that the re-learning of model 20 was performed due to the influence of the maintenance event identified by MaintenanceId1, and that the re-learning was then performed using the brightness sensor data for the above period.
[0067] FIG. 9 is a flowchart illustrating a process for identifying a model 20 to be re-learned because the sensor data has changed due to an event such as maintenance, and further presenting the sensor data to be used for the re-learning, and managing the information shown in FIGS. 7 and 8.
[0068] When an event such as maintenance occurs, the event management unit 23 records a maintenance event identifier, such as a maintenance ID or a maintenance name, and a date and time in the event-sensor association table 24 (step 901).
[0069] Next, the event management unit 23 records the degree of influence calculated by the calculation unit 28 for each sensor identifier in the event-sensor association table 24 (step 902).
[0070] If the impact recorded in step 902 is higher than a preset threshold value (YES in step 903), the association management unit 25 refers to the model-sensor association table 22 for the sensor identifier having that impact, and identifies a model whose input includes the sensor identifier (step 904). On the other hand, if the impact is equal to or lower than the preset threshold value (NO in step 903), the process ends.
[0071] It is recommended that the threshold value be set in advance by checking with the data scientist who created the model about changes in the sensor data that would require the model to be retrained, but this is not necessarily required.
[0072] Next, the model management unit 21 acquires the tolerance of the model identified in step 904, and further compares the degree of abnormality output by the model, and if the degree of abnormality is higher than the tolerance (YES in step 905), the association processing unit 27 refers to the event-sensor association table 24 and searches for a maintenance event similar to the maintenance event shown in step 901 (step 906). On the other hand, if the degree of abnormality is equal to or lower than the tolerance (NO in step 905), the process ends.
[0073] The degree of anomaly shown in step 905 is the degree to which the model 20 judges the input sensor data as an anomaly as a result of an anomaly detection diagnosis, as in the first embodiment. The degree of anomaly is described in Japanese Patent No. 5480440.
[0074] The association processing unit 27 obtains the result of step 906, and if, for example, there was a similar maintenance event with the same maintenance ID in the past (step 907), and the influence for each sensor managed in the event-sensor association table 24 has a similar tendency to the past influence (step 908), presents the learning period of the sensor data exceeding the threshold value corresponding to step 908 from the event-sensor association table 24 (step 909). Here, the tendency similar to the past influence shown in step 908 does not have to be the exact same numerical value as the past value, such as the change in the value of the sensor data from before the past maintenance was performed to after the maintenance was performed increasing or decreasing by about 20 to 30 percent, as long as the rough tendency is similar.
[0075] If re-learning is performed based on the determination in step 909, for example by inputting the sensor data for which the model was presented (step 910), the event management unit 23 records the period used for re-learning for each sensor for the event ID in the event-sensor association table 24 (step 911), and ends the process. Note that if the branches of steps 907 and 908 are No, the process proceeds to step 910, and if the branch of step 910 is No, the process ends. EXAMPLES
[0076] Third Embodiment Next, a machine learning system according to a third embodiment will be described.
[0077] FIG. 10 is a flowchart illustrating a process of managing the information shown in FIG. 7 and FIG. 8, in which model 20 to be re-learned is identified because the sensor data has changed due to the influence of an event such as maintenance, and further, sensor data to be used for re-learning is presented.
[0078] 9 are the processes indicated by double lines in step 921, step 922, step 923, and step 924. In this process, even if a new model 20 is created during operation as described at the end of the explanation of FIG. 1, it is possible to present sensor data that is to be input for re-learning the new model 20.
[0079] Since the processes up to step 908 are the same as those shown in steps 901 to 907 in FIG. 9, the processes after step 908 will be explained.
[0080] If the impact of each sensor shows the same tendency as the past impact (YES in step 908), and the model creation date of the new model 20 is after the past maintenance event (YES in step 921), and further the sensor data input to the new model 20 was used for re-learning after the past maintenance event (YES in step 922), the correspondence processing unit 27 presents the learning period of the sensor data exceeding the threshold value corresponding to step 908 from the event-sensor correspondence table 24 (step 909).
[0081] If the process of step 921 is YES (step 923), the new model 20 did not exist at the time of the past maintenance event, so the event management unit 23 may suggest reviewing the model 20, such as re-learning (step 924). The process after step 910 is also the same as that in Fig. 9. If the branch of step 921, step 922, or step 924 is NO, the process moves to step 910.
[0082] FIG. 11 is a diagram illustrating an example of a GUI 1001 in the machine learning system according to the embodiment.
[0083] The maintenance event list displays the date and time when the maintenance was performed (2017 / 08 / 27 XX:XX:XX - 2017 / 8 / 28 XX:XX:XX) for each maintenance identifier such as the maintenance ID (MaintenanceId1) or maintenance name (Part 1 rust prevention coating).
[0084] For example, the user selects the maintenance identifier for which details are to be confirmed, and presses the maintenance event details display button 1002. Then, detailed information about the maintenance identifier is displayed in a maintenance event details display field 1003.
[0085] The information displayed may include a maintenance identifier such as a maintenance ID or maintenance name, the date and time the maintenance was performed, and a description of the maintenance content (for example, part 1 of part XX of machine number XX has developed rust or other issues due to aging, so anti-rust paint has been applied to part 1).
[0086] In addition, the affected model identifier (ModelIdA) is displayed for that maintenance identifier, and detailed information about the model identifier may include the model ID, model name (Model A), the sensor identifier (Sensor1, etc.) included as an input in the model, the impact value, and the period during which the data was used as input sensor data for re-learning when re-learning was performed in the past with the same maintenance ID. In addition, the GUI may be used to change threshold values such as the impact level at which a model is assumed to be affected by maintenance, as shown in this embodiment.
[0087] In addition, the above-mentioned embodiments are described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Also, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.
[0088] As an example, in the present embodiment, a description has been given of re-learning using sensor data from the same maintenance identifier performed in the past to create the model 20, but instead of re-learning using past sensor data, the model 20 used in the past may be used as is.
[0089] In addition, in the present embodiment, an example has been described in which re-learning is performed using past sensor data before sensor data affected by maintenance is accumulated. However, after sensor data affected by maintenance is accumulated, the model 20 may be re-learned using current sensor data.
[0090] Furthermore, in the present embodiment, an example in which one type of maintenance event is performed during a period has been described, but in actual maintenance work, multiple types of maintenance events may be performed simultaneously. For example, a maintenance event with MaintenanceId1 and a maintenance event with MaintenanceId2 may be performed at the same time. In this case, if there is information on the past maintenance event with MaintenanceId1 (for example, information written in the MaintenanceId1 row of the event-sensor correspondence table 24) and information on the past maintenance event with MaintenanceId2 (for example, information written in the MaintenanceId2 row of the event-sensor correspondence table 24), it is possible to estimate the impact of MaintenanceId1 and MaintenanceId2 being performed at the same time by adding up their impacts. Therefore, even if multiple types of maintenance events are performed at the same time, it may be possible to identify the model 20 affected by the maintenance and present the sensor data used for past re-learning.
[0091] Furthermore, the branching of the conditions in the process flows shown in FIGS. 5, 9, and 10 of this embodiment may be automated using threshold value judgment or may involve human judgment.
[0092] Furthermore, in this embodiment, past sensor data may be used to perform re-learning, and the operator may check and determine whether or not to adopt the re-learned model in the system.
[0093] In this embodiment, the location where the sensor data that the model 20 uses as input for re-learning is stored is not particularly limited, but the sensor data affected by the maintenance event may be stored separately in an easily accessible location.
[0094] The calculation methods of the contribution degree, the abnormality degree, and the tolerance degree shown in this embodiment are merely examples.
[0095] The tables, process flows, and GUIs shown in the first to third embodiments are merely examples, and the contents do not necessarily have to be the same as those shown, and other information may also be included.
[0096] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.
[0097] Furthermore, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and Python.
[0098] Furthermore, all or part of the program code of the software that realizes the functions of each embodiment may be stored in advance in the storage of the machine learning system, or, if necessary, may be stored in the storage from a non-transitory storage device of another device connected to the network, or from a non-transitory storage medium via an external I / F (not shown) provided by the machine learning system.
[0099] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read out and execute the program code stored in the storage means or storage medium.
[0100] In the above-mentioned embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all the control lines and information lines are shown in the product. All the components may be connected to each other. [Explanation of symbols]
[0101] 20... Machine learning model 21... Model management unit 22... Model-sensor correspondence table 23... Event management unit 24... Event-sensor correspondence table 25... Correspondence management unit 26... Update candidate model information 27... Correspondence processing unit 28... Calculation unit 29... Display unit 1001... GUI
Claims
1. A machine learning system having one or more machine learning models to which a machine learning algorithm is applied, in which sensor data output from one or more sensors that detect the state of a device that is an anomaly detection target is used as input data, and the machine learning models detect an anomaly or the like of the device based on the input data, The machine learning system includes a processor. The processor, managing a model identifier specific to the machine learning model that identifies the machine learning model and a sensor identifier specific to the sensor that identifies the sensor that outputs the sensor data that becomes the input data of the machine learning model in association with each other; determining an impact degree representing a change in a trend of the sensor data before and after the time when the maintenance work was performed for each maintenance event identifier specific to the maintenance work to be performed on the device, and managing the impact degree in association with each sensor identifier; The sensor whose degree of influence satisfies a predetermined condition is deemed to have been influenced by the maintenance work, and the model identifier associated with the sensor identifier of the sensor that satisfies the condition is presented. A machine learning system characterized by:
2. The machine learning system according to claim 1, characterized in that the processor considers that the sensor whose impact degree exceeds a predetermined threshold has been affected by the maintenance work, and presents the model identifier associated with the sensor identifier of the sensor that satisfies the condition.
3. The machine learning system of claim 1, characterized in that the degree of impact is calculated based on a change in the degree of anomaly, which is the deviation between the normal value range of the sensor data obtained as a result of clustering the sensor data and the actual sensor data, before and after the time when the maintenance work was performed.
4. The processor, managing, for each model identifier, a time when the machine learning model was created, a tolerance that is a judgment criterion for judging whether or not the machine learning model is abnormal, and a learning period of the sensor data input for learning the machine learning model; When the maintenance work occurs, the time when the maintenance work occurred and, if the machine learning model is re-learned, a set of the sensor identifier used in the re-learning and the period of the re-learning are associated with each maintenance event identifier and managed. The machine learning system of claim 1 .
5. The processor, When an anomaly level, which is a deviation between a normal value range of the sensor data obtained as a result of clustering the sensor data that is the input data of the machine learning model associated with the tolerance, and the actual sensor data, exceeds the tolerance, searching for the maintenance work that is identical to the maintenance event identifier of the maintenance work associated with the anomaly level; If the same maintenance event identifier exists in the past and the trend of the impact degree for each sensor is similar to that during the same maintenance work, a re-learning period corresponding to the same maintenance event identifier is read, and the sensor data of the sensor identifier corresponding to the re-learning period is input to re-learn the machine learning model of the model identifier; On the other hand, if the same maintenance event identifier did not exist in the past, or the trend of the impact degree for each sensor is not the same as that at the time of the same maintenance event, and further the re-learning of the machine learning model is performed, a re-learning period of the sensor data of the sensor identifier that is an input of the re-learning of the machine learning model associated with the model identifier is recorded in association with the maintenance event identifier. The machine learning system of claim 4.
6. When a new machine learning model is registered in the machine learning system during operation of the machine learning system, The processor, When the creation date of the new machine learning model is after the past maintenance work, and the sensor data that is the input data of the new machine learning model is used in the re-learning in the past maintenance work, If the same maintenance event identifier exists in the past and the trend of the impact degree for each sensor is similar to that at the time of the same maintenance event, the re-learning period corresponding to the same maintenance event identifier is read, and the sensor data of the sensor identifier corresponding to the re-learning period is input to re-learn the new machine learning model. The machine learning system of claim 5 .
7. The machine learning system has a display unit, The processor, When it is determined that the sensor, the degree of influence of which satisfies the predetermined condition, has been affected by the maintenance work, the model identifier associated with the sensor identifier of the sensor that satisfies the condition, the sensor that outputs the input data of the machine learning model associated with the model identifier, and the degree of influence associated with the sensor are displayed on the display unit. The machine learning system of claim 1 .
8. A machine learning model management method for a machine learning system having one or more machine learning models to which a machine learning algorithm is applied, in which sensor data output from one or more sensors that detect a state of a device that is an anomaly detection target is used as input data, and the machine learning models detect an anomaly or the like of the device based on the input data, managing a model identifier specific to the machine learning model that identifies the machine learning model and a sensor identifier specific to the sensor that identifies the sensor that outputs the sensor data that becomes the input data of the machine learning model in association with each other; determining an impact degree representing a change in a trend of the sensor data before and after the time when the maintenance work was performed for each maintenance event identifier specific to the maintenance work to be performed on the device, and managing the impact degree in association with each sensor identifier; The sensor whose degree of influence satisfies a predetermined condition is deemed to have been influenced by the maintenance work, and the model identifier associated with the sensor identifier of the sensor that satisfies the condition is presented. A machine learning model management method using a machine learning system.
Citation Information
Patent Citations
JP1973032609A
JP1975081999A
Inspection maintenance data verification device, inspection maintenance data verification system, and inspection maintenance data verification method
JP2014049010A
Learned model update system, learned model update method, and program
WO2019150565A1
Diagnosis apparatus
WO2021157676A1
