Monitoring device, method for monitoring, and program

Abstract

To provide a technique of detecting degradation of the performance of a trained model at a right timing.SOLUTION: The monitoring device according to an aspect of the present invention includes: an acquisition unit for acquiring environmental data on an environment for operating a trained model generated by machine learning; a degradation estimation unit for estimating degradation of performance of the trained model by shift of the environment from the acquired environmental data; and an information output unit for outputting information on the result of estimating the degradation of the performance of the trained model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a monitoring device, a monitoring method, and a program. [Background technology]

[0002] In recent years, trained models generated by machine learning are increasingly being used in situations where various inference tasks, such as defect detection, object detection, signal processing, motion control, and natural language processing, are being performed. In response to this, modeling of service maintenance using machine learning models is being advanced by a continuous learning framework called machine learning operation. As an example, Patent Document 1 proposes a technology for active learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-047751 Summary of the Invention [Problem to be solved by the invention]

[0004] The present inventors have found that conventional frameworks have the following problems. That is, conventional frameworks basically adopt push-type or pull-type methods as a maintenance method for dealing with degradation in the performance of a trained model. The push-type is a method for maintaining a trained model by encouraging annotation and additional learning when degradation in the performance of the trained model occurs in the operational scenario. The pull-type is a method for maintaining a trained model by periodically encouraging annotation and additional learning, regardless of degradation in performance in the operational scenario.

[0005] When the push type is adopted, the results of operation are collected and the performance of the trained model is evaluated retroactively from the collected results of operation, so that there is a risk that a trained model with degraded performance may be temporarily used. If a trained model with low performance continues to be used, there is a risk that losses such as the outflow of defective products due to false positives may occur. When the pull type is adopted, annotation and additional learning may be performed even if there is no actual degradation in performance that would interfere with operation, resulting in high maintenance costs. In other words, the conventional framework has a problem in that it is difficult to detect degradation in the performance of a trained model at an appropriate time.

[0006] In one aspect, the present invention has been made in consideration of the above circumstances, and its purpose is to provide a technique for detecting deterioration in the performance of a trained model at an appropriate time. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention employs the following configurations. Note that the following configurations of the invention can be combined as appropriate.

[0008] That is, a monitoring device according to one aspect of the present invention comprises an acquisition unit configured to acquire environmental data relating to the environment in which a trained model generated by machine learning is operated, a degradation estimation unit configured to estimate, from the acquired environmental data, degradation of performance of the trained model due to a shift in the environment, and an information output unit configured to output information relating to the result of estimating degradation of performance of the trained model.

[0009] One of the factors that cause the performance of a trained model to deteriorate is a shift in the environment in which the trained model is operated over time (domain shift). In this configuration, the deterioration of the performance of the trained model is predicted by focusing on this environmental shift. This makes it possible to detect the deterioration of the performance of the trained model at an appropriate timing. Note that predicting may include predicting. By detecting the deterioration of the performance of the trained model at an early timing, it is possible to reduce the situation in which a trained model with degraded performance is operated.

[0010] In the monitoring device according to the above aspect, the environmental data may include input data obtained from the environment during operation and provided to the trained model to perform inference by the trained model. Estimating degradation in performance of the trained model may include estimating a decrease in reliability of inference of the trained model for the input data. The input data provided to the trained model reflects the environment in which the trained model is operated. Therefore, according to this configuration, degradation in performance of the trained model caused by a shift in the environment can be appropriately determined.

[0011] In the monitoring device according to the above aspect, the inference may include class identification. The reliability of the inference may be calculated according to the entropy of the result of the class identification by the trained model for the input data. The entropy makes it possible to appropriately evaluate the reliability of the inference. Therefore, according to this configuration, it is possible to appropriately determine the deterioration of the performance of the trained model caused by an environment shift. Note that the entropy may be appropriately adjusted by calibration to reduce the difference between the entropy and the reliability of the inference, and the reliability of the inference may be calculated according to the adjusted entropy.

[0012] In the monitoring device according to the above aspect, the trained model may be configured to output a reliability score of the inference together with the result of the inference. The reliability of the inference may be calculated according to the score output from the trained model for the input data. By using a trained model configured to output a reliability score together with the inference result, the reliability of the inference can be appropriately evaluated. Therefore, according to this configuration, it is possible to appropriately determine the deterioration of the performance of the trained model caused by a shift in the environment.

[0013] In the monitoring device according to the above aspect, the reliability of the inference may be calculated according to the extent to which the input data is within a distribution range of training data used in the machine learning of the trained model. By using a distribution comparison method, the reliability of the inference can be appropriately evaluated. Therefore, according to this configuration, it is possible to appropriately determine the deterioration of the performance of the trained model caused by a shift in the environment.

[0014] In the monitoring device according to the above aspect, the environmental data may include condition data indicating conditions under which input data to be provided to the trained model during operation is obtained from the environment in order to execute inference by the trained model. Predicting deterioration in performance of the trained model may include predicting deterioration in accuracy of inference by the trained model according to a degree of deviation of the conditions indicated by the condition data from the conditions under which training data used in machine learning of the trained model was obtained. The conditions (condition data) under which input data is obtained from the environment reflect the environment in which the trained model is operated. Therefore, according to this configuration, deterioration in performance of the trained model caused by a shift in the environment can be appropriately determined.

[0015] In the monitoring device according to the above aspect, the input data may include observation data acquired by a sensor. The condition data may include observation condition data indicating a condition of observation by the sensor during operation. According to this configuration, in a scene where the sensor is used, It is possible to properly determine the degradation in performance of a trained model due to a shift in

[0016] In the monitoring device according to the above aspect, the sensor may include an imaging device, and the observation conditions may include imaging conditions of the imaging device. With this configuration, in a situation where the imaging device is used, it is possible to appropriately determine deterioration in performance of a trained model caused by a shift in the environment.

[0017] In the monitoring device according to the above aspect, the sensor may include a microphone, and the observation conditions may include recording conditions of the microphone. With this configuration, in a situation where the microphone is used, it is possible to appropriately determine deterioration of the performance of a trained model caused by a shift in the environment.

[0018] In the monitoring device according to the above aspect, the sensor may be mounted on a robot device, and the observation conditions may include conditions related to the operation of the robot device. With this configuration, in a situation where the robot device is used, it is possible to appropriately determine deterioration in performance of a trained model caused by a shift in the environment.

[0019] In the monitoring device according to the above aspect, estimating the deterioration of the performance of the trained model may be continuously performed, and the output information may include a result of predicting when the performance of the trained model will fall below a threshold. According to this configuration, a result of predicting when the performance of the trained model will fall below a threshold is provided. This makes it possible to prompt maintenance of the trained model (e.g., annotation and additional learning) at an appropriate time.

[0020] In the monitoring device according to the above aspect, acquiring the environmental data may be configured by acquiring environmental data for each of the plurality of trained models. Inferring degradation of performance of the trained models may be configured by estimating degradation of performance of each of the plurality of trained models. With this configuration, in a situation where a plurality of trained models are used, degradation of performance of the trained models can be detected at an appropriate time.

[0021] In the monitoring device according to the above aspect, the output information may include a map showing the operation location of each of the trained models. Outputting the information may include superimposing and displaying information on the result of estimating the performance degradation of each trained model on the operation location of each trained model on the map. With this configuration, it is possible to visualize an evaluation of the performance degradation of each trained model together with the operation location of each trained model.

[0022] In the monitoring device according to the above aspect, the output information may include a result of estimating the degree to which the performance of the trained model will be improved by additional learning of a predetermined number of samples. With this configuration, it is possible to show a guide to the performance improvement by additional learning, thereby encouraging the implementation of appropriate additional learning.

[0023] In the monitoring device according to the above aspect, estimating the deterioration of the performance of the trained model may be continuously performed. The output information may include at least one of the timing of additional learning optimized within a predetermined budget and the number of annotations for the additional learning samples. This configuration can encourage the execution of additional learning optimized within a predetermined budget.

[0024] The present invention is not limited to the above-mentioned monitoring device. As another embodiment of the monitoring device according to each of the above-mentioned aspects, one aspect of the present invention is a monitoring device comprising all or part of each of the above-mentioned configurations. The present invention may be an information processing method for realizing the above, a program, or a storage medium that stores such a program and is readable by a machine such as a computer. A storage medium that is readable by a machine such as a computer is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action.

[0025] For example, a monitoring method according to one aspect of the present invention may be an information processing method in which a computer acquires environmental data regarding an environment in which a trained model generated by machine learning is operated, estimates from the acquired environmental data a deterioration in performance of the trained model due to a shift in the environment, and outputs information regarding the result of the estimation of the deterioration in performance of the trained model.

[0026] Furthermore, for example, a program relating to one aspect of the present invention may be a program for causing a computer to execute the following steps: acquiring environmental data regarding the environment in which a trained model generated by machine learning is operated; estimating, from the acquired environmental data, a deterioration in performance of the trained model due to a shift in the environment; and outputting information regarding the result of estimating the deterioration in performance of the trained model. Effect of the Invention

[0027] According to the present invention, deterioration of the performance of a trained model can be detected at an appropriate time. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 shows a schematic diagram of an example of a situation in which the present invention is applied. [Diagram 2] FIG. 2 is a diagram for explaining the degradation of the performance of a trained model caused by an environment shift (domain shift). [Diagram 3] FIG. 3 shows an example of a calculation process of the inference device according to the embodiment. [Figure 4A]FIG. 4A illustrates an example of a reasoning task according to an embodiment. [Figure 4B] FIG. 4B illustrates an example of a reasoning task according to an embodiment. [Figure 4C] FIG. 4C illustrates an example of a reasoning task according to an embodiment. [Diagram 5] FIG. 5 illustrates an example of a method (method using input data) for estimating performance degradation due to an environmental shift according to the embodiment. [Figure 6A] FIG. 6A illustrates a schematic diagram of an example of a method for estimating performance degradation using input data according to an embodiment. [Figure 6B] FIG. 6B illustrates a schematic diagram of an example of a method for estimating performance degradation using input data according to an embodiment. [Figure 6C] FIG. 6C illustrates a schematic diagram of an example of a method for estimating performance degradation using input data according to an embodiment. [Figure 7] FIG. 7 illustrates an example of a method (method using condition data) for estimating performance degradation due to an environmental shift according to the embodiment. [Figure 8] FIG. 8 illustrates an example of a hardware configuration of a monitoring device according to an embodiment. [Figure 9] FIG. 9 illustrates an example of a software configuration of the monitoring device according to the embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of a processing procedure of the monitoring device according to the embodiment. [Figure 11] FIG. 11 illustrates an example of output information according to the embodiment. [Figure 12] FIG. 12 illustrates an example of output information according to the embodiment. [Figure 13A] FIG. 13A illustrates an example of output information according to an embodiment. [Figure 13B] FIG. 13B illustrates an example of output information according to the embodiment. [Figure 14] FIG. 14 illustrates an example of output information according to the embodiment. [Figure 15] FIG. 15 is a diagram for explaining optimization of annotations according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] An embodiment according to one aspect of the present invention (hereinafter, also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in every respect. Various improvements or modifications may be made without departing from the scope of the present invention. In carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0030] §1 Examples of application 1 is a schematic diagram showing an example of a situation in which the present invention is applied. The system according to this embodiment includes an inference device 2 and a monitoring device 1.

[0031] The inference device 2 is one or more computers configured to execute an inference process using a trained model 5. Specifically, the inference device 2 acquires input data 50. The inference device 2 then provides the acquired input data 50 to the trained model 5 to execute a computation process of the trained model 5. As a result of this computation process, the inference device 2 derives an inference result from the input data 50. The derived inference result may be utilized as appropriate.

[0032] On the other hand, the monitoring device 1 is one or more computers configured to monitor the performance of the trained model 5. Specifically, the monitoring device 1 acquires environmental data 3 related to the environment in which the trained model 5 generated by machine learning is operated. The monitoring device 1 estimates the deterioration of the performance of the trained model 5 due to a shift in the environment from the acquired environmental data 3. The shift in the environment may also be referred to as a domain shift. Then, the monitoring device 1 outputs information related to the result of estimating the deterioration of the performance of the trained model 5.

[0033] FIG. 2 is a diagram for explaining the deterioration of the performance of a trained model caused by an environmental shift (domain shift) over time. When training for an operating environment is completed and the trained model is put into that environment (T1), the latest data is reflected in the training. Therefore, the difference between the distribution of input data expected by the trained model (machine learning model) (solid line) and the distribution of input data obtained in the operating environment (dotted line) is small. Therefore, in that environment, the trained model can demonstrate high performance.

[0034] However, the environment in which the trained model is operated is not always constant, and shifts occur over time (T2). As an example, when a sensor is used to acquire input data, a shift in the environment may occur due to a change in the observation conditions of the sensor. The change in the observation conditions may be, for example, a change in the inclination of the sensor, deterioration of the sensor over time, replacement of the sensor, or a change in the situation of the observation range of the sensor (for example, a change in brightness). This shift may occur gradually over time, or it may occur suddenly in a short period of time. When such an environmental shift occurs, the difference between the distribution assumed by the trained model (solid line) and the distribution of the operating environment (dotted line) becomes large. As a result, the performance of the trained model deteriorates.

[0035] Therefore, we collect training samples from the post-shift environment, annotate the collected training samples, and use the resulting dataset to perform additional learning of the trained model. By applying the trained model to the post-shift environment, we can reduce the difference between the distribution predicted by the trained model (solid line) and the distribution in the operating environment (dotted line). The performance of the trained model can be restored (T3).

[0036] As described above, when the push type is adopted, the results of operation are collected and the performance of the trained model is evaluated retroactively from the collected results of operation, so there is a possibility that the timing of detecting the deterioration of the performance of the trained model by T2 will be delayed. This may result in a significant delay in the timing of T3, and a trained model with degraded performance may continue to be used. Continuing to use a trained model with low performance may result in losses such as the outflow of defective products due to false positives. In addition, when the pull type is adopted, annotation and additional learning may be performed even if there is no actual deterioration in performance that would interfere with operation (a deviation in distribution like T2), resulting in high maintenance costs.

[0037] In contrast, in this embodiment, the monitoring device 1 uses the environmental data 3 to estimate the deterioration of the performance of the trained model 5 due to a deviation such as T2. This makes it possible to quickly detect a deviation in distribution such as T2 and to estimate the deterioration of the performance of the trained model 5 at an early stage. Therefore, according to this embodiment, the deterioration of the performance of the trained model 5 can be detected at an appropriate timing.

[0038] [A Inference Processing] The inference performed by the trained model 5 may include at least one of regression and identification. The inference may include prediction. The inference task may not be particularly limited and may be appropriately selected according to the embodiment. In one example, the inference task may be a comparison of the similarity of the input data, a calculation of the causal relationship of the input data, matching of the input data, extraction of a range including a feature in the input data, identification of a feature included in the input data, generation of information for the input data, etc. As a specific example, the inference task may be detection of defects in a factory, detection of a person, detection of an object (vehicle, etc.), identification of an object, detection of an abnormal sound, voice recognition, evaluation of a state, route search, derivation of a control command, generation of information (image, voice, control command, etc.), etc. The input data 50 (explanatory variables) used for the inference may be appropriately selected according to the inference target (object variable). The input data 50 may be obtained by any method.

[0039] Fig. 3 shows a schematic diagram of an example of a calculation process of the inference device 2 according to this embodiment. In the example of Fig. 3, it is assumed that the inference device 2 acquires at least a part of the input data 50 from one or more sensors S. That is, in this example of this embodiment, the input data 50 may include observation data 50S acquired by the sensor S. The inference task may include any inference process for the observation data 50S, and the trained model 5 may be generated to acquire the ability to perform the inference task. The observation data acquired by the sensor may be referred to as sensing data, measurement data, or the like.

[0040] The connection method and acquisition method may be appropriately selected depending on the embodiment. In one example, the inference device 2 may be connected to the sensor S via a wired or wireless connection. The inference device 2 may acquire the observation data 50S directly from the sensor S. In another example, the inference device 2 may acquire the observation data 50S indirectly from the sensor S via at least one of another computer and a storage medium.

[0041] The type of the sensor S may be appropriately selected depending on the inference task. In one example, the sensor S may be, for example, an imaging device, a microphone, an encoder, an environmental sensor, a vital sensor, a medical testing device, an in-vehicle sensor, a home security sensor, etc. The environmental sensor may be, for example, a barometer, a thermometer, a hygrometer, a sound pressure meter, a sound sensor, an ultraviolet sensor, a light meter, a rain gauge, a gas sensor, a light sensor, an earthquake sensor, a magnetic sensor, etc. The vital sensor may be, for example, a blood pressure meter, a pulse meter, a heart rate meter, an electrocardiograph, an electromyograph, a thermometer, a skin electrodermal response meter, a microwave sensor, an electroencephalograph, a magnetoencephalograph, an activity meter, a blood glucose meter, an electrooculography sensor, an eye movement meter, a posture sensor, etc. The medical examination device may be, for example, a CT (Computed Tomography) device, an MR The in-vehicle sensor may be, for example, an image sensor, a Lidar (light detection and ranging) sensor, a millimeter wave radar, an ultrasonic sensor, an infrared sensor, an acceleration sensor, etc. The home security sensor may be, for example, an image sensor, an infrared sensor, an activity (sound) sensor, a gas (CO2, etc.) sensor, a current sensor, a smart meter (a sensor that measures the power consumption of home appliances, lighting, etc.), etc. The observation data 50S may be image data, sound data, numerical data, text data, and other sensing data obtained by various sensors. The image data may include, for example, photographic data such as satellite photographs. In one example, the inference device 2 may acquire the observation data 50S from multiple sensors S. The observation data 50S may be composed of multiple types of sensing data.

[0042] (A1) Use of imaging devices 4A is a schematic diagram of an example of an inference task according to the present embodiment. As illustrated in FIG. 4A, the sensor S may include an image capture device SA. In this case, the input data 50 may include image data 50SA obtained by one or more image capture devices SA.

[0043] The inference task performed by the trained model 5 may be any inference process for the image data 50SA. The inference process may include, for example, image analysis processes such as object extraction and object identification. As a specific example, the inference process may be extraction of defective parts in a work, quality determination of a work, person detection, person identification, object detection (vehicle, etc.), object identification, symptom part extraction, symptom identification, etc. The work may be any of materials, intermediate products of a product, and final products.

[0044] The type of the imaging device SA is not particularly limited and may be appropriately selected depending on the embodiment of the inference task, etc. The imaging device SA may be a general camera, or may be a depth sensor, a CT device, an MRI device, a Lidar sensor, an infrared sensor, or the like that can observe in an image format (for example, two-dimensional or three-dimensional). The present invention may include any sensor that obtains measurement data.

[0045] (A2) Use of microphones 4B is a schematic diagram showing an example of an inference task according to the present embodiment. As illustrated in FIG. 4B, the sensor S may include a microphone SB. In this case, the input data 50 may include sound data 50SB obtained by one or more microphones SB.

[0046] The inference task performed by the trained model 5 may be any inference process for the sound data 50SB. The inference process may include, for example, sound analysis processes such as feature extraction and feature identification. As a specific example, the inference process may be, for example, detection of abnormal sounds, voice recognition, evaluation of health conditions from vital data obtained from sounds, etc. The type of microphone SB is not particularly limited and may be appropriately selected depending on the embodiment of the inference task, etc.

[0047] (A3) Robot device case 4C is a schematic diagram illustrating an example of an inference task according to the present embodiment. As illustrated in FIG. 4C, the sensor S may include a sensor SC mounted on the robot device R. In this case, the input data 50 may include observation data 50 SC obtained by the sensor SC mounted on the robot device R.

[0048] The robot device R may be, for example, an industrial robot in a production line, an autonomous robot configured to be able to operate autonomously, a mobile body configured to be able to move, etc. The industrial robot may be, for example, a vertical articulated robot, a horizontal articulated robot (SCARA robot), a parallel link robot, an orthogonal robot, etc. The autonomous robot may be, for example, a humanoid robot, The autonomous robot may be a guide robot, an agricultural robot, a care robot, a security robot, a transport robot, etc. The content of the autonomous processing may be appropriately selected depending on the embodiment. The moving body may include, for example, a cleaning robot, the autonomous robot configured to be movable, a vehicle configured to be capable of automatic driving, an aircraft (drone, etc.) capable of automatic flight, etc.

[0049] The sensor SC is not particularly limited as long as it is related to the robot device R, and may be appropriately selected depending on the embodiment. The sensor SC may be, for example, an imaging device, a microphone, an encoder, or other sensors. The sensor SC may be mounted on the robot device R in any manner. The sensor SC may be built into the robot device R, may be attached to the outside of the robot device R, or may be disposed at a position remote from the robot device R. A plurality of sensors SC may be mounted on the robot device R, and the observation data 50SC may include a plurality of types of sensing data.

[0050] The inference task performed by the trained model 5 may be any inference processing for the observation data 50SC. The inference processing may include information processing related to the robot device R, such as image recognition, voice recognition, other recognition, path search, and derivation of control commands. When the inference device 2 is involved in controlling the operation of the robot device R, the inference device 2 may be referred to as a control device.

[0051] The information source of the input data 50 is not limited to the sensor S, and may be appropriately selected depending on the embodiment. The information source of the input data 50 may include an information source other than the sensor, and at least a part of the input data 50 may be obtained from an information source other than the sensor. An example of the information source other than the sensor is a user input, etc.

[0052] [B Pre-trained model] The trained model 5 is configured by a machine learning model. The machine learning model is configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used for the calculation of the target inference. Machine learning is to adjust (optimize) the value of the calculation parameter using a training sample. The machine learning model may be configured by, for example, a neural network, a regression model, a decision tree model, a support vector machine, other function formulas (calculation models), etc. The machine learning method may be appropriately selected according to the machine learning model to be adopted (for example, backpropagation method, etc.).

[0053] In one example, the trained model 5 may include a neural network. The structure of the neural network may be appropriately determined depending on the embodiment, and may be specified, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationship between the nodes in each layer, etc. In one example, the neural network may have a recursive structure. In addition, the neural network may include any layer, such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, an unpooling layer, a normalization layer, a dropout layer, or a Long short-term memory (LSTM). The neural network may include any type of model, such as a diffusion model or a generative model. The neural network The weight of the connections between the nodes included in the network and the threshold value of each node are examples of calculation parameters.

[0054] In addition, the form of input and output of the trained model 5 is not particularly limited, and may be appropriately selected depending on the embodiment. In one example, the input data 50 may be provided to the trained model 5 as it is obtained from the environment, such as the sensor S. In another example, any pre-processing (which may include feature extraction, other analysis, inference, and other computational processing) may be applied to at least a portion of the input data 50 before it is provided to the trained model 5. That is, in order to perform an inference task, the pre-processed input data 50 may be provided to the trained model 5. A trained model other than the trained model 5 may be used for the pre-processing. In one example, the trained model 5 may be provided to the trained model 5 as it is. In one example, the output of the trained model 5 may be configured to directly indicate the inference result. In another example, the output of the trained model 5 may be configured to indirectly indicate the inference result. In this case, the inference result may be obtained by performing any information processing (interpretation processing) on ​​the output of the trained model 5. The inference task may be performed in real time on data obtained at the present time (which may include past data) or may be performed retrospectively on data obtained in the past.

[0055] The number of trained models 5 whose performance is monitored by the monitoring device 1 is not limited to one, and may be two or more. In one example, the monitoring device 1 may monitor a plurality of trained models 5. In this case, acquiring the environmental data 3 may be configured by acquiring the environmental data 3 of each of the plurality of trained models 5. Inferring the deterioration of the performance of the trained model 5 may be configured by inferring the deterioration of the performance of each of the plurality of trained models 5. According to one example of the present embodiment, in a situation where a plurality of trained models 5 are operated, the deterioration of the performance of each of the trained models 5 can be detected at an appropriate timing. In another example, the monitoring device 1 may monitor a single trained model 5.

[0056] [C environment] The environment may include any events related to the situation in which the trained model 5 performs inference. The type of the environmental data 3 is not particularly limited as long as the data can represent the environment, and may be appropriately selected according to the embodiment. The environment may be at least one of a real environment and a virtual environment.

[0057] In one example, the input data 50 used for inference in the trained model 5 is obtained from the environment. Therefore, the input data 50 may reflect the influence of the environment, and thus a shift in the environment can be detected from the input data 50. Thus, the environmental data 3 may include the input data 50 obtained from the environment during operation and provided to the trained model 5 to perform inference by the trained model 5. The monitoring device 1 may detect a shift in the environment from the input data 50.

[0058] Furthermore, since the influence of the environment is reflected in the input data 50, the influence of the environment is also reflected in the inference result (output data) obtained by the trained model 5. Therefore, the environmental data 3 may include output data obtained from the trained model 5 by providing the input data 50 to the trained model 5 and executing a calculation process (inference process) by the trained model 5. The monitoring device 1 may detect a shift in the environment from the output data.

[0059] Furthermore, the conditions under which the input data 50 is obtained from the environment, such as the observation conditions of the sensor S, may directly indicate the environment. Therefore, it is clear that a shift in the environment can be detected from the conditions. Thus, the environmental data 3 may include condition data indicating the conditions under which the input data 50 to be provided to the trained model 5 during operation is obtained from the environment in order to execute inference by the trained model 5. The monitoring device 1 may detect a shift in the environment from the condition data. The environmental data 3 may be composed of a combination of the input data 50, the output data, and the condition data.

[0060] [D Example of a method for inferring A method for estimating the deterioration of the performance of the trained model 5 due to an environmental shift may be appropriately selected depending on the data used as the environmental data 3. In one example, the monitoring device 1 may directly or indirectly evaluate the degree of the environmental shift from the environmental data 3, and as a result of the evaluation, calculate that the smaller the degree of the shift, the smaller the deterioration of the performance, and vice versa.

[0061] (D1) Use input data FIG. 5 is a schematic diagram showing an example of a method (a method using input data 50) for estimating performance degradation due to an environmental shift according to the present embodiment. As illustrated in FIG. 5, the environmental data 3 may include the input data 50. In accordance with this, estimating the performance degradation of the trained model 5 may include estimating a decrease in the reliability (which may also be referred to as confidence) of the inference of the trained model 5 for the input data 50. In one example, the monitoring device 1 may obtain a plurality of samples of the input data 50, statistically evaluate the reliability of the inference for each of the obtained samples, and estimate the degree of performance degradation of the trained model 5 according to the evaluation result. In each of the following examples, a configuration for statistically evaluating the degree of performance degradation of the trained model 5 from a plurality of samples may also be adopted.

[0062] As described above, the input data 50 reflects the environment in which the trained model 5 is operated. Therefore, according to an example of this embodiment (FIG. 5), it is possible to appropriately determine the deterioration of the performance of the trained model 5 caused by a shift in the environment. The reliability of the inference indicates the degree of certainty of the result of inference by the trained model 5. The reliability of the inference may be evaluated by an appropriate method so that the reliability becomes low enough to infer that a shift in the environment has occurred. In the examples of FIG. 3 and FIG. 4A to FIG. 4C, the input data 50 may include observation data 50S (image data 50SA, sound data 50SB, observation data 50SC) acquired by the sensor S (imaging device SA, microphone SB, sensor SC).

[0063] In addition, in one example, the monitoring device 1 may first calculate the reliability of the inference from the input data 50, and infer the deterioration of the performance of the trained model 5 according to the calculated reliability of the inference. That is, the monitoring device 1 may indirectly infer the deterioration of the performance of the trained model 5 from the input data 50. In another example, the monitoring device 1 may directly infer the deterioration of the performance of the trained model 5 from the input data 50 by using an evaluation model including an operation for evaluating the reliability of the inference.

[0064] (D1-1) Entropy-based method FIG. 6A is a schematic diagram showing an example of a method (entropy-based method) for inferring performance degradation using input data 50. As illustrated in FIG. 6A, inference by a trained model 5 may include class identification. In one example, the result of class identification is expressed as the probability of each class. The higher the certainty of identification, the higher the probability of only a specific class (i.e., the more deterministic the identification result is), and the smaller the entropy. On the other hand, the lower the certainty of identification, the less deterministic the identification result is, and the more sparse the probability of each class is, and the higher the entropy is.

[0065] Therefore, the reliability of the inference may be calculated according to the entropy in the result of class identification by the trained model 5 for the input data 50. That is, the monitoring device 1 may calculate the entropy from the class identification result output from the trained model 5 by giving the input data 50 to the trained model 5 and executing the arithmetic processing of the trained model 5. Then, the monitoring device 1 may calculate the degree of deterioration of the performance of the trained model 5 according to the calculated entropy. Specifically, it may be calculated that the smaller the entropy, the higher the reliability of the inference, and the less the deterioration of the performance of the trained model 5. On the other hand, it may be calculated that the larger the entropy, the lower the reliability of the inference, and the more the deterioration of the performance of the trained model 5. As described above, according to the entropy, the reliability of the inference can be appropriately evaluated. Therefore, according to an example of the present embodiment, it is possible to appropriately determine the deterioration of the performance of the trained model 5 due to the environmental shift. Note that there may be a difference between the entropy and the reliability of the inference. In such a case, for example, the difference between the entropy and the reliability of the inference can be reduced by adjusting by a calibration method such as Reference 1 (Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger, "On Calibration of Modern Neural Ne tworks", [online], [searched on June 23, Reiwa 5], Internet <URL:https: / / dl.acm.org / doi / pdf / 10.5555 / 3305381.<3305518>). The entropy adjusted by an arbitrary calibration method may be used for the above determination.

[0066] (D1-2) A method of adopting a model that outputs reliability FIG. 6B is a schematic diagram showing an example of a method for predicting performance degradation using input data 50 (a method for adopting a model that outputs confidence). For example, see Reference 2 (Terrance DeVries, Graham W. Taylor, “Learning Confidence for Out-of-Distribution Detection in Neural Networks”, [online], [Retrieved June 23, 2023], Internet<URL:https: / / arxiv.org / abs / 1802.04865> ) makes it possible to generate trained models that have the ability to output inference results as well as the confidence level of the inference results.

[0067] Therefore, as shown in FIG. 6B, the trained model 5 may be configured to output a score of the reliability of the inference together with the result of the inference. The reliability of the inference may be calculated according to the score output from the trained model 5 for the input data 50. That is, the monitoring device 1 may obtain a reliability score output from the trained model 5 by providing the input data 50 to the trained model 5 and executing the arithmetic processing of the trained model 5. The reliability score may be used as the reliability of the inference as it is, or may be used as the reliability of the inference after being numerically converted by any arithmetic processing. According to the obtained reliability score, the monitoring device 1 may calculate that the deterioration of the performance of the trained model 5 is small when the reliability of the inference is evaluated to be high, and may calculate that the deterioration of the performance of the trained model 5 is large when the reliability of the inference is evaluated to be low. By using the trained model 5 configured to output the reliability score of the inference, the reliability of the inference can be appropriately evaluated. Therefore, according to one example of the present embodiment, the deterioration of the performance of the trained model 5 caused by the shift in the environment can be appropriately determined.

[0068] (D1-3) Distribution Comparison Method FIG. 6C illustrates an example of a method (a distribution comparison method) for inferring performance degradation using input data 50.

[0069] In general, a trained model has acquired the ability to perform a target inference task within the scope of training data (training samples) used in machine learning. If the input data is within the scope of the training data, the input data is known, and the trained model can derive an inference result with high reliability. On the other hand, if the input data falls outside the scope of the training data, the reliability of the inference result of the trained model may decrease. Therefore, as the input data deviates from the scope of the training data due to a shift in the environment, the opportunities to perform an inference task for unknown input data may increase, and the performance of the trained model may deteriorate.

[0070] Therefore, as illustrated in FIG. 6C, the reliability of the inference may be calculated according to the extent to which the input data 50 is within the range of the distribution of the training data used in the machine learning of the trained model 5. That is, the monitoring device 1 may calculate the degree of deterioration of the performance of the trained model 5 according to the extent to which the input data 50 is within the range of the distribution of the training data. Specifically, the monitoring device 1 may calculate that the performance of the trained model 5 is less deteriorated as the input data 50 is within the range of the distribution of the training data, and that the performance of the trained model 5 is more deteriorated as the input data 50 is outside the range of the distribution of the training data. By using a method based on distribution comparison, the reliability of the inference can be appropriately evaluated. Therefore, according to one example of the present embodiment, the deterioration of the performance of the trained model 5 caused by the shift in the environment can be appropriately determined.

[0071] The method of evaluating whether the input data 50 is within the range of the distribution of the training data is not particularly limited, and may be appropriately selected according to the embodiment. The distribution comparison may be performed on a probability density basis. In one example, the monitoring device 1 may access the training data used for the machine learning of the trained model 5. The training data may be held in the monitoring device 1, may be stored in a storage medium, or may be held in another computer to which the monitoring device 1 can be connected. The monitoring device 1 may directly compare the input data 50 with the training data, and calculate the degree of deterioration of the performance of the trained model 5 according to the comparison result. A statistical method such as an outlier test may be adopted as the method of direct comparison.

[0072] In another example, the monitoring device 1 may indirectly compare the input data 50 with the training data and calculate the degree of deterioration in performance of the trained model 5 according to the comparison result. In one example, the indirect comparison may be to evaluate the input data 50 using a computational model obtained from the training data. As an example of a method for indirect comparison, the monitoring device 1 may use a feature-based determination method proposed in Reference 3 (JP 2021-081814 A).

[0073] As another example of the indirect comparison method, the monitoring device 1 may use a method using a reconstruction error by an autoencoder, a principal component vector, or the like. As a specific example, when the autoencoder method is adopted, the autoencoder may be generated together with the trained model 5. The autoencoder is composed of an encoder and a decoder. The autoencoder can be generated by machine learning using training data (training samples) as input data and also as true values ​​of output data. By this machine learning, the autoencoder acquires the ability to accurately restore input data within the range of the training data. That is, the more input data is given that is farther away from the training data, the larger the reconstruction error between the output data (restored data) output from the autoencoder and the input data may be. Therefore, the monitoring device 1 may obtain output data (restored data) from the autoencoder by providing the input data 50 to the autoencoder and executing the arithmetic processing of the autoencoder. The monitoring device 1 may calculate the reconstruction error between the acquired output data and input data, and calculate the degree of deterioration of the performance of the trained model according to the calculated reconstruction error. Specifically, the monitoring device 1 may calculate that the smaller the calculated reconstruction error is, the smaller the shift in the environment is, and the less the deterioration in the performance of the trained model is, and that the larger the calculated reconstruction error is, the larger the shift in the environment is, and the more the deterioration in the performance of the trained model is. The same applies to a method using a principal component vector obtained by performing principal component analysis on training data.

[0074] (D2) Use output data As described above, the influence of the environment is also reflected in the output data (inference result) of the trained model 5, so the degree of the environmental shift can be evaluated from the output data. Therefore, in one example, the environmental data 3 may include the output data (inference result) of the trained model 5 for the input data 50. Accordingly, inferring the deterioration of the performance of the trained model 5 may include inferring the decrease in the reliability of the inference that appears in the output data of the trained model 5. For example, the monitoring device 1 may acquire the output data of the trained model 5 by the above-mentioned methods (D1-1), (D1-2), etc., and calculate the degree of deterioration of the performance of the trained model from the acquired output data.

[0075] (D3) Use condition data 7 is a schematic diagram showing an example of a method (a method using condition data 55) for predicting performance degradation due to an environment shift according to the present embodiment. As shown in FIG. 7, the environment data 3 is an input to be provided to the trained model 5 during operation in order to execute inference by the trained model 5. The condition data 55 may include condition data 55 indicating conditions under which the data 50 was obtained from the environment. In accordance with this, predicting a deterioration in performance of the trained model 5 may include predicting a deterioration in accuracy of inference by the trained model 5 according to the degree of deviation of the conditions indicated by the condition data 55 from the conditions under which the training data used for the machine learning of the trained model 5 was obtained.

[0076] Specifically, the more the conditions under which the input data 50 was obtained during operation (conditions indicated by the condition data 55) deviate from the conditions under which the training data was obtained from the environment, the more the environmental shift occurs, and the performance of the trained model 5 can be evaluated as degraded. Therefore, the smaller the deviation of the conditions indicated by the condition data 55 from the conditions under which the training data was obtained from the environment, the less the deterioration of the performance of the trained model 5 may be calculated. On the other hand, the larger the deviation of the conditions indicated by the condition data 55 from the conditions under which the training data was obtained from the environment, the more the deterioration of the performance of the trained model 5 may be calculated. According to an example of this embodiment, by using the condition data 55, the deterioration of the performance of the trained model 5 caused by the shift in the environment can be appropriately determined. The condition data 55 may be referred to as meta information, external information, or the like.

[0077] Hereinafter, for convenience of explanation, the conditions when the training data used for the machine learning of the trained model 5 in operation was obtained will be referred to as "original conditions", and the conditions indicated by the condition data 55 will be referred to as "new conditions". In one example, the monitoring device 1 may first calculate the degree of deviation between the new conditions and the original conditions from the condition data 55, and estimate the deterioration of the performance of the trained model 5 according to the calculated degree of deviation. That is, the monitoring device 1 may indirectly estimate the deterioration of the performance of the trained model 5 from the condition data 55. In another example, the monitoring device 1 may directly estimate the deterioration of the performance of the trained model 5 from the condition data 55 by using an evaluation model including a calculation for evaluating the degree of deviation between the new conditions and the original conditions.

[0078] (D3-1) Cases where sensors are used In one example of this embodiment, as illustrated in Fig. 3 above, the input data 50 may include observation data 50S acquired by the sensor S. Accordingly, the condition data 55 may include observation condition data indicating the conditions of observation by the sensor S during operation. The sensor S may include a sensor provided in a virtual environment, such as a virtual camera. According to one example of this embodiment, in a situation where the sensor S is used, deterioration of the performance of the trained model 5 caused by a shift in the environment can be appropriately determined.

[0079] The parameters of the observation conditions for evaluating the performance degradation caused by the environmental shift may be appropriately selected according to the embodiment. The parameters of the observation conditions may be appropriately selected from items that may change over time. The parameters of the observation conditions may be defined, for example, by the attributes of the sensor S, the conditions of the observation target, the conditions of the observation environment, and the like. The attributes of the sensor S may include attributes related to the observation capability, such as the measurement range, the resolution (resolution, etc.) of the measurement range, and the sampling frequency. The conditions of the observation target may be defined, for example, by the attributes of the target, such as the individuality of the target (for example, a specific person, a specific work, etc.), the dimensions, the material, the quality, and the state. The conditions of the observation target may include rules that restrict the behavior of the observation target (for example, traffic rules, competition rules, smoking rules, language rules, etc.). The conditions of the observation environment may be defined, for example, by the attributes of the observation environment, such as the time period, the season (year), the weather, the location, and the attitude of the sensor S (installation angle, etc.). The time period may be defined, for example, by an expression method such as morning, noon, and night, or may be defined as an exact time interval, such as from 1:00 to 3:00. The weather may be defined by weather conditions such as sunny, cloudy, rainy, snowy, etc. Each observation condition may be obtained from the sensor S, may be obtained manually such as by user input, may be provided from an external computer (such as a server), or may be obtained as a result of observing the observation condition by a sensor other than the sensor S.

[0080] (D3-1-1) Use of imaging devices In one example of the present embodiment, as illustrated in FIG. 4A above, the sensor S may include an imaging device SA, and the input data 50 may include image data 50SA. In this case, the observation conditions indicated by the condition data 55 may include the imaging conditions of the imaging device SA. According to this example of the present embodiment, in a scene where the imaging device SA is used, it is possible to appropriately determine the deterioration of the performance of the trained model 5 caused by a shift in the environment.

[0081] In addition, the parameters of the imaging conditions for evaluating the performance deterioration caused by the environmental shift may be appropriately selected according to the embodiment. The parameters of the imaging conditions may be appropriately selected from items that may change over time. In one example, the imaging conditions may include weather conditions, lighting conditions, installation conditions of the imaging device SA, attributes of the imaging device SA, the degree of dirt on the lens, attributes related to the appearance of the subject, conditions related to aging deterioration of the imaging equipment including the imaging device SA, and the like. The installation conditions may be defined, for example, by the attitude (position, direction (angle of view), etc.). The attributes of the imaging device SA may be defined, for example, by conditions of imaging parameters such as the shutter speed, the aperture value, the sampling rate, the number of pixels, and the compression algorithm. The conditions related to aging deterioration may be defined, for example, by conditions of aging deterioration such as the presence or absence of noise. Each imaging condition may be obtained from the imaging device SA, may be obtained manually such as by user input, may be provided from an external computer, or may be obtained as a result of observing the imaging conditions by a sensor other than the imaging device SA.

[0082] (D3-1-2) Use of microphones In one example of the present embodiment, as illustrated in FIG. 4B above, the sensor S may include a microphone SB, and the input data 50 may include sound data 50SB. In this case, the observation conditions indicated by the condition data 55 may include the recording conditions of the microphone SB. According to one example of the present embodiment, in a scene where the microphone SB is used, the deterioration of the performance of the trained model 5 caused by a shift in the environment can be appropriately determined.

[0083] The parameters of the recording conditions for evaluating the performance deterioration caused by the environmental shift may be appropriately selected according to the embodiment. The parameters of the recording conditions may be appropriately selected from items that can change over time. In one example, the recording conditions may include a positional relationship with a sound source, an attribute of the sound source, an attribute of an environmental sound other than the sound source, an attribute of the microphone SB, and a condition related to deterioration over time of a recording device including the microphone SB. The positional relationship with a sound source may be specified by the distance, direction, etc. between the microphone SB and the sound source. The attribute of the sound source may be specified by, for example, the type, size, and frequency of the generated sound of the sound source. The attribute of the environmental sound may be specified by, for example, the presence or absence, type, size, and frequency of the generated sound of the environmental sound. The attribute of the microphone SB may be specified by, for example, the condition of the recording parameters such as the sampling rate, the number of bits, and directivity (for example, sound collection parameters such as the presence or absence of beamforming). The condition related to deterioration over time may be specified by, for example, the condition of deterioration over time such as the presence or absence of noise. Each recording condition may be obtained from the microphone SB, may be obtained manually, such as by user input, may be provided from an external computer, or may be obtained as a result of observing the recording conditions by a sensor other than the microphone SB.

[0084] (D3-1-3) Robot device case In one example of this embodiment, as illustrated in Fig. 4C above, the sensor S may include a sensor SC mounted on the robot device R, and the input data 50 may include observation data 50SC obtained by the sensor SC. In this case, the observation conditions indicated by the condition data 55 may include conditions related to the operation of the robot device R (hereinafter also referred to as operation conditions). According to this example of the present embodiment, in a scene where the robot device R is used, deterioration of the performance of the trained model 5 caused by a shift in the environment can be appropriately determined.

[0085] The parameters of the operating conditions for evaluating the performance deterioration caused by the environmental shift may be appropriately selected depending on the embodiment. Items that may change over time may be appropriately selected as the parameters of the operating conditions. In one example, the operating conditions may include position conditions, weather conditions, conditions of internal parameters, attributes of the floor, attributes of obstacles, attributes of the robot device R, attributes of the target of the task, conditions related to deterioration of the robot device R over time, and the like. The position conditions may be defined by, for example, position information obtained by a GPS (Global Positioning System) sensor or the like. The conditions of the internal parameters may be defined by, for example, operating conditions such as a moving speed and a driving speed. The attributes of the floor may be defined by, for example, a friction coefficient, a shape, a material, and the like. The attributes of the obstacle may be defined by, for example, the presence or absence of the obstacle, its type, dimensions, posture, whether it is dynamic or static (whether it moves or not), and the like. The attributes of the robot device R may be defined by, for example, its dimensions (footprint), the type of moving means (wheels, caterpillars, multiple legs, and the like), the driving range of the joints, the performance of the motor, the type of the end effector, and the like. The attributes of the target of the task may be appropriately selected according to the task. For example, if the robot device R is an industrial robot and the task is to transport an object by the industrial robot, the attributes of the target of the task may be defined by the weight, shape, dimensions, durability, and the like of the object. Also, for example, if the robot device R is a guide robot and the task is to guide a user, the attributes of the target of the task may be defined by the attributes of the user, such as the user's age, sex, whether or not the user has a disability, nationality, and the number of people to be guided. Each operating condition may be obtained from the robot device R, may be obtained manually by user input, may be provided from an external computer, may be obtained from the sensor SC, or may be obtained as a result of observing the operating conditions by a sensor other than the sensor SC.

[0086] The observation conditions (condition data 55) may include conditions related to the sensor S (sensor SC) in addition to the drive conditions of the robot apparatus R. If the sensor SC includes an imaging device, the observation conditions may include the imaging conditions. If the sensor SC includes a microphone, the observation conditions may include the recording conditions.

[0087] (D3-2) Cases where input data is obtained from sources other than sensors As described above, the information source of the input data 50 need not be limited to the sensor S. In one example of the present embodiment, at least a portion of the input data 50 may be obtained from an information source other than a sensor, such as, for example, a user input. In this case, the conditions of the observation indicated by the condition data 55 may include the conditions of the information source.

[0088] In one example, the trained model 5 may be a generative model that has acquired the ability to perform natural language processing, such as generating a character string of a response to a user input. In this case, the observation conditions may include conditions such as topic (current events), language, cultural area, generation, gender, purpose, writing style, rules, posting platform (e.g., media such as social network services, newspapers, and distribution services), etc. Each condition may be obtained by any means.

[0089] [E guess] In each of the above estimation methods, the deterioration of the performance of the trained model 5 may be derived from each of the above indices by any method. The deterioration of the performance of the trained model 5 may be derived directly from the environmental data 3, or may be derived indirectly via an evaluation index (e.g., the reliability, the degree of deviation of the conditions, etc.). In addition, the method of expressing the performance may be appropriately selected according to the embodiment. In one example, estimating the deterioration of the performance may be configured by at least one of estimating the performance (e.g., the inference accuracy, etc.) and estimating the difference in the performance (e.g., the decrease in the inference accuracy, etc.). That is, the deterioration of the performance may be expressed by at least one of the performance itself (absolute value) and the difference in the performance (relative value). The deterioration of the performance may be expressed by any score.

[0090] Inferring may or may not involve predicting. When predicting deterioration, in one example, future performance deterioration may be predicted directly from at least one of the environmental data 3 and the evaluation index. In another example, current performance deterioration may be predicted from at least one of the environmental data 3 and the evaluation index obtained at the present time, and future performance deterioration may be predicted from the result of predicting the current performance deterioration. In this case, the prediction of future performance deterioration may further use the past prediction result of performance deterioration. In other words, future performance deterioration may be predicted from the prediction result of performance deterioration up to the present.

[0091] Any computational model (estimation model) may be used to estimate the deterioration of performance. In one example, the estimation model may be appropriately configured to estimate the deterioration of performance of the trained model 5 from at least one of the environmental data 3 and the evaluation index. A machine learning model may be used as the estimation model. The machine learning model may be configured, for example, by a neural network, a regression model, a decision tree model, a support vector machine, or other function formulas. The machine learning model is as described above.

[0092] Each data set used for machine learning of the inference model may be appropriately collected from the past operational performance of the trained model 5. In one example, each data set may be configured by a combination of at least one sample of the environmental data and the evaluation index and the true value of the performance in the environment. The true value of the performance may be obtained from the actual value. For example, the true value of the performance may be obtained from the accuracy rate of inference for the evaluation sample (the actual value of the inference accuracy). The inference model may be appropriately generated by machine learning using the collected data sets.

[0093] In another example, an inference model generated for another trained model similar to the target trained model 5 may be diverted to infer the performance degradation of the target trained model 5. Whether or not they are similar may be evaluated depending on whether inference conditions such as the inference task performed by the trained model, the operation environment, and the input data are similar. In yet another example, when multiple other trained models exist, the monitoring device 1 may statistically derive the performance degradation of the target trained model 5 according to the tendency of each of the other trained models. As an example, the monitoring device 1 may apply the same environmental data 3 to each trained model, and calculate the average value of the estimated values ​​of performance degradation obtained for each trained model as the estimated value of performance degradation of the target trained model 5.

[0094] When estimating current performance degradation and predicting future performance degradation from the result of estimating performance degradation up to the present, an estimation model for estimating current performance degradation and a prediction model for predicting future performance degradation may be configured integrally or may be prepared separately. The prediction model may be appropriately configured to predict future performance degradation at one or more points in time from the result of estimating performance degradation up to the present. When the estimation model and the prediction model are prepared separately, the prediction model may be configured by any computation model such as a machine learning model. In one example, the prediction model may be configured by a regression model. By using the regression model, the prediction model can be generated easily.

[0095] The prediction model may be generated as appropriate, similar to the above-mentioned estimation model. In one example, the prediction model may be generated by performing regression analysis on performance results obtained from the past operation results of the trained model 5. In another example, a prediction model generated for a similar other trained model may be diverted to predict the deterioration of the performance of the target trained model 5. In yet another example, when multiple other trained models exist, the predicted value of the performance deterioration of the target trained model 5 may be statistically derived according to the tendency of each of the other trained models (e.g., average value).

[0096] [F Information Output] The content and format of the output information are not particularly limited and may be appropriately determined depending on the embodiment. The format of the information may be appropriately selected depending on the content of the information and the output form. For example, the information may be composed of text, images, audio, or a combination of these. The text may include a numerical expression and a sentence expression. The image may include a graph expression.

[0097] (F1) Content of information (F1-1) Output the guess result In one example, the output information may include a result of estimating the deterioration of the performance of the trained model 5. That is, the monitoring device 1 may directly output the result of estimating the deterioration of the performance of the trained model 5. The estimated result may include a predicted result.

[0098] In one example, estimating the degradation of the performance of the trained model 5 may be performed continuously. The output information may include a result of predicting when the performance of the trained model 5 will fall below a threshold. The threshold may be given as appropriate. According to one example of the present embodiment, a result of predicting when the performance of the trained model 5 will fall below a threshold is provided. This makes it possible to prompt maintenance of the trained model 5 at an appropriate time.

[0099] (F1-2) Output the degree of shift Inferring the deterioration of the performance of the trained model 5 may include evaluating the degree of the shift of the environment. In this case, the output information may include information indicating the degree of the evaluated shift of the environment. In one example, the monitoring device 1 may determine a distance between a representation of the original environment (the environment assumed by the trained model 5) and a representation of the current environment (the environment identified from the environmental data 3) according to the degree of the shift. Specifically, the monitoring device 1 may determine the distance between the original environment and the current environment such that the distance between the representation of the original environment and the representation of the current environment is large and the smaller the degree of the shift, the smaller the distance is. Then, the monitoring device 1 may output information indicating the degree of the shift by illustrating each representation with the determined distance. According to one example of the present embodiment, the degree of the environmental shift that affects the deterioration of the performance of the trained model 5 can be visualized.

[0100] (F1-3) Output the factors of environmental shift In one example, the output information may include at least one of information regarding the cause of the environmental shift and information related to the cause.

[0101] The cause of the environmental shift may be identified as appropriate. In one example, the cause of the environmental shift may be identified from environmental data 3 such as weather conditions. For example, the cause of the environmental shift may be identified from a condition item that has changed significantly among multiple condition items indicated by the environmental data 3. In another example, the cause of the environmental shift may be identified from indirect information. For example, in a production facility, the environment may fluctuate depending on the state of the facility (e.g., temperature, humidity, torque, etc.). The cause of the environmental shift may be identified from the measurement value of a sensor that observes the state of the facility.

[0102] In addition, information related to the cause of the environmental shift (hereinafter also referred to as related information) may be appropriately acquired. In one example, the related information may include any information that may be related to the cause of the environmental shift, such as a uniform resource locator (URL) and event information that is likely to be related to the cause of the environmental shift. As a specific example, if a weather change is identified as the cause of the environmental shift, the related information may include a URL of meteorological information (e.g., a weather forecast website). As another specific example, if the trained model 5 is used to monitor traffic conditions (e.g., detect vehicles) and the occurrence of traffic congestion (changes in traffic conditions) is identified as the cause of the environmental shift, the related information may include a URL of traffic information, event information in the surrounding area that may be related to the traffic congestion, and the like. The related information may be obtained by searching for information using a known search engine.

[0103] In one example, the output information includes information regarding the severity of maintenance for factors of an environmental shift. The information may include the degree of stringency. The degree of stringency may be evaluated appropriately. For example, a factor that repeatedly causes an environmental shift may be evaluated to have a high degree of stringency. This can prompt maintenance (e.g., annotation and additional learning) of the trained model 5 for factors with high stringency.

[0104] (F1-4) Output information about annotations In one example, the information that is output may include information about the annotation, which may include any information that may be related to the annotation.

[0105] (F1-4-1) Budget Output In a simple example, the information on the annotation may include an annotation budget (balance). The monitoring device 1 may output at least one of an estimated value and a predicted value of the performance of the trained model 5 as a predicted result of the performance deterioration of the trained model 5, and may output the annotation budget. In this way, the monitoring device 1 may allow an operator to decide whether or not to perform annotation. The annotation budget may be expressed in the form of, for example, an amount of money, the number of items that can be annotated, etc.

[0106] (F1-4-2) Gain output In one example, the output information may include, as information about annotations, a result of estimating the degree to which the performance of the trained model 5 is improved by additionally learning a predetermined number of samples (hereinafter, also referred to as gain). According to one example of the present embodiment, it is possible to show a guide to the performance improvement by additional learning. This makes it possible to encourage the implementation of appropriate additional learning.

[0107] In addition, in estimating (predicting) the gain, the additional learning may be assumed to be performed by a known method such as supervised learning, semi-supervised learning, unsupervised learning, etc. In one example, additional learning of a predetermined number of samples may include manually annotating all of the predetermined number of samples, manually annotating only a portion of the predetermined number of samples, or subjecting all of the predetermined number of samples to unsupervised learning. The predetermined number may be specified by any method, such as by a user, a program-specified value (variable value, fixed value), etc.

[0108] The gain may be estimated by an appropriate method. In estimating the gain, an arbitrary computation model (gain estimation model) may be used, as in the estimation of the deterioration of the performance. The gain estimation model may be a machine learning model. The machine learning model may be configured, for example, by a neural network, a regression model, a decision tree model, a support vector machine, or other function formulas. The machine learning model is as described above. In one example, the gain estimation model may be configured to estimate a predicted value of the performance of the trained model 5 after additional learning from the performance and the number of annotations of the trained model 5 up to the present. Each data set used for machine learning of such a gain estimation model may be appropriately collected from the past additional learning results of the trained model 5. In one example, each data set may be configured by a combination of the performance results and the number of annotations of the trained model 5 at one or more points before the additional learning and the performance results of the trained model 5 after the additional learning. The gain estimation model may be appropriately generated by machine learning using the collected data set. In the same manner as the prediction model, a regression model may be used for the gain estimation model.

[0109] In another example, a gain estimation model generated for other trained models similar to the subject trained model 5 may be repurposed to estimate the gain of the subject trained model 5. In another example, when multiple other trained models exist, the gain of the subject trained model 5 may be statistically derived (e.g., average value) according to the trends of each other trained model.

[0110] In another example, the gain prediction model may be generated separately during machine learning of the trained model 5. For example, a subset (e.g., 10, 50, 100, etc.) of the training data may be constructed during machine learning of the model (generation of the trained model 5), and a learning curve for each subset may be obtained by training the model with each subset. Then, a model (gain prediction model) that predicts the attained performance (accuracy) according to the number of samples can be generated by fitting each learning curve to a function. According to this gain prediction model (fitted function), the gain can be predicted by calculating the difference between the final performance predicted by the gain prediction model and the current performance. In another example, additional learning may actually be performed on a number of samples (i.e., a part of the samples specified by the predetermined number) that is smaller than the predetermined number. The gain of the trained model 5 may be estimated from the result of this additional learning (gain in additional learning of a small number of samples).

[0111] (F1-4-3) Sample output In one example, the output information may include, as information regarding the annotation, a sample of the input data 50 for which an annotation is recommended (hereinafter, also referred to as a recommended sample). The recommended sample may be, for example, a sample of the input data 50 that is difficult to infer with the currently trained model 5 due to a shift in the environment. As an example, the recommended sample (a sample that is difficult to infer) may be a sample that is unknown in the training data (for example, a sample that may belong to a category not set in the training data).

[0112] The recommended sample may be selected from the samples of the input data 50 (i.e., candidate samples for annotation) by any method. The method of selecting the recommended sample includes, for example, the method described in Reference 4 (Kaidi Cao, Maria Brbic, Jure Leskovec, “Open-World Semi-Supervised Learning”, [online], [searched June 23, 2023], Internet<URL:https: / / arxiv.org / abs / 2102.03526> ), Reference 5 (Sagar Vaze, Kai Han, Andrea Vedaldi, Andrew Zisserman, “Generalized Category Discovery”, [online], [June 2, 2020) 3 days search], Internet<URL:https: / / arxiv.org / abs / 2201.02609> ) etc. The Openset Detection technique, which is widely used in this case, may be adopted.

[0113] In another example, an active learning method may be adopted for selecting (sampling) the recommended sample. The recommended sample may be, for example, the one described in Reference 6 (Ozan Sener, Silvio Savarese, “Active Learning for Convolutional Neural Networks: A Core-Set Approach”, [online], [Retrieved June 23, 2023], Internet<URL:https: / / arxiv.org / abs / 1708.00489> ), Reference 7 (Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, Alekh Agarwal, “Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds”, [online], [Retrieved June 23, 2023], In Internet<URL:https: / / arxiv.org / abs / 1906.03671> ), Reference 8 (Yarin Gal, Riashat Islam, Zoubin Ghahramani, “Deep Bayesian Active Learning with Image Data”, [online], [Retrieved June 23, 2023], Internet<URL:https: / / arxiv.org / abs / 1703.02910> ) may be arbitrarily selected from samples extracted by an active learning technique such as the above.

[0114] In one example, the information on the annotation may include candidate samples of the annotation other than the recommended sample. In addition, in one example, when a sample (recommended sample / candidate sample) is output, the information on the annotation may further include at least one of explanatory information and related information of the sample. The explanatory information and related information may be obtained by any method. In one example, the explanatory information may be automatically generated using a known generative model. For example, the explanatory information may be composed of sentences explaining the contents of the sample, which are generated by a sentence generation model using natural language processing. The related information is based on features extracted from the sample and the explanatory information. At least one of the included words may be used as a query to perform an information search using a known search engine.

[0115] (F1-4-4) Optimization In addition, in one example, estimating the deterioration of the performance of the trained model 5 may be continuously executed. The output information may include at least one of the timing of the additional learning optimized within a predetermined budget and the number of annotations for the additional learning sample as information regarding the annotation. The predetermined budget may be specified by any method such as specification by a user. According to one example of the present embodiment, it is possible to encourage the performance of the additional learning optimized within a predetermined budget. Note that any computation model (optimization model) may be used for optimization. The optimization model may be generated as appropriate. In one example, the optimization model may be obtained by a method such as reinforcement learning or manual design. When unsupervised learning is selected as the method of the additional learning, outputting the number of optimized annotations may include outputting 0. A specific example of a method of obtaining an optimization model will be described below.

[0116] (I) Problem definition As an example of optimization, the following two optimization goals may be set. One goal is to obtain the minimum budget that achieves the condition that the accuracy of the trained model 5 does not fall below a lower limit. The other goal is to achieve the maximum average accuracy of the trained model 5 with the set budget. Hereinafter, the first goal is also referred to as budget minimization, and the second goal is also referred to as accuracy maximization.

[0117] To formulate the optimization problem, the following parameters are defined. For simplicity, the cost per annotation is defined as 1. The cost (c) other than annotation per additional learning is, for example, the server cost for additional learning, the drive cost for storing the training data and the model, etc. c: Costs other than annotation per additional learning session B: Total budget T: The period covered by the budget τ: total number of learning times (integer) D tρ : Average deterioration of model performance per unit time during the period g(n): A function that predicts the increase in accuracy of the model for n annotations (budget n).

[0118] c, B, and T are constants given by external constraints and may be appropriately specified by an operator's input, etc. τ is a variable. D tρ The initial value of may be a predicted value given appropriately. During the period, the actual value of the deterioration of the performance of the trained model 5 may be calculated, and D tρ may be updated based on the obtained performance value. As an example, g(n) may use a function such as the following formula 1 or formula 2. In another example, g(n) may be obtained by prior learning. D tρ Similarly, g(n) may be updated based on actual values ​​during the period.

[0119]

number

number

[0120] Under the above definition, the number of annotations N possible throughout the entire period can be expressed as the following formula 3. Furthermore, the total number of learning iterations τ can be expressed as the following formula 4 from formula 3.

[0121]

number

number

[0122] n ais the average number of annotations during additional training. If additional training is performed on average, the period from one additional training to the next is (T / τ). The degree of deterioration of the model's performance during this period (D tρ ×(T / τ)) can be expressed as the following Equation 5.

[0123]

number

[0124] Budget minimization is equivalent to staying as close as possible to this equilibrium state. Therefore, budget minimization is the same as if g(n)>g d (n)" under the condition (|g(n)-g d (n)|) is minimized. Since n is finite, this budget minimization problem is solvable. In one example, the budget minimization problem is d It may be solved by brute-force searching for n that minimizes |(n)|.

[0125] In addition, accuracy maximization can be achieved simply by selecting n such that g(n) / n is maximized. When considering the increase or decrease in model performance during additional training, the accuracy maximization problem becomes (|g(n)-g d This can be solved by exhaustively searching for n that maximizes |(n)|.

[0126] In addition, "g(n)>g dIf there is no n that satisfies "n (n)", the trained model 5 will deteriorate throughout the period under the total budget B. In other words, the total budget B is small to achieve the above goal. Therefore, an increase in the total budget B may be encouraged. Hereinafter, n obtained by solving the budget minimization or accuracy maximization by offline optimization will be referred to as "n s " should be written.

[0127] (II) Extension to online optimization In reality, the true accuracy improvement margin g gt (n) is the unknown number of annotations n that is optimal to achieve budget minimization or accuracy maximization. * and s There is a gap between To minimize this deviation, n s and the actual number of annotations n from the observed values r Consider generating an agent (i.e., the above optimization model) that determines the following. Below, we will show two specific examples of how to obtain an agent.

[0128] (II-1) Acquiring an Agent through Reinforcement Learning In one example, the agent may be obtained by reinforcement learning. Two regrets (penalties) can be defined for the agent to minimize: Regret due to extra annotations r · Regret for lack of accuracy due to lack of annotations s

[0129] The observed values ​​are, for example, τ, D tρ , g(n), etc. When using Equation 1 or 2 above for g(n), the observation value may include λ.

[0130] As an example of how to obtain an agent, we consider the two types of regret (R r , R s) can be implemented. In this case, the behavior of the agent (optimization model) in the reinforcement learning is rt The goal is to determine the accuracy of the model, ρ t The agent's behavior may further include selecting whether or not to perform additional learning at each time (e.g., may be expressed by a binary variable). This allows the optimal timing for additional learning to be learned through reinforcement learning.

[0131] The data to be subjected to reinforcement learning may be a data set in which all samples are given annotations and information on the time of observation. By proceeding with reinforcement learning by virtually determining annotation targets and simulating the implementation of additional learning, an agent that has acquired the ability to adopt a strategy to minimize regret can be obtained. The obtained agent may be used as the optimization model described above. Note that each regret (R r , R s ) may be calculated using the functions of the following Equations 6 to 9.

[0132]

number

number

number

number

[0133] Each regret (R r , R s ) when calculating n * and g gt(n) may be obtained in any manner. For example, training may be performed for each number of annotations by simulation, and the accuracy of the resulting trained model may be evaluated. * and g gt (n) may be derived from the results of this evaluation. gt The values ​​of hyperparameters (n) and n * A provisional value derived from past performance may be used for .

[0134] (II-2) Manually Designed Agents In another example, an agent may be manually designed by using an online optimization algorithm that provides accuracy guarantees on the ratio of the objective function value achieved assuming the availability of ex-post observed information to the objective function value achieved by the algorithm without that information (competitive ratio). Typically, performance is measured by the worst-case competitive ratio.

[0135] The agent may employ any online optimization algorithm. In one example, the agent may employ an algorithm for the ski rental problem. In this case, the problem the agent solves may be budget minimization. The number of ski trips is determined by the accuracy improvement margin g gt (n). The rental cost of one ski may be replaced by one annotation (m rentals cost m). The cost of purchasing skis may be replaced by n obtained by offline optimization. s may be replaced by g gt Assuming that (n) is a monotonically increasing function, then, following the algorithm for the ski rental problem, m=2n s By determining the number of annotations m per additional learning, the worst-case competitive ratio can be suppressed to 2. In other words, the agent can reduce the number of annotations m by the offline optimization described above in (I). Rin s Calculate the n sThe number of annotations output may be twice as many as the optimized number of annotations.

[0136] (III) g(n) and D tρ Update During operation, each time additional learning is performed on the trained model 5, partial observations of g(n) can be obtained. For example, if m annotations are applied in a certain additional learning, the actual value of g(n) can be obtained for the range of 0 to m. That is, by performing additional learning for each case of 0 to m annotations and measuring the increase in performance of the trained model 5, the actual value of g(n) for the range of 0 to m can be obtained. The values ​​of the hyperparameters of g(n) may be updated using these actual values. In addition, by calculating the accuracy of the trained model 5 before additional learning for the m annotated samples, D tρ The actual value of D can also be obtained. tρ The values ​​of the parameters of g(n) and D may be updated using the actual values. tρ It is possible to reduce the error in the above equation, and as a result, it is possible to obtain an agent (optimization model) that can be optimized with high accuracy.

[0137] (F1-4-5)Other In one example, the information regarding the annotation may include an instruction to perform the annotation and additional learning. At least a part of the annotation and additional learning may be automatically performed by computer processing in response to the instruction. The computer that performs the annotation and additional learning may be selected arbitrarily. In one example, the annotation and additional learning may be performed in the monitoring device 1. In another example, at least one of the annotation and additional learning may be performed in a computer other than the monitoring device 1.

[0138] (F2) Monitoring multiple trained models When the monitoring device 1 monitors a plurality of trained models 5, information on the results of estimating performance degradation may be output for each trained model 5, or information on each of two or more trained models 5 may be output collectively. In one example, information on each trained model 5 may be output in a list.

[0139] In another example, the output information may include a map showing the operation locations of each of the multiple trained models 5. In this case, outputting the information includes superimposing information on the results of estimating the deterioration of the performance of each trained model 5 on the operation location of each trained model 5 on the map. In one example of the present embodiment, an evaluation of the deterioration of the performance of each trained model 5 can be visualized together with the operation location of each trained model 5. On the map, the performance of each trained model 5 may be displayed collectively using a gradation (heat map), a contour line, or the like.

[0140] (F3) Output destination / output timing The output destination and output timing of the information may be appropriately selected according to the embodiment. In one example, the output destination of the information may be at least one of the memory resource of the monitoring device 1, an output device connected to the monitoring device 1, and an external computer. The external computer may include a user terminal. The output timing may be push type or pull type. In one example, the monitoring device 1 may execute the above-mentioned inference process at any timing and output information related to the obtained inference result. In another example, the monitoring device 1 may execute the above-mentioned inference process in response to any action such as a user's inquiry via the user terminal, and output information related to the obtained inference result to the source of the action.

[0141] [G System Configuration] In one example, as illustrated in FIG. 1, the monitoring device 1 may be configured by any computer separate from the inference device 2. As a specific example, the monitoring device 1 may be configured by a server device such as an edge server or a server on a cloud. The inference device 2 may be any server device, a personal computer (PC), a terminal device, or the like. In another example, The monitoring device 1 may be configured integrally with the inference device 2. That is, one or more integrated computers may operate as both the monitoring device 1 and the inference device 2.

[0142] Assume that the monitoring device 1 is configured by a computer separate from the inference device 2, and infers the deterioration of the performance of the trained model 5 from the inference result of the trained model 5 for the input data 50 (for example, the above cases (D1-1) and (D1-2)). In this case, in one example, the monitoring device 1 may also hold a trained model 5 that is the same as the trained model 5 operated in the inference device 2. The monitoring device 1 may obtain the inference result of the trained model 5 by providing the input data 50 to the trained model 5 held by the monitoring device 1. In another example, the monitoring device 1 may not hold a trained model 5, and may obtain the inference result of the trained model 5 by having the inference device 2 execute an inference process.

[0143] In addition, in one example, the monitoring device 1 and the inference device 2 may be connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like. However, the method of data exchange between the monitoring device 1 and the inference device 2 is not limited to this example, and may be appropriately selected depending on the embodiment. In another example, data exchange may be performed between the monitoring device 1 and the inference device 2 via a storage medium.

[0144] §2 Configuration Example [Hardware configuration] Fig. 8 illustrates a schematic example of a hardware configuration of the monitoring device 1 according to this embodiment. In the example of Fig. 8, the monitoring device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an input device 14, an output device 15, and a drive 16 are electrically connected.

[0145] The control unit 11 includes a CPU (Central Processing Unit) which is a hardware processor, The control unit 11 (CPU) includes a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and is configured to execute information processing based on programs and various data. , are examples of processor resources. The storage unit 12 is configured with, for example, a hard disk drive, a solid state drive, etc. The storage unit 12, the RAM, and the ROM are examples of memory resources. In this embodiment, the storage unit 12 stores various information such as a program 81.

[0146] The program 81 is a program for causing the monitoring device 1 to execute information processing (FIG. 10 described later) related to estimating performance degradation of the trained model 5. The program 81 includes a series of instructions for the information processing. When the trained model 5 is stored, the storage unit 12 may further store learning result data configured to indicate information related to the trained model 5.

[0147] The communication interface 13 is configured to perform wired or wireless communication via a network. The communication interface 13 may be configured, for example, by a wired LAN (Local Area Network) module, a wireless LAN module, etc. The monitoring device 1 may perform data communication with another computer (for example, the inference device 2, etc.) via the communication interface 13.

[0148] The input device 14 is a device for performing input, such as a mouse, a keyboard, etc. The output device 15 is a device for performing output, such as a display, a speaker, etc. An operator can operate the monitoring device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be connected via an external interface. The input device 14 and the output device 15 may be integrally configured, for example, by a touch panel display or the like.

[0149] The drive 16 is a device for reading various information such as a program stored in a storage medium 91. The program 81 may be stored in the storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is configured to accumulate various information (stored programs, etc.) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The monitoring device 1 may acquire the program 81 from the storage medium 91. The same applies to the learning result data. The storage medium 91 may be a disk-type storage medium such as a CD or DVD, or may be a non-disk type storage medium such as a semiconductor memory (e.g., a flash memory). The type of the drive 16 may be appropriately selected according to the type of the storage medium 91. The drive 16 may be connected via an external interface. The external interface may be appropriately configured to connect to an external device by wire or wirelessly, for example, via a Universal Serial Bus (USB) port, a dedicated port, or the like.

[0150] In addition, with regard to the specific hardware configuration of the monitoring device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), The storage unit 12 may be configured with a RAM and a ROM included in the control unit 11. At least one of the communication interface 13, the input device 14, the output device 15, and the drive 16 may be omitted. The monitoring device 1 may be configured with a plurality of computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the monitoring device 1 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0151] The inference device 2 may be configured in the same manner as the monitoring device 1. The inference device 2 may be an information processing device designed specifically for the service provided, a general-purpose server device, a general-purpose PC, a tablet, etc. It may be a laptop PC, a terminal device, etc. When the monitoring device 1 and the inference device 2 are integrally configured and a sensor S is used to acquire the input data 50, the sensor S may be connected to the monitoring device 1 via at least one of the external interface and the communication interface 13.

[0152] [Software configuration] 9 illustrates a schematic example of a software configuration of the monitoring device 1 according to this embodiment. The control unit 11 of the monitoring device 1 loads the program 81 stored in the storage unit 12 into the RAM, and executes instructions included in the program 81 using the CPU. This causes the monitoring device 1 to operate as a computer including an acquisition unit 111, a deterioration estimation unit 112, an information output unit 113, and an additional learning unit 114 as software modules. That is, in this embodiment, each software module of the monitoring device 1 is realized by the control unit 11 (CPU).

[0153] The acquisition unit 111 is configured to acquire environmental data 3 related to an environment in which the trained model 5 generated by machine learning is operated. The deterioration estimation unit 112 is configured to estimate, from the acquired environmental data 3, deterioration of performance of the trained model 5 due to a shift in the environment. In one example, the acquisition of the environmental data 3 by the acquisition unit 111 and the estimation of deterioration of performance of the trained model 5 by the deterioration estimation unit 112 may be continuously and repeatedly executed. The information output unit 113 is configured to output information related to a result of estimating deterioration of performance of the trained model 5.

[0154] When the additional learning is selected as a result of estimating the deterioration of the performance of the trained model 5, the additional learning unit 114 is configured to perform the additional learning of the trained model 5. Specifically, the additional learning unit 114 is configured to collect a plurality of training samples 60 from the input data 50 provided to the trained model 5 being operated. The additional learning unit 114 is configured to perform the additional learning of the trained model 5 using the collected plurality of training samples 60. In one example, the additional learning unit 114 may be configured to accept annotations for at least a portion of the collected plurality of training samples 60. The additional learning unit 114 may be configured to perform the additional learning of the trained model 5 using the plurality of training samples 60, at least a portion of which is annotated. The additional learning unit 114 is configured to generate learning result data 500 configured to indicate information about the updated trained model 5 as a result of the additional learning, and to store the generated learning result data 500 in a predetermined storage area. The learning result data 500 may be provided to the inference device 2 at any timing.

[0155] In this embodiment, an example is described in which each software module of the monitoring device 1 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors or chipsets. Each of the above modules may be realized as a hardware module. With regard to the software configuration of the monitoring device 1, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0156] §3 Example of operation Fig. 10 is a flowchart showing an example of a processing procedure of the monitoring device 1 according to this embodiment. The following processing procedure is an example of a monitoring method executed by a computer. However, the following processing procedure of the monitoring device 1 is merely an example, and each step may be changed as much as possible. In addition, steps may be omitted, replaced, or added to the following processing procedure as appropriate depending on the embodiment.

[0157] (Step S101) In step S101, the control unit 11 operates as the acquisition unit 111 and acquires environmental data 3 related to the environment in which the trained model 5 generated by machine learning is operated.

[0158] In one example of this embodiment, the control unit 11 may acquire at least one of the input data 50, the output data, and the condition data 55 as the environmental data 3. The control unit 11 may acquire the environmental data 3 directly from an information source, or indirectly via another computer, a storage medium, or the like. The information source may be, for example, the inference device 2, the sensor S, another sensor, human hands, an external computer, or the like. Upon acquiring the environmental data 3, the control unit 11 proceeds to the next step S102.

[0159] (Step S102) In step S102, the control unit 11 operates as the deterioration estimation unit 112 and estimates, from the acquired environmental data 3, the deterioration of the performance of the trained model 5 due to the shift in the environment.

[0160] In one example of this embodiment, the control unit 11 may estimate the performance degradation of the trained model 5 by at least one of the methods described in [D. An example of an estimation method] above. That is, when the environmental data 3 includes the input data 50, the control unit 11 may estimate the performance degradation of the trained model 5 according to a decrease in the reliability of the inference of the trained model 5 for the input data 50. When the inference by the trained model 5 includes class identification, the control unit 11 may estimate the performance degradation of the trained model 5 according to an increase in entropy calculated from the inference result. When the trained model 5 is configured to output a reliability score together with the inference result, the control unit 11 may estimate the performance degradation of the trained model 5 according to a decrease in the reliability score output from the trained model 5. In addition, the control unit 11 may estimate the performance degradation of the trained model 5 according to the deviation of the input data 50 from the distribution range of the training data. When the environmental data 3 includes output data of the trained model 5 for the input data 50, the control unit 11 may infer a deterioration in the performance of the trained model 5 in accordance with a decrease in the reliability of inference reflected in the output data.

[0161] When the environmental data 3 includes the condition data 55, the control unit 11 may infer a deterioration in performance of the trained model 5 in response to a deviation of the new conditions from the original conditions. When the input data 50 includes the observation data 50S of the sensor S, the control unit 11 may infer a deterioration in performance of the trained model 5 in response to a deviation of the new observation conditions from the original observation conditions of the sensor S. The observation conditions may include at least any of the imaging conditions of the imaging device SA, the recording conditions of the microphone SB, and the driving conditions of the robot device R.

[0162] In addition, in one example of this embodiment, the control unit 11 may derive a prediction result of the performance deterioration of the trained model 5 from each of the above-mentioned indicators by the method described in [E Prediction] above. Predicting may include predicting. In one example, the control unit 11 may use a prediction model to predict the current or future performance of the trained model 5 from the current environmental data 3. In another example, the control unit 11 may use a prediction model to predict the current performance of the trained model 5 from the current environmental data 3. Next, the control unit 11 may use a prediction model to predict the future performance of the trained model 5 from the prediction result of the performance up to the present. When the deterioration of the performance of the trained model 5 is predicted, the control unit 11 proceeds to the next step S103.

[0163] (Step S103) In step 103, the control unit 11 operates as the information output unit 113 and outputs information related to the result of estimating the deterioration of the performance of the trained model 5.

[0164] In one example of this embodiment, the control unit 11 performs the method according to the above [F. Output of Information]. The control unit 11 may output information on the result of estimating the deterioration of the performance of the trained model 5 by at least one of the methods (hereinafter, also referred to as "information on the estimation result"). That is, the control unit 11 may output the result of estimating the deterioration of the performance of the trained model 5 as it is. By step S109 described later, the control unit 11 may continuously repeatedly execute the processes of steps S101 and S102. In response to this, the control unit 11 may output the result of predicting the time when the performance of the trained model 5 will fall below the threshold. When the process of step S102 includes evaluating the degree of the environmental shift, the control unit 11 may output information indicating the evaluated degree of the environmental shift. The degree of the environmental shift may be appropriately evaluated based on the change in the reliability of the inference, the change in the conditions, and the like. The control unit 11 may output at least one of information on the cause of the environmental shift and related information. The control unit 11 may output information on the urgency of maintenance for the cause of the environmental shift. The control unit 11 may output information on the annotation. The information on the annotation may include at least one of the following: the annotation budget, the estimated gain, samples (recommended samples, candidate samples), sample information (at least one of explanatory information and related information), the estimated results of additional learning optimized within a given budget (at least one of the timing of additional learning and the number of annotations), and instructions. When monitoring multiple trained models 5, the control unit 11 may output a map showing the operation locations of each of the multiple trained models 5. In addition, the control unit 11 may output information on the estimated results of the deterioration of the performance of each trained model 5 so as to be superimposed and displayed on the operation location of each trained model 5 on the map.

[0165] (1) First output example FIG. 11 is a schematic diagram showing an example of output information in the case where a form is adopted in which a result of predicting when the performance of the trained model 5 falls below a threshold is output in this embodiment. In the screen I1 of FIG. 11, the current prediction result of the performance of the trained model 5 and the future prediction result are output in a line graph. The threshold is displayed as a tolerance line, and the time when the performance of the trained model 5 falls below the threshold is expressed by the intersection of the line graph and the tolerance line. The time when the performance of the trained model 5 falls below the threshold is shown in a balloon extending from the intersection by the date ("**** / ** / **") when the performance falls below the threshold. Furthermore, in the balloon, information on the cause of the environmental shift ("Probability of snowfall: 90%) and information indicating the source of the information ("Source of information: XXXX") are output. The balloon may be configured so that whether or not to display it is switched by a user operation. In one example, the control unit 11 may generate information having the components exemplified in the screen I1 of FIG. 11 as information on the prediction result, and output the generated information.

[0166] The components of screen I1 may be modified as appropriate. The method of expressing the performance estimation result is not limited to the form of a line graph. The performance estimation result may be expressed in any form, such as a line graph, other graph form, numerical form, table form, etc. The time when the performance of the trained model 5 falls below the threshold may be expressed in a form other than a date, for example, a period during which performance above the threshold can be maintained. The time may be expressed as a date and time. The information displayed in the speech bubble may be modified as appropriate.

[0167] (2) Second output example Fig. 12 shows an example of output information when a form for outputting a gain estimation result is adopted in this embodiment. In screen I2 in Fig. 12, the gain (dotted line) obtained by additionally learning 10, 50, and 100 samples is output in a line graph together with the current estimation result (solid line) of the performance of the trained model 5. As in screen I1, a threshold is displayed as a tolerance line. In one example, the control unit 11 may generate information having components exemplified in screen I2 in Fig. 12 as information related to the estimation result, and output the generated information.

[0168] The components of screen I2 may be modified as appropriate. The method of expressing the performance prediction results and gains need not be limited to the form of a line graph. The performance prediction results and gains may be expressed in any form, such as a line graph, other graph form, numerical form, or table form. Although three cases of gains (10 cases, 50 cases, and 100 cases) are displayed in screen I2, the number of gains displayed is not limited to three, and may be two or less, or four or more. The predetermined number may be determined as appropriate other than 10 cases, 50 cases, and 100 cases. The predetermined number may be specified by the user.

[0169] (3) Third output example FIG. 13A is a schematic diagram showing an example of output information in the case where information on the estimation results of each of a plurality of trained models 5 operated in a factory is displayed on a map MA in this embodiment. In a screen I3A in FIG. 13A, a map MA showing the operation locations of each of a plurality of trained models 5 (models A to G) is output. In this map MA, the estimation or prediction results of the performance of each trained model 5 are collectively displayed by gradation (heat map) as information on the estimation results. The gradation may be appropriately generated from the values ​​of the estimation or prediction results of each trained model 5. The button A101 is used to switch the location of the factory. The button A102 is used to display the estimation or prediction results of the performance of the trained model 5. The slider SLA is used to specify the target time point for estimation or prediction. In the screen I3A in FIG. 13A, the button A102 is operated and the present is specified by the slider SLA, so that the estimation results of the current performance of each trained model 5 are displayed on the map MA. The input field A104 is used to specify the number of samples (predetermined number) in the additional learning to be used for gain prediction. The button A103 is used to display on the map MA the gain resulting from additional learning of the number of samples specified in the input field A104.

[0170] FIG. 13B is a schematic diagram showing an example of output information in the case where information on the estimation results of each of a plurality of trained models 5 operated in various parts of Japan is displayed on a map MB in this embodiment. In the example of FIG. 13B, a case is assumed where a plurality of trained models 5 are distributed and operated in various parts of Japan for the purpose of inferring traffic conditions, for example. In the screen I3B of FIG. 13B, a map MB (map of Japan) showing the operation locations of each of the plurality of trained models 5 is output. On this map MB, the estimation or prediction results of the performance of each trained model 5 are displayed collectively by contour lines as information on the estimation results. The contour lines may be appropriately generated from the values ​​of the estimation or prediction results of each trained model 5. The button B102 is used to display the estimation or prediction results of the performance of the trained model 5. The slider SLB is used to specify the target time point for the estimation or prediction. In the screen I3B of FIG. 13B, the button B102 is operated and the present is specified by the slider SLB, so that the estimation results of the current performance of each trained model 5 are displayed on the map MB. The input field B104 is used to specify the number of samples (predetermined number) in additional learning to be used for gain prediction. The button B103 is used to display, on the map MB, the gain resulting from additional learning of the number of samples specified in the input field B104.

[0171] In one example, the control unit 11 may generate information having components exemplified in screen I3A of Fig. 13A or screen I3B of Fig. 13B as information on the estimation result, and output the generated information. Note that the components of screen I3A and screen I3B may be changed as appropriate. The performance estimation or prediction result may be output for each model in any format such as a graph format, a numerical format, or a table format.

[0172] (4) Example of the fourth output 14 is a schematic diagram showing an example of output information when a form for outputting information indicating the degree of an environmental shift is adopted in this embodiment. In the screen I4 of FIG. The heart indicates the original environment (the environment assumed by the trained model 5). The point I41 indicates the current environment (the environment identified from the environmental data 3). That is, the distance between the target I40 and the point I41 represents the degree of the environmental shift. In one example, the control unit 11 may generate information having the components exemplified in the screen I4 of FIG. 14 as information on the estimation result, and output the generated information. The representation of the target I40 and the point I41 may be changed as appropriate.

[0173] (5) Fifth output example FIG. 15 is a diagram for explaining optimization of annotations in this embodiment. In one example of this embodiment, the control unit 11 may output at least one of the timing of additional learning optimized within a predetermined budget (recommended timing) and the number of annotations for the sample of additional learning (recommended number of annotations). In one example, the control unit 11 may determine at least one of the recommended timing and the number of annotations recommended within a predetermined budget using an agent (optimization model) acquired by any of the above-mentioned reinforcement learning and manual design methods. The agent may be acquired by executing at least a part of the processing in the monitoring device 1, or may be generated by another computer. In the case of being generated by another computer, the monitoring device 1 may appropriately acquire the agent from the other computer via a network, a storage medium 91, or the like. In addition, the cost (c) other than annotation per additional learning, the total budget (B), and the target period of the budget (T) may be appropriately given. The goal of optimization may be the above-mentioned budget minimization or accuracy maximization. FIG. 15 illustrates a case of budget minimization.

[0174] The destination and timing of output of the information may be appropriately selected depending on the embodiment. In one example, the control unit 11 may output the information on the inference result to at least one of the RAM, the storage unit 12, the storage medium 91, the output device 15, and an external computer. The output timing may be either push type or pull type. After outputting the information on the inference result, the control unit 11 proceeds to the next step S104.

[0175] (Step S104) In step S104, the control unit 11 selects whether or not to perform additional learning. In one example, whether or not to perform additional learning may be manually selected by an operator (user) who refers to the information output in step S103. In another example, the control unit 11 may automatically select whether or not to perform additional learning depending on whether or not the estimation result in step S102 satisfies a predetermined condition (for example, it is predicted that the performance of the trained model 5 will fall below a threshold after a predetermined number of days). If it is selected to perform additional learning, the control unit 11 proceeds to the next step S105. On the other hand, if it is selected not to perform additional learning, the control unit 11 proceeds to the next step S109.

[0176] (Steps S105 to S108) In steps S105 to S108, the control unit 11 operates as the additional learning unit 114 and performs additional learning of the trained model 5. The additional learning may include re-learning and active learning.

[0177] In step S105, the control unit 11 collects a plurality of training samples 60 from the input data 50 provided to the trained model 5 being operated. The plurality of training samples 60 to be subjected to additional learning may be appropriately selected. For example, the control unit 11 may collect the plurality of training samples 60 by any of the methods in References 4 to 8 mentioned above.

[0178] In step S106, the control unit 11 accepts annotations for at least a portion of the collected training samples 60. The annotation may be at least partially automated. The annotation may also be performed manually. In step S107, The control unit 11 then performs additional learning of the trained model 5 using a plurality of training samples 60, at least some of which have been annotated.

[0179] In one example, the additional learning may be configured in machine learning by generating a new trained model 5 different from the trained model 5 in operation, using original training data (the multiple training samples used to generate the trained model 5 in operation) and the multiple training samples 60 obtained up to step S107. In this case, the control unit 11 may generate learning result data 500 corresponding to the new trained model 5 as a result of the additional learning.

[0180] In another example, the additional learning may be configured by readjusting parameters of the trained model 5 using at least a plurality of training samples 60 in machine learning. In this case, the control unit 11 may generate the learning result data 500 as a result of updating the learning result data corresponding to the original trained model 5 by the additional learning.

[0181] The configuration of the learning result data 500 is not particularly limited and may be appropriately determined depending on the embodiment. As an example, the learning result data 500 may be configured to include information indicating values ​​of calculation parameters adjusted by machine learning. In some cases, the learning result data 500 may be configured to include information indicating the configuration of the trained model 5 (e.g., the structure of a neural network, etc.).

[0182] In step S108, the control unit 11 stores the generated learning result data 500 in a predetermined memory area. The predetermined memory area may be, for example, a RAM in the control unit 11, the memory unit 12, an external storage device, a storage medium, or a combination of these. The storage medium may be, for example, a CD, a DVD, a semiconductor memory, etc. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). The external storage device may be, for example, an external storage device. The learning result data 500 may be provided to the inference device 2 as appropriate. When the storage of the learning results is completed, the control unit 11 proceeds to the next step S109.

[0183] (Step S109) In step S109, the control unit 11 selects whether or not to repeat a series of information processing. Whether or not to repeat may be selected as appropriate. In one example, the monitoring device 1 may be configured to repeatedly execute the information processing unless an end operation is performed by the operator. In this case, the control unit 11 may select whether or not to repeat the information processing depending on the presence or absence of an end operation by the operator. That is, the control unit 11 may select to repeat the information processing while an end operation is not performed by the operator, and may select not to repeat the information processing after an end operation is performed. If repeating is selected, the control unit 11 returns the processing to step S101 and executes the processing again from step S101. On the other hand, if not repeating is selected, the control unit 11 ends the processing procedure of the monitoring device 1 according to this operation example.

[0184] [Features] In this embodiment, the monitoring device 1 acquires the environmental data 3 in step S101, and in step S102, uses the environmental data 3 to estimate the deterioration of the performance of the trained model 5 caused by a shift in the environment. This makes it possible to quickly detect a shift in distribution such as T2 in Fig. 2 above, and to estimate the deterioration of the performance of the trained model 5 at an early stage. Therefore, according to this embodiment, the deterioration of the performance of the trained model 5 can be detected at an appropriate timing.

[0185] § 4 Variations Although the embodiment of the present invention has been described above in detail, the above description is in all respects inaccurate. The above-described embodiment is merely an example of the present invention. Various improvements or modifications may be made as appropriate to the above-described embodiment. For example, the following modifications are possible. In the following, the same reference numerals are used for the same components as those in the above-described embodiment, and the description of the same points as those in the above-described embodiment is omitted as appropriate. The following modifications can be combined as appropriate.

[0186] In the processing procedure of the monitoring device 1 according to the above embodiment, at least one of omission, replacement, and addition of steps may be performed. For example, the processing of steps S104 to S108 may be omitted. In this case, the additional learning unit 114 may be omitted from the software configuration of the monitoring device 1. The processing of steps S104 and S105 may be performed before step S104. At least a part of the processing of steps S105 to S108 may be performed by another computer. In a case where the processing of steps S105 to S108 is performed by another computer, when the monitoring device 1 selects to perform additional learning in step S104, it may output an instruction for additional learning to the other computer, thereby causing the other computer to perform additional learning. Step S109 may be omitted. When at least a part of the processing of additional learning is automated, the processing of step S103 may be interpreted as including at least a part of the processing of steps S104 to S108.

[0187] This specification includes the following disclosure. [Appendix 1] An acquisition unit (111) configured to acquire environmental data (3) related to an environment in which a trained model (5) generated by machine learning is operated; a degradation estimation unit (112) configured to estimate, from the acquired environmental data (3), a degradation in performance of the trained model (5) due to a shift in the environment; An information output unit (113) configured to output information about a result of estimating a deterioration in performance of the trained model (5); Equipped with Monitoring equipment (1). [Appendix 2] The environmental data (3) includes input data (50) obtained from the environment during operation, the input data (50) being provided to the trained model (5) to perform inference by the trained model (5); Inferring a degradation in the performance of the trained model (5) includes inferring a decrease in the reliability of the inferences of the trained model (5) on the input data (50). 2. The monitoring device (1) according to claim 1. [Appendix 3] The inference includes identifying a class; The reliability of the inference is calculated according to the entropy of the result of the class classification by the trained model (5) for the input data (50). 2. The monitoring device (1) according to claim 2. [Appendix 4] The trained model (5) is configured to output a result of the inference together with a score of the confidence of the inference; The reliability of the inference is calculated according to the score output from the trained model (5) for the input data (50). 4. The monitoring device (1) according to claim 2 or 3. [Appendix 5] The reliability of the inference is calculated according to the extent to which the input data (50) is within the range of the distribution of training data used in the machine learning of the trained model (5). 5. A monitoring device (1) according to any one of appendices 2 to 4. [Appendix 6] The environmental data (3) includes condition data (55) indicating conditions for obtaining input data (50) to be provided to the trained model (5) during operation from the environment in order to execute inference by the trained model (5); Predicting the deterioration of the performance of the trained model (5) includes predicting the deterioration of the accuracy of inference by the trained model (5) according to the degree of deviation of the conditions indicated by the condition data (55) from the conditions under which training data used in the machine learning of the trained model (5) was obtained. 6. A monitoring device (1) according to any one of appendices 1 to 5. [Appendix 7] The input data (50) includes observation data acquired by a sensor (S); The condition data (55) includes observation condition data indicating conditions of observation by the sensor (S) during operation. 6. The monitoring device (1) according to claim 6. [Appendix 8] The sensor (S) includes an imaging device (SA), The observation conditions include imaging conditions of the imaging device (SA), 7. The monitoring device (1) according to claim 7. [Appendix 9] The sensor (S) includes a microphone (SB), The observation conditions include recording conditions of the microphone (SB); 9. The monitoring device (1) according to claim 7 or 8. [Appendix 10] The sensor (S) includes a sensor (SC) mounted on a robot device (R), The observation conditions include conditions regarding the operation of the robot device (R). 10. The monitoring device (1) according to any one of appendixes 7 to 9. [Appendix 11] Inferring the degradation of the performance of the trained model (5) is performed continuously; The output information includes a prediction result of when the performance of the trained model (5) will fall below a threshold. 11. The monitoring device (1) according to any one of appendixes 1 to 10. [Appendix 12] obtaining the environmental data (3) comprises obtaining environmental data (3) for each of a plurality of the trained models (5); Inferring the degradation of the performance of the trained model (5) comprises inferring the degradation of the performance of each of the plurality of trained models (5). 12. A monitoring device (1) according to any one of appendixes 1 to 11. [Appendix 13] The output information includes a map (MA, MB) showing the operation location of each of the plurality of trained models (5), Outputting the information includes superimposing and displaying information on the results of estimating the deterioration of the performance of each of the trained models (5) on the operation location of each of the trained models (5) on the map (MA, MB). 13. The monitoring device (1) according to claim 12. [Appendix 14] The output information includes a result of estimating the degree to which the performance of the trained model (5) will be improved by additionally learning a predetermined number of samples. 14. The monitoring device (1) according to any one of appendixes 1 to 13. [Appendix 15] Inferring the degradation of the performance of the trained model (5) is performed continuously; The output information includes at least one of the timing of additional learning optimized within a predetermined budget and the number of annotations for the additional learning samples. 15. The monitoring device (1) according to any one of appendixes 1 to 14. [Appendix 16] A computer (1) Obtaining environmental data (3) about an environment in which a trained model (5) generated by machine learning is operated; Inferring from the acquired environmental data (3) a degradation in performance of the trained model (5) due to a shift in the environment; and Outputting information regarding the result of estimating the deterioration of the performance of the trained model (5); Monitoring methods. [Appendix 17] On the computer (1), Obtaining environmental data (3) about an environment in which a trained model (5) generated by machine learning is operated; Inferring from the acquired environmental data (3) a degradation in performance of the trained model (5) due to a shift in the environment; and and outputting information regarding the result of estimating the deterioration of the performance of the trained model (5). program. [Explanation of symbols]

[0188] 1...Monitoring device, 11: control unit, 12: storage unit, 13: communication interface, 14...input device, 15...output device, 16...drive, 81... program, 91... storage medium, 111...acquisition section, 112...deterioration estimation section, 113: information output unit; 114: additional learning unit; 2... Reasoning device, 3...environmental data, 5...trained model, 50...input data, 50S...observation data, S: sensor, 55...Condition data

Claims

1. An acquisition unit configured to acquire environmental data related to an environment in which the trained model generated by machine learning is operated; a degradation estimation unit configured to estimate degradation of performance of the trained model due to a shift in the environment from the acquired environmental data; an information output unit configured to output information regarding a result of estimating a deterioration in performance of the trained model; Equipped with Monitoring equipment.

2. The environmental data includes input data obtained from the environment during operation, the input data being provided to the trained model to perform inference using the trained model; Inferring degradation of the performance of the trained model includes inferring a decrease in confidence in the trained model's inferences on the input data. The monitoring device of claim 1 .

3. The inference includes identifying a class; The reliability of the inference is calculated according to the entropy of the result of the class discrimination by the trained model for the input data. The monitoring device of claim 2.

4. The trained model is configured to output a result of the inference together with a score of confidence in the inference; The reliability of the inference is calculated according to the score output from the trained model for the input data. The monitoring device of claim 2.

5. The reliability of the inference is calculated according to the degree to which the input data is within the range of the distribution of training data used in the machine learning of the trained model. The monitoring device of claim 2.

6. The environmental data includes condition data indicating conditions for obtaining input data to be given to the trained model during operation from the environment in order to execute inference using the trained model; Predicting the deterioration of the performance of the trained model includes predicting the deterioration of the accuracy of inference by the trained model according to the degree of deviation of the conditions indicated by the condition data from the conditions under which training data used in the machine learning of the trained model was obtained. The monitoring device of claim 1 .

7. The input data includes observation data acquired by a sensor; The condition data includes observation condition data indicating conditions of observation of the sensor during operation.

7. The monitoring device of claim 6.

8. the sensor includes an imaging device; The observation conditions include imaging conditions of the imaging device.

8. The monitoring device of claim 7.

9. The sensor includes a microphone. The observation conditions include the microphone recording conditions.

8. The monitoring device of claim 7.

10. The sensor is mounted on a robotic device, the observation conditions include conditions related to the operation of the robot device; 8. The monitoring device of claim 7.

11. Inferring the degradation of the performance of the trained model is performed continuously; The output information includes a prediction of when the performance of the trained model will fall below a threshold. The monitoring device of claim 1 .

12. obtaining the environmental data comprises obtaining environmental data for each of a plurality of the trained models; Inferring a degradation in performance of the trained models comprises inferring a degradation in performance of each of the plurality of trained models. The monitoring device of claim 1 .

13. The output information includes a map showing the location of operation of each of the plurality of trained models; The outputting of the information includes superimposing and displaying information on the result of estimating the deterioration of the performance of each of the trained models on the map at the operation location of each of the trained models.

13. The monitoring device of claim 12.

14. The output information includes an estimate of the degree to which the performance of the trained model will be improved by additionally learning a predetermined number of samples. The monitoring device of claim 1 .

15. Inferring the degradation of the performance of the trained model is performed continuously; The output information includes at least one of the timing of additional learning optimized within a predetermined budget and the number of annotations for the additional learning samples. The monitoring device of claim 1 .

16. The computer Obtaining environmental data about an environment in which the trained model generated by machine learning is operated; Inferring from the acquired environmental data a degradation in performance of the trained model due to a shift in the environment; and outputting information related to the estimated degradation of the performance of the trained model; Execute Monitoring methods.

17. On the computer, Obtaining environmental data about an environment in which the trained model generated by machine learning is operated; Inferring from the acquired environmental data a degradation in performance of the trained model due to a shift in the environment; and outputting information related to the estimated degradation of the performance of the trained model; In order to execute program.

Images