Anomaly detection system and anomaly detection method

Abstract

To provide an anomaly detection system and an anomaly detection method configured to execute re-learning at appropriate timing to prevent accuracy degradation (data drift) of an AI model.SOLUTION: A learning unit 10 generates an anomaly determination AI_1 for determining an anomaly based on normal imaging data. An inference unit 20 infers an anomaly of an inspection object from imaging data in an imaging data DB 22 using the anomaly determination AI_1, and displays, when detecting degradation (data drift) of the anomaly determination AI_1, a model state including a degree of degradation and a degraded portion of the anomaly determination AI_1 on a model state display UI 24. A re-learning unit 30 performs, on receipt of an instruction for re-learning from a user, re-learning based on the imaging data from which the anomaly and degradation have been detected to generate an anomaly determination AI_2, and displays a model evaluation result on a model evaluation result display UI 33. The inference unit 20 replaces, on receipt of an instruction for replacement from the user, the anomaly determination AI_1 by the anomaly determination AI_2.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an anomaly detection system and an anomaly detection method. [Background technology]

[0002] When using AI (Artificial Intelligence) in production sites, etc., the accuracy of the AI ​​model can decrease due to changes in the data characteristics and distribution of the usage environment over time (data drift). As part of addressing such data drift, it is important to retrain the model. Therefore, it is very important to notify users when it is time to retrain.

[0003] For example, Patent Document 1 discloses a technology for automating the timing of relearning an AI model. Patent Document 2 discloses a system for selecting a trained model. Patent Document 3 discloses the order of data when relearning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 024985 [Patent Document 2] Japanese Patent Application Publication No. 2020-42669 [Patent Document 3] Japanese Patent Application Publication No. 2019-95217 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned Patent Document 1 does not provide the timing of re-learning due to data drift in an AI model for detecting anomalies, the specific parts that have deteriorated (where data drift has occurred), or the degree of model deterioration over time, so the reason and cause for re-learning are unclear. Furthermore, although Patent Document 2 allows the selection of a trained model, no consideration is given to re-learning. Furthermore, Patent Document 3 does not clearly state the timing of re-learning. Therefore, Patent Documents 1-3 have the problem of being unable to perform re-learning at an appropriate time to prevent deterioration in the accuracy of the AI ​​model (data drift).

[0006] Therefore, an object of the present invention is to provide an anomaly detection system and an anomaly detection method that can perform re-learning at an appropriate time to prevent deterioration in the accuracy of an AI model (data drift). [Means for solving the problem]

[0007] In order to solve the above problems, the present invention employs the following means. That is, the anomaly detection system according to the first invention is an anomaly detection system for performing visual inspection, comprising: a learning unit that creates a first AI model for determining an anomaly based on normal image data of the inspection object photographed in advance; an inference unit that infers an anomaly of the inspection object from the image data photographed using the first AI model, and, if deterioration of the first AI model is detected, presents to a user the model state including at least the degree of deterioration of the first AI model and the location of the deterioration; and a re-learning unit that, upon receiving a re-learning instruction from the user, re-learns using the image data in which the anomaly was detected and the image data in which deterioration was detected to create a second AI model, evaluates the first AI model and the second AI model, and presents the model evaluation results to the user, and is characterized in that, upon receiving a replacement instruction from the user, the first AI model is replaced with the second AI model.

[0008] In addition, an anomaly detection system according to a second invention is an anomaly detection system for performing visual inspection, comprising: a learning unit that creates a first AI model for determining an anomaly based on normal image data of the inspection object photographed in advance; an inference unit that infers an anomaly of the inspection object from the image data photographed using the first AI model, and, if deterioration of the first AI model is detected, presents a model state including at least the degree of deterioration of the first AI model and the location of the deterioration to a client device via a network; and a re-learning unit that, when a re-learning instruction is received from the client device via the network, re-learns using the image data in which the anomaly was detected and the image data in which deterioration was detected to create a second AI model, evaluates the first AI model and the second AI model, and presents the model evaluation results to the client device via the network, and is characterized in that the first AI model is replaced with the second AI model when a replacement instruction is received from the client device via the network.

[0009] In addition, an anomaly detection system according to a third invention is an anomaly detection system for performing visual inspection, comprising: a learning unit that creates a first AI model for determining an anomaly based on normal image data of the inspection object photographed in advance; an inference unit that infers an anomaly of the inspection object from the image data photographed using the first AI model and, if deterioration of the first AI model is detected, instructs the system to re-learn; and a re-learning unit that, upon receiving an instruction to re-learn from the inference unit, creates a second AI model by re-learning using the image data in which an anomaly was detected and the image data in which deterioration was detected, evaluates the first AI model and the second AI model, and outputs a model evaluation result, and is characterized in that, if, based on the model evaluation result, the evaluation value of the second AI model is equal to or greater than the evaluation value of the first AI model, the first AI model is replaced with the second AI model.

[0010] Furthermore, an anomaly detection method according to a fourth invention is an anomaly detection method for performing visual inspection, comprising: creating a first AI model for determining an anomaly based on normal image data of an inspection object photographed in advance; inferring an anomaly of the inspection object from the image data photographed using the first AI model; and, when deterioration of the first AI model is detected, presenting to a user a model state including at least the degree of deterioration of the first AI model and the location of the deterioration; when a re-learning instruction is received from the user, creating a second AI model by re-learning using the image data in which an anomaly was detected and the image data in which deterioration was detected, evaluating the first AI model and the second AI model, and presenting the model evaluation results to the user; and when a replacement instruction is received from the user, replacing the first AI model with the second AI model.

[0011] Furthermore, an anomaly detection method according to a fifth invention is an anomaly detection method for performing visual inspection, comprising: creating a first AI model for determining an anomaly based on normal image data of an inspection object photographed in advance; inferring an anomaly of the inspection object from the image data photographed using the first AI model; and, when degradation of the first AI model is detected, presenting a model state including at least the degree of degradation of the first AI model and the location of the degradation to a client device via a network; when a re-learning instruction is received from the client device via the network, creating a second AI model by re-learning using the image data in which an anomaly was detected and the image data in which degradation was detected, evaluating the first AI model and the second AI model, and presenting the model evaluation results to the client device via the network; and when a replacement instruction is received from the client device via the network, replacing the first AI model with the second AI model.

[0012] In addition, an anomaly detection method according to a sixth invention is an anomaly detection method for performing visual inspection, comprising: creating a first AI model for determining an anomaly based on normal image data of the inspection object photographed in advance; inferring an anomaly of the inspection object from the image data photographed using the first AI model; and, if deterioration of the first AI model is detected, issuing an instruction for re-learning; when the instruction for re-learning is received, creating a second AI model by re-learning based on the image data in which the anomaly was detected and the image data in which deterioration was detected, evaluating the first AI model and the second AI model and outputting a model evaluation result; and, if, based on the model evaluation result, the evaluation value of the second AI model is equal to or greater than the evaluation value of the first AI model, replacing the first AI model with the second AI model. [Effects of the Invention]

[0013] According to this invention, it is possible to prevent deterioration of the accuracy of the AI ​​model (data drift) by executing re-learning at an appropriate timing. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a configuration of an anomaly detection system according to a first embodiment of the present invention. [Figure 2] 5 is a flowchart for explaining the operation of the anomaly detection system (inference unit and relearning unit) according to the first embodiment. [Figure 3] FIG. 2 is a schematic diagram showing an example of a model state display UI according to the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of a trend of abnormal values ​​presented on a model state display UI according to the first embodiment. [Figure 5] FIG. 2 is a schematic diagram showing an example of visualization of the location where data drift occurs in the anomaly detection system according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram showing an example of a model evaluation result display UI in the anomaly detection system according to the first embodiment. [Figure 7]FIG. 10 is a block diagram showing the configuration of an anomaly detection system according to a second embodiment of the present invention. [Figure 8] FIG. 10 is a block diagram showing the configuration of an anomaly detection system according to a third embodiment of the present invention. [Figure 9] 10 is a flowchart for explaining the operation of the anomaly detection system (inference unit and relearning unit) according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. A. First embodiment The first embodiment is a system that displays the locations where data drift is occurring and the trends of the degree of deterioration, and based on the results, allows the user to instruct whether re-learning and the replacement of the anomaly detection model are necessary, and is characterized by the fact that the model replacement is performed on-site.

[0016] FIG. 1 is a block diagram showing the configuration of an anomaly detection system according to the first embodiment. In FIG. 1, the anomaly detection system 1 includes a learning unit 10, an inference unit 20, and a re-learning unit 30. The learning unit 10 includes a learning data database 11 storing previously captured normal images (images in which no abnormalities are detected), and a learning processing unit 12 for training an anomaly detection model using the normal images stored in the learning data database 11. The learning processing unit 12 reads normal images from the learning data database 11, trains a deep learning model for anomaly detection, and creates an anomaly detection (anomaly detection model) AI_1 (first AI model). The anomaly detection model used may be a model such as Patch Core or CS-Flow that performs end-to-end learning for anomalies, or a model that uses reconstruction error such as AutoEncoder or VAE. However, any model type is acceptable as long as it can output an anomaly score and a visualization of the anomaly location after inference. The same applies to supervised learning and semi-supervised learning that uses abnormal data for training.

[0017] The inference unit 20 comprises a camera 21, a photographed data DB 22, an inference processing unit 23, a model state display UI 24, an abnormality data DB 25, and a data drift DB 26. The camera 21 sequentially photographs an object for abnormality detection at a predetermined timing and outputs the photographed data. The photographed data DB 22 stores the photographed data sequentially photographed by the camera 21. The inference processing unit 23 reads out the photographed data from the photographed data DB 22 and determines whether there is an abnormality in the photographed data using an abnormality determination AI_1.

[0018] More specifically, the inference processing unit 23 determines that an abnormality has occurred when the abnormal value (abnormality score) inferred using the abnormality determination AI_1 learned by the learning processing unit 12 of the learning unit 10 is equal to or greater than an abnormality determination threshold (threshold A). If the abnormal value is equal to or greater than threshold A, the inference processing unit 23 determines that an abnormality has occurred, notifies the user of the abnormality, and stores the photographed data determined to be abnormal (photographed data in which an abnormality has been detected) in the abnormality data DB 25.

[0019] Furthermore, if the inferred abnormal value is not equal to or greater than threshold A but is equal to or greater than a data drift determination threshold (threshold B; A>B), the inference processing unit 23 determines that data drift has occurred, although the value is not abnormal. If the abnormal value is equal to or greater than threshold B, the inference processing unit 23 presents (displays) to the user via the model state display UI 24 the model state including the abnormal value calculated for threshold determination and the visualization result (location where data drift has occurred), and saves the photographed data determined to have data drift (photographed data in which deterioration has been detected) in the data drift DB 26 for re-learning.

[0020] The inference processing unit 23 also counts the number of times (data drift count) that the inferred abnormal value is not equal to or greater than threshold A but equal to or greater than threshold B, and when the count value (data drift count) is equal to or greater than a data drift judgment threshold (threshold B; A>B) and the count value (data drift count) is equal to or greater than a predetermined threshold C, it determines that data drift (accuracy degradation) has occurred in the abnormality judgment AI_1, and presents (notifies) the user via the model state display UI 24 a re-learning recommendation indicating that the abnormality judgment AI_1 should be re-learned. When the user instructs (operates) to re-learn via the model state display UI 24, the inference processing unit 23 instructs the re-learning unit 30 to perform re-learning. When the model evaluation unit 32 instructs model replacement, the inference processing unit 23 replaces the abnormality judgment AI_1 of the learning unit 10 with an abnormality judgment AI_2 (second AI model) described below.

[0021] The relearning unit 30 comprises a relearning processing unit 31, a model evaluation unit 32, and a model evaluation result display UI 33. When a user who has confirmed the model state presented by the inference unit 20 instructs relearning, the relearning processing unit 31 reads the photographed data in which an abnormality has been detected in the abnormal data DB 25 and the photographed data in which data drift has been detected in the data drift DB 26, and trains a deep learning model that performs an abnormality judgment based on these data to create an abnormality judgment AI_2 (relearning). At this time, some data (evaluation data) in the abnormal data DB 25 and the data drift DB 26 are not used in the learning in order to evaluate the AI ​​model in the subsequent stage.

[0022] The model evaluation unit 32 evaluates the abnormality determination AI_1 and the abnormality determination AI_2 using the evaluation data. The evaluation is performed on the visualization results of the evaluation data and numerical results such as histograms and ROC (Receiver Operating Characteristic). The model evaluation unit 32 also presents (displays) the model evaluation results to the user via the model evaluation result display UI 33.

[0023] 2 is a flowchart for explaining the operation of the anomaly detection system (the inference unit 20 and the relearning unit 30) according to the first embodiment. The inference unit 20 first receives image data from the camera 21 (step S10) and stores the image data in the image data DB 22 (step S12). Next, the inference processing unit 23 reads the image data from the image data DB 22, infers whether the image data contains an anomaly using the anomaly judgment AI_1 (step S14), and determines whether the inferred anomaly value (anomaly score) is equal to or greater than the anomaly judgment threshold (threshold A) (step S16).

[0024] If the inferred abnormal value is not equal to or greater than threshold A (NO in step S16), the inference processing unit 23 determines that there is no abnormality, and determines whether the inferred abnormal value is equal to or greater than a data drift determination threshold (threshold B) (step S22). If the inferred abnormal value is not equal to or greater than threshold B (NO in step S22), the inference processing unit 23 determines that there is no data drift, and determines whether or not imaging has ended (step S42). If imaging has not ended (NO in step S42), the inference processing unit 23 returns to step S10 and repeats the same process for the next imaging data.

[0025] On the other hand, if the result of inferring whether the photographed data has an abnormality is that the inferred abnormal value is equal to or greater than the threshold value A (YES in step S16), the inference processing unit 23 determines that an abnormality exists, notifies the user of the abnormality via the model state display UI 24 (step S18), and stores the photographed data determined to be abnormal in the abnormal data DB 25 (step S20). Thereafter, the inference processing unit 23 determines whether or not photographing has ended (step S42), and if photographing has not ended (NO in step S42), returns to step S10 and repeats the same process for the next photographed data.

[0026] Furthermore, if, as a result of inferring whether there is an abnormality in the photographed data, the inferred abnormal value is not equal to or greater than threshold value A (NO in step S16) but is equal to or greater than threshold value B (YES in step S22), the inference processing unit 23 determines that data drift has occurred, and presents (displays) the abnormal value calculated for threshold determination and the visualization result (location of data drift occurrence) to the user via the model status display UI 24 (step S24).

[0027] 3 is a schematic diagram showing an example of the model state display UI 24 according to the first embodiment. The model state display UI 24 displays a trend 40 of abnormal values ​​calculated for threshold determination, a location of data drift (location of model degradation) 41, a re-learning recommendation 42, and a re-learning button 43.

[0028] 4 is a schematic diagram showing an example of an abnormal value trend 40 presented on the model state display UI 24 according to the first embodiment. As shown in FIG. 4, the abnormal value trend 40 is displayed in chronological order by a line segment L1, and a threshold value B, which is a data drift determination threshold, is also displayed. When the line segment L1 showing the abnormal value trend exceeds the threshold value B, it is determined that data drift has occurred.

[0029] 5 is a schematic diagram showing an example of visualization of a location 41 where data drift has occurred in the anomaly detection system according to the first embodiment. When there is no data drift, as shown in FIG. 5(a), at the location 41 where data drift has occurred (location where deterioration has occurred in the model), the photographed data to be inspected is displayed in the image to be analyzed (area) 51, but nothing is displayed in the anomaly location visualization result (area) 52. In this way, when there is no data drift (the data is similar to the training data), no abnormality is detected by the anomaly judgment AI_1, and therefore the location where the abnormality has occurred (location where data drift has occurred) is not visualized.

[0030] On the other hand, if there is data drift, that location is detected as an abnormality, and as shown in Fig. 5(b), at the location where data drift occurred (location where deterioration occurred in the model) 41, the photographed data of the inspection target is displayed in the image (area) to be analyzed 61, and the location where drift occurred is displayed (visualized) in the abnormal location visualization result (area) 52. Causes of data drift include, for example, lighting fluctuations due to equipment deterioration.

[0031] In this way, the user can visually recognize the degree of deterioration in the accuracy of the AI ​​model (data drift) and the location of the data drift from the trend 40 of the abnormal values ​​and the location 41 of the data drift.

[0032] Next, the inference processing unit 23 stores the photographic data determined to have data drift in the data drift DB 26 for re-learning (step S26) and counts the number of data drifts (step S28). The inference processing unit 23 determines whether the count value (number of data drifts) is equal to or greater than a predetermined threshold C (step S30), and if the count value (number of data drifts) is not equal to or greater than the predetermined threshold C (NO in step S30), it determines that the degree of data drift is low. In this case, the inference processing unit 23 determines whether photographing has ended (step S42) without performing re-learning, which will be described later. If photographing has not ended (NO in step S42), the process returns to step S10 and the same process is repeated for the next photographic data.

[0033] On the other hand, if the count value (number of data drifts) is equal to or greater than the predetermined threshold C (YES in step S30), the inference processing unit 23 determines that the degree of data drift is large, and notifies the model state display UI 24 of a recommendation to re-learn the abnormality judgment AI_1 as a re-learning recommendation 42 (see FIG. 3) (step S32). The user confirms that re-learning is necessary from the re-learning recommendation 42 in the model state display UI 24, or from the trend of abnormal values, the location of data drift, etc., and instructs (operates) re-learning using the re-learning button 43.

[0034] The inference processing unit 23 determines whether or not a re-learning instruction (operation) has been given by the user (step S34), and if a re-learning instruction has not been given (NO in step S34), determines whether or not shooting has ended (step S42), and if shooting has not ended (NO in step S42), returns to step S10 and repeats the same processing for the next shooting data. On the other hand, if a re-learning instruction (operation) has been given by the user (YES in step S34), the inference processing unit 23 instructs the re-learning unit 30 to perform re-learning.

[0035] When the inference processing unit 23 instructs the relearning unit 30 to perform relearning (step S36), the relearning processing unit 31 executes relearning. Specifically, the relearning processing unit 31 reads out the photographed data determined to be abnormal from the abnormal data DB 25 and the photographed data determined to have data drift from the data drift DB 26. In addition, for a supervised anomaly detection method (a method using abnormal data and normal data) for model generation, the relearning processing unit 31 also reads out the abnormal data DB 25 while taking into account the abnormal data in the learning data DB 11. The relearning processing unit 31 creates an abnormality determination AI_2 using the read-out photographed data. Next, the model evaluation unit 32 evaluates the abnormality determination AI_1 and the abnormality determination AI_2 using the evaluation data, and presents (displays) the model evaluation result, which is the evaluation result, to the user via the model evaluation result display UI 33.

[0036] FIG. 6 is a schematic diagram showing an example of the model evaluation result display UI 33 in the anomaly detection system according to the first embodiment. As shown in FIG. 6, the model evaluation result display UI 33 displays visualization results 70 and 71 and numerical results (histograms) 72 and 73 for anomaly determination AI_1 before relearning and anomaly determination AI_2 after relearning, respectively. The visualization results 70 and 71 display the analysis target image and the visualization results of the abnormal area. In the anomaly determination AI_1 before relearning, it can be seen that the abnormal area was determined as a location where data drift occurred. On the other hand, in the anomaly determination AI_2 after relearning, it can be seen that the location where data drift occurred is learned as normal and therefore not determined as an abnormal area. Similarly, in the numerical results, it can be seen that in the anomaly determination AI_1 before relearning, the histogram determined as data drift (gray area) is close to the anomaly detection threshold (threshold A), while in the anomaly determination AI_2 after relearning, the histogram determined as data drift (gray area) is smaller than threshold B.

[0037] The user determines whether to replace the abnormality determination AI_1 by viewing the display of the model evaluation result display UI 33. For example, if the user determines that the abnormality determination AI_1 needs to be replaced, the user instructs (operates) the model replacement button 74 displayed on the model evaluation result display UI 33 shown in FIG.

[0038] The inference processing unit 23 determines whether or not the user has instructed to change the model (step S38), and if the user has not instructed to change the model (NO in step S38), the process returns to step S10 and repeats the same process for the next photographed data.

[0039] On the other hand, if the user instructs to switch models (YES in step S38), the inference processing unit 23 switches the abnormality determination AI_1 of the learning unit 10 to the abnormality determination AI_2 (step S40). After that, the process returns to step S14, and the above-described abnormality detection process is performed again using the new abnormality determination AI_1 (abnormality determination AI_2).

[0040] Then, when the photographing is completed (YES in step S42), the inference processing unit 23 ends the processing.

[0041] According to the first embodiment described above, in the AI ​​model that performs visual inspection, deterioration (data drift) of the anomaly judgment AI_1 during operation is detected, and the degree of deterioration and the location of the deterioration are visualized, thereby clearly indicating to the user that it is time to re-learn, and the anomaly detection model can be replaced by performing re-learning at an appropriate time in response to instructions from the user.

[0042] B. Second embodiment The second embodiment is characterized in that model degradation (occurrence of data drift) is notified to a remote client device via a network, and the client device issues instructions to re-learn and to replace the anomaly detection model.

[0043] 7 is a block diagram showing the configuration of an anomaly detection system according to a second embodiment of the present invention. The following description will focus on only components having functions different from those of the first embodiment. In the inference unit 20, the inference processing unit 23 infers whether an anomaly exists in the captured data. If the inferred anomaly value (anomaly score) is determined to be not greater than the anomaly determination threshold (threshold A) but greater than the data drift determination threshold (threshold B), the inference processing unit 23 outputs the anomaly value calculated for threshold determination and the visualization result (location of data drift occurrence) as a model state to the client device 81 via the network 80. The client device 81 displays the information presented in the model state display UI 24 shown in FIGS. 3 to 5.

[0044] Furthermore, in the relearning unit 30, the model evaluation unit 32 evaluates the abnormality determination AI_1 and the abnormality determination AI_2 created by the relearning processing unit 31 using the evaluation data, and outputs the evaluation result, that is, the model evaluation result, to the client device 81 via the network 80. The client device 81 displays the information presented in the model evaluation result display UI 33 shown in FIG.

[0045] The user checks the information presented on the model status display UI 24 displayed on the client device 81 and determines whether to perform re-learning. The user also checks the information presented on the model evaluation result display UI 33 displayed on the client device 81 and determines whether to replace the anomaly judgment (anomaly detection model) AI_1.

[0046] For example, if the user determines that relearning is necessary, the user instructs (operates) relearning from client device 81 via network 80. When inference processing unit 23 receives the relearning instruction from client device 81 via network 80, it instructs relearning unit 30 to relearning, and causes relearning unit 30 to execute relearning.

[0047] Furthermore, if the user determines that it is necessary to replace the abnormality determination AI_1, the user instructs (operates) the model replacement from the client device 81 via the network 80. When the inference processing unit 23 receives the model replacement from the client device 81 via the network 80, it replaces the abnormality determination AI_1 of the learning unit 10 with the abnormality determination AI_2.

[0048] In the second embodiment described above, when deterioration (data drift) of the abnormality judgment AI_1 in operation is detected in an AI model that performs visual inspections at a remote location, the degree of deterioration and the location where the deterioration has occurred are visualized on the client device 81 via the network 80, so that the timing for re-learning can be clearly indicated to the user, and re-learning can be performed at an appropriate time to replace the anomaly detection model in response to instructions from the remote client device 81.

[0049] C. Third embodiment The third embodiment is characterized in that the determination of replacement of anomaly detection models and the replacement operation, which have been performed by the user, are automated.

[0050] FIG. 8 is a block diagram showing the configuration of an anomaly detection system according to a third embodiment of the present invention. Note that the following description will only focus on components having functions different from those of the first embodiment. In FIG. 8, the model evaluation unit 32 uses evaluation data to evaluate each of the anomaly determinations (anomaly detection models) AI_1 and AI_2, and supplies the model evaluation results, which are the evaluation results, to the inference processing unit 23. Based on the model evaluation results from the model evaluation unit 32, the inference processing unit 23 determines whether to replace the anomaly determinations AI_1 and AI_2 of the learning unit 10. If the evaluation value of the re-learned anomaly determination AI_2 is equal to or greater than the current anomaly determination AI_1, the inference processing unit 23 replaces the models.

[0051] 9 is a flowchart for explaining the operation of the anomaly detection system (the inference unit 20 and the relearning unit 30) according to the third embodiment. The inference unit 20 first inputs the photographed data from the camera 21 (step S70) and stores the photographed data in the photographed data DB 22 (step S72). Next, the inference processing unit 23 in the inference unit 20 reads the photographed data from the photographed data DB 22, infers whether the photographed data contains an anomaly using the anomaly judgment AI_1 (step S74), and determines whether the inferred anomaly value (anomaly score) is equal to or greater than the anomaly judgment threshold (threshold A) (step S76).

[0052] If the inferred abnormal value is not equal to or greater than threshold A (NO in step S76), the inference processing unit 23 determines that there is no abnormality, and determines whether the inferred abnormal value is equal to or greater than a data drift determination threshold (threshold B) (step S82). If the inferred abnormal value is not equal to or greater than threshold B (NO in step S82), the inference processing unit 23 determines that there is no data drift, and determines whether or not imaging has ended (step S96). If imaging has not ended (NO in step S96), the process returns to step S10, and the same process is repeated for the next imaging data.

[0053] On the other hand, if the result of inferring whether the photographed data has an abnormality is that the inferred abnormal value is equal to or greater than the threshold value A (YES in step S76), the inference processing unit 23 determines that an abnormality exists, notifies the user of the abnormality via the model state display UI 24 (step S78), and stores the photographed data determined to be abnormal in the abnormal data DB 25 (step S80). Thereafter, the inference processing unit 23 determines whether or not photographing has ended (step S96), and if photographing has not ended (NO in step S96), returns to step S70 and repeats the same process for the next photographed data.

[0054] Furthermore, if, as a result of inferring whether or not there is an abnormality in the photographed data, the inferred abnormal value is not equal to or greater than threshold value A (NO in step S76) but is equal to or greater than threshold value B (YES in step S82), the inference processing unit 23 determines that data drift has occurred, stores the photographed data in the data drift DB 26 for re-learning (step S84), and counts the number of data drifts (step S86). The inference processing unit 23 determines whether or not the count value (number of data drifts) is equal to or greater than a predetermined threshold value C (step S88), and if the count value (number of data drifts) is not equal to or greater than threshold value C (NO in step S88), determines that the degree of data drift is low. Thereafter, the inference processing unit 23 determines whether or not photographing has ended (step S96), and if photographing has not ended (NO in step S96), returns to step S70 and repeats the same process for the next photographed data.

[0055] On the other hand, if the count value (number of data drifts) is equal to or greater than the predetermined threshold C (YES in step S88), the inference processing unit 23 determines that the degree of data drift is large, and instructs the re-learning unit 30 to perform re-learning.

[0056] When the inference processing unit 23 instructs the relearning unit 30 to perform relearning, the relearning unit 30 executes relearning (step S90). Specifically, the relearning processing unit 31 reads out the imaging data determined to be abnormal from the abnormal data DB 25 and the imaging data determined to have data drift from the data drift DB 26, and creates the abnormality determination AI_2 using the read imaging data. Next, the model evaluation unit 32 evaluates the abnormality determination AI_1 and the abnormality determination AI_2 using the evaluation data, and supplies the evaluation result, which is the model evaluation result, to the inference processing unit 23 of the inference unit 20.

[0057] The inference processing unit 23 determines whether to replace the model, i.e., whether the evaluation value of the re-learned abnormality determination (model) AI_2 is equal to or greater than the evaluation value of the current abnormality determination AI_1, based on the model evaluation result from the model evaluation unit 32 (step S92). If the evaluation value of the abnormality determination AI_2 is not equal to or greater than the evaluation value of the abnormality determination AI_1, the process returns to step S10 without replacing the abnormality determination AI_1, and the same process is repeated for the next captured data.

[0058] On the other hand, if the evaluation value of the abnormality determination AI_2 is equal to or greater than the evaluation value of the abnormality determination AI_1 (YES in step S92), the inference processing unit 23 replaces the abnormality determination AI_1 of the learning unit 10 with the abnormality determination AI_2 (step S94). Thereafter, the process returns to step S74, and the abnormality detection process described above is repeated using the replaced new abnormality determination AI_1 (abnormality determination AI_2).

[0059] Then, when the photographing is completed (YES in step S42), the inference processing unit 23 ends the processing.

[0060] According to the third embodiment described above, when deterioration (data drift) of the anomaly detection AI_1 during operation is detected in an AI model that performs visual inspection, the anomaly detection model can be automatically re-learned and replaced at an appropriate time.

[0061] In addition to this, the present invention is not limited to the above-mentioned embodiments and each modified example described with reference to the drawings, and it is possible to select and discard the configurations listed in the above-mentioned embodiments and each modified example, or to change them to other configurations as appropriate, as long as this does not deviate from the gist of the present invention. [Explanation of symbols]

[0062] 1. Anomaly detection system 10 Learning Department 11 Learning Data DB 12 Learning processing unit 20 Reasoning section 21 Camera 22 Shooting Data DB 23 Inference processing unit 24 Model status display UI 25 Abnormal Data DB 26 Data Drift DB 30 Re-learning section 31 Re-learning processing unit 32 Model Evaluation Section 33 Model evaluation result display UI 40 Outlier Trends 41 Locations where data drift occurs (locations where model degradation occurs) 42 Re-learning recommended 43 Relearn button 51, 61 Image to be analyzed 52, 62 Visualization results of abnormal areas 70, 71 Visualization results 72, 73 Numerical results 80 Network 81 Client Devices A, B, C thresholds

Claims

1. An anomaly detection system that performs visual inspection, a learning unit that creates a first AI model for determining an abnormality based on normal image data of the inspection object that has been photographed in advance; an inference unit that infers an abnormality of an inspection object from photographed data captured using the first AI model, and when degradation of the first AI model is detected, presents a model state including at least the degree of degradation of the first AI model and a location where degradation has occurred to a user; a re-learning unit that, when a re-learning instruction is received from the user, re-learns using the photographed data in which an abnormality is detected and the photographed data in which deterioration is detected to create a second AI model, evaluates the first AI model and the second AI model, and presents a model evaluation result to the user; An anomaly detection system characterized in that, when a replacement instruction is received from the user, the first AI model is replaced with the second AI model.

2. An anomaly detection system that performs visual inspection, a learning unit that creates a first AI model for determining an abnormality based on normal image data of the inspection object that has been photographed in advance; an inference unit that infers an abnormality in an inspection object from photographed data captured using the first AI model, and when degradation of the first AI model is detected, presents a model state including at least the degree of degradation of the first AI model and the location of degradation to a client device via a network; a re-learning unit that, when receiving a re-learning instruction from the client device via the network, re-learns using the photographed data in which an abnormality is detected and the photographed data in which deterioration is detected to create a second AI model, evaluates the first AI model and the second AI model, and presents a model evaluation result to the client device via the network; An anomaly detection system characterized in that, when a replacement instruction is received from the client device via the network, the first AI model is replaced with the second AI model.

3. An anomaly detection system that performs visual inspection, a learning unit that creates a first AI model for determining an abnormality based on normal image data of the inspection object that has been photographed in advance; an inference unit that infers an abnormality of an inspection object from photographed data captured using the first AI model, and instructs re-learning when degradation of the first AI model is detected; a re-learning unit that, when instructed to re-learn by the inference unit, performs re-learning using the photographed data in which an abnormality is detected and the photographed data in which deterioration is detected to create a second AI model, evaluates the first AI model and the second AI model, and outputs a model evaluation result; An anomaly detection system characterized in that, based on the model evaluation result, if the evaluation value of the second AI model is equal to or greater than the evaluation value of the first AI model, the first AI model is replaced with the second AI model.

4. An anomaly detection method for performing visual inspection, comprising: Creating a first AI model for determining an abnormality based on normal imaging data of the inspection object captured in advance; Inferring an abnormality in an inspection object from photographed data photographed using the first AI model, and when deterioration of the first AI model is detected, presenting to a user a model state including at least the degree of deterioration of the first AI model and the location of the deterioration; When a re-learning instruction is received from the user, re-learning is performed using the photographed data in which an abnormality is detected and the photographed data in which deterioration is detected to create a second AI model, the first AI model and the second AI model are evaluated, and a model evaluation result is presented to the user; and replacing the first AI model with the second AI model when the user instructs the model to be replaced.

5. An anomaly detection method for performing visual inspection, Creating a first AI model for determining an abnormality based on normal imaging data of the inspection object captured in advance; Inferring an abnormality in an inspection object from photographed data photographed using the first AI model, and when deterioration of the first AI model is detected, presenting a model state including at least the degree of deterioration of the first AI model and the location of the deterioration to a client device via a network; When a re-learning instruction is received from the client device via the network, re-learning is performed using the photographed data in which an abnormality is detected and the photographed data in which deterioration is detected to create a second AI model, the first AI model and the second AI model are evaluated, and a model evaluation result is presented to the client device via the network; and replacing the first AI model with the second AI model when a replacement instruction is received from the client device via the network.

6. An anomaly detection method for performing visual inspection, comprising: Creating a first AI model for determining an abnormality based on normal imaging data of the inspection object captured in advance; Inferring an abnormality of the inspection object from the photographed data using the first AI model, and instructing re-learning when deterioration of the first AI model is detected; When an instruction to re-learn is received, re-learning is performed based on the photographed data in which an abnormality is detected and the photographed data in which deterioration is detected to create a second AI model, and the first AI model and the second AI model are evaluated to output a model evaluation result; and if, based on the model evaluation result, the evaluation value of the second AI model is equal to or greater than the evaluation value of the first AI model, replacing the first AI model with the second AI model.

Images