Abnormality determination device, abnormality determination method, and abnormality determination program

Abstract

To improve accuracy of determining whether a state of the observed object is normal or abnormal to use a variational autoencoder.SOLUTION: An anomaly determination device is provided with a variational autoencoder trained using training data including observation data of a normal state of an observation target. A storage unit stores, as reference distribution, distribution of latent variables extracted from normal observation data of the observation target by an encoder of the variational autoencoder. A distribution acquisition unit acquires, as a distribution for determination, a distribution of latent variables extracted from observation data for state determination of an observation target by the encoder of the variational autoencoder. A determination unit determines whether the observation target is in an abnormal state based on the difference between the reference distribution calculated by a first calculation unit and the distribution for determination.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for determining anomalies in an observed object by utilizing a variational autoencoder (VAE). [Background technology]

[0002] Conventionally, there has been a device that utilizes a variational autoencoder (hereinafter referred to as a VAE (Variational Autoencoder)) to determine whether the state of an observed object is normal or abnormal (see, for example, Patent Document 1).

[0003] A VAE is a model that includes an encoder that calculates latent variables from observed data, which is input data, and a decoder that restores observed data from the latent variables. A VAE is generated by machine learning that mainly uses observed data from normal observations of the object to be observed. Here, the observed data restored from the latent variables is called restored data.

[0004] The device described in Patent Document 1 is configured to calculate the difference between observed data and restored data, and determine whether the state of the observed object is normal or abnormal based on this difference. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6740247 Summary of the Invention [Problem to be solved by the invention]

[0006] In many fields, there is a demand for improving the accuracy of determining whether the state of an observed object is normal or abnormal. VAE is attracting attention as a technology that can meet this demand.

[0007] An object of the present invention is to provide a technique for improving the accuracy of determining whether the state of an observed object is normal or abnormal by utilizing a variational autoencoder. [Means for solving the problem]

[0008] In order to achieve the above object, the abnormality determining device of the present invention is configured as follows.

[0009] The anomaly determination device includes a variational autoencoder (VAE) that is trained using training data including observation data of the object under normal conditions. The observation data is sensing data sensed by a sensor. For example, the observation data may be image data of a frame image capturing an image of the object under observation, sound collection data capturing the sound of the object under observation, or other sensing data.

[0010] The storage unit stores, as a reference distribution, the distribution of latent variables extracted by the encoder of the variational autoencoder from the observation data of the observation target in a normal state. This reference distribution may be, for example, one obtained during learning of the variational autoencoder, or one obtained after learning of the variational autoencoder. The reference distribution may be, for example, an n-dimensional normal distribution.

[0011] The distribution acquisition unit acquires, as a distribution for judgment, a distribution of latent variables extracted by the encoder of the variational autoencoder from the observation data for judging the state of the observation target. The first calculation unit calculates a difference between the reference distribution and the distribution for judgment. The first calculation unit calculates, for example, a degree of deviation of the distribution for judgment from the reference distribution as the difference between the reference distribution and the distribution for judgment. For example, the first calculation unit calculates, as the difference between the reference distribution and the distribution for judgment, a difference between the distribution mean value of the reference distribution and the distribution mean value of the distribution for judgment.

[0012] The determination unit determines whether the observation target is in an abnormal state based on the difference calculated by the first calculation unit.

[0013] With this configuration, it is possible to determine whether the state of the observed object is normal or abnormal without being affected by the accuracy of the decoder of the variational autoencoder.

[0014] Furthermore, when the observation data of the observation target is image data related to a frame image capturing the observation target, the frame image may be divided into each divided region. In this case, the storage unit may store a reference distribution for each divided region, the distribution acquisition unit may acquire a distribution for judgment for each divided region, the first calculation unit may calculate a difference between the reference distribution and the distribution for judgment for each divided region, and the judgment unit may judge whether the observation target is in an abnormal state for each divided region.

[0015] With this configuration, it is possible to determine whether or not an abnormal state exists for each region of the frame image (that is, for each part of the observation target imaged in that imaging region).

[0016] In addition, for example, the encoder of the variational autoencoder extracts latent variables from the observed data for state determination of the observed object, and the decoder However, the configuration may also include a second calculation unit that calculates the difference between the restored observation data restored from the observation data for determining the state of the observed object and the restored observation data, and the determination unit may be configured to determine whether the observed object is in an abnormal state based on the difference calculated by the first calculation unit and the difference calculated by the second calculation unit.

[0017] In this case, for example, the judgment unit judges that the observed object is not in an abnormal state based on the difference calculated by the first calculation unit, and judges that the observed object is not in an abnormal state based on the difference calculated by the second calculation unit.

[0018] With this configuration, it is possible to prevent the erroneous determination that the observation target is in an abnormal state when it is actually in a normal state. Effect of the Invention

[0019] According to the present invention, it is possible to improve the accuracy of determining whether the state of an observation target is normal or abnormal. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a schematic diagram illustrating a variational autoencoder. [Diagram 2] 2 is a block diagram showing the configuration of a main part of the abnormality determination device of this example. FIG. [Diagram 3] 4 is a flowchart showing the operation of the abnormality determination device of this example. [Figure 4] 10 is a block diagram showing a configuration of a main part of an abnormality determination device according to a first modified example. FIG. [Diagram 5] 6 is a flowchart showing the operation of the abnormality determination device of the first modification. [Figure 6] 10 is a flowchart showing the operation of the abnormality determination device of the second modification. [Figure 7] FIG. 7(A) is a diagram showing an example of observed data, and FIG. 7(B) is a diagram showing an example in which a frame image is divided into four regions. [Figure 8] FIG. 11 is a block diagram showing a configuration of a main part of an abnormality determination device according to a third modified example. [Figure 9] 13 is a flowchart showing the operation of the abnormality determination device of the third modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, an embodiment of the present invention will be described.

[0022] <1. Application Examples> FIG. 1 is a schematic diagram for explaining a variational autoencoder. As is well known, a variational autoencoder 1 (hereinafter referred to as VAE1 (Variational Autoencoder)) is composed of a computer. The VAE1 is a model generated by deep learning using training data including observation data of an observation target under normal conditions, and includes an encoder 2 and a decoder 3. The encoder 2 and the decoder 3 are neural networks. The encoder 2 extracts a latent variable z from the input observation data. The decoder 3 receives the latent variable z as an input and restores the observation data. Here, the observation data restored by the decoder 3 from the latent variable z is referred to as restored data.

[0023] The observation data is sensing data sensed by a sensor. For example, the observation data may be image data of a frame image of an observation target, sound data of a sound of the observation target, or other sensing data.

[0024] The observation data may be a time series of numerical values, or may be a frequency image generated by performing processing such as Fourier transform.

[0025] In deep learning, the encoder 2 learns the conditional probability q(z|x) of extracting a latent variable z from observed data, and the decoder 3 learns the conditional probability p(x|z) of restoring the observed data from the latent variable z.

[0026] In this example, the encoder 2 and the decoder 3 learn the conditional probability q(z|x) and the conditional probability p(x|z) so that the latent variable z has a specific distribution. The specific distribution may be, for example, an n-dimensional normal distribution, a distribution on the surface of a hypersphere with a radius r, or another distribution.

[0027] The anomaly determination device of this example includes a VAE 1. In addition, the anomaly determination device of this example stores, as a reference distribution, the distribution of latent variables z extracted by the encoder 2 of the VAE 1 from the observation data of the observation target when the target is normal. The reference distribution may be generated when the encoder 2 and the decoder 3 are made to learn the conditional probability q(z|x) and the conditional probability p(x|z), or may be generated by additionally processing the observation data of the observation target when the target is normal after the encoder 2 and the decoder 3 have learned the conditional probability q(z|x) and the conditional probability p(x|z).

[0028] The anomaly determination device of this example calculates the difference between the distribution of latent variables z extracted from the observation data input by the encoder 2 of the VAE 1 and a stored reference distribution. That is, the anomaly determination device of this example calculates the degree of deviation between the distribution of latent variables z extracted from the observation data input by the encoder 2 of the VAE 1 and the reference distribution. The anomaly determination device of this example determines whether the observation target is in a normal state or an abnormal state based on the calculated difference.

[0029] Therefore, the abnormality determination device of this example can determine whether the observed object is in a normal state or an abnormal state without being affected by the accuracy of the decoder 3 of the VAE 1. This improves the accuracy of determining whether the observed object is in a normal state or an abnormal state.

[0030] <2. Configuration example> 2 is a block diagram showing the configuration of the main parts of the abnormality determination device of this example. The abnormality determination device 10 of this example includes a control unit 11, an input section 12, an output section 13, and a storage section 14.

[0031] The control unit 11 of the abnormality determination device 10 is configured with a hardware CPU, a memory, and other electronic circuits. The control unit 11 has a VAE 1, a distribution acquisition unit 21, a first calculation unit 22, and a determination unit 23.

[0032] The input unit 12 receives observation data of an observation target to be determined as being in a normal state or an abnormal state. The observation data is sensing data sensed by a sensor, as described above. For example, the observation data may be image data of a frame image of an observation target, sound collection data of a sound of the observation target, or other sensing data.

[0033] The output unit 13 outputs the determination result (normal state or abnormal state) made by the control unit 11 regarding the observation target to a higher-level device (not shown).

[0034] The storage unit 14 stores the reference distribution of the latent variable z described above. The storage unit 14 is preferably configured with a non-volatile memory, but may be a volatile memory. The storage unit 14 may be a removable memory such as a memory card for the main body of the anomaly determination device 10. The storage unit 14 may utilize a part of the storage area of ​​a hard disk drive (HDD) or a solid state drive (SSD).

[0035] Next, the VAE 1, the distribution acquisition unit 21, the first calculation unit 22, and the determination unit 23 included in the control unit 11 will be described.

[0036] As described above, VAE1 is a model generated by deep learning using training data including observation data of the object under normal conditions.

[0037] The distribution acquisition unit 21 acquires the distribution of the latent variable z of the observation data extracted by the encoder 2 of the VAE 1 as a distribution for determination.

[0038] The first calculation unit 22 calculates the difference (degree of deviation) between the distribution for judgment acquired by the distribution acquisition unit 21 and the reference distribution stored in the storage unit 14. For example, the first calculation unit 22 calculates the distance between the distribution average value of the reference distribution and the distribution average value of the distribution for judgment as the difference. The first calculation unit 22 may also calculate the distance for each element of the latent variable z, and calculate the sum, average value, median, or the like as the difference (degree of deviation) between the distribution for judgment and the reference distribution. The first calculation unit 22 may also calculate the feature amount of the distribution for judgment and the feature amount of the reference distribution, and calculate the difference between these feature amounts as the difference between the distribution for judgment and the reference distribution.

[0039] The determination unit 23 determines whether the observation target is in a normal state or an abnormal state based on the difference between the reference distribution and the distribution for judgment calculated by the first calculation unit 22. For example, the determination unit 23 determines that the observation target is in a normal state if the difference between the reference distribution and the distribution for judgment is less than a predetermined judgment threshold. The determination unit 23 determines that the observation target is in an abnormal state if the difference between the reference distribution and the distribution for judgment is equal to or greater than the predetermined judgment threshold.

[0040] When the hardware CPU constituting the control unit 11 of the abnormality determination device 10 executes the abnormality determination program according to the present invention, it operates as the VAE 1, the distribution acquisition unit 21, the first calculation unit 22, and the determination unit 23. The memory has an area for expanding the abnormality determination program according to the present invention and an area for temporarily storing data generated during execution of the abnormality determination program. The control unit 11 may be an LSI that integrates the hardware CPU, memory, and the like. The hardware CPU is a computer that executes the abnormality determination method according to the present invention.

[0041] <3. Example of operation> 3 is a flowchart showing the operation of the abnormality determination device of this example. The abnormality determination device 10 has an observation data storage unit (not shown) that temporarily stores observation data of an observation target input to the input unit 12. The observation data may be input at regular time intervals, may be input at the timing when a predetermined specific event is detected, or may be input at other timings. The timing when a specific event is detected is, for example, the timing when a specific operation is performed, or the timing when a state change is detected by a sensor other than the sensor that senses the observation data (for example, the timing when a moving object is detected by an infrared sensor, or the timing when a change in the open / closed state of a door is detected by a door open / close sensor).

[0042] The abnormality determination device 10 selects observation data to be processed from among the observation data temporarily stored in the observation data storage unit (s1). For example, when the observation data is input to the input unit 12 at 100 msec intervals, the input observation data may be selected as the observation data to be processed in the order in which they were input, or the m-th (m>1) observation data from the previously selected observation data to be processed may be selected as the observation data to be processed. For example, when the observation data is a moving image of the observation target captured by an imaging device, the frame images of this moving image may be selected in order as the observation data to be processed, or the m-th frame image from the frame image previously selected as the observation data to be processed may be selected as the observation data to be processed.

[0043] The above-described method for selecting the observation data to be processed is merely an example, and other methods may be used for selection.

[0044] The anomaly determination device 10 encodes the observation data to be processed selected in s1 using the encoder 2 of the VAE 1, and extracts a latent variable z from the observation data (s2). At this time, the VAE 1 may or may not restore the observation data from the latent variable z extracted by the decoder 3.

[0045] The distribution acquisition unit 21 acquires the distribution of the latent variable z extracted from the observed data as a distribution for judgment (s3). The first calculation unit 22 calculates the difference between the distribution for judgment acquired in s3 and a reference distribution of the latent variable z stored in the storage unit 14 as the degree of deviation (s4). For example, the first calculation unit 22 calculates the distance between the distribution average value of the reference distribution and the distribution average value of the distribution for judgment as the difference.

[0046] The first calculation unit 22 may calculate the distance for each element of the latent variable z, and calculate the sum, average, median, or the like of the distance as the difference (degree of deviation) between the judgment distribution and the reference distribution.

[0047] The determination unit 23 determines whether the observation target is in a normal state or an abnormal state based on the difference between the reference distribution and the distribution for judgment calculated by the first calculation unit 22 (s5). For example, the determination unit 23 determines that the observation target is in a normal state if the difference between the reference distribution and the distribution for judgment is less than a predetermined judgment threshold. The determination unit 23 determines that the observation target is in an abnormal state if the difference between the reference distribution and the distribution for judgment is equal to or greater than the predetermined judgment threshold.

[0048] In the abnormality determination device 10, the output unit 13 outputs the determination result in s5 to the higher-level device (s6), and the process returns to s1. In s6, if the determination result is a normal state, the result is not output to the higher-level device, and only if the determination result is an abnormal state, the result may be output to the higher-level device.

[0049] In this way, the abnormality determination device 10 of this example can determine whether the observed object is in a normal state or an abnormal state without being affected by the accuracy of the decoder 3 of the VAE 1. This improves the accuracy of determining whether the observed object is in a normal state or an abnormal state.

[0050] <4. Modifications> Variation 1 Fig. 4 is a block diagram showing the configuration of the main parts of the abnormality determination device of Modification 1. The abnormality determination device 10A of Modification 1 differs from the abnormality determination device 10 shown in Fig. 2 in that it includes a second calculation unit 24 and a determination unit 25. Furthermore, the determination unit 25 is configured to determine whether the observation target is in a normal state or an abnormal state, but the process differs from that of the determination unit 23 of the abnormality determination device 10 shown in Fig. 2.

[0051] The second calculation unit 24 calculates the difference between the input observation data and the restored data restored from the latent variable z by the decoder 3 of the VAE1. The second calculation unit 24 may calculate the distance between each element of the input observation data and the corresponding element of the restored data, and calculate the sum of the distances as the difference between the input observation data and the restored data restored from the latent variable z by the decoder 3 of the VAE1. The second calculation unit 24 may also calculate the feature amount of the input observation data and the feature amount of the restored data, and calculate the difference between these feature amounts as the difference between the observation data and the restored data. The second calculation unit 24 may also calculate the likelihood correlated with the degree of anomaly in each dimension between the input observation data and the restored data restored from the latent variable z by the decoder 3 of the VAE1, as described in the above-mentioned Patent Document 1 (Japanese Patent No. 6740247), and calculate the score based on this likelihood as the difference between the observation data and the restored data, or may calculate it by other methods.

[0052] Furthermore, the determination unit 25 determines whether the observation target is in a normal state or an abnormal state based on the difference between the reference distribution and the determination distribution calculated by the first calculation unit 22 and the difference between the observation data and the restored data calculated by the second calculation unit 24. In this modification example 1, the determination unit 25 determines that the observation target is in a normal state when it determines that the observation target is in a normal state based on the difference between the reference distribution and the determination distribution calculated by the first calculation unit 22 and also determines that the observation target is in a normal state based on the difference between the observation data and the restored data calculated by the second calculation unit 24. In other words, the determination unit 25 determines that the observation target is in an abnormal state when it determines that the observation target is not in a normal state based on one or both of the difference between the reference distribution and the determination distribution calculated by the first calculation unit 22 and the difference between the observation data and the restored data calculated by the second calculation unit 24.

[0053] FIG. 5 is a flowchart showing the operation of the abnormality determination device of the first modification.

[0054] The abnormality determination device 10A of this modified example 1 selects observation data to be processed from the observation data temporarily stored in the observation data storage unit (s11). s11 is the same process as s1 in the above example. The abnormality determination device 10A encodes the observation data to be processed selected in s11 using the encoder 2 of the VAE1, and extracts a latent variable z of this observation data (s12). s12 is the same process as s2 in the above example. In addition, the abnormality determination device 10A decodes the latent variable z extracted in s12 using the decoder 3 of the VAE1, and generates restored data of this observation data (s13).

[0055] The distribution acquisition unit 21 acquires the distribution of the latent variable z extracted from the observation data as a distribution for judgment (s14). S14 is the same process as s3 in the above example. The first calculation unit 22 calculates the difference between the distribution for judgment acquired in s14 and a reference distribution of the latent variable z stored in the storage unit 14 as the degree of deviation (s15). S15 is the same process as s4 in the above example. The judgment unit 25 performs a first judgment process for judging whether the observation target is in a normal state or an abnormal state based on the difference between the reference distribution and the distribution for judgment calculated by the first calculation unit 22 (s16). S16 is the same process as s5 in the above example.

[0056] When abnormality determination device 10A determines in the first determination process of s16 that the observation target is in an abnormal state, output unit 13 outputs the abnormal state of the observation target to the higher-level device (s17, s22), and the process returns to s11.

[0057] Furthermore, when the abnormality determination device 10A determines that the observation target is in a normal state in the first determination process of s16, the second calculation unit 24 calculates the difference between the observation data and the restored data generated in the decoding process of s3 (s17, s18). The determination unit 25 performs a second determination process (s19) to determine whether the observation target is in a normal state or an abnormal state based on the difference between the observation data calculated in s18 and the restored data.

[0058] When abnormality determination device 10A determines in the second determination process of s19 that the observation target is in an abnormal state, output unit 13 outputs the abnormal state of the observation target to the higher-level device (s20, s22) and returns to s11. When abnormality determination device 10A determines in the second determination process of s19 that the observation target is in a normal state, output unit 13 outputs the normal state of the observation target to the higher-level device (s20, s21) and returns to s11.

[0059] In this manner, the abnormality determination device 10A of the first modification determines whether the observation target is in a normal state or an abnormal state based on the difference between the determination distribution of the latent variable z acquired from the observation data and the reference distribution of the latent variable z, and the difference between the observation data and the restored data. Therefore, it is possible to improve the accuracy of determining whether the observation target is in a normal state or an abnormal state.

[0060] In addition, in this modified example 1, if it is determined that the observation target is not in a normal state in the first determination process (determination based on the difference between the determination distribution of the latent variable z acquired from the observation data and the reference distribution of the latent variable z), the processes from s18 to s21 are not performed. Therefore, the processing load of the abnormality determination device 10A is also reduced.

[0061] In the above description, the first judgment process (judgment based on the difference between the judgment distribution of the latent variable z obtained from the observed data and the reference distribution of the latent variable z) is performed before the second judgment process (judgment based on the difference between the observed data and the restored data), but the second judgment process may be performed before the first judgment process. In this case, if the observed target is judged to be not in a normal state in the second judgment process executed first, the first judgment process may not be executed.

[0062] Similarly to the above example, abnormality determination device 10A may be configured such that output unit 13 does not execute the process of outputting the normal state of the observation target to a higher-level device (the process related to s21).

[0063] Variation 2 The abnormality determination device of the second modification has a similar configuration to the abnormality determination device 10A of the first modification shown in FIG.

[0064] 6 is a flowchart showing the operation of the abnormality determination device of Modification 2. The abnormality determination device 10A of this modification executes the processes of s11 to s16 described in Modification 1 (s31 to s36). After finishing the process of s36, the device executes the processes of s18 and s19 described in Modification 1 (s37, s38).

[0065] In the abnormality determination device 10A of this variant example 2, when it is determined in both the first determination process and the second determination process that the object to be observed is in a normal state, the output unit 13 outputs the normal state of the object to the higher-level device (s40) and the process returns to s31.

[0066] Furthermore, when the abnormality judgment device 10A of this variant example 2 judges that the observed object is in a normal state in either the first judgment process or the second judgment process, and judges that the observed object is in an abnormal state in the other, the output unit 13 outputs a caution-requiring state of the observed object to the higher-level device (s41, s42), and returns to s31.

[0067] In addition, in the abnormality judgment device 10A of this variant example 2, if it is judged that the object to be observed is in an abnormal state in both the first judgment process and the second judgment process, the output unit 13 outputs the abnormal state of the object to be observed to the higher-level device (s41, s43) and returns to s31.

[0068] The abnormality determination device 10A of the second modification can determine the state of the observation target in three stages, including a normal state, an abnormal state, and a state requiring attention. of State management can be performed more finely.

[0069] Similarly to the above example, the abnormality determination device 10A may be configured so that the output unit 13 does not execute the process of outputting the normal state of the observation target to the higher-level device (the process related to s40).

[0070] Variation 3 The abnormality determination device 10B of the third modification receives frame images of moving images captured by an imaging device such as a video camera as observation data. Fig. 7(A) is a diagram showing an example of the observation data. The observation data shown in Fig. 7 is an example in which the observation target is a road on which a vehicle travels.

[0071] The abnormality determination device 10B of the third modification determines that the road to be observed is in an abnormal state when, for example, there is an obstacle (a fallen object) on the road.

[0072] The abnormality determination device 10B of the third modification divides a frame image, which is observation data, into a plurality of predetermined divided regions. Fig. 7(B) is a diagram showing an example in which a frame image is divided into four divided regions A to D (two in the vertical direction and two in the horizontal direction).

[0073] Fig. 8 is a block diagram showing the configuration of the main part of the abnormality determination device of this modification 3. In Fig. 8, the same components as those in the abnormality determination device 10 shown in Fig. 2 are given the same reference numerals.

[0074] The abnormality determination device 10B of the third modification is different in that the control unit 11 includes a division unit 30 and that the control unit 11 includes VAEs 1a to 1d for the divided regions A to D shown in Fig. 7. In addition, the storage unit 14a stores a reference distribution of the latent variable z for each of the divided regions A to D.

[0075] The division unit 30 divides a frame image, which is observation data, into four divided areas A to D shown in FIG. 7(B).

[0076] VAE1a is a model generated by deep learning using division area A of the observation data when the object is normal. VAE1b is a model generated by deep learning using division area B of the observation data when the object is normal. VAE1c is a model generated by deep learning using division area C of the observation data when the object is normal. VAE1d is a model generated by deep learning using division area D of the observation data when the object is normal.

[0077] 9 is a flowchart showing the operation of the abnormality determination device of Modification 3. The abnormality determination device 10B selects observation data to be processed (s51). S51 is the same process as s1 and the like described above.

[0078] The division unit 30 divides the frame image, which is the selected observation data, into four divided regions A to D (s52). The abnormality determination device 10B selects one divided region from among the unprocessed divided regions A to D that have not been subjected to the processes from s54 onward (s53). The abnormality determination device 10B performs the same processes as s2 to s5 described above on the divided regions A to D selected in s53 (s54 to s57).

[0079] At this time, in the encoding process in s54, the abnormality determination device 10B uses the VAEs 1a to 1d corresponding to the divided region selected in s53. In addition, in the deviation degree calculation process in s56, the first calculation unit 22 reads out the reference distribution of the latent variable z corresponding to the divided region selected in s53 from the storage unit 14 and calculates the deviation degree.

[0080] When the abnormality determination device 10B completes the processes of s54 to s57 for the divided regions A to D selected in s53, it determines whether or not there are any unprocessed divided regions A to D (s58). If there are any unprocessed divided regions A to D, the abnormality determination device 10B returns to s53 and repeats the above-mentioned processes.

[0081] The abnormality determination device 10B determines in s58 that there are no unprocessed divided regions A to D, outputs the determination result in s57 for each of the divided regions A to D (s59), and returns to s51.

[0082] In this way, the abnormality determination device 10B of the third modification can output whether each divided area of ​​the frame image, which is the observation data, is in a normal state or an abnormal state. Therefore, the staff can be notified of the locations of the abnormal state in detail. In other words, the staff can respond quickly and appropriately to the detected abnormal state.

[0083] In addition, in this modification 3, an example has been described in which a frame image, which is observation data, is divided into four divided regions A to D, but the number of divided regions into which the frame image is divided is not limited to four and may be any number. Also, in this modification 3, an example has been described in which each divided region A to D is divided so as not to overlap with other adjacent divided regions A to D, but each divided region may be divided so as to partially overlap with other adjacent divided regions A to D.

[0084] Furthermore, when the observed data is a frame image, the position of the abnormal state may be acquired by using a latent variable z that holds position information through convolution processing such as CNN (Convolution Neural Network).

[0085] It should be noted that the present invention is not limited to the above-described embodiment as it is, and that the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

[0086] Furthermore, the correspondence between the configuration according to the present invention and the configuration according to the above-mentioned embodiment can be described as follows. <Additional Notes> In an abnormality determination device (10) including a variational autoencoder (1) trained using training data including observation data of an object under normal conditions, a storage unit (14) for storing, as a reference distribution, a distribution of a latent variable (z) extracted by the encoder (2) of the variational autoencoder (1) from the observation data of the object under normal conditions; a distribution acquisition unit (21) that acquires, as a judgment distribution, a distribution of a latent variable (z) extracted by the encoder (2) of the variational autoencoder (1) from the observation data for judging the state of the observation target; a first calculation unit (22) that calculates a difference between the reference distribution and the judgment distribution; An abnormality determination device (10) comprising: a determination unit (23) that determines whether or not the observation target is in an abnormal state based on the difference calculated by the first calculation unit (22). [Explanation of symbols]

[0087] 1, 1a~1d...variational autoencoder (VAE) 2. Encoder 3. Decoder 10, 10A, 10B…Abnormality determination device 11…Control unit 12...Input section 13...Output section 14, 14a...Storage section 21…Distribution acquisition part 22...First calculation section 23, 25...judgment section 24…Second calculation section 30...Divided part

Claims

1. 1. An anomaly detection device including a variational autoencoder trained using training data including observation data of an object under normal conditions, a storage unit that stores a distribution of latent variables extracted by an encoder of the variational autoencoder from normal observation data of the object to be observed as a reference distribution; a distribution acquisition unit that acquires, as a judgment distribution, a distribution of latent variables extracted by an encoder of the variational autoencoder from observation data for judging a state of the observation target; a first calculation unit that calculates a difference between the reference distribution and the judgment distribution; a determination unit that determines that the observation target is in an abnormal state if the difference calculated by the first calculation unit is equal to or greater than a determination threshold.

2. The abnormality determination device according to claim 1 , wherein the reference distribution is an n-dimensional normal distribution.

3. 3 . The abnormality determination device according to claim 1 , wherein the first calculation unit calculates a degree of deviation of the determination distribution from the reference distribution as a difference between the reference distribution and the determination distribution.

4. 4. The abnormality determination device according to claim 1, wherein the first calculation unit calculates a difference between a distribution average value of the reference distribution and a distribution average value of the determination distribution as a difference between the reference distribution and the determination distribution.

5. The observation data of the observation target is each divided area obtained by dividing a frame image of the observation target, the storage unit stores the reference distribution for each of the divided regions; the distribution acquisition unit acquires the judgment distribution for each of the divided regions; the first calculation unit calculates a difference between the reference distribution and the judgment distribution for each of the divided regions; 5. The abnormality determination device according to claim 1, wherein the determination unit determines whether the observation target is in an abnormal state for each of the divided regions.

6. a second calculation unit that calculates a difference between observed data for state determination of the object from which an encoder of the variational autoencoder extracts latent variables and reconstructed observed data reconstructed from the observed data for state determination of the object by a decoder of the variational autoencoder; The abnormality determination device according to any one of claims 1 to 5, wherein the determination unit determines whether the observation target is in an abnormal state based on a determination based on a difference calculated by the second calculation unit in addition to a determination based on the difference calculated by the first calculation unit.

7. 7. The abnormality determination device according to claim 6, wherein the determination unit determines that the observed object is not in an abnormal state when the determination based on the difference calculated by the first calculation unit determines that the observed object is not in an abnormal state and the determination based on the difference calculated by the second calculation unit determines that the observed object is not in an abnormal state.

8. a computer of an anomaly determination device including an encoder of a variational autoencoder trained using training data including observation data of an object under normal conditions, the encoder including a memory unit storing a distribution of latent variables extracted from the observation data of the object under normal conditions as a reference distribution, a distribution acquisition step in which an encoder of the variational autoencoder acquires a distribution of latent variables extracted from observation data for determining a state of the observation target as a determination distribution; a first calculation step of calculating a difference between the reference distribution and the judgment distribution; a determination step of determining that the observation target is in an abnormal state if the difference calculated in the first calculation step is equal to or greater than a determination threshold.

9. A computer of an anomaly determination device including a storage unit in which an encoder of a variational autoencoder trained using training data including observation data of an object under normal conditions stores a distribution of latent variables extracted from the observation data of the object under normal conditions as a reference distribution, a distribution acquisition step in which an encoder of the variational autoencoder acquires a distribution of latent variables extracted from observation data for determining a state of the observation target as a determination distribution; a first calculation step of calculating a difference between the reference distribution and the judgment distribution; and a determination step of determining that the observation target is in an abnormal state if the difference calculated in the first calculation step is equal to or greater than a determination threshold.

Images