Anomaly detection device and anomaly detection method

Abstract

The anomaly detection device (1) of the present invention comprises: an object feature value extraction unit (121), an object feature value conversion unit (122), a normal distance calculation unit (123), an anomaly distance calculation unit (124), and a determination unit (125). The object feature value extraction unit (121) generates a first feature vector set with feature vectors of a plurality of partial regions of the object image as a requirement. The object feature value conversion unit (122) converts the first feature vector set into a second feature vector set belonging to the same feature space as the normal feature vector set. The normal distance calculation unit (123) calculates the normal distance one by one according to the set requirements corresponding to the normal feature vector set and the second feature vector set. The anomaly distance calculation unit (124) calculates the anomaly distance one by one according to the set requirements corresponding to the anomaly feature vector set and the second feature vector set. The determination unit (125) determines whether the object is normal or abnormal based on the normal distance set and the anomaly distance set and the requirements of the set corresponding to the object image.

Description

Abnormal Detection Device and Abnormal Detection Method The present invention relates to an abnormal detection device and an abnormal detection method. In recent years, the appearance inspection of products has been evolving towards automation. For example, Patent Document 1 describes an abnormal determination method for automatically determining whether an object to be inspected is abnormal from the appearance of the object to be inspected. This method uses machine learning with a plurality of normal image data, and generates a normal object learning model that constructs a feature space of multivariate normal distribution with the feature values of the normal image data. According to the output results when known normal image data and known abnormal image data are input to the normal object learning model, the identification information for identifying whether the product is normal or abnormal is determined. Regarding the output result when the image data of the object to be inspected is input to the normal object learning model, the object to be inspected is identified as normal or abnormal according to the identification information. [Prior Art Documents] [Patent Documents] [Patent Document 1] Japanese Patent Application Laid-Open No. 2021-174456 [Problems to be Solved by the Invention] However, the conventional abnormal detection method performs abnormal detection of the object to be inspected based on integrating the feature values extracted from the entire image, so there is a problem that it is impossible to specify exactly which position is abnormal for the inspection object image of the object to be inspected. The present invention has been completed to solve the above problems, and the object is to obtain: an abnormal detection device that can specify exactly which position is abnormal for the inspection object image of the object to be inspected. [Technical Means for Solving the Problems] The abnormal detection device of the present invention includes: an inspection object feature value extraction unit, an inspection object feature value conversion unit, a normal distance calculation unit, and a determination unit; and the inspection object feature value extraction unit extracts respective first inspection object feature vectors from a plurality of partial regions divided from an inspection object image of an inspection object, and then generates a first inspection object feature vector set with the first inspection object feature vectors as elements; the inspection object feature value conversion unit converts the first inspection object feature vector set into a second inspection object feature vector set belonging to the same feature space as the feature vector set of the relevant normal state inspection object; the normal feature vector group set is a plurality of normal feature vector groups calculated for partial regions at the same position in all normal object images from a plurality of partial regions divided from a normal object image of a normal state inspection object, and collects these normal feature vectors of all partial regions; the normal distance calculation unit calculates normal distances one by one for the set elements corresponding between the normal feature vector group set and the second inspection object feature vector set from a plurality of partial regions divided from a normal object image of a normal state inspection object, and generates a normal distance set with the normal distances as elements, where the above normal feature vector group set is a plurality of normal feature vector groups calculated for partial regions at the same position in all normal object images and are collected for all partial regions; the abnormal distance calculation unit calculates abnormal distances one by one for the set elements corresponding between the abnormal feature vector group set and the second inspection object feature vector set from a plurality of partial regions divided from an abnormal object image of an abnormal state inspection object, and generates an abnormal distance set with the abnormal distances as elements, where the above abnormal feature vector group set is a plurality of abnormal feature vector groups calculated for partial regions at the same position in all abnormal object images and are collected for all partial regions; the determination unit determines whether it is normal or abnormal one by one for the set elements corresponding to the inspection object image according to the normal distance set and the abnormal distance set. [Advantages of the Invention] According to the present invention, for a set of inspection object feature vectors that require inspection object images of an inspected object to be divided into a plurality of partial regions, and a set of normal feature vector groups that require normal object images of an inspected object in a normal state to be divided into a plurality of partial regions, the normal distances are calculated one by one for the corresponding elements of the sets. For the set of inspection object feature vectors, the abnormal distances are calculated one by one for the corresponding elements of the set between the set of abnormal feature vector groups that require abnormal object images of an inspected object in an abnormal state to be divided into a plurality of partial regions. Based on the normal distances and the abnormal distances, for the elements of the set corresponding to the inspection object image, it is determined one by one whether the inspected object is normal or abnormal. The elements of the set correspond to the partial regions of the image. Since it is possible to determine whether there is an abnormality for each partial region of the image, the abnormality detection device of the present invention can specify which position of the inspection object image (partial region) has an abnormality. Embodiment 1. FIG. 1 shows a block diagram of a configuration example of the abnormality detection device 1 according to Embodiment 1. In FIG. 1, the abnormality detection device 1 acquires an inspection object image of an inspected object, divides the inspection object image into a plurality of partial regions, and then automatically determines whether the inspected object is normal or abnormal for each of the plurality of partial regions. Thereby, the abnormality detection device 1 can specify which position (partial region) of the inspection object image has an abnormality. When the abnormality detection device 1 determines the abnormality of the inspected object, it uses feature vectors representing the features of each partial region in the image. For example, in order for the abnormality detection device 1 to extract feature vectors from the image, a neural network that has been learned is used. The abnormality detection device 1 can be implemented using, for example, a tablet terminal, a smartphone, or a notebook personal computer (PC). The abnormality detection device 1 can also be connected to a camera device (not shown in FIG. 1) via wire or wireless. The camera device is not limited to an external device, and the abnormality detection device 1 can also have a built-in camera device. When performing an appearance inspection of the inspected object, the inspected object is photographed using this camera device. The abnormality detection device 1 can also have a component of a server capable of communicating with a terminal device. For example, the terminal device can be provided in the form of SaaS (Software as a Service), and an appearance inspection of the inspected object is performed. When performing the inspection in the SaaS form, the inspection application software may not be installed in the terminal device. The inspection application software is executed using the above-mentioned server, and the terminal device provides measurement result information on a general-purpose web browser. The inspection application software is stored in the memory unit provided in the server. Furthermore, the inspection application software may be installed in the terminal device. The terminal device with the inspection application software installed can perform an appearance inspection of the inspected object by executing the application software. The abnormality detection device 1 is implemented using a computer having an arithmetic unit and a memory unit. The memory unit is the memory unit 13 in FIG. 1, which is the memory device of the above computer. The memory unit 13 includes, for example, a storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or a memory 104 in FIG. 8B described later. In addition, the memory unit 13 only needs to be accessible by the abnormality detection device 1 and may also be provided outside the abnormality detection device 1. The arithmetic unit controls the overall operation of the abnormality detection device 1. The arithmetic unit includes: a learning processing unit 11 and an inspection processing unit 12. The functions of the learning processing unit 11 and the inspection processing unit 12 are realized by the arithmetic unit executing the inspection application software stored in the memory unit 13. The learning processing unit 11 includes: a normal eigenvalue extraction unit 111, a transformation processing matrix generation unit 112, a normal eigenvalue transformation unit 113, an abnormal eigenvalue extraction unit 114, and an abnormal eigenvalue transformation unit 115. The inspection processing unit 12 includes: an inspection object eigenvalue extraction unit 121, an inspection object eigenvalue transformation unit 122, a normal distance calculation unit 123, an abnormal distance calculation unit 124, and a determination unit 125. The learning processing unit 11 performs learning processing. During the learning processing, the learning processing unit 11 obtains: a plurality of normal object image groups of the inspection object in a normal state and one or more abnormal object image groups of the inspection object in an abnormal state, and generates a set of transformation processing matrices, a set of second normal image eigenvector groups, and a set of second abnormal image eigenvector groups. The set of transformation processing matrices is a set of transformation processing matrices collected by performing transformation based on the eigenvalues of the normal object image group distribution. The set of second normal image eigenvector groups is a set of second normal image eigenvectors representing the eigenvalues after the normal object image group is subjected to transformation processing. The set of second abnormal image eigenvector groups is a set of second abnormal image eigenvectors representing the eigenvalues after the abnormal object image group is subjected to transformation processing. The set of transformation processing matrices, the set of second normal image eigenvector groups, and the set of second abnormal image eigenvector groups are all stored in the memory unit 13 by the learning processing unit 11. The inspection processing unit 12 performs inspection processing that belongs to the appearance inspection of the inspection object. During the inspection processing, if the inspection object image of the inspection object is obtained by the inspection processing unit 12, the set of transformation processing matrices, the set of second normal image eigenvector groups, and the set of second abnormal image eigenvector groups stored in the memory unit 13 are used to determine whether the inspection object in the inspection object image is normal or abnormal. First, the learning process of Embodiment 1 will be described. The flowchart of the learning process performed by the anomaly detection device of Embodiment 1 shown in FIG. 2 represents the learning process performed by the learning processing unit 11. The normal eigenvalue extraction unit 111 extracts the first set of normal eigenvector groups, where the first set of normal eigenvector groups is a set of a plurality of normal eigenvector groups collected for all partial regions (step ST1). For example, the normal eigenvalue extraction unit 111 uses a large image dataset such as ImageNet to learn a neural network, and by inputting a set of normal object images to the eigenvalue extractor of the neural network, the first set of normal eigenvector groups is extracted from the set of normal object images. The schematic diagram of the classification task learning of the neural network B using the large image dataset A shown in FIG. 3. The neural network B includes: an eigenvalue extractor B1 and a classifier B2. In step ST1, the normal eigenvalue extraction unit 111 uses the eigenvalue extractor B1. The classifier B2 outputs the classification result C. Each element of the first set of normal eigenvector groups corresponds to dividing the normal object image into a plurality of partial regions, and the first normal eigenvector corresponds to each partial region. The first normal eigenvector of the first normal eigenvector group element is an eigenvector representing the characteristics of the partial region within one normal object image contained in the set of normal object images. The schematic diagram of the eigenvector extraction process performed by the eigenvalue extractor B1 of the neural network shown in FIG. 4. As shown in FIG. 4, the normal object image A1 is divided into a plurality of partial regions P. The eigenvalue extractor B1 is composed of, for example, a convolutional neural network (CNN) with one layer (L1), two layers (L2), and three layers (L3), and extracts the eigenvectors representing the characteristics of the partial regions from each layer. Then, the eigenvector V1 of the first layer of the partial region P, the eigenvector V2 of the second layer of the partial region P, and the eigenvector V3 of the third layer of the partial region P are integrated to form the first normal eigenvector VC corresponding to the partial region P. In addition, although the eigenvalue extractor B1 is illustrated in the case of a three-layer CNN, it is not limited to a three-layer CNN as long as it can extract the eigenvectors of the partial regions. The schematic diagram of the correspondence relationship between the partial regions P1 to PN within an image A1 and the set of eigenvectors shown in FIG. 5. As shown in FIG. 5, among the plurality of partial regions P1 to PN divided from the image A1, one eigenvector is calculated from one of the partial regions. The eigenvectors VC1 to VCN of the partial regions P1 to PN are collected to form the set of eigenvectors VG1. The schematic diagram of the correspondence relationship between the partial region P1 at the same position within a plurality of images A and the set of eigenvector groups VG2 shown in FIG. 6. As shown in FIG. 6, the set of images A contains a plurality of images of the same inspection object. The set of images A is, for example, a series of images taken from the same field of view angle for the inspection object products flowing sequentially on the production line. The eigenvalue extractor B1 extracts eigenvectors VC-1 to VC-m from partial regions P1 to Pm at the same position in the image group A. The integrated eigenvectors VC-1 to VC-m form an eigenvector group VG2. In FIGS. 5 and 6, each image of the image group A is divided into a plurality of partial regions, and the eigenvector group VG2 formed by the plurality of eigenvectors VC-1 to VC-m calculated for the partial regions at the same positions P1 to Pm of all the images is collected for all the partial regions P1 to PN to form an eigenvector group set. The conversion processing row and column generation unit 112 uses the first normal eigenvector group set to calculate the conversion processing rows and columns of the feature space for each partial region of the normal object image one by one, and generates a plurality of conversion processing row and column sets with the conversion processing rows and columns as elements (step ST2). For example, the conversion processing row and column generation unit 112 calculates the conversion processing row and column set by performing singular value decomposition (SVD) on each first normal eigenvector group belonging to the set elements of the first normal eigenvector group set. The conversion processing rows and columns of the conversion processing row and column set elements are calculated for each partial region one by one. When the number of elements of the first normal eigenvector is P and the number of elements of the first normal eigenvector group is N, P singular values and a singular value decomposition matrix of P rows and P columns can be obtained when performing singular value decomposition. The conversion processing row and column generation unit 112 extracts the top K from the P singular values, and extracts the columns corresponding to the top K singular values from the singular value decomposition matrix to form a matrix of P rows and K columns, and calculates it as the conversion processing row and column. Usually, P is a value around one hundred to several hundred. In contrast, K is a value around several tens. The conversion processing row and column set is stored in the storage unit 13. The normal eigenvalue conversion unit 113 uses the conversion processing row and column set to convert to the normal eigenvector set used when calculating the normal distance in the same feature space as the first normal eigenvector group set (step ST3). For example, the normal eigenvalue conversion unit 113 uses the conversion processing row and column set to convert each first normal eigenvector group set according to the set elements one by one, and calculates the second normal eigenvector group set. The second normal eigenvector group set is stored in the storage unit 13. The conversion processing matrix is calculated for each partial area one by one. The conversion processing matrices of all partial areas within an image are collected to form a conversion processing matrix set. In addition, the normal eigenvector is calculated once for one partial area of an image. The normal eigenvectors corresponding to the partial areas at the same position of a plurality of images are used as elements to form a normal eigenvector group. Furthermore, the normal eigenvector groups are collected for all partial areas to form a normal eigenvector group set. The normal eigenvalue conversion unit 113 performs a feature space conversion from an image number × P-dimensional vector to an image number × K-dimensional vector by multiplying the conversion processing matrix corresponding to one partial area in the conversion processing matrix set by the normal eigenvector group corresponding to the partial area at the same position. The abnormal eigenvalue extraction unit 114 extracts a first abnormal eigenvector group set, where the first abnormal eigenvector group set is a set in which a plurality of abnormal eigenvector groups are collected for all partial areas (step ST4). For example, the abnormal eigenvalue extraction unit 114 extracts the first abnormal eigenvector group set by inputting an abnormal object image group to the eigenvalue extractor B1 of the learned classification neural network. The abnormal eigenvalue conversion unit 115 uses the conversion processing matrix set to convert the abnormal eigenvector set used when calculating the abnormal distance belonging to the same feature space as the first abnormal eigenvector group set (step ST5). For example, the abnormal eigenvalue conversion unit 115 uses the conversion processing matrix set to convert the first abnormal eigenvector group set one by one according to the set elements to calculate a second abnormal eigenvector group set. The second abnormal eigenvector group set is stored in the storage unit 13. In addition, by also using the conversion processing matrix calculated using the normal eigenvector for the conversion of the abnormal eigenvector, both the normal eigenvector and the abnormal eigenvector are converted to the same feature space. Next, the abnormal detection method of Embodiment 1 will be described. FIG. 7 shows a flowchart of the inspection process performed by the abnormal detection device 1 of Embodiment 1, which is the inspection process performed by the inspection processing unit 12. This inspection process is the abnormal detection method of Embodiment 1. The inspection object eigenvalue extraction unit 121 extracts a feature vector from the inspection object image and generates a first inspection object feature vector set based on it (step ST1A). For example, the inspection object eigenvalue extraction unit 121 uses the eigenvalue extractor B1 to calculate the first inspection object feature vector set from the inspection object image captured by the inspection object. The inspection object feature value conversion unit 122 converts the first inspection object feature vector set into a second feature vector set (step ST2A). For example, the inspection object feature value conversion unit 122 uses the conversion processing matrix set read from the memory unit 13 to perform conversion on the first inspection object feature vector set one by one according to the set requirements, and calculates the second inspection object feature vector set. The normal distance calculation unit 123 calculates the normal distance one by one according to the corresponding set requirements between the normal feature vector group set and the second inspection object feature vector set, and generates a normal distance set with the normal distance as the requirement (step ST3A). For example, the normal distance calculation unit 123 calculates the normal distance set one by one according to the set requirements for the distance between the second normal feature vector group set read from the memory unit 13 and the second inspection object feature vector set. When calculating the normal distance, for example, the average vector and covariance matrix of the vectors are calculated from the second normal feature vector group, and then the Mahalanobis distance between the second inspection object feature vector is calculated. The abnormal distance calculation unit 124 calculates the abnormal distance one by one according to the corresponding set requirements between the abnormal feature vector group set and the second inspection object feature vector set, and generates an abnormal distance set with the abnormal distance as the requirement (step ST4A). For example, the abnormal distance calculation unit 124 calculates the distance between the second abnormal feature vector group set read from the memory unit 13 and the second inspection object feature vector set one by one according to the set requirements, and calculates the abnormal distance set. When calculating the abnormal distance, the inner product of the lower L-dimensional requirements of the second inspection object feature vector and the lower L-dimensional requirements of each second abnormal feature vector is calculated, and the value obtained by multiplying the inner product by "-1" is set as the "abnormal distance". Also, among the abnormal distances between each second abnormal feature vector, the smallest abnormal distance is set as the abnormal distance of the corresponding second inspection object feature vector. By performing the inner product operation limited to the "lower L-dimension" to obtain the abnormal distance, it is experimentally confirmed that the correlation between the actual abnormal state and the abnormal distance is higher than that in the case without limitation. The determination unit 125 determines whether it is normal or abnormal one by one according to the normal distance set and the abnormal distance set for the requirements of the set corresponding to the inspection object image (step ST5A). For example, the determination unit 125 calculates the determination result set of normal and abnormal by performing conditional determination on the normal distance set and the abnormal distance set one by one according to the set requirements. For example, the determination condition is to determine that it belongs to "normal" only when the set requirements of the two conditions that the normal distance is below the first threshold value and the abnormal distance satisfies above the second threshold value, and determine that it belongs to "abnormal" for the set requirements other than this. By learning and inspecting the image as described above, the following effects can be obtained. Since a learned classification neural network is used for eigenvalue extraction, it is not necessary to relearn the neural network for each object one by one (Effect 1). Since each partial area in the image is determined to be normal or abnormal one by one, the abnormal position in the image can be specified (Effect 2). By using a conversion processing matrix that converts the P-dimensional first normal eigenvalue into a K-dimensional second normal eigenvalue with P≥K by singular value decomposition, the dimension can be reduced, thereby shortening the time required for appearance inspection (Effect 3). By using a conversion processing matrix that converts the P-dimensional first normal eigenvalue into a K-dimensional second normal eigenvalue with P≥K by singular value decomposition, eigenvalues that can well represent the characteristics of a normal object image can be extracted, thereby improving the accuracy of appearance inspection (Effect 4). Since the K-dimensional second eigenvalue that can well represent the characteristics of a normal object image is used to calculate the normal distance, the determination of whether it is normal can be accurately performed (Effect 5). Since the conversion processing matrix extracts the K-dimension that can well represent the characteristics of a normal object image, for an abnormal object image, the upper K-dimension represents the characteristics similar to a normal object image, and the lower part represents the characteristics not similar to a normal object image. When calculating the abnormal distance, since the lower L-dimension of the K-dimensional second eigenvector is used, an inspection object image similar to the abnormal object image can be accurately searched (Effect 6). Next, the hardware configuration for implementing the functions of the abnormality detection device 1 will be described. The functions of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13 provided in the abnormality detection device 1 are implemented by a processing circuit. That is, the abnormality detection device 1 is provided with a processing circuit for executing each process from step ST1A to step ST5A shown in FIG. 7. The processing circuit can be dedicated hardware or a CPU (Central Processing Unit) that executes a program stored in a memory. FIG. 8A shows a hardware configuration block diagram for implementing the functions of the abnormality detection device 1. FIG. 8B shows a hardware configuration block diagram for the software that implements the functions of the abnormality detection device 1 and the hardware that executes the software. In FIGS. 8A and 8B, the input interface 100 is an interface that relays the output of image information from an external device to the abnormality detection device 1. The output interface 101 is an interface that relays the output of the abnormality inspection result from the abnormality detection device 1 to the outside. When dealing with the case of the processing circuit 102 of the dedicated hardware shown in FIG. 8A, the processing circuit 102 can be, for example: a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13 provided in the abnormality detection device 1 can be realized by respective processing circuits, or the functions thereof can be integrated and realized by a single processing circuit. When the processing circuit is the processor 103 shown in FIG. 8B, the functions of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13 provided in the abnormality detection device 1 are realized by software, firmware, or a combination of software and firmware. In addition, the software or firmware is described in the form of a program and stored in the memory 104. The memory 104 is, for example, the memory unit 13 shown in FIG. 1. By reading and executing the application software program for abnormality detection stored in the memory 104 by the processor 103, the functions of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13 provided in the abnormality detection device 1 are realized. For example, when the abnormality detection device 1 is executed using the processor 103, there is a memory 104 for storing a program for processing the steps ST1A to ST5A shown in FIG. 7. This program causes the computer to execute the processing sequence or method of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13. The memory 104 can also be a memory medium that stores a program for enabling the computer to have the functions of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13 and is readable by the computer. The memory 104 can be, for example: non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), or magnetic disks, floppy disks, video optical discs, CDs, mini discs, DVDs, etc. The functions of the learning processing unit 11, the inspection processing unit 12, and the memory unit 13 provided in the abnormality detection device 1 may be realized in part by dedicated hardware, and the remaining parts may be realized by software or firmware. For example, the memory unit 13 may use the processing circuit 102 of dedicated hardware to realize this function, and the learning processing unit 11 and the inspection processing unit 12 may realize this function by the processor 103 reading and executing the program stored in the memory 104. Accordingly, the above functions can be realized by using hardware, software, firmware, or a combination of these. So far, the case where the abnormality detection device 1 includes the learning processing unit 11 and the inspection processing unit 12 has been illustrated, but the abnormality detection device 1 may also include only the inspection processing unit 12. The learning processing unit 11 and the memory unit 13 may also be devices provided separately from the abnormality detection device 1. In this case, the abnormality detection device 1 accesses these devices to obtain the learning results. As described above, the abnormality detection device 1 of Embodiment 1 includes: an inspection object feature value extraction unit 121, an inspection object feature value conversion unit 122, a normal distance calculation unit 123, an abnormal distance calculation unit 124, and a determination unit 125; the inspection object feature value extraction unit 121 generates a first inspection object feature vector set that requires a first inspection object feature vector extracted from the inspection object image. The inspection object feature value conversion unit 122 converts the first inspection object feature vector set into a second inspection object feature vector set in the same feature space as the feature vector set for the inspection object in the normal state. The normal distance calculation unit 123 calculates the normal distance one by one for the set requirements corresponding between the normal feature vector group set and the second inspection object feature vector set, and generates a normal distance set that requires the normal distance. The abnormal distance calculation unit 124 calculates the abnormal distance one by one for the set requirements corresponding between the abnormal feature vector group set and the second inspection object feature vector set, and generates an abnormal distance set that requires the abnormal distance. The determination unit 125 determines whether it is normal or abnormal one by one for the set requirements corresponding to the inspection object image based on the normal distance set and the abnormal distance set. Since the set requirements correspond to partial regions of the image, the abnormality detection device 1 can determine which position of the inspection object image for photographing the inspection object has an abnormality because it can determine whether there is an abnormality one by one for the partial regions of the image. The abnormality detection device 1 according to Embodiment 1 includes: a normal eigenvalue extraction unit 111, a conversion processing matrix generation unit 112, a normal eigenvalue conversion unit 113, an abnormal eigenvalue extraction unit 114, and an abnormal eigenvalue conversion unit 115. The normal eigenvalue extraction unit 111 extracts a first set of normal eigenvector groups from a plurality of normal object images. The conversion processing matrix generation unit 112 uses the first set of normal eigenvector groups to calculate the conversion processing matrices of the feature space one by one for each partial region of the normal object image, and generates a plurality of sets of conversion processing matrices with the conversion processing matrices as elements. The normal eigenvalue conversion unit 113 uses the set of conversion processing matrices to convert them into a set of normal eigenvectors that belong to the same feature space as the first set of normal eigenvector groups and are used when calculating the normal distance. The abnormal eigenvalue extraction unit 114 obtains a plurality of sets of abnormal object image groups with abnormal object images as elements, and extracts a first set of abnormal eigenvector groups when dividing the abnormal object images into a plurality of partial regions. Among them, the above-mentioned first set of abnormal eigenvector groups is a plurality of abnormal eigenvector groups calculated for the partial regions at the same positions of all abnormal object images, and are collected for all partial regions. The abnormal eigenvalue conversion unit 115 uses the set of conversion processing matrices to convert them into a set of abnormal eigenvectors that belong to the same feature space as the first set of abnormal eigenvector groups and are used when calculating the abnormal distance. Thereby, the abnormality detection device 1 can learn the eigenvalue extraction and conversion processing matrix calculation necessary for the abnormality detection of the inspection object. In the abnormality detection device 1 according to Embodiment 1, the inspection object eigenvalue extraction unit 121 is a learning completed neural network that outputs a first set of inspection object eigenvectors when an image is input. By using the above-mentioned learning completed neural network, the inspection object eigenvalue extraction unit 121 can correctly extract the first set of inspection object eigenvectors from the image. In the abnormality detection device 1 according to Embodiment 1, either one or both of the normal eigenvalue extraction unit 111 and the abnormal eigenvalue extraction unit 114 are learning completed neural networks that, when an image is input, divide the image into a plurality of partial regions and output a set of eigenvectors representing the regional features one by one. By using the learning completed neural network, the normal eigenvalue extraction unit 111 can correctly extract a set of normal eigenvectors from the normal image. By using the learning completed neural network, the abnormal eigenvalue extraction unit 114 can correctly extract a set of abnormal eigenvectors from the abnormal image. In the abnormality detection device 1 according to Embodiment 1, the inspection object eigenvalue conversion unit 122 uses the set of conversion processing matrices to convert the first set of eigenvectors into a second set of eigenvectors that belong to the same feature space as the set of eigenvectors of the inspection object in the normal state for each partial region one by one. Thereby, the inspection object eigenvalue conversion unit 122 can correctly convert the first set of eigenvectors into the second set of eigenvectors for each partial region one by one. In the abnormality detection device 1 according to Embodiment 1, the conversion processing matrix generation unit 112 calculates a conversion processing matrix by performing singular value decomposition on a first normal eigenvector group belonging to the requirements of the first normal eigenvector group set. By using this conversion processing matrix, the conversion processing matrix generation unit 112 can correctly convert the first inspection target eigenvector set into a second inspection target eigenvector set belonging to the same feature space as the eigenvector set of the normal state inspection target object. In the abnormality detection device 1 according to Embodiment 1, the normal distance calculation unit 123 calculates the Mahalanobis distance or the Euclidean distance used as the normal distance one by one according to the corresponding set requirements between the normal eigenvector group set and the second eigenvector set. Thereby, the normal distance calculation unit 123 can correctly calculate the normal distance. In the abnormality detection device 1 according to Embodiment 1, the abnormal distance calculation unit 124 calculates the inner product of the lower L - dimensional requirements of the second inspection target eigenvector belonging to the requirements of the second inspection target eigenvector set and the lower L - dimensional requirements of the abnormal eigenvector belonging to the requirements of the abnormal eigenvector group set, and sets the value obtained by multiplying the inner product value by - 1 as the abnormal distance. Thereby, the abnormal distance calculation unit 124 can correctly calculate the abnormal distance. In the abnormality detection device 1 according to Embodiment 1, when the normal distance is below the first threshold value and the abnormal distance is above the second threshold value, the determination unit 125 determines that the inspection target object is in a normal state. Thereby, the determination unit 125 can determine whether the inspection target object is in a normal state one by one according to the partial regions of the image. In the abnormality detection device 1 according to Embodiment 1, when the normal distance is above the first threshold value and the abnormal distance is below the second threshold value, the determination unit 125 determines that the inspection target object is in an abnormal state. Thereby, the determination unit 125 can determine whether the inspection target object is in an abnormal state one by one according to the partial regions of the image. The abnormality detection method of Embodiment 1 includes the following steps: Step ST1A in which the feature vector extraction unit 121 of the inspection object extracts the feature vector from the inspection object image and generates the first inspection object feature vector set based on the extracted feature vector; Step ST2A in which the inspection object feature value conversion unit 122 converts the first inspection object feature vector set into the second feature vector set; Step ST3A in which the normal distance calculation unit 123 calculates the normal distance one by one according to the corresponding set elements between the normal feature vector group set and the second inspection object feature vector set, and generates the normal distance set; Step ST4A in which the abnormal distance calculation unit 124 calculates the abnormal distance one by one according to the corresponding set elements between the abnormal feature vector group set and the second inspection object feature vector set, and generates the abnormal distance set; and Step ST5A in which the determination unit 125 determines whether it is normal or abnormal one by one according to the normal distance set and the abnormal distance set for the set elements corresponding to the inspection object image. By executing this method, the abnormality detection device 1 can specify which position of the inspection object image is abnormal. The abnormality detection method of Embodiment 1 includes the following steps: Step ST1 in which the normal feature value extraction unit 111 extracts the first normal feature vector group set from a plurality of normal feature vector groups collected from all partial regions; Step ST2 in which the conversion processing matrix generation unit 112 uses the first normal feature vector group set to calculate the conversion processing matrix of the feature space one by one according to the partial regions of the normal object image, and generates a plurality of conversion processing matrix sets based on the conversion processing matrix; Step ST3 in which the normal feature value conversion unit 113 uses the conversion processing matrix set to convert it into the normal feature vector set used when calculating the normal distance in the same feature space as the first normal feature vector group set; Step ST4 in which the abnormal feature value extraction unit 114 extracts the first abnormal feature vector group set from a plurality of abnormal feature vector groups collected from all partial regions; and Step ST5 in which the abnormal feature value conversion unit 115 uses the conversion processing matrix set to convert it into the abnormal feature vector set used when calculating the abnormal distance in the same feature space as the first abnormal feature vector group set. By executing this method, the abnormality detection device 1 can learn the feature vector extraction and conversion processing matrix calculation necessary for the abnormality detection of the inspection object. In addition, the constituent elements of the embodiment can be arbitrarily changed, or any constituent elements of the embodiment can be omitted. [Industrial Applicability] The abnormality detection device of the present invention can be used, for example, for the appearance inspection of products on a manufacturing production line. 1: Abnormal detection device 11: Learning processing unit 12: Inspection processing unit 13: Memory unit 100: Input interface 101: Output interface 102: Processing circuit 103: Processor 104: Memory 111: Normal eigenvalue extraction unit 112: Conversion processing matrix generation unit 113: Normal eigenvalue conversion unit 114: Abnormal eigenvalue extraction unit 115: Abnormal eigenvalue conversion unit 121: Feature value extraction unit for inspection object 122: Feature value conversion unit for inspection object 123: Normal distance calculation unit 124: Abnormal distance calculation unit 125: Judgment unit B: Neural network B1: Eigenvalue extractor B2: Classifier C: Classification result P: Partial area VC: First normal eigenvector VC-1~VC-m: Eigenvectors VC1~VCN: Eigenvectors of partial areas P1~PN VG1: Set of eigenvectors VG2: Group of eigenvectors FIG. 1 is a block diagram showing a configuration example of the abnormal detection device according to Embodiment 1. FIG. 2 is a flowchart of the learning process performed by the abnormal detection device according to Embodiment 1. FIG. 3 is a schematic diagram showing an overview of the learning of the neural network classification task using a large image dataset. FIG. 4 is a schematic diagram of the eigenvector extraction process performed by the eigenvalue extractor of the neural network. FIG. 5 is a schematic diagram showing the correspondence between a partial area in an image and a set of eigenvectors. FIG. 6 is a schematic diagram showing the correspondence between partial areas at the same position in a plurality of images and a group of eigenvectors. FIG. 7 is a flowchart of the inspection process performed by the abnormal detection device according to Embodiment 1. In FIG. 8, FIGS. 8A and 8B are block diagrams of the hardware configuration for implementing the functions of the abnormal detection device according to Embodiment 1. 1: Abnormal detection device 11: Learning processing unit 12: Inspection processing unit 13: Memory unit 111: Normal eigenvalue extraction unit 112: Conversion processing matrix generation unit 113: Normal eigenvalue conversion unit 114: Abnormal eigenvalue extraction unit 115: Abnormal eigenvalue conversion unit 121: Feature value extraction unit for inspection object 122: Feature value conversion unit for inspection object 123: Normal distance calculation unit 124: Abnormal distance calculation unit 125: Judgment unit

Claims

1. An anomaly detection device, comprising: an object feature value extraction unit, which extracts first object feature vectors from a plurality of partial regions divided from an image of the object being inspected, and generates a first object feature vector set based on the first object feature vectors; an object feature value conversion unit, which converts the first object feature vector set into a second object feature vector set belonging to the same feature space as the feature vector set of an object in a normal state; and a normal distance calculation unit, which calculates normal distances from a plurality of partial regions divided from an image of an object in a normal state, based on a set of normal feature vector groups and a set of corresponding elements with the second object feature vector set, and generates a normal distance set based on the normal distance element, wherein... The aforementioned normal feature vector set is a plurality of normal feature vector sets calculated for partial regions at the same location in all the aforementioned normal object images, and collected for all partial regions; the abnormal distance calculation unit divides the abnormal object image of the abnormal state inspection object into a plurality of partial regions, and calculates the abnormal distance one by one according to the abnormal feature vector set and the corresponding set requirements between the second inspection object feature vector set, generating an abnormal distance set with abnormal distance as a requirement, wherein the aforementioned abnormal feature vector set is a plurality of abnormal feature vector sets calculated for partial regions at the same location in all the aforementioned abnormal object images, and collected for all partial regions; the determination unit determines whether it is normal or abnormal one by one according to the aforementioned normal distance set and the aforementioned abnormal distance set, based on the requirements of the set corresponding to the inspection object image; The normal feature value extraction unit acquires a set of normal object images consisting of a plurality of normal object images, and extracts a first set of normal feature vectors from a plurality of partial regions divided from the normal object images. The first set of normal feature vectors is a collection of a plurality of normal feature vectors calculated for partial regions at the same location in all the normal object images, covering all partial regions. The transformation processing column generation unit uses the first set of normal feature vectors to calculate transformation processing columns of the feature space for each partial region of the normal object images, generating a set of transformation processing columns consisting of a plurality of such transformation processing columns. The normal feature value conversion unit uses the transformation processing column set to convert it into a set of normal feature vectors belonging to the same feature space as the first set of normal feature vectors and used when calculating normal distances. The abnormal feature value extraction unit acquires an abnormal object image group consisting of a plurality of the aforementioned abnormal object images, and extracts a first abnormal feature vector set from a plurality of partial regions divided from the aforementioned abnormal object images. The first abnormal feature vector set is a plurality of abnormal feature vector sets calculated for partial regions at the same location in all the aforementioned abnormal object images, and is collected for all partial regions. The abnormal feature value conversion unit uses the aforementioned conversion processing column set to convert it into an abnormal feature vector set that belongs to the same feature space as the aforementioned first abnormal feature vector set and is used when calculating the abnormal distance.

2. As in request item 1, the anomaly detection device, wherein, If the aforementioned feature value extraction system of the inspection object is input into an image, it will output the learning class neural network of the first set of feature vectors of the inspection object.

3. As in request item 1, the anomaly detection device, wherein, The above-mentioned normal feature value extraction unit and the above-mentioned abnormal feature value extraction unit are, if they are input images, a learning neural network that outputs a set of feature vectors representing the features of a region according to a plurality of partial regions divided by the image.

4. As in request item 1, the anomaly detection device, wherein, The aforementioned inspection object feature value conversion unit uses a conversion processing row and column set to convert the aforementioned first inspection object feature vector set into a second inspection object feature vector set belonging to the same feature space as the feature vector set of the inspection object in the normal state, according to a partial region.

5. The anomaly detection device as described in request item 1, wherein, The transformation processing column generation unit calculates the transformation processing column by performing singular value decomposition on the first normal eigenvector group of the first normal eigenvector group set requirement.

6. An anomaly detection device as described in any of requests 1 to 5, wherein, The aforementioned normal distance calculation unit calculates the Mahalanobis distance or Euclidean distance used as the normal distance one by one, based on the corresponding set requirements between the aforementioned normal feature vector set and the aforementioned second inspection object feature vector set.

7. An anomaly detection device as described in any of requests 1 to 5, wherein, The above-mentioned abnormal distance calculation unit calculates the inner product of the lower L-dimensional element of the feature vector of the second inspection object that belongs to the feature vector set of the second inspection object and the lower L-dimensional element of the feature vector of the abnormal feature vector that belongs to the set of abnormal feature vectors, and then multiplies the inner product value by -1 to set it as the abnormal distance.

8. An anomaly detection device as described in any of requests 1 to 5, wherein, The aforementioned determination unit determines that the object under inspection is in a normal state when the normal distance is below the first threshold and the abnormal distance is above the second threshold.

9. An anomaly detection device as described in any of requests 1 to 5, wherein, The aforementioned determination unit determines that the object under inspection is in an abnormal state when the normal distance reaches or exceeds the first threshold and the abnormal distance is below the second threshold.

10. An anomaly detection method, which utilizes an anomaly detection device, includes the following steps: First, an inspection object feature vector is extracted from a plurality of partial regions divided from an image of the inspection object by an inspection object feature value extraction unit, and a first inspection object feature vector set is generated, with the first inspection object feature vector as a requirement; Second, the first inspection object feature vector set is converted into a second inspection object feature vector set belonging to the same feature space as a feature vector set of a related normal state inspection object by an inspection object feature value conversion unit; Third, a normal distance calculation unit calculates normal distances for each of the plurality of partial regions divided from the normal object image of the normal state inspection object, based on a set of normal feature vector groups and a set of corresponding elements with the second inspection object feature vector set, generating a normal distance set, with normal distance as a requirement, wherein... The aforementioned normal feature vector set is a collection of multiple normal feature vector sets calculated for partial regions at the same location in all the aforementioned normal object images, covering all partial regions; the abnormal distance calculation unit divides the abnormal object image from the abnormal object image of the abnormal state inspection object into multiple partial regions, and calculates the abnormal distance one by one according to the abnormal feature vector set and the corresponding set requirements between the second inspection object feature vector set, generating an abnormal distance set with abnormal distance as a requirement, wherein the aforementioned abnormal feature vector set is a collection of multiple abnormal feature vector sets calculated for partial regions at the same location in all the abnormal object images, covering all partial regions; the determination unit determines one by one whether it is normal or abnormal based on the aforementioned normal distance set and the aforementioned abnormal distance set, and according to the requirements of the set corresponding to the inspection object image; The steps include: obtaining a set of normal object images (comprising a plurality of normal object images) from a normal feature value extraction unit; extracting a first set of normal feature vectors from a plurality of partial regions divided from the normal object images; wherein the first set of normal feature vectors is a collection of a plurality of normal feature vectors calculated for partial regions at the same location in all the normal object images; generating a set of conversion processing columns (comprising a plurality of such conversion processing columns) using the first set of normal feature vectors; and converting the conversion processing columns into a set of normal feature vectors that belongs to the same feature space as the first set of normal feature vectors and is used when calculating normal distances. The steps include: obtaining an anomaly image group consisting of a plurality of the aforementioned anomaly images by the anomaly feature value extraction unit; extracting a first set of anomaly feature vector groups from a plurality of partial regions divided from the aforementioned anomaly images; wherein the first set of anomaly feature vector groups is a plurality of anomaly feature vector groups calculated for partial regions at the same location in all the aforementioned anomaly images and collected for all partial regions; and converting the aforementioned conversion processing row and column set by the anomaly feature value conversion unit into an anomaly feature vector set belonging to the same feature space as the first set of anomaly feature vector groups and used when calculating the anomaly distance.

Images