Abnormal Detection Device and Abnormal Detection Method

Abstract

The abnormality detection device (1) includes an inspection target feature amount extraction unit (121) that generates a first feature amount vector set having feature amount vectors of a plurality of partial regions of an inspection target object image as elements, an inspection target feature amount conversion unit (122) that converts the first feature amount vector set into a second feature amount vector set that is the same feature space as the normal feature amount vector set, a normal distance calculation unit (123) that calculates a normal distance for each element of the corresponding sets in the normal feature amount vector group set and the second feature amount vector set, an abnormal distance calculation unit (124) that calculates an abnormal distance for each element of the corresponding sets in the abnormal feature amount vector group set and the second feature amount vector set, and a determination unit (125) that determines whether each element of the set corresponding to the inspection target object image is normal or abnormal based on the normal distance set and the abnormal distance set.

Description

Technical Field

[0001] The present disclosure relates to an abnormality detection device and an abnormality detection method.

Background Art

[0002] In recent years, the automation of product appearance inspection has been progressing. For example, Patent Document 1 describes an abnormality determination method for automatically determining an abnormal product from the appearance of a product to be inspected. In this method, a normal product learning model configured in a feature space in which the feature amounts of normal image data generated by machine learning using a plurality of normal image data are made to follow a multivariate normal distribution is used. Identification information for identifying whether the product is normal or abnormal is determined based on the output results when known normal image data and known abnormal image data are input to the normal product learning model. Based on the identification information, it is determined whether the product to be inspected is normal or abnormal with respect to the output result when the image data of the product to be inspected is input to the normal product learning model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since the conventional abnormality detection method detects the abnormality of the product to be inspected based on the feature amounts extracted collectively from the entire image, there is a problem that it is impossible to specify at which position of the inspection target image in which the product to be inspected is photographed an abnormality has occurred.

[0005] The present disclosure solves the above problems, and an object thereof is to obtain an abnormality detection device capable of specifying at which position of the inspection target image in which the inspection target object is photographed an abnormality has occurred.

Means for Solving the Problems

[0006] The abnormality detection device according to the present disclosure extracts a first inspection target feature amount vector for each of a plurality of partial regions that partition an inspection target object image of an inspection target object, and generates a first inspection target feature amount vector set having the first inspection target feature amount vectors as elements, an inspection target feature amount extraction unit; a inspection target feature amount conversion unit that converts the first inspection target feature amount vector set into a second inspection target feature amount vector set that is the same feature space as the feature amount vector set for the inspection target object in a normal state; among a plurality of partial regions that partition a normal object image of the inspection target object in a normal state, a normal distance is calculated for each element of the corresponding sets between the normal feature amount vector group set in which a plurality of normal feature amount vector groups calculated for the partial regions at the same position in all the normal object images are collected for all the partial regions and the second inspection target feature amount vector set, and a normal distance set having the normal distances as elements is generated, a normal distance calculation unit; among a plurality of partial regions that partition an abnormal object image of the inspection target object in an abnormal state, an abnormal distance is calculated for each element of the corresponding sets between the abnormal feature amount vector group set in which a plurality of abnormal feature amount vector groups calculated for the partial regions at the same position in all the abnormal object images are collected for all the partial regions and the second inspection target feature amount vector set, and an abnormal distance set having the abnormal distances as elements is generated, an abnormal distance calculation unit; and a determination unit that determines whether each element of the set corresponding to the inspection target object image is normal or abnormal based on the normal distance set and the abnormal distance set.

Advantages of the Invention

[0007] According to the present disclosure, a set of inspection target feature vectors having, as elements, feature vectors of partial regions obtained by partitioning an inspection target image in which an inspection target is photographed into a plurality of parts, and a set of normal feature vector groups having, as elements, feature vectors of partial regions obtained by partitioning a normal object image in which a normal-state inspection target is photographed into a plurality of parts are used to calculate a normal distance for each corresponding element of the sets. An abnormal distance is calculated for each corresponding element of the sets between the set of inspection target feature vectors and a set of abnormal feature vector groups having, as elements, feature vectors of partial regions obtained by partitioning an abnormal object image in which an inspection target in an abnormal state is photographed into a plurality of parts. Based on the normal distance and the abnormal distance, it is determined whether the inspection target is normal or abnormal for each element of the set corresponding to the inspection target image. Since the elements of the set correspond to partial regions of the image, and the presence or absence of an abnormality can be determined for each partial region of the image, the abnormality detection device according to the present disclosure can specify at which position in the inspection target image photographed an abnormality has occurred.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0009] Embodiment 1. FIG. 1 is a block diagram showing a configuration example of an abnormality detection device 1 according to Embodiment 1. In FIG. 1, the abnormality detection device 1 acquires an inspection object image in which an inspection object is photographed, and automatically determines whether the inspection object is normal or abnormal for each partial region obtained by partitioning the inspection object image into a plurality of parts. Thereby, the abnormality detection device 1 can specify at which position (partial region) of the inspection target image an abnormality has occurred.

[0010] For the abnormality determination of the inspection object by the abnormality detection device 1, a feature amount vector representing the features for each partial region in the image is used. For example, the abnormality detection device 1 uses a learned neural network to extract a feature amount vector from the image. The abnormality detection device 1 is realized, for example, using a tablet terminal, a smartphone, or a notebook-type personal computer (PC).

[0011] A camera device (not shown in FIG. 1) may be connected to the abnormality detection device 1 via wired or wireless means. The abnormality detection device 1 may not be limited to a device with an external camera device, and may incorporate a camera device. In the appearance inspection of the inspection object, the inspection object is photographed by this camera device.

[0012] The abnormality detection device 1 may be a component provided in a server capable of communicating with a terminal device. For example, the terminal device can perform an appearance inspection of an object to be inspected in the form of SaaS (Software as a Service). When performing the inspection in the form of SaaS, it is not necessary to install an inspection application on the terminal device. The inspection application is executed on the above server, and the terminal device is provided with measurement result information on a general-purpose web browser. The inspection application is stored in a storage unit provided in the server. Also, an inspection application may be installed on the terminal device. In a terminal device on which the inspection application is installed, the appearance inspection of the object to be inspected can be performed by executing the application.

[0013] The abnormality detection device 1 is realized by a computer including an arithmetic unit and a storage unit. The storage unit is the storage unit 13 in FIG. 1 and is a storage device provided in the above computer. The storage unit 13 includes a storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), or a memory 104 in FIG. 8B described later. Note that the storage unit 13 only needs to be accessible by the abnormality detection device 1 and may be provided outside the abnormality detection device 1.

[0014] The arithmetic unit controls the overall operation of the abnormality detection device 1. The arithmetic unit includes a learning processing unit 11 and an inspection processing unit 12. By executing the inspection application stored in the storage unit 13 by the arithmetic unit, various functions of the learning processing unit 11 and the inspection processing unit 12 are realized. The learning processing unit 11 includes a normal feature amount extraction unit 111, a conversion processing matrix generation unit 112, a normal feature amount conversion unit 113, an abnormal feature amount extraction unit 114, and an abnormal feature amount conversion unit 115. The inspection processing unit 12 includes an inspection target feature amount extraction unit 121, an inspection target feature amount conversion unit 122, a normal distance calculation unit 123, an abnormal distance calculation unit 124, and a determination unit 125.

[0015] The learning processing unit 11 performs learning processing. In the learning processing, the learning processing unit 11 acquires a plurality of normal object image groups in which the inspection object in a normal state is photographed and one or more abnormal object image groups in which the inspection object in an abnormal state is photographed, and generates a conversion processing matrix set, a second normal image feature vector group set, and a second abnormal image feature vector group set.

[0016] The conversion processing matrix set is a collection of conversion processing matrices for performing conversion of feature amounts according to the distribution of the normal object image group. The second normal image feature vector group set is a collection of second normal image feature vectors representing the feature amounts after the conversion processing of the normal object image group. The second abnormal image feature vector group set is a collection of second abnormal image feature vectors representing the feature amounts after the conversion processing of the abnormal object image group. The conversion processing matrix set, the second normal image feature vector group set, and the second abnormal image feature vector group set are stored in the storage unit 13 from the learning processing unit 11.

[0017] The inspection processing unit 12 performs inspection processing, which is an appearance inspection of the inspection object. In the inspection processing, when the inspection processing unit 12 acquires an inspection object image in which the inspection object is photographed, it determines whether the inspection object in the inspection object image is normal or abnormal by using the conversion processing matrix set, the second normal image feature vector group set, and the second abnormal image feature vector group set stored in the storage unit 13.

[0018] First, the learning processing according to Embodiment 1 will be described. FIG. 2 is a flowchart showing the learning processing by the abnormality detection apparatus according to Embodiment 1, and shows the learning processing by the learning processing unit 11. The normal feature amount extraction unit 111 extracts a first normal feature amount vector group set in which a plurality of normal feature amount vector groups are collected for all partial regions (step ST1). For example, the normal feature extraction unit 111 learns a neural network using a large-scale image dataset such as ImageNet, and extracts a first set of normal feature vector groups from the normal object image group by inputting the normal object image group to the feature extractor of this neural network.

[0019] FIG. 3 is a schematic diagram showing an outline of the learning of the classification task of the neural network B using the large-scale image dataset A. The neural network B includes a feature extractor B1 and a classifier B2. In step ST1, the normal feature extraction unit 111 uses the feature extractor B1. The classifier B2 outputs a classification result C. Each element of the first set of normal feature vector groups corresponds to a partial region obtained by partitioning a normal object image into a plurality of parts, and a first set of normal feature vector groups corresponds to each partial region. The first normal feature vector, which is an element of the first set of normal feature vectors, is a feature vector representing the features of a certain partial region within one normal object image included in the normal object image group.

[0020] FIG. 4 is a schematic diagram schematically showing the extraction process of the feature vector by the feature extractor B1 of the neural network. As shown in FIG. 4, the normal object image A1 is partitioned into a plurality of partial regions P. The feature extractor B1 is composed of, for example, a one-layer (L1), a two-layer (L2), and a three-layer (L3) convolutional neural network (CNN), and feature vectors representing the features of the partial regions are extracted in each layer. The combination of the first-layer feature vector V1 of the partial region P, the second-layer feature vector V2 of the partial region P, and the third-layer feature vector V3 of the partial region P is the first normal feature vector VC corresponding to the partial region P. Although the case where the feature extractor B1 is a three-layer CNN is shown, it is not limited to a three-layer CNN as long as the feature vectors of the partial regions can be extracted.

[0021] FIG. 5 is a schematic diagram showing the correspondence between sub-regions P1 to PN within a single image A1 and a set of feature vectors. As shown in FIG. 5, one feature vector is calculated from one of the sub-regions P1 to PN obtained by partitioning the image A1 into a plurality of parts. The set of feature vectors VC1 to VCN of the sub-regions P1 to PN is the feature vector set VG1.

[0022] FIG. 6 is a schematic diagram showing the correspondence between the sub-region P1 at the same position within a plurality of images A and the feature vector group VG2. As shown in FIG. 6, the image group A includes a plurality of images of the same inspection object. The image group A is, for example, a series of images of inspection target products flowing in order on a production line, taken from the same shooting angle.

[0023] The feature extractor B1 extracts feature vectors VC-1 to VC-m from the sub-regions P1 to Pm at the same position within the image group A. The collection of the feature vectors VC-1 to VC-m is the feature vector group VG2. In FIGS. 5 and 6, among the sub-regions partitioning each image of the image group A, the feature vector group VG2 composed of a plurality of feature vectors VC-1 to VC-m calculated for the sub-regions at the same positions P1 to Pm in all the images, and the collection of all the sub-regions P1 to PN is the set of feature vector groups.

[0024] The transformation processing matrix generation unit 112 calculates a transformation processing matrix for the feature space for each sub-region of the normal object image using the first set of normal feature vector groups, and generates a set of transformation processing matrices having a plurality of transformation processing matrices as elements (step ST2). For example, the transformation processing matrix generation unit 112 calculates the set of transformation processing matrices by performing singular value decomposition on the first set of normal feature vector groups for each first normal feature vector group that is an element of the set. The transformation processing matrix that is an element of the set of transformation processing matrices is calculated for each sub-region.

[0025] When the number of elements of the first normal feature vector is P and the number of elements of the first normal feature vector group is N, in singular value decomposition, P singular values and a singular value decomposition matrix of P rows and P columns are obtained. The conversion processing matrix generation unit 112 extracts the top K out of the P singular values, and calculates, as a conversion processing matrix, a matrix of P rows and K columns obtained by extracting the columns corresponding to the top K singular values from the singular value decomposition matrix. Usually, P is a value on the order of a hundred to several hundred. In contrast, K uses a value on the order of several tens. The set of conversion processing matrices is stored in the storage unit 13.

[0026] The normal feature vector conversion unit 113 converts, using the set of conversion processing matrices, into a set of normal feature vectors used for calculating the normal distance, which is the same feature space as the set of the first normal feature vector groups (step ST3). For example, the normal feature vector conversion unit 113 uses the set of conversion processing matrices to convert each element of the set of the first normal feature vector groups, and calculates a set of second normal feature vector groups. The set of second normal feature vector groups is stored in the storage unit 13.

[0027] The conversion processing matrix is calculated for each sub-region. The collection of conversion processing matrices for all sub-regions within one image is the set of conversion processing matrices. Note that one normal feature vector is calculated for one sub-region of one image. A normal feature vector group is a group having, as elements, normal feature vectors corresponding to sub-regions at the same position in a plurality of images. Furthermore, the set of normal feature vector groups is the collection of normal feature vector groups for all sub-regions. The normal feature vector conversion unit 113 performs a conversion of the feature space from an image number × P-dimensional vector to an image number × K-dimensional vector by multiplying a conversion processing matrix corresponding to one sub-region in the set of conversion processing matrices by the normal feature vector group corresponding to the sub-region at the same position.

[0028] The abnormal feature quantity extraction unit 114 extracts a first set of abnormal feature quantity vector groups in which a plurality of abnormal feature quantity vector groups are collected for all partial regions (step ST4). For example, the abnormal feature quantity extraction unit 114 inputs the abnormal object image group to the feature quantity extractor B1 of the learned neural network to extract the first set of abnormal feature quantity vector groups.

[0029] The abnormal feature quantity conversion unit 115 converts, using the conversion processing matrix set, into a set of abnormal feature quantity vectors used for calculating the abnormal distance, which is the same feature space as the first set of abnormal feature quantity vector groups (step ST5). For example, the abnormal feature quantity conversion unit 115 uses the conversion processing matrix set to convert the first set of abnormal feature quantity vector groups for each element of the set to calculate a second set of abnormal feature quantity vector groups. The second set of abnormal feature quantity vector groups is stored in the storage unit 13. Note that by also using the conversion processing matrix calculated using the normal feature quantity vector for the conversion of the abnormal feature quantity vector, the normal feature quantity vector and the abnormal feature quantity vector are both converted into the same feature space.

[0030] Next, the abnormal detection method according to Embodiment 1 will be described. FIG. 7 is a flowchart showing the inspection process by the abnormal detection device 1 according to Embodiment 1, and shows the inspection process by the inspection processing unit 12. The said inspection process is the abnormal detection method according to Embodiment 1. The inspection target feature quantity extraction unit 121 generates a first set of inspection target feature quantity vectors having as elements the feature quantity vectors extracted from the inspection target object image (step ST1A). For example, the inspection target feature quantity extraction unit 121 uses the feature quantity extractor B1 to calculate a first set of inspection target feature quantity vectors from the inspection target image in which the inspection target object is photographed.

[0031] The inspection target feature quantity conversion unit 122 converts the first inspection target feature quantity vector set into a second feature quantity vector set (step ST2A). For example, the inspection target feature quantity conversion unit 122 calculates the second inspection target feature quantity vector set by converting the first inspection target feature quantity vector set for each element of the set using the conversion processing matrix set read from the storage unit 13.

[0032] The normal distance calculation unit 123 calculates the normal distance for each corresponding element of the set between the normal feature quantity vector group set and the second inspection target feature quantity vector set, and generates a normal distance set having the normal distances as elements (step ST3A). For example, the normal distance calculation unit 123 calculates the distance between the second normal feature quantity vector group set read from the storage unit 13 and the second inspection target feature quantity vector set for each element of the set, and calculates the normal distance set. In the calculation of the normal distance, for example, the average vector and the covariance matrix of the vectors are calculated from the second normal feature quantity vector group, and the Mahalanobis distance from the second inspection target feature quantity vector is calculated.

[0033] The abnormal distance calculation unit 124 calculates the abnormal distance for each corresponding element of the set between the abnormal feature quantity vector group set and the second inspection target feature quantity vector set, and generates an abnormal distance set having the abnormal distances as elements (step ST4A). For example, the abnormal distance calculation unit 124 calculates the distance between the second abnormal feature quantity vector group set read from the storage unit 13 and the second inspection target feature quantity vector set for each element of the set, and calculates the abnormal distance set. In the calculation of the abnormal distance, the inner product between the lower L-dimensional elements of the second inspection target feature quantity vector and the lower L-dimensional elements of each second abnormal feature vector is calculated, and the value obtained by multiplying this inner product by "-1" is taken as the abnormal distance. Further, among the abnormal distances between the individual second abnormal feature vectors, the smallest abnormal distance is taken as the abnormal distance of the corresponding second inspection target feature quantity vector. By calculating the abnormal distance by performing the inner product operation limited to the "lower L dimensions" in this way, it has been experimentally confirmed that the correlation between the actual abnormal state and the abnormal distance is higher than when not limited.

[0034] The determination unit 125 determines whether each element of the set corresponding to the inspection object image is normal or abnormal based on the normal distance set and the abnormal distance set (step ST5A). For example, the determination unit 125 calculates a set of determination results of normal and abnormal by performing conditional determination on the normal distance set and the abnormal distance set for each element of the set. For example, the determination condition is to determine that only the elements of the set that satisfy both conditions that the normal distance is less than or equal to the first threshold value and the abnormal distance is greater than or equal to the second threshold value are normal, and the other elements of the set are determined to be abnormal.

[0035] By learning and inspecting the image as described above, the following effects can be obtained. Since a pre-trained neural network is used for feature extraction, it is not necessary to re-learn the neural network for each object (Effect 1). In order to determine whether each partial region in the image is normal or abnormal, the position of the abnormality in the image can be specified (Effect 2). By using a transformation processing matrix that transforms the first normal feature amount of P dimensions into the second normal feature amount of K dimensions where P≥K by singular value decomposition, the time required for appearance inspection can be shortened by dimensionality reduction (Effect 3). By using a transformation processing matrix that transforms the first normal feature amount of P dimensions into the second normal feature amount of K dimensions where P≥K by singular value decomposition, a feature amount that well represents the features of the normal object image can be extracted, and the accuracy of appearance inspection can be improved (Effect 4). Since the normal distance is calculated using the second feature amount of K dimensions that well represents the features of the normal object image, it is possible to accurately determine whether it is normal or not (Effect 5). In the transformation processing matrix, since the K dimensions that well represent the features of the normal object image are extracted, for the abnormal object image, features similar to the normal object image appear in the upper part of the K dimensions, and features not similar to the normal object image appear in the lower part. In the calculation of the abnormal distance, since the lower L dimensions among the K dimensions of the second feature vector are used, it is possible to accurately search for an inspection target image similar to the abnormal object image (Effect 6).

[0036] Next, the hardware configuration for realizing the functions of the abnormality detection device 1 will be described. The functions of the learning processing unit 11, the inspection processing unit 12, and the storage unit 13 included in the abnormality detection device 1 are realized by a processing circuit. That is, the abnormality detection device 1 includes a processing circuit for executing each process from step ST1A to step ST5A shown in FIG. 7. The processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.

[0037] FIG. 8A is a block diagram showing the hardware configuration for realizing the functions of the abnormality detection device 1. FIG. 8B is a block diagram showing the hardware configuration for executing the software for realizing the functions of the abnormality detection device 1. In FIGS. 8A and 8B, the input interface 100 is an interface for relaying image information output from an external device to the abnormality detection device 1. The output interface 101 is an interface for relaying the abnormality inspection result output from the abnormality detection device 1 to the outside.

[0038] When the processing circuit is the dedicated hardware processing circuit 102 shown in FIG. 8A, the processing circuit 102 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the learning processing unit 11, the inspection processing unit 12, and the storage unit 13 included in the abnormality detection device 1 may be realized by separate processing circuits, or these functions may be realized by a single processing circuit together.

[0039] When the processing circuit is the processor 103 shown in FIG. 8B, the functions of the learning processing unit 11, the inspection processing unit 12, and the storage unit 13 included in the abnormality detection device 1 are realized by software, firmware, or a combination of software and firmware. Note that the software or firmware is described as a program and stored in the memory 104. The memory 104 is, for example, the storage unit 13 shown in FIG. 1.

[0040] The processor 103 reads and executes the program of the abnormality detection application stored in the memory 104, thereby realizing the functions of the learning processing unit 11, the inspection processing unit 12, and the storage unit 13 included in the abnormality detection device 1. For example, the abnormality detection device 1 includes a memory 104 for storing a program that, when executed by the processor 103, results in the execution of the processing from step ST1A to step ST5A shown in FIG. 7. This program causes the computer to execute the procedures or methods of the processing performed by the learning processing unit 11, the inspection processing unit 12, and the storage unit 13. The memory 104 may be a computer-readable storage medium storing a program for causing a computer to function as the learning processing unit 11, the inspection processing unit 12, and the storage unit 13.

[0041] The memory 104 corresponds to, for example, non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), magnetic disks, flexible disks, optical disks, compact disks, mini-disks, DVDs, and the like.

[0042] Part of the functions of the learning processing unit 11, the inspection processing unit 12, and the storage unit 13 included in the abnormality detection device 1 may be realized by dedicated hardware, and the remaining part may be realized by software or firmware. For example, the function of the storage unit 13 is realized by a processing circuit 102 which is dedicated hardware, but the learning processing unit 11 and the inspection processing unit 12 may be such that their functions are realized by the processor 103 reading and executing a program stored in the memory 104. In this way, the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.

[0043] So far, the case where the abnormality detection device 1 includes the learning processing unit 11 and the inspection processing unit 12 has been shown, but the abnormality detection device 1 may include only the inspection processing unit 12. The learning processing unit 11 and the storage unit 13 may be provided in a device provided separately from the abnormality detection device 1. In this case, the abnormality detection device 1 accesses these devices to obtain the learning results.

[0044] As described above, the abnormality detection device 1 according to Embodiment 1 includes an inspection target feature amount extraction unit 121 that generates a first inspection target feature amount vector set having as elements first inspection target feature amount vectors extracted from an inspection target object image, an inspection target feature amount conversion unit 122 that converts the first inspection target feature amount vector set into a second inspection target feature amount vector set that is the same feature space as the feature amount vector set for a normal inspection target object, a normal distance calculation unit 123 that calculates a normal distance for each element of the corresponding sets between the normal feature amount vector group set and the second inspection target feature amount vector set and generates a normal distance set having the normal distances as elements, an abnormal distance calculation unit 124 that calculates an abnormal distance for each element of the corresponding sets between the abnormal feature amount vector group set and the second inspection target feature amount vector set and generates an abnormal distance set having the abnormal distances as elements, and a determination unit 125 that determines whether each element of the set corresponding to the inspection target object image is normal or abnormal based on the normal distance set and the abnormal distance set. The elements of the set correspond to partial regions of the image, and since the presence or absence of abnormalities can be determined for each partial region of the image, the abnormality detection device 1 can identify the location where an abnormality has occurred in the inspection target image of the object to be inspected.

[0045] The abnormality detection device 1 according to Embodiment 1 includes a normal feature amount extraction unit 111 that extracts a first set of normal feature amount vectors from a plurality of normal object images, a conversion processing matrix generation unit 112 that calculates a conversion processing matrix for each partial region of the normal object image using the first set of normal feature amount vectors and generates a set of conversion processing matrices having the plurality of conversion processing matrices as elements, a normal feature amount conversion unit 113 that converts, using the set of conversion processing matrices, into a set of normal feature amount vectors used for calculating a normal distance, which is the same feature space as the first set of normal feature amount vectors, an abnormal feature amount extraction unit 114 that acquires a group of abnormal object images having a plurality of abnormal object images as elements and extracts a first set of abnormal feature amount vectors in which a plurality of abnormal feature amount vector groups calculated for partial regions at the same position in all the abnormal object images are collected for all the partial regions, and an abnormal feature amount conversion unit 115 that converts, using the set of conversion processing matrices, into a set of abnormal feature amount vectors used for calculating an abnormal distance, which is the same feature space as the first set of abnormal feature amount vectors. Thereby, the abnormality detection device 1 can learn the extraction of the feature amount vectors necessary for abnormality detection of the object to be inspected and the calculation of the conversion processing matrix.

[0046] In the abnormality detection device 1 according to Embodiment 1, the inspection target feature amount extraction unit 121 is a learned neural network that outputs a first set of inspection target feature amount vectors when an image is input. By using the learned neural network, the inspection target feature amount extraction unit 121 can accurately extract the first set of inspection target feature amount vectors from the image.

[0047] In the abnormality detection device 1 according to Embodiment 1, one or both of the normal feature amount extraction unit 111 and the abnormal feature amount extraction unit 114 are trained neural networks that output a set of feature vectors representing the features of a region for each partial region that divides the image into a plurality of parts when an image is input. By using the trained neural network, the normal feature amount extraction unit 111 can accurately extract a set of normal feature vectors from a normal image. By using the trained neural network, the abnormal feature amount extraction unit 114 can accurately extract a set of abnormal feature vectors from an abnormal image.

[0048] In the abnormality detection device 1 according to Embodiment 1, the inspection target feature amount conversion unit 122 uses a set of conversion processing matrices to convert the first set of feature vectors into a second set of feature vectors in the same feature space as the set of feature vectors for the inspection target object in a normal state for each partial region. Thereby, the inspection target feature amount conversion unit 122 can accurately convert the first set of feature vectors into the second set of feature vectors for each partial region.

[0049] In the abnormality detection device 1 according to Embodiment 1, the conversion processing matrix generation unit 112 calculates a conversion processing matrix by performing singular value decomposition on a first normal feature vector group that is an element of the first set of normal feature vector groups. By using this conversion processing matrix, the conversion processing matrix generation unit 112 can accurately convert the first set of inspection target feature vectors into a second set of inspection target feature vectors in the same feature space as the set of feature vectors for the inspection target object in a normal state.

[0050] In the abnormality detection device 1 according to Embodiment 1, the normal distance calculation unit 123 calculates the Mahalanobis distance or the Euclidean distance as the normal distance for each corresponding element of the set between the set of normal feature vector groups and the second set of feature vectors. Thereby, the normal distance calculation unit 123 can accurately calculate the normal distance.

[0051] In the abnormality detection device 1 according to Embodiment 1, the abnormal distance calculation unit 124 calculates the inner product of the lower L-dimensional elements of the second inspection target feature amount vector, which is an element of the second inspection target feature amount vector set, and the lower L-dimensional elements of the abnormal feature amount vector, which is an element of the abnormal feature amount vector group set, and multiplies the inner product value by -1 to calculate it as the abnormal distance. As a result, the normal distance calculation unit 123 can accurately calculate the abnormal distance.

[0052] In the abnormality detection device 1 according to Embodiment 1, when the normal distance is equal to or less than the first threshold value and the abnormal distance is equal to or greater than the second threshold value, the determination unit 125 determines that the inspection target object is in a normal state. As a result, the determination unit 125 can determine whether the inspection target object is in a normal state for each partial region of the image.

[0053] In the abnormality detection device 1 according to Embodiment 1, when the normal distance is equal to or greater than the first threshold value and the abnormal distance is equal to or less than the second threshold value, the determination unit 125 determines that the inspection target object is in an abnormal state. As a result, the determination unit 125 can determine whether the inspection target object is in an abnormal state for each partial region of the image.

[0054] In the abnormality detection method according to Embodiment 1, a step ST1A of generating a first inspection target feature amount vector set including, as elements, feature amount vectors extracted from an inspection target object image by an inspection target feature amount extraction unit 121; a step ST2A of converting the first inspection target feature amount vector set into a second feature amount vector set by an inspection target feature amount conversion unit 122; a step ST3A of calculating a normal distance for each element of corresponding sets between a normal feature amount vector group set and the second inspection target feature amount vector set, and generating a normal distance set by a normal distance calculation unit 123; a step ST4A of calculating an abnormal distance for each element of corresponding sets between an abnormal feature amount vector group set and the second inspection target feature amount vector set, and generating an abnormal distance set by an abnormal distance calculation unit 124; and a step ST5A of determining whether each element of the set corresponding to the inspection target object image is normal or abnormal based on the normal distance set and the abnormal distance set by a determination unit 125. By executing this method, the abnormality detection device 1 can identify at which position of the inspection target image an abnormality has occurred.

[0055] In the abnormality detection method according to Embodiment 1, a normal feature quantity extraction unit 111 extracts a first set of normal feature quantity vectors in which a plurality of normal feature quantity vector groups are collected for all partial regions in step ST1; a conversion processing matrix generation unit 112 uses the first set of normal feature quantity vectors to calculate a conversion processing matrix for the feature space for each partial region of the normal object image, and generates a set of conversion processing matrices having a plurality of conversion processing matrices as elements in step ST2; a normal feature quantity conversion unit 113 uses the set of conversion processing matrices to convert to a set of normal feature quantity vectors used for calculating a normal distance, which is the same feature space as the first set of normal feature quantity vectors, in step ST3; an abnormal feature quantity extraction unit 114 extracts a first set of abnormal feature quantity vectors in which a plurality of abnormal feature quantity vector groups are collected for all partial regions in step ST4; and an abnormal feature quantity conversion unit 115 uses the set of conversion processing matrices to convert to a set of abnormal feature quantity vectors used for calculating an abnormal distance, which is the same feature space as the first set of abnormal feature quantity vectors, in step ST5. By executing this method, the abnormality detection device 1 can learn the extraction of feature quantity vectors and the calculation of conversion processing matrices necessary for the abnormality detection of an object to be inspected.

[0056] Note that deformation of any component of the embodiment or omission of any component of the embodiment is possible.

Industrial Applicability

[0057] The abnormality detection device according to the present disclosure can be used, for example, for the appearance inspection of products on a production line.

Explanation of Signs

[0058] 1 Abnormality detection device, 11 Learning processing unit, 12 Inspection processing unit, 13 Memory unit, 100 Input interface, 101 Output interface, 102 Processing circuit, 103 Processor, 104 Memory, 111 Normal feature quantity extraction unit, 112 Conversion processing matrix generation unit, 113 Normal feature quantity conversion unit, 114 Abnormal feature quantity extraction unit, 115 Abnormal feature quantity conversion unit, 121 Feature quantity extraction unit for inspection target, 122 Feature quantity conversion unit for inspection target, 123 Normal distance calculation unit, 124 Abnormal distance calculation unit, 125 Determination unit.

Claims

1. An inspection object feature amount extraction unit that extracts a first inspection object feature amount vector for each of a plurality of partial regions that partition an inspection object image in which an inspection object is photographed, and generates a first inspection object feature amount vector set having the first inspection object feature amount vectors as elements; An inspection object feature amount conversion unit that converts the first inspection object feature amount vector set into a second inspection object feature amount vector set that is the same feature space as the feature amount vector set for an inspection object in a normal state; Among a plurality of partial regions that partition a normal object image in which a normal state inspection object is photographed, a normal feature amount vector group in which a plurality of normal feature amount vectors calculated for partial regions at the same position in all the normal object images are collected for all partial regions A normal distance calculation unit that calculates a normal distance for each element of the corresponding sets between the normal feature amount vector group set and the second inspection object feature amount vector set, and generates a normal distance set having the normal distances as elements; Among a plurality of partial regions that partition an abnormal object image in which an inspection object in an abnormal state is photographed, an abnormal feature amount vector group in which a plurality of abnormal feature amount vectors calculated for partial regions at the same position in all the abnormal object images are collected for all partial regions An abnormal distance calculation unit that calculates an abnormal distance for each element of the corresponding sets between the abnormal feature amount vector group set and the second inspection object feature amount vector set, and generates an abnormal distance set having the abnormal distances as elements; A determination unit that determines whether each element of the set corresponding to the inspection object image is normal or abnormal based on the normal distance set and the abnormal distance set; A normal feature amount extraction unit that acquires a normal object image group having a plurality of the normal object images as elements, and extracts a first normal feature amount vector group set in which a plurality of normal feature amount vectors calculated for partial regions at the same position in all the normal object images are collected for all partial regions; A conversion processing matrix generation unit that calculates a conversion processing matrix of the feature space for each partial region of the normal object image using the first normal feature amount vector group set, and generates a conversion processing matrix set having the plurality of conversion processing matrices as elements; Using the set of transformation processing matrices, a normal feature quantity conversion unit that converts the set of first normal feature quantity vectors into a set of normal feature quantity vectors used for calculating a normal distance, which is in the same feature space as the set of first normal feature quantity vectors; An abnormal feature quantity extraction unit that acquires a group of abnormal object images including a plurality of the abnormal object images, and extracts a first set of abnormal feature quantity vectors calculated for sub-regions at the same position in all the abnormal object images among a plurality of sub-regions that partition the abnormal object images; Using the set of transformation processing matrices, an abnormal feature quantity conversion unit that converts the first set of abnormal feature quantity vectors into a set of abnormal feature quantity vectors used for calculating an abnormal distance, which is in the same feature space as the first set of abnormal feature quantity vectors; and An abnormal detection device characterized by the above.

2. The inspection target feature quantity extraction unit is a learned neural network that outputs the first set of inspection target feature quantity vectors when an image is input. The abnormal detection device according to claim 1, characterized by the above.

3. Either one or both of the normal feature quantity extraction unit and the abnormal feature quantity extraction unit is a learned neural network that outputs a set of feature quantity vectors representing the features of a region for each sub-region that partitions the image when an image is input. The abnormal detection device according to claim 1, characterized by the above.

4. The inspection target feature quantity conversion unit uses a set of transformation processing matrices to convert the first set of inspection target feature quantity vectors into a second set of inspection target feature quantity vectors that are in the same feature space as the set of feature quantity vectors for the inspection target object in a normal state for each sub-region. The abnormal detection device according to claim 1, characterized by the above.

5. The transformation processing matrix generation unit calculates the transformation processing matrix by performing singular value decomposition on the first normal feature quantity vector group, which is an element of the first set of normal feature quantity vectors. The abnormality detection device according to claim 1, characterized in that...

6. The normal distance calculation unit calculates, for each element of the corresponding set between the normal feature vector group set and the second inspection target feature vector set, the Mahalanobis distance or the Euclidean distance as the normal distance. The abnormality detection device according to any one of claims 1 to 5, characterized in that...

7. The abnormality distance calculation unit calculates the inner product between the lower L-dimensional elements of the second inspection target feature vector, which is an element of the second inspection target feature vector set, and the lower L-dimensional elements of the abnormality feature vector, which is an element of the abnormality feature vector group set, and calculates, as the abnormality distance, the value obtained by multiplying the inner product value by -1. The abnormality detection device according to any one of claims 1 to 5, characterized in that...

8. When the normal distance is less than or equal to the first threshold value and the abnormality distance is greater than or equal to the second threshold value, the determination unit determines that the inspection target object is in a normal state. The abnormality detection device according to any one of claims 1 to 5, characterized in that...

9. When the normal distance is greater than or equal to the first threshold value and the abnormality distance is less than or equal to the second threshold value, the determination unit determines that the inspection target object is in an abnormal state. The abnormality detection device according to any one of claims 1 to 5, characterized in that...

10. An abnormality detection method by an abnormality detection device, comprising: A step in which an inspection target feature amount extraction unit extracts first inspection target feature vectors of each of a plurality of partial regions partitioning an inspection target object image of an inspection target object, and generates a first inspection target feature vector set having the first inspection target feature vectors as elements; A step in which the inspection target feature quantity conversion unit converts the first inspection target feature quantity vector set into a second inspection target feature quantity vector set that is in the same feature space as the feature quantity vector set for the inspection target in a normal state. A step in which the normal distance calculation unit calculates a normal distance for each corresponding element of a set between a set of normal feature quantity vector groups, in which a plurality of normal feature quantity vectors calculated for partial regions at the same position in all of the normal object images among a plurality of partial regions partitioning the normal object image of the inspection target in a normal state are collected for all of the partial regions, and the second inspection target feature quantity vector set, and generates a normal distance set having the normal distances as elements. A step in which the abnormal distance calculation unit calculates an abnormal distance for each corresponding element of a set between a set of abnormal feature quantity vector groups, in which a plurality of abnormal feature quantity vectors calculated for partial regions at the same position in all of the abnormal object images among a plurality of partial regions partitioning the abnormal object image of the inspection target in an abnormal state are collected for all of the partial regions, and the second inspection target feature quantity vector set, and generates an abnormal distance set having the abnormal distances as elements. A step in which the determination unit determines whether each element of the set corresponding to the inspection target object image is normal or abnormal based on the normal distance set and the abnormal distance set. A step in which the normal feature quantity extraction unit acquires a normal object image group having a plurality of the normal object images as elements, and extracts a first normal feature quantity vector group set in which a plurality of normal feature quantity vector groups calculated for partial regions at the same position in all of the normal object images among a plurality of partial regions partitioning the normal object image are collected for all of the partial regions. A step in which the conversion processing matrix generation unit calculates a conversion processing matrix for the feature space for each partial region of the normal object image using the first normal feature quantity vector group set, and generates a conversion processing matrix set having a plurality of the conversion processing matrices as elements. A step in which the normal feature quantity conversion unit converts the first normal feature quantity vector group set into a normal feature quantity vector set used for calculating a normal distance that is in the same feature space as the first normal feature quantity vector group set using the conversion processing matrix set. The abnormal feature quantity extraction unit acquires a group of abnormal object images including a plurality of the abnormal object images, and extracts a first set of abnormal feature quantity vector groups collected for all partial regions, which are located at the same position in all the abnormal object images, among the plurality of partial regions partitioning the abnormal object images. The abnormal feature quantity conversion unit includes a step of converting, using the conversion processing matrix set, into a set of abnormal feature quantity vectors used for calculating an abnormal distance, which is the same feature space as the first set of abnormal feature quantity vector groups. An abnormal detection method characterized by the above.

Images