Inspection system and inspection method

Abstract

To provide an inspection system and inspection method capable of efficiently inspecting an image to be inspected.SOLUTION: An inspection system 1 comprises: feature extraction circuits 22, 32 that extract multiple types of features from an image; dimension reduction circuits 24, 34 that reduce the dimensions of feature vectors due to the multiple types of features; and a determination circuit 36 that determines the presence or absence of an abnormality by comparing distribution of feature vectors derived from multiple normal images with a feature vector derived from an inspection image. The dimension reduction circuits 24, 34 reduce the dimensions of the feature vectors by selecting an axis on which a variance value of the features across the multiple normal images is smaller than a threshold value.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an inspection system and an inspection method for determining whether or not an inspection object has an abnormality. [Background technology]

[0002] Conventionally, image inspection devices are known that process captured images of inspection targets such as products and parts, medical X-ray images, etc., to determine whether the inspection targets are pass or fail, whether they have a disease, etc. For example, Patent Document 1 listed below describes an image inspection device that uses a trained neural network to extract image features. This image inspection device uses the trained neural network to extract multidimensional feature vectors from the captured images of the inspection targets, and evaluates these to determine whether the inspection targets are pass or fail.

[0003] When the number of dimensions of a feature vector is large, problems arise, such as a large amount of data, excessive memory usage, and high computational costs for evaluation. Therefore, Patent Document 2 listed below describes an image inspection device that evaluates feature vectors whose dimensions have been reduced, so that multidimensional feature vectors can be evaluated efficiently. This document describes that reducing the dimensions of the feature vectors can reduce memory usage and computational costs, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-87181 [Patent Document 2] Patent Publication No. 2021-176070 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional image inspection device has the following problem: The computational load required for dimension reduction may make it difficult to sufficiently reduce inspection costs.

[0006] The present invention has been made in consideration of the above-mentioned conventional problems, and aims to provide an inspection system and an inspection method that can efficiently inspect an object to be inspected. [Means for solving the problem]

[0007] One aspect of the present invention is a feature extraction circuit that extracts multiple types of feature amounts from an image; a dimension reduction circuit that reduces the dimension of an original feature vector composed of the plurality of types of feature amounts to convert it into a low-dimensional feature vector; a determination circuit for determining whether or not an abnormality exists in the inspection object by comparing a distribution of a plurality of low-level feature vectors derived from a plurality of normal images with a low-level feature vector derived from an inspection image that is a captured image of the inspection object, or by comparing each feature vector among the plurality of low-level feature vectors derived from the plurality of normal images with a low-level feature vector derived from the inspection image, the low-level feature vector is a vector belonging to a multidimensional space defined by some of a plurality of axes that define the multidimensional space to which the original feature vector belongs, In an inspection system, the some axes are, among multiple axes that define a multidimensional space to which the original feature vector belongs, axes on which at least one of the average value and variance value of the feature across multiple normal images is smaller than a threshold value.

[0008] One aspect of the present invention is an inspection method for determining whether or not an inspection object has an abnormality, the method comprising: A process of extracting multiple types of features from an image; a process of reducing the dimension of the original feature vector composed of the plurality of types of feature amounts and converting it into a low-dimensional feature vector; a process of determining whether or not an abnormality exists in the inspection object by comparing a distribution of a plurality of low-level feature vectors derived from a plurality of normal images with a low-level feature vector derived from an inspection image, which is an image captured of the inspection object, or by comparing each feature vector among the plurality of low-level feature vectors derived from a plurality of normal images with a low-level feature vector derived from the inspection image, the low-level feature vector is a vector belonging to a multidimensional space defined by some of a plurality of axes that define the multidimensional space to which the original feature vector belongs, The inspection method is such that the certain axes are, among a plurality of axes defining a multidimensional space to which the original feature vector belongs, axes on which at least one of the average value and variance value of the feature across a plurality of normal images is smaller than a threshold value. [Effects of the Invention]

[0009] In the present invention, the dimensions of an original feature vector composed of multiple types of feature amounts extracted from an image are reduced and converted into a low-level feature vector. Then, the presence or absence of an abnormality in the inspection object is determined by comparing the distribution of the multiple low-level feature vectors derived from multiple normal images with the low-level feature vector derived from the inspection image, or by comparing each feature vector among the multiple low-level feature vectors derived from multiple normal images with the low-level feature vector derived from the inspection image.

[0010] The low-level feature vector in the present invention belongs to a multidimensional space defined by some of the axes that define the multidimensional space to which the original feature vector belongs. These some of the axes are axes on which at least one of the average value and variance value of the features across a plurality of normal images is smaller than a threshold value.

[0011] According to the present invention, it is possible to efficiently determine abnormalities in an inspection object in a dimension-reduced multidimensional space. Furthermore, the present invention can achieve dimension reduction by utilizing at least one of the average value and variance value of feature quantities across multiple normal images, thereby reducing the computational load required for dimension reduction. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 3 is an explanatory diagram showing how the surface of a leather seat, which is an object to be inspected, is imaged in the first embodiment. [Figure 2] FIG. 1 is a block diagram showing the configuration of an inspection system according to a first embodiment. [Figure 3] 5A to 5C are explanatory diagrams of a procedure for acquiring statistical information representing variations among a plurality of normal images in the first embodiment. [Figure 4] FIG. 3 is an explanatory diagram of a procedure for processing an inspection image in the first embodiment. [Figure 5] FIG. 4 is a flowchart showing the flow of an abnormality determination process in the first embodiment. [Figure 6(a)] FIG. 4 is a front view of a normal image in the first embodiment. [Figure 6(b)] FIG. 4 is a front view of an abnormal image in the first embodiment. [Figure 6(c)] FIG. 4 is a front view of a determination image in the first embodiment. [Figure 7] FIG. 10 is a block diagram showing the configuration of an inspection system according to a second embodiment. [Figure 8] FIG. 11 is a distribution diagram showing the frequency of the number of axes where the average value of the feature amount is zero in the second embodiment. [Figure 9] FIG. 10 is an explanatory diagram showing an inspection area set in an inspection image in the second embodiment. [Figure 10] FIG. 11 is a diagram showing the comparison results of abnormality determination methods in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] The embodiments of the present invention will be specifically described using the following examples. Example 1 This example relates to an inspection system 1 that judges abnormalities in an inspection object using a captured image. The details of this will be described with reference to Figs. 1 to 6(c).

[0014] The inspection system 1 of this example is, for example, a system for inspecting products or parts processed in a factory. The inspection object of this example is a long leather sheet 10 as shown in FIG. 1. The leather sheet 10 to be inspected is, for example, transported by a transport device 101 and wound up in a roll. The inspection system 1 processes an inspection image, which is a captured image of the surface of the leather sheet 10 during transportation, and determines whether or not there is an abnormality in the leather sheet 10 to be inspected.

[0015] The inspection system 1 (Fig. 2) is composed of a learning unit 2 that learns normal images 11 (see Fig. 3), which are images of an inspection object without abnormalities, and a judgment unit 3 that judges whether or not an inspection object has an abnormality. The learning unit 2 is a subsystem that acquires statistical information that represents the variation of multiple normal images 11. The judgment unit 3 is a subsystem that uses this statistical information to judge whether or not an inspection object has an abnormality.

[0016] (Learning section configuration) The learning unit 2 (FIG. 2) is configured to include an image DB 20 storing multiple normal images 11, a feature extraction circuit 22 that extracts multiple types of feature values ​​from the normal images 11, a dimension reduction circuit 24 that obtains low-order feature vectors by reducing the dimensions of original feature vectors consisting of multiple types of feature values, a statistical processing circuit 26 that obtains statistical information representing the distribution of multiple low-order feature vectors derived from the multiple normal images 11, and a storage device 25 that stores statistical information, etc., of the multiple low-order feature vectors. In the following description, the original feature vectors will be referred to as high-order feature vectors.

[0017] The image DB 20 (Fig. 2) is a database of images. The image DB 20 stores a plurality of normal images 11. The plurality of normal images 11 are images used for comparison with the inspection image. Variation in quality is inevitable among inspection objects. An inspection object without abnormalities means an inspection object whose quality is within an acceptable range. The plurality of normal images 11 are captured images of a plurality of inspection objects whose quality is within an acceptable range.

[0018] As shown in FIGS. 2 and 3, the feature extraction circuit 22 is a circuit that extracts multiple types of features (512 types in this example) from an image using a CNN (Convolutional Neural Network) 221. In this example, the CNN 221 is trained using ImageNet, a large-scale image dataset. The feature extraction circuit 22 applies the CNN 221 to the normal image 11 and extracts a high-order feature vector 225 composed of 512 types of features. The high-order feature vector 225 is a 512-dimensional multidimensional vector. The high-order feature vector 225 belongs to a 512-dimensional multidimensional space and is composed of 512 types of features for each of the 512 axes.

[0019] The CNN 221 (FIG. 3) of this example is configured to coarse-grain the normal image 11, which is an input image, and convert it into a low-resolution feature map 220. The high-order feature vector 225 extracted by the CNN 221 is vector data of each pixel 223 in the feature map 220. The feature map 220 is a two-dimensional map. Each pixel 223 in the feature map 220 corresponds to a region 111 of multiple pixels in the normal image 11, and the correspondence is known. Note that FIG. 3 shows how, for a specific pixel 223 on the feature map 220, a high-order feature vector 225 is extracted from the corresponding region 111 in the normal image 11. The extraction of such a high-order feature vector 225 is performed for each region in the normal image 11 corresponding to each pixel 223 in the feature map 220.

[0020] 2 and 3, the dimension reduction circuit 24 is a circuit that reduces the dimensions of the high-order feature vector 225 to obtain a low-order feature vector 226. The low-order feature vector 226 is a multidimensional vector that belongs to a 50-dimensional multidimensional space and is composed of 50 types of feature values ​​for each of 50 axes. The dimension reduction circuit 24 performs dimension reduction from the high-order feature vector 225 to the low-order feature vector 226 for each pixel 223 of the feature map 220. A 50-dimensional feature vector can reduce the amount of memory required for storage and can suppress the calculation cost when determining the presence or absence of an abnormality, thereby improving inspection efficiency.

[0021] The dimension reduction circuit 24 (FIG. 2) of this example reduces the dimension by selecting, from among 512 axes defining the multidimensional space to which the high-order feature vector 225 belongs, axes whose feature variance across multiple normal images 11 (see FIG. 3) is equal to or less than a threshold. The threshold is the 50th smallest variance value among the variance values ​​related to the 512 axes. By setting the threshold in this manner, 50 axes with smallest variance values ​​can be selected from the 512 axes. Information on the 50 axes selected by the dimension reduction circuit 24 is stored as selected axis information in the storage device 25.

[0022] The statistical processing circuit 26 (FIG. 2) is a circuit that acquires statistical information representing the distribution of a plurality of low-level feature vectors 226 derived from a plurality of normal images 11. In this example, the statistical processing circuit 26 calculates statistical information for each pixel 223 of the feature map 220 (see FIG. 3). In this example, the statistical information is a covariance matrix Σ and a mean μ when the variance of the plurality of low-level feature vectors 226 is considered as a multivariate normal distribution.

[0023] In addition to the selection axis information, the storage device 25 (FIG. 2) stores statistical information (covariance matrix Σ and mean μ) acquired for each pixel 223 (see FIG. 3) of the feature map 220. In the configuration of this example, this statistical information is used to determine whether or not an abnormality exists in the inspection target. The selection axis information and statistical information stored in the storage device 25 can be read out by the determination unit 3.

[0024] (Configuration of the judgment unit) As shown in FIGS. 2 and 4, the judgment unit 3 is configured to include an imaging camera 30 that acquires an inspection image 13, which is an image of the inspection object; a feature extraction circuit 32 that extracts a high-order feature vector 325 from the inspection image 13; a dimension reduction circuit 34 that converts the high-order feature vector 325 into a low-order feature vector 326; a judgment circuit 36 ​​that judges whether or not the inspection object has an abnormality; and an output device 38 that outputs the judgment result.

[0025] The imaging camera 30 (FIGS. 1 and 2) is installed so as to capture an image of the surface of the leather sheet 10 (an example of an object to be inspected) being transported by, for example, the transport device 101. Instead of the imaging camera 30, an image acquisition circuit may be used that reads out inspection images one by one from a storage device (not shown) that stores the inspection images in advance, or an image acquisition circuit that acquires inspection images via wireless communication from an external imaging camera or storage device (not shown).

[0026] As shown in FIGS. 2 and 4, the feature extraction circuit 32 is a circuit that extracts a 512-dimensional high-order feature vector 325 from the inspection image 13. The feature extraction circuit 32 has the same specifications as the feature extraction circuit 22 of the learning unit 2. The feature extraction circuit 32 generates a feature map 320 in which data for each pixel 323 is a high-dimensional feature vector. The trained CNN 321 included in the feature extraction circuit 32 has the same specifications as the CNN 221 described above. Note that in this example, for convenience of explanation, the feature extraction circuit 22 of the learning unit 2 and the feature extraction circuit 32 of the determination unit 3 are shown as separate entities. In actual operation, it is preferable that the learning unit 2 and the determination unit 3 share a feature extraction circuit including a trained CNN.

[0027] 2 and 4, the dimension reduction circuit 34 is a circuit that reduces the dimension of the high-order feature vector 325 and converts it into a low-order feature vector 326. The low-order feature vector 326 is a feature vector that is composed of feature quantities of 50 axes out of the feature quantities of 512 axes that make up the high-order feature vector 325. The dimension reduction from the high-order feature vector 325 to the low-order feature vector 326 is performed for each pixel 323 of the feature map 320.

[0028] 2 and 4, the determination circuit 36 ​​is a circuit for determining whether or not an abnormality exists in the inspection object. The determination circuit 36 ​​determines whether or not an abnormality exists in the inspection object by comparing the distribution of a plurality of low-level feature vectors 226 derived from a plurality of normal images 11 with a low-level feature vector 326 derived from the inspection image (FIG. 4). The determination circuit 36 ​​can determine whether or not an abnormality exists for each pixel 323 in a feature map 320 based on the inspection image. Note that, as will be described in more detail later, in this example, the above comparison is performed using the well-known Mahalanobis distance, which represents the degree to which one piece of data deviates from the distribution of a data group.

[0029] The output device 38 (FIG. 2) is a device that outputs the determination result by the determination circuit 36. For example, the output device 38 for notifying an operator of an abnormality can be a display device such as a liquid crystal display or a light, or a sound output device such as a speaker. For example, the output device 38 for notifying the production system of an abnormality can be a circuit that outputs an abnormality occurrence signal.

[0030] Next, the inspection procedure by the inspection system 1 of this example will be described with reference to Figs. 4 and 5. Fig. 5 is a flow chart showing the flow of the inspection process. In the following explanation, the inspection procedure will be described assuming that the selected axis information and statistical information acquired by the learning unit 2 are stored in advance in the storage device 25. The selected axis information is information on 50 axes selected when reducing the dimensions of the high-order feature vector 225 extracted from the normal image 11 (see Fig. 3) and converting it into the low-order feature vector 226, as described above. The statistical information is information representing the distribution of the multiple low-order feature vectors 226 derived from the multiple normal images 11, as described above.

[0031] In order to determine whether or not there is an abnormality in the inspection object, first, an inspection image 13, which is a captured image of the inspection object, is acquired (S101). In this example, the inspection object is a long leather sheet 10 (see FIG. 1). As described above, in this example, the imaging camera 30 is installed so that it can capture images of the inspection object while it is being transported. The imaging camera 30 captures images of the surface of the leather sheet 10 while it is being transported, and sequentially acquires the inspection images 13.

[0032] The feature extraction circuit 32 applies the trained CNN 321 to the inspection image 13 to extract a high-order feature vector 325 from the inspection image 13 (S102). This high-order feature vector 325 is a vector belonging to a multidimensional space defined by 512 axes. The high-order feature vector 325 is extracted for each pixel 323 of the feature map 320 based on the inspection image 13.

[0033] 4 shows how a high-order feature vector 325 is extracted from an area 131 in the inspection image 13 for a specific pixel 323 on the feature map 320. The extraction of such a high-order feature vector 325 is performed for each area in the inspection image 13 that corresponds to each pixel 323 on the feature map 320.

[0034] The high-order feature vector 325 is supplied to the dimension reduction circuit 34, where its dimensions are reduced and it is converted into a low-order feature vector 326. To reduce the dimensions of the high-order feature vector 325, the dimension reduction circuit 34 reads the selected axis information from the storage device 25 (S103). As described above, the selected axis information is information indicating 50 axes selected to reduce the dimensions of the high-order feature vector 225 (see FIG. 3 ) derived from the normal image 11. The dimension reduction circuit 34 selects the 50 axes indicated by the selected axis information and generates a low-order feature vector 326 composed of 50 types of features along the selected 50 axes (S104).

[0035] The low-level feature vector 326 of each pixel 323 of the feature map 320 is supplied to the determination circuit 36, which determines whether or not there is an abnormality in the inspection object (S105). The determination circuit 36 ​​calculates a Mahalanobis distance, which indicates the degree to which the low-level feature vector 326 derived from the inspection image 13 deviates from the distribution of the plurality of low-level feature vectors 226 derived from the plurality of normal images 11.

[0036] The Mahalanobis distance is a common index used to quantify the degree to which specific data deviates from a data distribution. In this example, the Mahalanobis distance is used as an example of the result of comparing the distribution of multiple low-level feature vectors 226 derived from multiple normal images 11 with a low-level feature vector 326 derived from the test image 13. The Mahalanobis distance is calculated for each pixel 323 in the feature map 320.

[0037] To calculate the Mahalanobis distance, the determination circuit 36 ​​reads statistical information (covariance matrix Σ and mean μ) of each pixel 223 (see FIG. 3) in the feature map 220 from the storage device 25. As described above, this statistical information represents the distribution of a plurality of low-level feature vectors 226 derived from a plurality of normal images 11. The Mahalanobis distance can be calculated using the covariance matrix Σ and mean μ, which are the statistical information read from the storage device 25. The determination circuit 36 ​​then applies threshold processing to the Mahalanobis distance of each pixel 323 in the feature map 320 to determine whether or not an abnormality exists. The determination result is output by an output device 38, such as a liquid crystal display, as shown in FIGS. 6(a) to 6(c) (S106).

[0038] For example, Figures 6(a) to 6(c) show the results of determining whether or not there is an abnormality in a leather seat 10 that is the subject of inspection. Figure 6(a) is a normal image 11 captured of an abnormality-free leather seat 10. Figure 6(b) is an inspection image 13 in which an abnormality exists, in which there is a defect on the surface of the leather seat 10. This inspection image 13 includes a defect such as a wart in the upper left. Figure 6(c) is a judgment image that displays the abnormality judgment result on the feature map 320. This judgment image is displayed by adjusting the scale of the feature map 320 output by the CNN 321 to the same size as the inspection image 13, etc.

[0039] In the judgment image of Fig. 6(c), pixels 323 containing an abnormality are shown brightly on the feature map 320. In the judgment image of the same figure, pixels 323 corresponding to defects in the inspection image 13 of Fig. 6(b) are judged to contain an abnormality. Based on the judgment image, it is possible to not only determine that an abnormality exists in the inspection object, but also to grasp the abnormal portion in the inspection image 13.

[0040] In the inspection system 1 of this example configured as described above, the presence or absence of an abnormality is efficiently determined by reducing the dimensions of the multidimensional feature vector extracted from the image. In particular, in this example, when reducing the dimensions of the feature vector, an axis with a small variance value of the feature is selected. Compared to principal component analysis, which is a common dimension reduction method, the amount of calculation required to calculate the variance value of the feature is significantly less. The dimension reduction method of this example is more efficient than a dimension reduction method using principal component analysis.

[0041] In this example, the presence or absence of an abnormality is determined for each pixel 323 in the feature map 320. Alternatively, the pixel with the largest Mahalanobis distance may be selected from the pixels 323 in the feature map 320. The pixel with the largest Mahalanobis distance may be treated as a representative pixel of the inspection image 13, and threshold processing may be applied to the Mahalanobis distance of the representative pixel to determine the presence or absence of an abnormality.

[0042] In addition, in the configuration of this example, the presence or absence of an abnormality is determined for each pixel 323 of the feature map 320 (see FIG. 4). Alternatively, the presence or absence of an abnormality may be determined using a single feature vector that represents the features of the entire image. In this case, too, when reducing the dimension of the feature vector, it is advisable to select a feature with a small variance, as in this example.

[0043] In this example, when selecting axes to reduce the dimension of the feature vector, 50 axes with small variance values ​​of the feature across multiple normal images 11 are selected. Alternatively, or in addition, 50 axes with small average feature values ​​may be selected to reduce the dimension of the feature vector. Similar to variance values, calculation of the average feature value requires significantly less computational effort than principal component analysis. Here, an axis with small variance values ​​or average feature values ​​across multiple normal images 11 refers to an axis with small variance values ​​or average feature values ​​among multiple feature vectors derived from multiple normal images 11.

[0044] This example is a configuration example in which the presence or absence of an abnormality is determined using the Mahalanobis distance, which represents the degree of deviation between the distribution of multiple low-level feature vectors 226 derived from multiple normal images 11 (see FIG. 3) and the low-level feature vector 326 derived from the test image. Alternatively, the presence or absence of an abnormality may be determined using an algorithm such as KNN (K Nearest Neighbors). KNN can determine the presence or absence of an abnormality by comparing each low-level feature vector 226 derived from multiple normal images 11 with the low-level feature vector 326 derived from the test image.

[0045] In this example, the presence or absence of an abnormality is determined using distances such as the Mahalanobis distance in the multidimensional space to which the feature vector belongs. However, it is also possible to determine the presence or absence of an abnormality using a model such as a decision tree that does not use the Mahalanobis distance or Euclidean distance.

[0046] In this example, 512-dimensional feature vectors are shown as the high-order feature vectors 225 and 325, and 50-dimensional feature vectors are shown as the post-dimensionality reduced low-order feature vectors 226 and 326. The number of dimensions of the high-order feature vectors 225 and 325 and the number of dimensions of the low-order feature vectors 226 and 326 are not limited to the number of dimensions in this example and can be changed as appropriate.

[0047] In this example, the inspection object is a leather seat 10. Other objects that can be inspected include mechanical parts such as screws and bolts, electronic parts such as printed patterns on electronic circuit boards and circuit boards with electronic parts mounted thereon, captured images of urban or natural environments, medical images such as X-ray images, and captured images of food packaging and contents.

[0048] In this example, the covariance matrix Σ and mean μ that can be used to calculate the Mahalanobis distance are used as examples of statistical information representing the distribution of feature vectors. The statistical information representing the distribution of feature vectors is not limited to the covariance matrix Σ and mean μ. By using the statistical information representing the distribution of feature vectors, it is possible to accurately compare the distribution of feature vectors derived from multiple normal images 11 with the feature vectors derived from the test image 13.

[0049] Example 2 This example is an example of an inspection system 1 that employs a CNN that uses a ReLU function as an activation function as a CNN that extracts feature vectors from an inspection image based on the configuration of Example 1. The contents of this example will be described with reference to Figs. 7 to 9.

[0050] The inspection system 1 of this example (FIG. 7) differs from the configuration of the first embodiment in that the statistical processing circuit that acquires statistical information representing the distribution of low-level feature vectors derived from normal images is omitted from the learning unit 2.

[0051] Similar to the first embodiment, the feature extraction circuits are provided in the learning unit 2 and the determination unit 3. The feature extraction circuit 22 of the learning unit 2 and the feature extraction circuit 32 of the determination unit 3 have common specifications. As described above, the feature extraction circuits 22 and 32 of this embodiment are circuits that use a CNN that uses a ReLU function as an activation function. The CNN of this embodiment has common specifications with the CNN of the first embodiment, except that the ReLU function is used as the activation function. Similar to the CNN of the first embodiment, the CNN of this embodiment extracts a 512-dimensional high-order feature vector (an example of an original feature vector) from an image.

[0052] The ReLU function, also known as a ramp function, outputs zero and positive values ​​as they are, while replacing negative values ​​with zero. The CNNs that make up the feature extraction circuits 22 and 32 in this example use the ReLU function as an activation function, and therefore do not output negative features.

[0053] The feature extraction circuit 22 of the learning unit 2 extracts a high-order feature vector from each of the multiple normal images. The high-order feature vector extracted by a CNN with a ReLU function as its activation function contains many zero features. Therefore, among the multiple high-order feature vectors derived from multiple normal images, an axis appears on which all features are zero. On an axis on which all features across multiple normal images are zero, both the mean and variance are naturally zero.

[0054] FIG. 8 is a diagram showing the number of axes whose average feature value is zero for each pixel 223 in a feature map 220 generated by the CNN of this example. In the feature map 220 in the figure, pixels 223 with more axes whose average feature value is zero are shown darker. The darker the pixel 223 appears, the more axes whose average feature value is zero, and the brighter the pixel 223 appears, the fewer axes whose average feature value is zero. Note that the figure does not explicitly show the number of axes whose average feature value is zero, but the number of such axes in the figure is approximately 0 to 20.

[0055] The dimension reduction circuit 24 in this example selects an axis on which the average value of the feature is zero from among the 512 axes that define the 512-dimensional multidimensional space to which the high-order feature vector (original feature vector) belongs. Such axis selection is performed for each pixel 223 in the feature map 220. Information on the axis selected in this way is stored in the storage device 25 as selected axis information for each pixel 223.

[0056] Unlike in the first embodiment, where the number of axes selected is fixed at 50, the number of axes selected in this embodiment varies for each pixel 223 in the feature map 220. In this way, the dimension reduction circuit 24 converts each of the multiple high-order feature vectors extracted from the multiple normal images into a low-order feature vector. Note that the feature values ​​for each axis of each low-order feature vector are all zero. Therefore, the multiple low-order feature vectors derived from the multiple normal images are all zero vectors.

[0057] Next, the feature extraction circuit 32 of the determination unit 3 extracts a high-order feature vector from the inspection image. The CNN used by this feature extraction circuit 32 is a CNN that employs the ReLU function as an activation function, similar to the feature extraction circuit 22 of the learning unit 2. The high-order feature vector is extracted for each pixel of the feature map of the inspection image.

[0058] The dimension reduction circuit 34 performs dimensionality reduction on the high-order feature vector extracted from the inspection image for each pixel of the feature map. The dimension reduction circuit 34 reads the selected axis information for each pixel 223 of the feature map 220 of the normal image from the storage device 25, and generates a low-order feature vector by selecting an axis indicated by the selected axis information from the high-order feature vector derived from the inspection image. As described above, the number of axes indicated by the selected axis information differs for each pixel 223 of the feature map 220. Therefore, the number of axes (number of dimensions) of the low-order feature vector differs for each pixel of the feature map of the inspection image. The low-order feature vector for each pixel of the feature map of the inspection image is input to the determination circuit 36.

[0059] As in the first embodiment, the determination circuit 36 ​​determines whether or not an abnormality exists in the inspection image by comparing the distribution of the plurality of low-level feature vectors derived from the plurality of normal images with the low-level feature vector derived from the inspection image. Such comparison of the low-level feature vectors is performed for each pixel in the feature map of the inspection image. As described above, the plurality of low-level feature vectors derived from the plurality of normal images are all zero vectors. Therefore, the plurality of low-level feature vectors are not distributed in multidimensional space but are concentrated at the zero point. Therefore, the distance by which the low-level feature vector derived from the inspection image deviates from the distribution of the plurality of low-level feature vectors derived from the plurality of normal images is the scalar (length) of the low-level feature vector derived from the inspection image.

[0060] The determination circuit 36 ​​calculates the scalar (length) of the low-level feature vector for each pixel of the feature map of the inspection image, then performs threshold processing on the scalar of each pixel, and determines pixels whose low-level feature vector scalar exceeds the threshold as abnormal pixels.

[0061] According to the configuration of this example, the presence or absence of an abnormality can be determined simply by applying threshold processing to the scalar of the low-order feature vector of each pixel in the feature map of the inspection image. This configuration further reduces the amount of calculation required to determine the presence or absence of an abnormality in the inspection object.

[0062] As mentioned above, Figure 8 shows the frequency of axes with all zero feature values ​​across multiple normal images. As can be seen from the figure, the number of such axes is reduced on the periphery of the feature map 220. If the number of axes with all zero feature values ​​is reduced, the dimensionality of the low-level feature vector used for anomaly detection will be excessive, and the number of feature types constituting the low-level feature vector will be excessively reduced. In such cases, the accuracy of determining whether or not an anomaly exists may be impaired. One of the reasons for the reduced number of axes with all zero feature values ​​on the periphery of the feature map 220 is the padding process that occurs during the convolution operation used by the CNN to calculate the features.

[0063] Convolution is a process in which a grid-like weight matrix called a kernel or filter is superimposed on a window in an image to obtain a filter value. The kernel has a two-dimensional area, so the kernel may extend outside the image at the edge of the image. In convolution, padding is commonly used so that feature values ​​can be calculated even if part of the kernel extends outside the image. Padding is a process that expands the image area by setting a predetermined value, such as zero, outside the image.

[0064] Therefore, it is also possible to set an inspection area 228 on the feature map 220 as shown in Figure 9, and exclude the outer periphery of the feature map 220 (corresponding to the outer periphery of the inspection image), which is susceptible to the effects of padding, from the target of anomaly detection (inspection object). Excluding the outer periphery of the feature map 220 from the target of anomaly detection can suppress the effect of padding on anomaly detection, and reduce erroneous detections caused by padding. The inspection area 228 in the figure excludes the entire outer periphery of the inspection image from the inspection object. Alternatively, an inspection area may be set that excludes at least a portion of the outer periphery of the inspection image from the inspection object. The other configurations and effects are the same as those of the first embodiment.

[0065] Example 3 This example compares the calculation cost and accuracy of anomaly detection. In this example, the following four anomaly detection methods are compared. Method 1, Method 2, and Method 3 are anomaly detection methods that utilize the present invention. Method 4 is a conventional anomaly detection method that utilizes principal component analysis. This will be described with reference to FIGS. 9 and 10.

[0066] (Method 1) An anomaly determination method according to the first embodiment, which reduces the dimensions of the high-order feature vector by selecting 50 axes with small variance values. (Method 2) An anomaly determination method described as another example in the first embodiment, in which 50 axes with small average values ​​are selected to reduce the dimensions of the high-order feature vector. (Method 3) Anomaly determination method according to Example 2. Note that an inspection area 228 (see FIG. 9) is set on the feature map 220 so as to avoid the influence of padding processing. (Method 4) A reference example of a conventional anomaly detection method that reduces the dimensions of high-order feature vectors by selecting 50 axes with small variance through principal component analysis.

[0067] In proposed method 1 and method 2, and conventional method 4, 50 axes are selected from 512 axes through dimensionality reduction. The low-level feature vector after dimensionality reduction is a vector composed of features on 50 axes. In proposed method 3, the number of axes selected from the 512 axes is indefinite. The low-level feature vector after dimensionality reduction in method 3 is a vector composed of features on 10 to 20 axes. The number of axes of the low-level feature vector varies depending on the position on the feature map.

[0068] In this example, as shown in Figure 10, four anomaly detection methods are compared based on two criteria: detection accuracy and computational cost. AUC (Area Under the Curve, also known as AUROC) is used as an index of anomaly detection accuracy. AUC is well known as an evaluation index for binary classification tasks (problems) such as the presence or absence of anomalies. AUC can take a value between 0 and 1, with values ​​closer to 1 indicating higher accuracy. Additionally, the processing time per inspection image is used as an index of computational cost.

[0069] As can be seen from Fig. 10, the anomaly determination accuracy of methods 1 to 3 is comparable to that of method 4, which employs dimensionality reduction by principal component analysis. Meanwhile, the processing time per image of methods 1 to 3 is reduced to about one-third of that of method 4. In other words, the calculation costs of methods 1 to 3 are significantly reduced compared to method 4. In particular, the calculation cost of method 3 according to the second embodiment is further reduced than that of methods 1 and 2 according to the first embodiment.

[0070] Although specific examples of the present invention have been described in detail as examples, these examples merely disclose examples of the technology encompassed by the claims. Needless to say, the scope of the claims should not be interpreted as being limited by the configurations, numerical values, etc. of the specific examples. The claims encompass technologies that are obtained by variously modifying, changing, or appropriately combining the specific examples using publicly known technology and the knowledge of those skilled in the art. [Explanation of symbols]

[0071] 1. Inspection system 10 Leather seats (subject to inspection) 11 Normal image 13 Inspection images 2. Learning Department 20 Image DB 22, 32 Feature extraction circuit 220, 320 feature map 221, 321 CNN 223, 323 pixels 225, 325 High-dimensional feature vector (original feature vector) 226, 326 Low-level feature vector 24, 34 Dimensionality reduction circuit 25 Storage device 26 Statistical Processing Circuit 3 Judgment section 30 Imaging camera 36 Judgment circuit 38 Output Devices

Claims

1. a feature extraction circuit that extracts multiple types of feature amounts from an image; a dimension reduction circuit that reduces the dimension of an original feature vector composed of the plurality of types of feature amounts to convert it into a low-dimensional feature vector; a determination circuit for determining whether or not an abnormality exists in the inspection object by comparing a distribution of a plurality of low-level feature vectors derived from a plurality of normal images with a low-level feature vector derived from an inspection image that is a captured image of the inspection object, or by comparing each feature vector among the plurality of low-level feature vectors derived from the plurality of normal images with a low-level feature vector derived from the inspection image, the low-level feature vector is a vector belonging to a multidimensional space defined by some of a plurality of axes that define the multidimensional space to which the original feature vector belongs, The part of the axes is an axis among a plurality of axes that define a multidimensional space to which the original feature vector belongs, and at least one of the mean value and variance value of the feature across a plurality of normal images is smaller than a threshold value.

2. 2. The inspection system according to claim 1, wherein the judgment circuit is configured to judge the presence or absence of an abnormality in the inspection object by threshold processing on the distance in multidimensional space between a distribution of the plurality of low-order feature vectors and a low-order feature vector derived from the inspection image, or the distance in multidimensional space between an individual feature vector among the plurality of low-order feature vectors and a low-order feature vector derived from the inspection image.

3. 3. The method according to claim 1, wherein the plurality of feature quantities output by the feature quantity extraction circuit are all feature quantities output via a ramp function that outputs zero values ​​and positive values ​​as they are, while replacing negative values ​​with zero values; An inspection system wherein the some axes are axes whose mean values ​​are zero.

4. 4. The method according to claim 3, wherein the feature extraction circuit is configured to extract the plurality of feature quantities by utilizing a convolution operation, An inspection system, wherein the judgment circuit is configured to set an inspection area on the inspection image that excludes at least a portion of the outer periphery of the inspection image and judge whether or not there is an abnormality in the inspection object.

5. A testing method for determining whether or not an abnormality is present in a test object, comprising: A process of extracting multiple types of features from an image; a process of reducing the dimension of the original feature vector composed of the plurality of types of feature amounts and converting it into a low-dimensional feature vector; a process of determining whether or not an abnormality exists in the inspection object by comparing a distribution of a plurality of low-level feature vectors derived from a plurality of normal images with a low-level feature vector derived from an inspection image, which is an image captured of the inspection object, or by comparing each feature vector among the plurality of low-level feature vectors derived from a plurality of normal images with a low-level feature vector derived from the inspection image, the low-level feature vector is a vector belonging to a multidimensional space defined by some of a plurality of axes that define the multidimensional space to which the original feature vector belongs, The part of the axes is an axis among a plurality of axes defining a multidimensional space to which the original feature vector belongs, on which at least one of the average value and variance value of the feature across a plurality of normal images is smaller than a threshold value.

6. 6. An inspection method according to claim 5, wherein the process of determining the presence or absence of an abnormality is a process of determining the presence or absence of an abnormality in the inspection object by threshold processing on the distance in multidimensional space between a distribution of the plurality of low-order feature vectors and a low-order feature vector derived from the inspection image, or the distance in multidimensional space between an individual feature vector of the plurality of low-order feature vectors and a low-order feature vector derived from the inspection image.

Images