Information processing device, method, and program

Abstract

Provided is a technology for enabling an AI model that accurately detects an abnormality to be built using a limited number of images. This information processing device comprises a memory and a processor. The memory stores image data. The processor inputs the image data into a diffusion model that has a forward process and a backward process each formed from a plurality of steps, extracts, as intermediate layer features, data of an intermediate layer in a prescribed step of the backward process, reduces the order of the intermediate layer features, generates fewer reduced-order feature amounts than the number of the intermediate layer features, and records the reduced-order feature amounts in the memory.

Description

Information processing device, method, and program

[0001] The present disclosure relates to techniques for detecting abnormal images using artificial intelligence.

[0002] There have been attempts to introduce artificial intelligence (AI) into the analysis of images acquired by scanning electron microscopes (SEMs). AI analyzing SEM images is required to perform an anomaly detection task, which is to determine whether an input image is abnormal or not. In recent years, a wide range of methods have been researched for this task, and recently, embedding-based methods have produced good results. In an embedding-based method, features of normal images are extracted and stored using a convolutional neural network (CNN) pre-trained on an image dataset. The image dataset is, for example, ImageNet. Next, features are extracted from the input image, and the extracted features are compared with the stored normal features to determine whether the input image is normal or abnormal.

[0003] Representative embedding-based anomaly detection methods are disclosed in Non-Patent Document 1 and Non-Patent Document 2.

[0004] The method disclosed in Non-Patent Document 1 is a method that uses feature comparison. In Non-Patent Document 1, a new teacher-student architecture consisting of a fixed teacher encoder E and a trainable student decoder D is introduced as Deng's model. In training, the student decoder D learns to imitate the behavior of the fixed teacher encoder E with normal samples. In inference, the fixed teacher encoder E and the student decoder D each extract features from images. A low similarity between these feature vectors indicates the presence of an abnormality.

[0005] The method in Non-Patent Document 2 is a method for generating abnormal features. In Non-Patent Document 2, SimpleNet, an anomaly detection and localization network that generates target-oriented features using feature adapters, is introduced. This reduces domain bias. SimpleNet generates abnormal features by introducing noise into normal features in the feature space, rather than directly synthesizing anomalies on the image. This enables significant improvements in both accuracy and efficiency in anomaly detection.

[0006] Deng H, Li X, “Anomaly detection via reverse distillation from one-class embedding”, In Proceedings of the IEEE / CVF CVPR 2022, pp. 9737-9746.Liu Z, Zhou Y, Xu Y, Wang Z. Simplenet, “A simple network for image anomaly detection and localization”, In Proceedings of the IEEE / CVF CVPR 2023, pp. 20402-20411.

[0007] In an image recognition system that uses AI to detect image anomalies, it is necessary to build an AI model that can accurately detect anomalies using a limited number of images.

[0008] One objective of the present disclosure is to provide a technology that enables the construction of an AI model that can accurately detect anomalies using a limited number of images.

[0009] An information processing device according to one aspect of the present disclosure includes a memory and a processor, wherein the memory stores image data, and the processor inputs the image data to a diffusion model having a forward process and a backward process, each of which has a plurality of steps, extracts intermediate layer data at a predetermined step of the backward process as intermediate layer features, reduces the dimension of the intermediate layer features, generates lower-dimensional features that are fewer in number than the intermediate layer features, and records the lower-dimensional features in the memory.

[0010] According to one aspect of the present disclosure, it is possible to check whether target image data is normal or abnormal using low-dimensional features obtained by reducing the dimension of intermediate layer features of image data.

[0011] 1 is a block diagram showing a functional configuration of an information processing device; FIG. 2 is a block diagram showing a hardware configuration of an information processing device; FIG. 3 is a flowchart of an image normality determination process; FIG. 4 is a flowchart of a training process; FIG. 5 is a flowchart of a feature quantity selection process; FIG. 6 is a flowchart of a test process; FIG. 7 is a diagram showing an example of a GUI for the training process; FIG. 8 is a diagram showing an example of a GUI for the test process; and FIG. 9 is a table recording experimental results.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] 1 is a block diagram showing the functional configuration of an information processing device according to this embodiment. The information processing device 10 of this embodiment is a device that detects abnormal images using artificial intelligence (AI). The images are, for example, images acquired by a SEM. The information processing device 10 includes an input / output unit 11, an information acquisition unit 12, a feature storage unit 13, a bus 14, a feature dimension reduction unit 15, a feature selection unit 16, a feature storage unit 17, and a distance-based image classification unit 18.

[0014] The input / output unit 11 inputs and outputs information and data from and to the outside. Input of information and data includes importing image data from an external device, input of information by user operation, etc. Output of information and data includes output of a graphical user interface (GUI) screen for accepting operations and presenting processing results, and transmission of processing results to an external device, etc.

[0015] The information acquisition unit 12 acquires information using AI. The AI ​​is a diffusion model equipped with a neural network and executing a forward process and a backward process, each of which consists of multiple steps. The diffusion model is a generative AI model generally used for image generation, etc., but in this embodiment, the diffusion model is used to detect abnormal images. The AI ​​may be provided externally to the information processing device 10 or may be implemented in the information processing device 10 itself. The information acquisition unit 12 inputs image data into the diffusion model and acquires data from each intermediate layer of the neural network at a predetermined step of the backward process. The number of intermediate layers is not particularly limited. For example, the number of intermediate layers may be determined depending on the resolution of the target image data. Furthermore, there is no particular limit to which step of the backward process data is acquired from. For example, the user may specify which step data is to be acquired, or a step at a predetermined position within the entire backward process may be identified as the step from which data is to be acquired based on the total number of steps in the backward process.

[0016] The feature storage unit 13 records the intermediate layer data acquired by the information acquisition unit 12 as intermediate layer feature amounts.

[0017] The bus 14 interconnects the input / output unit 11, the feature storage unit 13, the feature dimension reduction unit 15, the feature selection unit 16, the feature storage unit 17, and the distance-based image classification unit 18.

[0018] The feature dimension reduction unit 15 reduces the dimensions of the intermediate layer features recorded in the feature storage unit 13. There are multiple intermediate layer features. That is, the intermediate layer features are feature vectors of multiple dimensions. As an example, the feature dimension reduction unit 15 performs principal component analysis on the intermediate layer features to synthesize principal components that are uncorrelated with each other and whose number is less than the number of dimensions of the intermediate layer features. Hereinafter, the principal components obtained here may be referred to as synthesized features. The number of dimensions is reduced while retaining the information contained in the intermediate layer features.

[0019] The feature selection unit 16 acquires, from among the composite features generated by the feature dimension reduction unit 15, composite features whose variance values ​​exceed a predetermined threshold, and discards composite features whose variance values ​​are equal to or less than the threshold, thereby selecting features with even fewer dimensions than the composite features. The features selected here may be referred to as low-dimensional features hereinafter. Note that in this example, the variance value of a composite feature is used to determine whether to retain or discard the feature as a low-dimensional feature. However, as another example, an eigenvalue or a contribution rate may be used instead of the variance when selecting a low-dimensional feature from among the composite features.

[0020] The feature storage unit 17 stores information on the low-dimensional features extracted by the feature selection unit 16 .

[0021] The distance-based image classification unit 18 calculates the similarity between the low-dimensional features of an image known to be normal and the low-dimensional features of an image (target image) whose normality or abnormality is unknown and which is the subject of inference, and determines whether the target image is a normal image or an abnormal image based on the similarity.

[0022] FIG. 2 is a block diagram showing the hardware configuration of the information processing apparatus according to this embodiment.

[0023] The information processing device 10 in this embodiment has a hardware configuration similar to that of a typical computer, and includes at least a processor 21, a main memory device 22, an auxiliary memory device 23, an input device 24, an output device 25, and a communication device 26.

[0024] The processor 21 is a computing device that controls the overall operation of the information processing device 10, controls the transmission and reception of data between each element, and performs information processing necessary for executing applications. For example, the processor 21 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and performs various information processing by executing programs stored in the auxiliary storage device 23 and deployed in the main storage device 22. By executing the programs stored in the auxiliary storage device 23, the processor 21 functions as the input / output unit 11, the information acquisition unit 12, the feature dimension reduction unit 15, the feature selection unit 16, and the distance-based image classification unit 18 described with reference to FIG. 1 .

[0025] The main memory device 22 includes a volatile memory device such as a RAM (Random Access Memory), a DRAM (Dynamic Random Access Memory), etc. The main memory device 22 is used as a work area for the processor 21, etc.

[0026] The auxiliary storage device 23 includes a nonvolatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs such as application programs and programs for implementing the functions of this embodiment. The auxiliary storage device 23 also at least temporarily stores information acquired by the communication device 26 and the input device 24, and various data (including intermediate data) generated by processing operations by the processor 21. A database for storing such data may be constructed in the auxiliary storage device 23.

[0027] One or both of the main memory device 22 and the auxiliary memory device 23 constitute the feature amount saving unit 13 and the feature amount storage unit 17 shown in FIG.

[0028] The input device 24 is an input interface that accepts inputs by the user to operate the information processing device 10, and can be composed of, for example, a keyboard, a pointing device such as a mouse or joystick, a touch panel, an audio microphone, or any combination thereof.

[0029] The output device 25 is a display device that displays input image data and information generated by the processor 21 and provides it to the user.

[0030] The communication device 26 is a communication interface for communicating with external devices via a communication network such as the Internet, an intranet, a local area network (LAN), or a wide area network (WAN), or via a wired connection such as a universal serial bus (USB).

[0031] The information processing device 10 of this embodiment does not have to be a physical computer as illustrated in FIG. 2, but may be, for example, a virtual computer configured on a cloud.

[0032] 3 is a flowchart of the image normality determination process, which determines whether a target image is normal or abnormal.

[0033] 3 , in step 31, the information processing device 10 executes a training process. The training process is a preparatory process for determining the normality of a target image. The training process may be started by a user instruction operation. Subsequently, in step 32, the information processing device 10 executes a test process. The test process may be started by a user instruction.

[0034] 4 is a flowchart of the training process. When the training process is started (step 101), the input / output unit 11 selects a normal image from a database 102 that stores images known to be normal (hereinafter also referred to as normal images) as a sample to be used in this process (step 103), and uses this as input for the training process (step 104). One or more normal images may be selected as the sample, but here it is assumed that one normal image is selected.

[0035] Next, the information acquisition unit 12 repeats the forward process of the diffusion model until the number of iterations reaches a predetermined total number of steps (steps 105 to 108). In the forward process, first, random noise is added to the image data (step 105), and it is determined whether the number of iterations has reached the total number of steps (step 106). If the number of iterations has not reached the total number of steps, the image data to which the random noise has been added is returned to the next input (step 107). If the number of iterations has reached the total number of steps, the last image data to which noise has been added is passed to the next process (step 108).

[0036] Next, the information acquisition unit 12 repeats the backward process of the diffusion model until the number of iterations reaches a predetermined total number of steps (steps 109 to 111). In the backward process, the last image data to which noise has been added in the forward process is used as the first input, noise is removed from the image data (step 109), and it is determined whether the number of iterations has reached the total number of steps (step 110). If the number of iterations has not reached the total number of steps, the image data after noise removal is returned as the next input (step 111). If the number of iterations has reached the total number of steps, data from each intermediate layer of the neural network at a predetermined step of the backward process is acquired. In step 112, the feature storage unit 13 records the intermediate layer data acquired by the information acquisition unit 12 as intermediate layer features.

[0037] Next, in step 113, the feature dimension reduction unit 15 performs principal component analysis on the intermediate layer features recorded in the feature storage unit 13 to synthesize composite features having a smaller number of dimensions than the intermediate layer features. Furthermore, in step 114, the feature selection unit 16 selects low-dimensional features having an even smaller number of dimensions than the composite features through feature selection processing. Details of the feature selection processing will be described later. In step 115, the feature storage unit 17 stores the low-dimensional features extracted by the feature selection unit 16, and the training processing ends (step 116).

[0038] 5 is a flowchart of the feature selection process. When the feature selection process is initiated (step 301), the feature selection unit 16 measures the variance of each composite feature in step 302. Next, the variance of each composite feature is compared with a predetermined threshold (step 303), and composite features with variances below the threshold are discarded (step 304). Composite features with variances above the threshold are selected as low-dimensional features (step 305). The feature selection unit 16 then records the selected low-dimensional features (step 306) and terminates the feature selection process (step 307).

[0039] 6 is a flowchart of the test process. When the test process is started (step 201), the input / output unit 11 selects one of the target images from the database 202 storing the target images as a sample to be used in this process (step 203), and sets this as the input for the test process (step 204). It is assumed that one target image is selected as the sample.

[0040] Next, the information acquisition unit 12 repeats the forward process of the diffusion model until the number of iterations reaches a predetermined total number of steps (steps 205 to 208). In the forward process, first, random noise is added to the image data (step 205), and it is determined whether the number of iterations has reached the total number of steps (step 206). If the number of iterations has not reached the total number of steps, the image data to which the random noise has been added is returned to the next input (step 207). If the number of iterations has reached the total number of steps, the last image data to which noise has been added is passed to the next process (step 208).

[0041] Next, the information acquisition unit 12 repeats the backward process of the diffusion model until the number of iterations reaches a predetermined total number of steps (steps 209 to 211). In the backward process, the last image data to which noise was added in the forward process is used as the first input, noise is removed from the image data (step 209), and it is determined whether the number of iterations has reached the total number of steps (step 210). If the number of iterations has not reached the total number of steps, the image data after noise removal is returned as the next input (step 211). If the number of iterations has reached the total number of steps, data from each intermediate layer of the neural network at a predetermined step of the backward process is acquired. In step 212, the feature storage unit 13 records the intermediate layer data acquired by the information acquisition unit 12 as intermediate layer features.

[0042] Next, in step 213, the feature dimension reduction unit 15 reduces the dimensions of the intermediate layer features recorded in the feature storage unit 13 and synthesizes composite features having a smaller number of dimensions than the intermediate layer features. At this time, the feature dimension reduction unit 15 may project the intermediate layer features of the target image onto the same dimensions in the metric space as the composite features of the normal image synthesized in step 113 of the training process. Furthermore, in step 214, the feature selection unit 16 selects low-dimensional features having an even smaller number of dimensions than the composite features extracted in step 213. At this time, the feature dimension reduction unit 15 may select, as low-dimensional features, composite features of the target image having the same dimensions as the low-dimensional features of the normal image selected in step 114 of the training process.

[0043] In step 215, the distance-based image classifying unit 18 calculates the distance (D) between the position indicated by the vector representing the low-dimensional feature amount of the normal image stored in the feature amount storage unit 17 and the position indicated by the vector representing the low-dimensional feature amount of the target image extracted in step 214. The distance D is, for example, the Euclidean distance.

[0044] Next, in step 216, the distance-based image classification unit 18 determines whether the distance D calculated in step 215 is greater than a predetermined threshold. If the distance D is equal to or less than the threshold, the distance-based image classification unit 18 determines that the target image is an abnormal image (step 217), and if the distance D is greater than the threshold, the distance-based image classification unit 18 determines that the target image is a normal image (step 218).

[0045] Although the present embodiment exemplifies the case where one normal image is used as a sample in the training process, multiple normal images may be used as samples in the training process. In this case, the degree of similarity between the low-dimensional feature values ​​of each of the multiple normal images and the low-dimensional feature values ​​of the target image is calculated, the average value of these similarity values ​​is calculated, and the average value is compared with a threshold value to determine whether the target image is a normal image or an abnormal image. Furthermore, although the present embodiment exemplifies the case where one target image is used as a sample in the test process, multiple target images may be used as samples in the test process. In this case, the above-described determination may be performed for each of the multiple target images.

[0046] As described above, the training process (FIG. 4) and the test process (FIG. 6) may be performed by user operation.

[0047] 7 and 8 are diagrams showing examples of GUIs for training and testing processes, respectively.

[0048] 7 shows a GUI screen 501 for the training process. The GUI screen 501 includes an image type selection object 503, a time step selection object 504, a training image selection object 505, a diffusion model selection object 506, a size selection object 507, and a feature log object 508.

[0049] The image type selection object 503 is an object that allows the user to select the image type of the normal image to be used in the training process. Selectable image types are displayed in a pull-down menu, and the user can select one of the displayed image types. In the example of Figure 7, the image type BAR is selected.

[0050] The time step selection object 504 allows the user to select the total number of steps in the forward process and the backward process of the diffusion model used in the training process. The selectable total number of steps is displayed in a pull-down menu, and the user can select one of the displayed total number of steps. Also displayed in association with the time step selection object 504 are a "Time Step" graph representing the total number of time steps, a "Run Time" graph representing the execution time, and an "Error" graph representing the accuracy. The "Time Step," "Run Time," and "Error" graphs indicate the level of the selection made by the user for the time step selection object 504. In the example of FIG. 7 , the selectable total number of steps is displayed as "10," "50," and "100." Since no total number of steps has been selected, the "Time Step" graph shows zero, the "Run Time" graph shows zero, and the "Error" graph shows the maximum.

[0051] The training image selection object 505 is an object that allows the user to select the number of normal images to be used in the training process. The selectable number is displayed in a pull-down menu, and the user can select one from the displayed number. Also displayed in association with the training image selection object 505 are a "Training Images" graph showing the number of images, a "Run Time" graph showing the execution time, and an "Error" graph showing the accuracy. The "Training Images," "Run Time," and "Error" graphs indicate the level of the selection made by the user for the training image selection object 505. The example in FIG. 7 indicates that the selectable numbers of normal images are "10," "50," and "100." Since the number of normal images has not been selected, the "Training Images" graph shows zero, the "Run Time" graph shows zero, and the "Error" graph shows the maximum.

[0052] The diffusion model selection object 506 is an object that allows the user to select the size of the diffusion model to be used in the training process. Selectable diffusion model sizes are displayed in a pull-down menu, and the user can select one of the displayed sizes. In the example of FIG. 7 , the selectable diffusion model sizes are shown as "Small" and "Large." No diffusion model size is selected in this state.

[0053] The size selection object 507 is an object for selecting the size of the intermediate layer for acquiring features in the training process. Selectable intermediate layer sizes are displayed in a pull-down menu, and the user can select one of the displayed sizes. In the example of FIG. 7 , the selectable intermediate layer sizes are shown as "All" and "Few." No intermediate layer size is selected.

[0054] The feature log object 508 is an object that displays the points where features are acquired as logs in the training process. In the example of Fig. 7, features are acquired as logs from the encoder in the forward process and the decoder in the backward process.

[0055] 8, there is shown a test processing GUI screen 502. The GUI screen 502 includes an image type selection object 509, a time step selection object 510, and a test execution object 511.

[0056] The image type selection object 509 is an object that allows the user to select the image type of the target image to be used in the test process. Selectable image types are displayed in a pull-down menu, and the user can select one of the displayed image types. In the example of Figure 8, the image type BAR is selected.

[0057] The time step selection object 510 is an object that allows the user to select the total number of steps in the forward process and the backward process of the diffusion model to be used in the test process. The selectable total number of steps is displayed in a pull-down menu, and the user can select one from the displayed total number of steps. In the example of FIG. 8, the selectable total number of steps is shown to be "10," "50," and "100." The total number of steps is not selected.

[0058] The test execution object 511 is an object that allows the user to start the execution of a test process. When the user presses the test execution object 511 with a mouse or the like, the test process is executed.

[0059] According to the information processing device 10 of this embodiment described above, it is possible to check whether the target image data is normal or abnormal using low-dimensional features obtained by reducing the dimension of the intermediate layer features of the image data.

[0060] Although the present embodiment shows an example in which images acquired by an SEM are the target, the information processing device 10 can also handle images acquired in various other fields such as manufacturing, healthcare, and security.

[0061] A specific example will be described below.

[0062] In recent years, text-to-image models trained on large volumes of text-tagged image data have become capable of converting human imagination into high-quality images with high accuracy. Deep features extracted from these text-to-image models enable rich representation of the input content and style. However, the usefulness of these deep features for anomaly detection tasks has remained unclear.

[0063] Therefore, the inventors investigated the Stable Diffusion model, which is a state-of-the-art pre-trained, fixed text-to-image diffusion model (LDM: Latent Diffusion Model). The Stable Diffusion model is based on the U-Net architecture. Hereinafter, the Stable Diffusion model may be simply referred to as the model.

[0064] This study addresses the fundamental question of whether information about anomalous regions is encoded within the model. We explore in detail the intermediate spatial features generated during the image generation process using the Stable Diffusion model, analyze them empirically, and propose a new approach for anomaly detection based on spatial features obtained through direct manipulation.

[0065] This approach utilizes a pre-trained and fixed text-to-image diffusion model without the need to retrain or fine-tune the pre-trained model.

[0066] Specifically, self-attention features are obtained from spatial features extracted from a normal image as guidance. The self-attention mechanism uses the spatial features of the entire image as input and calculates attention features based on pixel relationships. Therefore, the self-attention feature map can be considered to capture the relationships between distant pixels in the image. This makes it possible to detect anomalies by utilizing the relationships between distant features through self-attention.

[0067] A dimension reduction technique such as principal component analysis (PCA) is applied to the feature quantities suitable for anomaly detection obtained from the Stable Diffusion model, thereby reducing the dimension of the feature quantities while maintaining important information.

[0068] The model used in this study is a pre-trained stable diffusion model, and its neural network is θ (x ｔ , P, t), where x ｔis the image generated at time step t, and is given by the following equation (1): P is the guiding prompt. In the learning stage, Gaussian noise is introduced into the normal sample x during the forward process. ０ will be gradually added to.

[0069] x ｔ =√(α ｔ ) x ０ +√(1-α ｔ ) z (1)

[0070] Here, z∼N(0,I) and {α ｔ} represents the schedule of noise added at each time step. The noise follows a standard normal distribution. I is x ｔ is the identity matrix of the same dimension as α ｔ weights the noise added to time step t.

[0071] When the time step t reaches a predetermined total number of time steps in the forward process, the neural network e θ current x ｔ to get a cleaner, noise-removed x （ｔ－１） In this backward process, we perform a backward process consisting of time steps to obtain the neural network e θ Collect the features of the intermediate layers for each layer.

[0072] Then, a self-attention mechanism and PCA-based dimensionality reduction are applied to these intermediate features to generate low-dimensional features that efficiently aggregate the features relevant to anomaly detection while minimizing irrelevant and redundant information, thereby reducing the computational load of anomaly detection. These refined features (low-dimensional features) are stored in memory.

[0073] In the test process, the low-dimensional features of a normal image sample and the low-dimensional features of a target image sample are simply compared using a distance-based classifier, and if the distance between them is below a threshold, the target image is determined to be an abnormal image.

[0074] In this embodiment, the neural network eθ We used PCA analysis to identify anomaly information present in the features of each layer. This analysis allows us to exploit key visual characteristics present in the pre-trained high-dimensional features of stable diffusion.

[0075] Specifically, we apply a diffusion process to each image in the dataset, generating the same image without the guiding prompt P. We then extract self-attention features from each layer at specific time steps. We then perform principal component analysis across the extracted hidden layer features, which allows us to reason about the visual properties present in the high-dimensional hidden layer features.

[0076] Here, we conducted experiments on anomaly detection tasks on SEM images of four image types: BCAT_L, GBL_L, BAR_L, and DOT_L, which correspond to different image categories.

[0077] The normal images used in the training process and the target images used in the test process are determined in advance to be abnormal or not. In the case of image type BCAT_L and image type GBL_L, 300 images determined to be abnormal are used as the normal images used in the training process, and a total of 581 images including both images with and without abnormalities are used as the target images used in the test process. In the case of image type BAR_L, 35 normal images are used in the training process and 145 target images are used in the test process. In the case of image type DOT_L, 34 normal images are used in the training process and 143 target images are used in the test process.

[0078] FIG. 9 is a table recording the experimental results. The table in FIG. 9 shows scores indicating the percentage of times a target image was correctly determined to be normal or abnormal (accuracy rate) for the method (SimpleNet) disclosed in Non-Patent Document 2 and the method of this embodiment. For image type BCAT_L, the accuracy rate for the SimpleNet method was 0.370, while the accuracy rate for the method of this embodiment was 0.652, a significant improvement of approximately 28%. Furthermore, the accuracy rate for the method of this embodiment was improved over the SimpleNet method for all image types. The experimental results in FIG. 8 show that the accuracy rate for the method of this embodiment is improved compared to SimpleNet, and that the method of this embodiment can achieve a practical accuracy rate depending on the image type and application.

[0079] The above-described embodiments and examples are merely illustrative examples of the present invention and are not intended to limit the scope of the present invention. Those skilled in the art can implement the present invention in various other forms without departing from the scope of the present invention.

[0080] The above-described embodiments and examples include the following features, but the features included in the above-described embodiments are not limited to those listed below.

[0081] (Item 1)

[0082] The information processing device includes a memory and a processor, wherein the memory stores image data, and the processor inputs the image data to a diffusion model having a forward process and a backward process, each of which has a plurality of steps, extracts intermediate layer data at a predetermined step of the backward process as intermediate layer features, reduces the dimension of the intermediate layer features, generates lower-dimensional features that are fewer in number than the intermediate layer features, and records the lower-dimensional features in the memory. This makes it possible to determine whether the target image data is normal or abnormal based on the lower-dimensional features obtained by reducing the dimension of the intermediate layer features of the image data.

[0083] (Item 2)

[0084] In the information processing device described in item 1, the memory stores image data of a normal image and image data of a target image, and the processor generates low-dimensional features of the image data of the normal image, generates low-dimensional features of the image data of the target image, and calculates a degree of similarity between the low-dimensional features of the normal image and the low-dimensional features of the target image. This makes it possible to determine whether the target image is normal or abnormal by comparing the low-dimensional features of the target image with the low-dimensional features of the normal image.

[0085] (Item 3)

[0086] In the information processing device described in item 2, the processor calculates a vector distance between a feature vector represented by the low-dimensional feature of the normal image and a feature vector represented by the low-dimensional feature of the target image, and if the vector distance is greater than a predetermined threshold, determines that the target image is an abnormal image. This makes it possible to easily make a judgment based on the vector distance of the low-dimensional feature.

[0087] (Item 4)

[0088] In the information processing device described in item 1, the processor converts the intermediate layer features into mutually orthogonal principal components by principal component analysis, selects principal components from the principal components whose variance is greater than a predetermined threshold, and sets the selected principal components as the low-dimensional features. This reduces the amount of information in the features while suppressing information loss through principal component analysis, and further reduces the dimension of the features effectively by selecting only principal components with large variance.

[0089] (Item 5)

[0090] In the information processing device described in item 1, the processor displays a screen that allows the user to specify the number of iterations of the forward and backward steps of the diffusion model, and acquires the intermediate layer feature values ​​through the forward and backward steps of the number of iterations accepted from the screen. This makes it possible to appropriately set the number of iterations and efficiently determine the desired accuracy. Increasing the number of iterations (number of time steps) increases accuracy but takes time. Reducing the number of iterations (number of time steps) decreases accuracy but saves time.

[0091] (Item 6)

[0092] In the information processing device described in item 1, the image data is data of an image acquired by a scanning electron microscope.

[0093] (Item 7)

[0094] In the information processing device described in item 2, the processor generates low-dimensional features of a plurality of normal images, calculates the degree of similarity between the low-dimensional features of each of the normal images and the low-dimensional features of the target image, and determines whether the target image is a normal image or an abnormal image based on the average value of the degrees of similarity.

[0095] (Item 8)

[0096] An information processing method executed by a computer includes storing image data, inputting the image data to a diffusion model having a forward process and a backward process, each of the forward process and the backward process consisting of a plurality of steps, extracting intermediate layer data at a predetermined step of the backward process as intermediate layer features, reducing the dimension of the intermediate layer features, generating lower-dimensional features that are fewer in number than the intermediate layer features, and recording the lower-dimensional features.

[0097] (Item 9)

[0098] The information processing program to be executed by a computer causes the computer to store image data, input the image data to a diffusion model having a forward process and a backward process, each of which has a plurality of steps, extract intermediate layer data at a predetermined step of the backward process as intermediate layer features, reduce the dimension of the intermediate layer features, generate lower-dimensional features that are fewer in number than the intermediate layer features, and record the lower-dimensional features.

[0099] 10...information processing device, 11...input / output unit, 12...information acquisition unit, 13...feature storage unit, 14...bus, 15...feature dimension reduction unit, 16...feature selection unit, 17...feature storage unit, 18...distance-based image classification unit, 21...processor, 22...main memory device, 23...auxiliary memory device, 24...input device, 25...output device, 26...communication device, 102...database, 202...database, 501...GUI screen, 502...GUI screen, 503...image type selection object, 504...time step selection object, 505...training image selection object, 506...diffusion model selection object, 507...size selection object, 508...feature log object, 509...image type selection object, 510...time step selection object, 511...test execution object

Claims

1. An information processing device having a memory and a processor, wherein the memory stores image data, and the processor inputs the image data to a diffusion model having a forward process and a backward process, each of which has a plurality of steps, extracts intermediate layer data at a predetermined step of the backward process as intermediate layer features, reduces the dimension of the intermediate layer features, generates lower-dimensional features that are fewer in number than the intermediate layer features, and records the lower-dimensional features in the memory.

2. The information processing device according to claim 1, wherein the memory stores image data of a normal image and image data of a target image, and the processor generates low-dimensional features of the image data of the normal image, generates low-dimensional features of the image data of the target image, and calculates the degree of similarity between the low-dimensional features of the normal image and the low-dimensional features of the target image.

3. The information processing device according to claim 2, wherein the processor calculates the inter-vector distance between a feature vector represented by the low-dimensional feature of the normal image and a feature vector represented by the low-dimensional feature of the target image, and if the inter-vector distance is greater than a predetermined threshold, determines that the target image is an abnormal image.

4. The information processing device according to claim 1, wherein the processor converts the intermediate layer features into mutually orthogonal principal components by principal component analysis, selects principal components from the principal components whose variance is greater than a predetermined threshold, and sets the selected principal components as the low-dimensional features.

5. The information processing device according to claim 1, wherein the processor displays a screen that enables the user to specify the number of iterations of the forward and backward steps of the diffusion model, and acquires the intermediate layer features through the forward and backward steps for the number of iterations accepted from the screen.

6. The information processing device according to claim 1, wherein the image data is data of an image acquired by a scanning electron microscope.

7. The information processing device described in claim 2, wherein the processor generates low-dimensional features of a plurality of normal images, calculates the degree of similarity between each low-dimensional feature of the normal images and the low-dimensional feature of the target image, and determines whether the target image is a normal image or an abnormal image based on the average value of the degrees of similarity.

8. An information processing method executed by a computer, comprising: storing image data; inputting the image data into a diffusion model having a forward process and a backward process, each of which has a plurality of steps; extracting intermediate layer data at a predetermined step of the backward process as intermediate layer features; reducing the dimension of the intermediate layer features; generating lower-dimensional features that are fewer in number than the intermediate layer features; and recording the lower-dimensional features.

9. An information processing program for causing a computer to execute the following steps: storing image data; inputting the image data into a diffusion model having a forward process and a backward process, each of which has a plurality of steps; extracting intermediate layer data at a predetermined step of the backward process as intermediate layer features; reducing the dimension of the intermediate layer features; generating lower-dimensional features that are fewer in number than the intermediate layer features; and recording the lower-dimensional features.

Images