Learning device, image generation device, learning method, and storage medium

Abstract

A learning device 1X mainly comprises a region-of-interest extraction means 30X, a pair generation means 32X, and a learning means 33X. The region-of-interest extraction means 30X extracts a region of interest from an abnormal region of an abnormal image, which includes an abnormal region of a target, and from a normal region other than the abnormal region. The pair generation means 32X generates a pair consisting of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region. The learning means 33X executes machine learning of an inference device that, on the basis of the pair, performs inference regarding a pseudo-abnormal image, in which a pseudo abnormality of the target appears, from a normal image representing a normal state of the target.

Description

Learning device, image generation device, learning method, and storage medium

[0001] The present disclosure relates to the technical fields of a learning device, an image generation device, a learning method, and a storage medium related to machine learning of a model for generating abnormal images.

[0002] There are known techniques for generating training data used in machine learning of models that detect abnormalities such as lesions. For example, Patent Literature 1 discloses a technique for generating fake tumor images from real normal images obtained by contrast-enhanced CT.

[0003] Japanese Patent Application Laid-Open No. 2023-180097

[0004] Training a model to detect anomalies requires images of abnormal conditions, but in the case of medical images, it is difficult to obtain realistic images of abnormal conditions. Therefore, it is possible to use a generative model based on deep learning to increase the number of pseudo-abnormal images. However, generative models that perform image transformation on the entire image have a limit to the amount of training data that can be increased.

[0005] In view of the above-mentioned problems, one of the objects of the present disclosure is to provide a learning device, an image generation device, a learning method, and a storage medium related to an inference device that performs inference regarding abnormal images with high accuracy.

[0006] One aspect of the learning device is a learning device having: a region of interest extraction means for extracting regions of interest from an abnormal region of an abnormal image having an abnormal region of an object and from a normal region other than the abnormal region; a pair generation means for generating pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region; and a learning means for executing machine learning of an inference device that makes inferences regarding a pseudo-abnormal image in which a pseudo-abnormality of the object is shown from a normal image that represents the normal state of the object, based on the pairs.

[0007] One aspect of the image generation device is an image generation device comprising: a region of interest extraction means for extracting a region of interest representing a normal state of an object from an image of the object; and a pseudo-abnormal image generation means for generating a pseudo-abnormal image, which is an image of the object in which the region of interest is replaced with a pseudo-abnormal region representing an abnormality in the object, based on the region of interest and a machine learning model, wherein the machine learning model is a model that has been machine-learned to determine the relationship between an image representing the normal state of the object and an image representing an abnormality in the object.

[0008] One aspect of the learning method is a learning method in which a computer extracts regions of interest from an abnormal region of an abnormal image of an object having the abnormal region and from a normal region other than the abnormal region, generates pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region, and performs machine learning of an inference device that makes inferences regarding a pseudo-abnormal image in which a pseudo-abnormality of the object is shown from a normal image that represents a normal state of the object, based on the pairs.

[0009] One aspect of the storage medium is a storage medium that stores a program that causes a computer to execute a process of extracting regions of interest from an abnormal region of an abnormal image having an abnormal region of an object and from a normal region other than the abnormal region, generating pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region, and performing machine learning of an inference device that makes inferences regarding a pseudo-abnormal image that shows a pseudo-abnormality of the object based on the pairs from a normal image that shows the normal state of the object.

[0010] As an example of an effect of the present disclosure, it is possible to train an inference device that can make inferences regarding abnormal data with high accuracy.

[0011] 1 shows a schematic configuration of a learning system; 2 shows a hardware configuration of a learning device; 3 shows an overview of the processing performed by the learning device; 4 shows an example of functional blocks of a learning device; 5 shows an abnormal image clearly indicating an abnormal ROI and a normal ROI; 6 shows a region including an extracted ROI in pixel units; 7 shows the extracted ROI and surrounding ROIs in pixel units; 8 shows the input and output of a neural network used in a denoising process when a normal ROI is used to condition the denoising process in training an abnormal image inferer; 9 shows the relationship between an input ROI in an abnormal replacement range of a normal image and a pseudo-abnormal region based on the inference result; 10 shows an example of a flowchart related to training an abnormal image inferer; 11 shows an example of a flowchart showing processing related to generating a pseudo-abnormal image using an abnormal image inferer; 12 shows a block diagram of a learning device; 13 shows an example of a flowchart executed by a learning device; 14 shows a block diagram of an image generation device; 15 shows an example of a flowchart executed by an image generation device.

[0012] Hereinafter, embodiments of a learning device, an image generating device, a learning method, and a storage medium will be described with reference to the drawings.

[0013] <First Embodiment> (1) System Configuration Fig. 1 shows a schematic configuration of a learning system 100. As shown in Fig. 1, the learning system 100 generates learning data (particularly, images showing abnormalities in an object) necessary for training a model that detects anomalies from images of the object in which anomalies are to be detected. Hereinafter, an image showing a normal state of an object will be referred to as a "normal image," and an image showing an abnormal state of an object will be referred to as an "abnormal image." The learning system 100 mainly includes a learning device 1 and a storage device 2.

[0014] Hereinafter, as a representative example, we will describe the generation of abnormal images used in training a lesion detection model that detects image regions (also referred to as "abnormal regions") representing areas suspected of being abnormal, such as lesions, from endoscopic images of an organ being examined during an endoscopic examination. Endoscopic images are one of the suitable applications for generating abnormal images based on this embodiment, in that it is difficult to obtain images containing abnormal regions. Examples of endoscopes that are applicable to the present disclosure include pharyngeal endoscopes, bronchoscopes, upper gastrointestinal endoscopes, duodenoscopes, small intestinal endoscopes, colonoscopes, capsule endoscopes, thoracoscopes, laparoscopes, cystoscopes, cholangioscopes, arthroscopes, spinal endoscopes, angioscopes, and epidural endoscopes. In this case, the target object is the organ being examined.

[0015] Furthermore, the training data generated by the learning system 100 is not limited to the training data used to train the lesion detection model described above. For example, the training data generated by the learning system 100 may be abnormal images used to train a model that detects lesion areas from any medical images other than endoscopic images (e.g., images obtained in ultrasound examinations, PET examinations, CT examinations, or MRI examinations). In another example, the abnormal images generated by the learning system 100 may be used to train a model that detects object areas where abnormalities (abnormalities other than lesions) appear in images of any object. Examples of models that detect object areas where abnormalities other than lesions appear include models that detect suspected abnormal areas from images (visual inspection images) taken during visual inspection of an article as the target object.

[0016] The learning device 1 performs learning of an abnormal image inference device, which is an inference device that performs inference regarding the generation of pseudo-abnormal images of an object (also called "pseudo-abnormal images") based on information stored in the storage device 2.

[0017] The abnormal image inference device is a machine learning model that converts an image representing a normal state of an object into an image representing an abnormal state of the object. In other words, the abnormal image inference device is a model that learns the relationship between an image representing a normal state of an object and an image representing an abnormal state of the object in the captured area of ​​the image. Specifically, the abnormal image inference device is a machine learning model that performs machine learning to infer an image showing an abnormality of the object based on a normal image of an object of a predetermined size that is input. In this embodiment, a region of interest (ROI) of a predetermined size extracted (sampled) from an endoscopic image is input to the abnormal image inference device. The abnormal image inference device performs machine learning using a pair of an ROI representing a normal state of the object and an ROI representing an abnormal state of the object. Here, the ROI represents a very small partial image (patch) extracted from a captured image of the object. As will be described later, the abnormal image inference device receives an ROI in which images of surrounding areas are combined in the channel direction.

[0018] The storage device 2 is a memory that stores various information necessary for the processing of the learning device 1. The storage device 2 has an abnormal image DB 21, a normal image DB 22, and inference device information 23.

[0019] The abnormal image DB21 is a database that stores abnormal images of objects. The abnormal image DB21 is used for training the abnormal image inference device. Here, each abnormal image stored in the abnormal image DB21 is associated with mask information (label information) that indicates an abnormal region in the abnormal image. The mask information may be, for example, information that indicates the size (vertical width and horizontal width) and position of a rectangular abnormal region, or information that indicates an arbitrary shape, position, and size that forms the outer frame of the abnormal region. Here, it is assumed that no abnormal part of the object exists in the region of the abnormal image other than the abnormal region indicated by the mask information, and that the region of the abnormal image other than the abnormal region is a normal region that indicates the normal state of the object.

[0020] Examples of pathologies detected as abnormal regions in endoscopic images include the following (a) to (f): (a) head and neck: pharyngeal cancer, malignant lymphoma, papilloma; (b) esophagus: esophageal cancer, esophagitis, hiatal hernia, Barrett's esophagus, esophageal varices, esophageal achalasia, esophageal submucosal tumor, benign esophageal tumor; (c) stomach: gastric cancer, gastritis, gastric ulcer, gastric polyp, gastric tumor; (d) duodenum: duodenal cancer, duodenal ulcer, duodenitis, duodenal tumor, duodenal lymphoma; (e) small intestine: small intestine cancer, small intestine neoplastic disease, small intestine inflammatory disease, small intestine vascular disease; (f) large intestine: large intestine cancer, large intestine neoplastic disease, large intestine inflammatory disease, large intestine polyp, large intestine polyposis, Crohn's disease, colitis, intestinal tuberculosis, hemorrhoids.

[0021] The normal image DB 22 is a database that stores normal images of an object, and each normal image is used by the abnormal image inference device to generate a pseudo-abnormal image. Each normal image is associated with mask information that specifies the range (also called the "abnormal replacement range") to be replaced with a pseudo-abnormal region by the abnormal image inference device. The mask information is, for example, information that indicates the position and size of the rectangular abnormal replacement range.

[0022] The inference device information 23 stores parameters of the abnormal image inference device necessary to configure the abnormal image inference device. The parameters of the abnormal image inference device stored in the inference device information 23 are updated by learning performed by the learning device 1. The abnormal image inference device may be, for example, any machine learning model used in image generation AI that generates an image from an image. Examples of machine learning models used in image generation AI include a diffusion model such as Leonardo.ai's Image2Image, a generative adversarial network (GAN), and a variational autoencoder (VAE). When the abnormal image inference device is configured using a neural network, the inference device information 23 includes various parameters (including hyperparameters), such as the layer structure, the neuron structure of each layer, the number and size of filters in each layer, and the weight of each element of each filter. The inference module information 23 stores the initial values ​​of the parameters of the abnormal image inference module before learning.

[0023] The storage device 2 may be an external storage device such as a hard disk connected to or built into the learning device 1, a storage medium such as a flash memory, or a server device that communicates data with the learning device 1. The storage device 2 may also be composed of multiple storage devices, and may have the above-mentioned storage units distributed among them.

[0024] The configuration of learning system 100 shown in Figure 1 is an example, and various modifications may be made. For example, learning device 1 and storage device 2 may be implemented as a single device. In another example, learning device 1 may be implemented as a plurality of devices. In this case, the plurality of devices that make up learning device 1 exchange information necessary to execute pre-assigned processing between them via direct wired or wireless communication or via communication via a network.

[0025] (2) Hardware Configuration Fig. 2 shows an example of the hardware configuration of the learning device 1. The learning device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 19.

[0026] The processor 11 executes a program stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the learning device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0027] Memory 12 is composed of various types of volatile and non-volatile memory, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. Memory 12 also stores programs executed by learning device 1. Some of the information stored in memory 12 may be stored in an external storage device, such as storage device 2, that can communicate with learning device 1, or in a storage medium that is detachable from learning device 1. Memory 12 may also store information stored in storage device 2 instead.

[0028] The interface 13 is an interface for electrically connecting the learning device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0029] The hardware configuration of the learning device 1 is not limited to the configuration shown in Fig. 2. For example, the learning device 1 may further include a display unit such as a display, an input unit such as a keyboard or a mouse, and an audio output unit such as a speaker.

[0030] (3) Overview Figure 3 shows an overview of the processing executed by the learning device 1. After the learning device 1 has trained the abnormal image inference device, it executes inference using the trained abnormal image inference device. Note that the inference using the abnormal image inference device is executed by a device that references inference device information 23 that stores the parameters of the abnormal image inference device after learning, and may be executed by any device other than the learning device 1.

[0031] First, in learning the abnormal image inference device, the learning device 1 randomly extracts (randomly samples) a normal ROI, which is an area of ​​a predetermined size representing the normal state of an object, and an abnormal ROI, which is an area of ​​a predetermined size representing the abnormal state of an object, from the abnormal images included in the abnormal image DB 21. The normal ROI and abnormal ROI are areas that are sufficiently small compared to the abnormal image, and are, for example, rectangular images of 3 pixels in both length and width (i.e., 3 × 3).

[0032] The advantages of setting a small area such as 3 × 3 as the ROI will be described below. Generally, if a lesion is considered to be cellular degeneration, it is possible to reproduce abnormal areas based only on local characteristics. Therefore, it is possible to convert normal areas into pseudo-abnormal areas using a small ROI such as 3 × 3. Furthermore, since multiple ROIs can be sampled from a single image, there is an advantage in that learning image transformation using a small ROI can generate more training data than learning image transformation using the entire image.

[0033] The learning device 1 then combines, in the channel direction, a surrounding ROI, which is a surrounding region of the same shape and size as each extracted ROI and allows overlap with the ROI. Hereinafter, a normal ROI combined with a surrounding ROI will be referred to as a "combined normal ROI," and an abnormal ROI combined with a surrounding ROI will be referred to as a "combined abnormal ROI." In this way, by overlapping the surrounding ROIs in the channel direction, the abnormal image inference device can be trained to perform inference taking into account information about the surrounding regions of each ROI. The abnormal ROI, normal ROI, surrounding ROI, and input ROI, which will be described later, have the same shape and size and are congruent.

[0034] Furthermore, the learning device 1 randomly generates pairs of combined normal ROIs and combined abnormal ROIs, thereby generating a plurality of pairs of combined normal ROIs and combined abnormal ROIs.

[0035] The learning device 1 then trains the abnormal image inferor based on multiple pairs of combined normal ROIs and combined abnormal ROIs. In this case, the learning device 1 updates the parameters of the abnormal image inferor for each pair of combined normal ROIs and combined abnormal ROIs so as to minimize the loss between the combined abnormal ROI and the inference result generated by the abnormal image inferor from the combined normal ROI. The learning device 1 then stores the parameters of the abnormal image inferor updated by learning in the inferor information 23.

[0036] In inference using the trained abnormal image inferencing device, the learning device 1 uses the trained abnormal image inferencing device to generate pseudo-abnormal images from normal images included in the abnormal image DB 21. In this case, as described below, the learning device 1 sequentially sets ROIs to be input to the abnormal image inferencing device so that they scan the abnormal replacement range of the normal image. Then, based on the inference results of the abnormal image inferencing device obtained by inputting each ROI within the set abnormal replacement range to the abnormal image inferencing device, the learning device 1 generates pseudo-abnormal images, which are normal images in which the abnormal replacement range has been replaced by the inference results of the abnormal image inferencing device. This makes it possible to increase the number of abnormal images that are generally difficult to obtain and to obtain a sufficient amount of learning data necessary for highly accurate training of a lesion detection model (anomaly detection model).

[0037] (4) Functional Blocks Figure 4 shows an example of functional blocks of the learning device 1. Functionally, the processor 11 of the learning device 1 has a first extraction unit 30, a first combination unit 31, a pair generation unit 32, a learning unit 33, a second extraction unit 34, a second combination unit 35, and a pseudo-abnormal image generation unit 36. Note that in Figure 4, blocks between which data is exchanged are connected by solid lines, but the combination of blocks between which data is exchanged is not limited to this. The same applies to other functional block diagrams described below.

[0038] The first extractor 30 performs a process of extracting (sampling) normal ROIs and abnormal ROIs by referring to the abnormal image DB 21. The first extractor 30 extracts abnormal images from the abnormal image DB 21 randomly or based on a predetermined rule. The first extractor 30 then extracts abnormal ROIs from the identified abnormal regions by referring to mask information associated with the extracted abnormal images. For example, the first extractor 30 randomly extracts abnormal ROIs from the abnormal regions of the extracted abnormal images.

[0039] The first extractor 30 also extracts normal ROIs from regions other than the abnormal region (i.e., normal regions) in the abnormal image. In this case, the first extractor 30 preferably samples the normal ROIs probabilistically so that regions closer to the abnormal region are more likely to be extracted as normal ROIs. For example, the first extractor 30 probabilistically samples the normal ROIs according to a normal distribution according to the distance to the abnormal region, so that ROIs within the normal region centered at a position closer to the abnormal region are more likely to be extracted. Furthermore, if mask information indicating a region of another tissue different from the target object is further associated with the abnormal image, the first extractor 30 may set the probability of a normal ROI being sampled in the region of the other tissue to 0.

[0040] The first extraction unit 30 may extract normal ROIs and abnormal ROIs of multiple sizes and then perform a reduction process so that the extracted normal ROIs and abnormal ROIs have a predetermined uniform size. Examples of the reduction process include bi-liner interpolation. For example, the first extraction unit 30 extracts normal ROIs and abnormal ROIs of 3x3, 5x5, and 7x7 sizes, and then normalizes the sizes so that all extracted normal ROIs and abnormal ROIs are 3x3 images. In this way, by sampling ROIs of multiple sizes, the first extraction unit 30 can effectively resolve issues such as the apparent size of an endoscopic image varying depending on the timing of capture and the fact that the appropriate ROI size is not self-evident.

[0041] The first combining unit 31 performs a process of combining each of the normal and abnormal ROIs supplied from the first extracting unit 30 with surrounding ROIs. In this case, for each of the extracted abnormal and normal ROIs, the first combining unit 31 extracts surrounding ROIs that are congruent with the ROI and allow overlap with the ROI, and combines the extracted surrounding ROI with each ROI in the channel direction. Specific examples of combining surrounding ROIs will be described later. The first combining unit 31 supplies the generated multiple combined abnormal ROIs and multiple combined normal ROIs to the pair generating unit 32.

[0042] The pair generation unit 32 randomly generates a plurality of pairs of the abnormally bound ROIs and the normal bound ROIs from the abnormally bound ROIs and the normal bound ROIs generated by the first combining unit 31. The pair generation unit 32 then supplies the generated pairs of the abnormally bound ROIs and the normal bound ROIs to the learning unit 33.

[0043] The learning unit 33 trains the abnormal image inferor based on the pair of the combined abnormal ROI and the combined normal ROI. For example, the learning unit 33 updates the parameters of the abnormal image inferor so as to minimize the loss (error) between the combined abnormal ROI and the inference result output by the abnormal image inferor when the combined normal ROI is input to the abnormal image inferor. The algorithm for determining the parameters to minimize the loss may be any learning algorithm used in machine learning, such as gradient descent or backpropagation. Furthermore, when the abnormal image inferor is a diffusion model, the learning unit 33 uses data obtained by adding noise for N (N is a positive integer) steps to the combined abnormal ROI as input to a neural network used in a denoising process (denoising process or reverse process). Furthermore, the learning unit 33 uses the combined normal ROI as input for setting conditions. Then, the learning unit 33 updates the parameters of the abnormal image inference device for each pair of the combined abnormal ROI and the combined normal ROI, and stores the updated parameters in the inference device information 23 .

[0044] The second extraction unit 34 references the normal image DB 22 and performs a process of extracting an area from a normal image within the abnormal replacement range specified by the mask information to be input to the abnormal image inference device. In this case, the second extraction unit 34 randomly or based on a predetermined rule extracts a normal image from the normal image DB 22 and identifies the abnormal replacement range within the normal image by referring to the mask information associated with the normal image. Next, the second extraction unit 34 sequentially sets an area within the abnormal replacement range to be input to the abnormal image inference device (also referred to as an "input ROI") so as to scan the entire abnormal replacement range. In this case, the shape and size of the input ROI are the same as the abnormal ROI and normal ROI used for learning and are consistent with the input format of the abnormal image inference device. The second extraction unit 34 supplies the extracted input ROI to the second combination unit 35.

[0045] The second combining unit 35 performs a process of combining each input ROI extracted by the second extracting unit 34 with its surrounding region. In this case, the second extracting unit 34 extracts a surrounding ROI that is congruent with the input ROI, allowing overlap with the input ROI, and combines the extracted surrounding ROI with the input ROI in the channel direction. The input ROI with the combined surrounding ROI is also referred to as a "combined input ROI." The method for setting the surrounding ROI is the same as the method for setting the surrounding ROI executed by the first combining unit 31. The second combining unit 35 supplies the generated combined input ROI to the pseudo-abnormal image generating unit 36.

[0046] After the abnormal image inference device has learned, the pseudo-abnormal image generation unit 36 ​​converts a normal image into a pseudo-abnormal image using the abnormal image inference device configured with reference to the inference device information 23 and the combined input ROI. In this case, the pseudo-abnormal image generation unit 36 ​​inputs the combined input ROI to the abnormal image inference device, and acquires a pseudo-abnormal image based on the inference result output by the abnormal image inference device.

[0047] The inference result output by the abnormal image inference unit is a multi-channel image in which images of the input ROI and surrounding ROIs, each converted into a pseudo-abnormal region, are superimposed in the channel direction. Therefore, the pseudo-abnormal image generation unit 36 ​​uses the multi-channel image obtained as the inference result of the abnormal image inference unit to replace the region of the normal image corresponding to the combined input ROI with a pseudo-abnormal region. In this case, the pseudo-abnormal image generation unit 36 ​​sequentially sets the input ROIs to scan the entire abnormal replacement range, and sequentially inputs all of the combined input ROIs generated by the second combining unit 35 to the abnormal image inference unit. The pseudo-abnormal image generation unit 36 ​​can then replace the abnormal replacement range of the normal image with a pseudo-abnormal region by using the inference result obtained by sequentially inputting the combined input ROIs to the abnormal image inference unit. In this way, the pseudo-abnormal image generation unit 36 ​​converts normal images into pseudo-abnormal images, thereby essentially increasing the number of abnormal images used for model learning.

[0048] The components of the first extraction unit 30, the first combination unit 31, the pair generation unit 32, the learning unit 33, the second extraction unit 34, the second combination unit 35, and the pseudo-abnormal image generation unit 36 ​​can be implemented, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to implement each component. At least some of these components may not necessarily be implemented by software programs, but may be implemented by any combination of hardware, firmware, and software. At least some of these components may be implemented using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to implement a program consisting of the above components. At least some of the components may be implemented using an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, each component may be implemented using various hardware. The same applies to other embodiments described below. Furthermore, each of these components may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0049] (5) Specific Examples Next, specific examples of the processing executed by each block shown in FIG. 4 will be described.

[0050] 5 shows an abnormal image extracted from the abnormal image DB 21 by the first extractor 30, clearly indicating the abnormal ROI and normal ROI set for the abnormal image by the first extractor 30. Here, an abnormal region 40 identified by referencing mask information associated with the abnormal image is indicated by a dashed frame, abnormal ROIs 41a to 41d are indicated by dashed-dotted frame, and normal ROIs 41e to 41k are indicated by solid-line frame.

[0051] As shown in FIG. 5 , the first extraction unit 30 randomly sets multiple abnormal ROIs 41a-41d within the abnormal region of the abnormal image. Furthermore, the first extraction unit 30 randomly sets multiple normal ROIs 41e-41k in regions other than the abnormal region. The first extraction unit 30 then sets a region closer to the abnormal region so that it is more likely to be extracted as a normal ROI. For example, when determining the center position of the normal ROI, the first extraction unit 30 uses a normal distribution and probabilistically samples the center position of the normal ROI so that the greater the distance to the abnormal region, the farther the center position of the normal ROI is from the peak of the normal distribution. Here, the abnormal ROIs 41a-41d and the normal ROIs 41e-41k are all square regions of the same size (3 × 3) and are set smaller than the abnormal region.

[0052] 6 is a diagram showing, in pixel units, a region including an ROI 41 corresponding to any of abnormal ROIs 41a to 41d and normal ROIs 41e to 41k. In FIG. 6, ROI 41 is a 3×3 rectangular region of 9 pixels, with x-y coordinates (i, j) representing the horizontal and vertical pixel position of the center pixel in the entire abnormal image. Sixteen pixels adjacent to the extracted ROI 41 are further shown as peripheral pixels. Each pixel is identified by x-y coordinates, with the center pixel position of the extracted ROI 41 being (i, j), with the x coordinate ranging from "i-2" to "i+2" and the y coordinate ranging from "j-2" to "j+2."

[0053] The first combining unit 31 sets peripheral ROIs at positions shifted by a predetermined number of pixels in each of the four diagonal directions (left, right, up, down, and diagonal directions) from the extracted ROI 41. In this way, the first combining unit 31 sets a total of eight peripheral ROIs.

[0054] FIG. 7 is a diagram showing the ROI 41 and the surrounding ROIs 42a-42h in pixel units. In FIG. 7, the ROI 41, whose central pixel position is (i, j), is set as the center, and the surrounding ROIs 42a-42h obtained by shifting the ROI 41 in each of the left, right, up, down, and diagonal directions are shown corresponding to the shifted directions. Here, as an example, the first combining unit 31 sets the surrounding ROIs 42a-42h by shifting the ROI 41 by one pixel in each direction. As a result, the surrounding ROIs 42a-42h are set with their centers at coordinates shifted one pixel left, right, up, down, and diagonally from the center coordinates (i, j) of the ROI 41.

[0055] The pair generator 32 then generates ROIs for nine channels by combining the surrounding ROIs 42a to 42h with the ROI 41 in the channel direction, as combined ROIs (here, combined abnormal ROIs or combined normal ROIs). In this way, the pair generator 32 can generate a combined ROI that includes information indicating its relationship with its surroundings by bundling the surrounding ROIs in the channel direction. The generation and combination of surrounding ROIs is similarly performed when generating a combined input ROI.

[0056] The method of setting the surrounding ROI is not limited to the examples shown in Figures 5 and 6. For example, instead of generating eight surrounding ROIs in eight directions (left, right, up, down, and diagonal), the surrounding ROIs may be generated in fewer directions (e.g., four directions (left, right, up, down)). In another example, instead of being a region shifted by one pixel from the extracted ROI, the surrounding ROI may be a region shifted by two or more pixels.

[0057] Next, the learning of the abnormal image inference device by the learning unit 33 when the abnormal image inference device is a diffusion model will be described.

[0058] 8 shows the input and output of a neural network used in the denoising process when a normal ROI is used to condition the denoising process in training an abnormal image inference device. Here, the denoising process is a process in which an original image is generated by a neural network from a noise image generated by a diffusion process (or forward process), which is a step in which a noise image is generated from an input image for training (here, a combined abnormal ROI).

[0059] Hereafter, the combined normal ROI whose center pixel coordinates are (i, j) is referred to as "X ０ {i, j}" and the noise image input to the neural network used in the denoising process is denoted as "X ｔ {i, j}". ｔ {i, j}" is a noise image in which N steps of noise are added to the combined abnormal ROI in the diffusion process, and X ０ It corresponds to an image in which t steps of noise have been added to {i, j}.

[0060] As shown in FIG. 8, the neural network used in the denoising process is ｔ When {i, j} is input, one step of noise (X ｔ {i, j} → X ｔ－１ {i, j}) and conditioned on the combined normal ROI, X ０ {i, j} is input. Note that the neural network does not output noise, but X ｔ {i, j} to X ｔ－１ A neural network of the type that outputs {i,j} (i.e., image to image) may be used.

[0061] Here, X ｔ {i, j} is the noise image based on the combined abnormal ROI, and X ０ {i, j} is the combined normal ROI, and the neural network is trained using the ROI, which is a small region in the image.

[0062] In the case of a general diffusion model that generates an abnormal image by conditioning with a normal image, a noise image in which N steps of noise are added to the abnormal image in the diffusion process is input to a neural network. The normal image is then input as conditioning to the neural network, and one step of noise is output. Details of the general diffusion model that generates an abnormal image by conditioning with a normal image are disclosed, for example, in the following literature (particularly Figure 3): "High-Resolution Image Synthesis with Latent Diffusion Models", Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Bjorn Ommer, CVPR 2022

[0063] Next, a specific example of inference using the abnormal image inference device will be described.

[0064] FIG. 9 shows the relationship between the input ROI in the abnormal replacement range of a normal image included in the abnormal image DB 21 and the pseudo abnormal region based on the inference result.

[0065] As shown in Fig. 9, the second extraction unit 34 recognizes the abnormal replacement range of the normal image by referring to mask information associated with the normal image extracted from the normal image DB 22, and sets a 3 x 3 input ROI within the abnormal replacement range. Furthermore, the second combination unit 35 sets a peripheral ROI corresponding to the input ROI. Here, as in the example of Fig. 7, the peripheral ROI is set to be an area shifted by one pixel to the left, right, up, down, and diagonally from the input ROI. Therefore, in Fig. 9, a 4 x 4 area obtained by expanding the input ROI by one pixel is clearly indicated as the "set range of the peripheral ROI."

[0066] The second combining unit 35 then combines the eight surrounding ROIs with the input ROI to generate a nine-channel combined input ROI, and the pseudo-abnormal image generating unit 36 ​​inputs the generated combined input ROI to the trained abnormal image inference device. As a result, the abnormal image inference device outputs an inference result that is a nine-channel 3x3 image. Here, the nine-channel 3x3 image output by the abnormal image inference device corresponds to an image obtained by converting the combined input ROI for each channel to represent a pseudo-abnormal region. In the abnormal image inference device, noise is input to the combined input ROI during the diffusion process, and in the denoising process, when a trained neural network is used, the combined input ROI is used for conditioning as a normal image. In this case, when an image is generated by gradually denoising noise, the normal image is used for conditioning, and an abnormal image is represented based on the normal image.

[0067] Then, the pseudo-abnormal image generation unit 36 ​​generates a pseudo-abnormal region within the same range as the "set range of the surrounding ROI" based on the inference results of the abnormal image inference unit. Here, the pseudo-abnormal region generated based on the combined input ROI is referred to as the "pseudo-abnormal region based on the inference results." In this case, for example, since the 3x3 image for nine channels output by the abnormal image inference unit contains overlapping pixels in the x-y coordinates of the normal image, the pseudo-abnormal image generation unit 36 ​​determines pixel values ​​for these overlapping pixels through statistical processing. For example, based on the 3x3 image for nine channels output by the abnormal image inference unit, the pseudo-abnormal image generation unit 36 ​​calculates a representative pixel value (such as the average or median) for each corresponding pixel within the "set range of the surrounding ROI."

[0068] The second extraction unit 34 then sets a new input ROI within the abnormal replacement range and continues the process of setting the input ROI so that each pixel in the abnormal replacement range is included in at least one of the set input ROIs (or its surrounding ROIs). This allows the entire abnormal replacement range to be scanned by the input ROI (or its surrounding ROIs). In this case, the second extraction unit 34 may set a new input ROI so that it does not overlap with an already set input ROI, or may set a new input ROI that allows overlap with an already set input ROI. Therefore, the second extraction unit 34 may slide the input ROI vertically or horizontally by one pixel, or by one pixel for each set of multiple pixels. In this way, the learning device 1 can replace the entire abnormal replacement range with a pseudo-abnormal region based on the inference result output by the abnormal image inference unit.

[0069] (6) Processing Flow Fig. 10 is an example of a flowchart relating to the learning of the abnormal image inference device. The learning device 1 repeatedly executes the processing of the flowchart shown in Fig. 10.

[0070] First, the learning device 1 extracts (samples) abnormal ROIs and normal ROIs from abnormal images contained in the abnormal image DB 21 (step S11). In this case, for example, the learning device 1 randomly extracts abnormal images from the normal image DB 22, refers to mask information related to the abnormal regions, extracts abnormal ROIs from the abnormal regions, and extracts normal ROIs from the normal regions.

[0071] Next, the learning device 1 sets a surrounding ROI for each of the extracted abnormal ROIs and normal ROIs, and combines the set surrounding ROIs in the channel direction (step S12), thereby generating a combined abnormal ROI and a combined normal ROI.

[0072] Next, the learning device 1 generates pairs of combined normal ROIs and combined abnormal ROIs (step S13). In this case, for example, the learning device 1 randomly selects a combined normal ROI and a combined abnormal ROI, and defines the selected combined normal ROI and combined abnormal ROI as a pair.

[0073] Next, the learning device 1 updates the parameters of the abnormal image inference device based on the pair of the combined normal ROI and the combined abnormal ROI (step S14). The learning device 1 stores the updated parameters in the inference device information 23. The learning device 1 then determines whether a predetermined learning termination condition is met, and if the learning termination condition is not met, executes the processing of the flowchart again.

[0074] The learning device 1 may execute steps S12 and S13 in the reverse order. That is, the learning device 1 may generate pairs of abnormal ROIs and normal ROIs, and then combine these abnormal ROIs and normal ROIs with surrounding ROIs to generate combined abnormal ROIs and combined normal ROIs. Instead of repeatedly executing the processing of the flowchart in FIG. 10 , the learning device 1 may generate as many pairs of combined normal ROIs and combined abnormal ROIs as necessary for learning the abnormal image inference device in step S13, and update the parameters of the abnormal image inference device based on these pairs in step S14.

[0075] 11 is an example of a flowchart showing a process for generating a pseudo-abnormal image using an abnormal image inference device. The learning device 1 repeatedly executes the process of the flowchart shown in FIG.

[0076] First, the learning device 1 sets multiple input ROIs so as to scan the abnormal replacement range of a normal image sampled from the normal image DB 22 (step S21). In this case, the learning device 1 acquires a normal image and mask information associated with the normal image from the normal image DB 22, and identifies the abnormal replacement range based on the mask information. Then, the learning device 1 sets multiple input ROIs so that the entire identified abnormal replacement range is scanned by the set range of the input ROI or surrounding ROI.

[0077] Next, the learning device 1 sets peripheral ROIs for each set input ROI and combines the set peripheral ROIs with each input ROI in the channel direction (step S22), thereby generating the same number of combined input ROIs as the number of input ROIs.

[0078] Next, the learning device 1 inputs each combined input ROI to the trained abnormal image inference device and obtains the inference result output by the abnormal image inference device (step S23).The learning device 1 then generates a pseudo-abnormal image by replacing the abnormal replacement range of the normal image with a pseudo-abnormal region based on the inference result obtained in step S23 (step S24).

[0079] Note that instead of setting multiple input ROIs in step S21, the learning device 1 may sequentially integrate surrounding ROIs and obtain an inference result each time an input ROI is set. In this case, the learning device 1 sets one input ROI in step S21, and after obtaining the corresponding inference results in steps S22 and S23, returns to step S21 to set the next input ROI.

[0080] (7) Modification Example: Instead of inputting a combined ROI, which is a small region extracted from an image and a surrounding ROI superimposed in the channel direction, to the abnormal image inference device, an ROI extracted from an image without combining the surrounding ROIs may be input to the abnormal image inference device. In this case, the abnormal image inference device serves as a machine learning model that converts the input ROI into an ROI representing a pseudo-abnormal state. Even in this case, it is possible to reproduce an abnormal location based on local features and to suitably increase the training data.

[0081] 12 is a block diagram of a learning device 1X. The learning device 1X mainly includes a region of interest extraction means 30X, a pair generation means 32X, and a learning means 33X. The learning device 1X may be composed of multiple devices.

[0082] The region of interest extraction unit 30X extracts regions of interest from the abnormal region of the abnormal image having the abnormal region of the object and from normal regions other than the abnormal region. The region of interest extraction unit 30X can be, for example, the first extraction unit 30 in the first embodiment.

[0083] The pair generating means 32X generates pairs of regions of interest extracted from abnormal regions and regions of interest extracted from normal regions. The pair generating means 32X can be, for example, the pair generating unit 32 in the first embodiment.

[0084] The learning means 33X executes machine learning of an inference device that performs inference regarding a pseudo-abnormal image showing a pseudo-abnormality of an object from a normal image showing a normal state of the object based on the pair. The learning means 33X can be, for example, the learning unit 33 in the first embodiment.

[0085] 13 is an example of a flowchart showing a processing procedure executed by the learning device 1X. The region of interest extraction means 30X extracts regions of interest from abnormal regions of an abnormal image having an abnormal region of an object and from normal regions other than the abnormal region (step S31). The pair generation means 32X generates pairs of regions of interest extracted from the abnormal region and regions of interest extracted from the normal region (step S32). The learning means 33X executes machine learning of an inference device that performs inference on pseudo-abnormal images showing pseudo-abnormalities of an object from normal images showing the normal state of the object based on the pairs (step S33).

[0086] According to the second embodiment, the learning device 1X can preferably perform learning of an inference device used to generate a pseudo-abnormal image in which a pseudo-abnormality is expressed from a normal image.

[0087] 14 is a block diagram of an image generating device 1Y. The image generating device 1Y mainly includes a region of interest extraction unit 34Y and a pseudo-abnormal image generating unit 36Y. The image generating device 1Y may be composed of multiple devices.

[0088] The region of interest extraction unit 34Y extracts a region of interest representing the normal state of the object from the image of the object. The region of interest extraction unit 34Y can be, for example, the second extraction unit 34 in the first embodiment.

[0089] The pseudo-abnormal image generating means 36Y generates a pseudo-abnormal image, which is a normal image in which the region of interest is replaced with a pseudo-abnormal region that represents an abnormality in the target object, based on the region of interest and the machine learning model. The pseudo-abnormal image generating means 36Y can be, for example, the pseudo-abnormal image generating unit 36 ​​in the first embodiment.

[0090] 15 is an example of a flowchart executed by the image generating device 1Y. The region of interest extraction unit 34Y extracts a region of interest representing a normal state of the object from an image of the object (step S41). The pseudo-abnormal image generating unit 36Y generates a pseudo-abnormal image, which is a normal image in which the region of interest is replaced with a pseudo-abnormal region representing an abnormality in the object, based on the region of interest and the machine learning model (step S42).

[0091] According to the third embodiment, the image generating device 1Y can suitably generate a pseudo-abnormal image in which a pseudo-abnormality is displayed from a normal image.

[0092] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor or the like. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and random access memories (RAMs)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable medium can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0093] In addition, part or all of the above-described embodiments (including variations, the same applies below) may also be described as, but are not limited to, the following supplementary notes. Furthermore, not only the devices, methods, and storage media described in the supplementary notes, but also various hardware, software, various recording means (including storage media) for recording software, or systems may similarly have part or all of the configurations described as supplementary notes subordinated to them, as long as they do not deviate from the above-described embodiments.

[0094] [Supplementary Note 1] A learning device comprising: a region of interest extraction means for extracting regions of interest from an abnormal region of an object having the abnormal region and a normal region other than the abnormal region of the object; a pair generation means for generating pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region, and a learning means for executing machine learning of an inference device that performs inference on a pseudo-abnormal image in which a pseudo-abnormality of the object appears from a normal image representing a normal state of the object, based on the pairs. [Supplementary Note 2] The learning device according to Supplementary Note 1 further comprises a combining means for extracting peripheral regions of the region of interest that are congruent with the region of interest and combining the region of interest with the peripheral regions in a channel direction, wherein the learning means performs machine learning of the inference device based on the pair of the region of interest in which the peripheral regions are combined in the channel direction. [Supplementary Note 3] The learning device according to Supplementary Note 2, wherein the combining means probabilistically extracts the peripheral regions based on a distance to the region of interest. [Supplementary Note 4] The learning device of Supplementary Note 1, wherein the region of interest extraction means extracts the region of interest of multiple sizes and converts the extracted region of interest to normalize the size of the region of interest. [Supplementary Note 5] The learning device of Supplementary Note 1, wherein mask information indicating the abnormal region is associated with the abnormal image, and the region of interest extraction means extracts a region of interest in the abnormal region and a region of interest in the normal region based on the mask information. [Supplementary Note 6] The learning device of Supplementary Note 1, wherein the inference unit is a diffusion model having a denoising process using a neural network that infers noise, and the learning means inputs a noise image of the region of interest extracted from the abnormal region to the neural network and uses the region of interest extracted from the normal region for conditioning in the denoising process. [Supplementary Note 7] The learning device of Supplementary Note 2, wherein the combining means extracts positions obtained by shifting the region of interest by a predetermined number of pixels in multiple directions as the peripheral regions and combines the peripheral regions extracted in each of the multiple directions to the region of interest.[Supplementary Note 8] An image generation device comprising: a region of interest extraction means that extracts a region of interest representing a normal state of an object from an image of the object; and a pseudo-abnormal image generation means that generates a pseudo-abnormal image, which is an image of the object in which the region of interest is replaced with a pseudo-abnormal region that represents an abnormality of the object, based on the region of interest and a machine learning model, wherein the machine learning model is a model that has been learned by machine learning to understand the relationship between an image representing the normal state of the object and an image in which the abnormality of the object appears. [Supplementary Note 9] The image of the object has a specified range to be replaced with the abnormal region, the region of interest extraction means sets multiple regions of interest such that each pixel within the range is included in at least one of the regions of interest, and the pseudo-abnormal image generation means generates the pseudo-abnormal image in which the range has been replaced with the abnormal region, based on the multiple regions of interest and the machine learning model. [Supplementary Note 10] The image generating device according to Supplementary Note 8, further comprising a combining means for extracting peripheral regions of the region of interest that are congruent with the region of interest and combining the region of interest with the peripheral regions in a channel direction, wherein the pseudo-abnormal image generating means generates the pseudo-abnormal image based on the region of interest with the peripheral regions combined in the channel direction and the machine learning model. [Supplementary Note 11] A learning method, wherein a computer extracts regions of interest from abnormal regions of an object having the abnormal region and normal regions other than the abnormal region, respectively, generates pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region, and performs machine learning on an inference device that makes inferences regarding a pseudo-abnormal image in which a pseudo-abnormality of the object appears from a normal image that represents a normal state of the object, based on the pairs. [Supplementary Note 12] A program that causes a computer to execute a process of performing machine learning of an inference device that extracts regions of interest from abnormal regions of an object having the abnormal regions and normal regions other than the abnormal regions, generates pairs of the regions of interest extracted from the abnormal regions and the regions of interest extracted from the normal regions, and performs inference on pseudo-abnormal images that show pseudo-abnormalities in the object based on the pairs, from normal images that show the normal state of the object.[Supplementary Note 13] An image generation method, wherein a computer extracts a region of interest representing a normal state of an object from an image of the object, and generates a pseudo-abnormal image, which is an image of the object in which the region of interest is replaced with a pseudo-abnormal region that represents an abnormality of the object, based on the region of interest and a machine learning model, wherein the machine learning model is a model learned through machine learning of the relationship between the image representing the normal state of the object and the image in which the abnormality of the object appears. [Supplementary Note 14] A storage medium storing a program, wherein a computer executes a process of extracting a region of interest representing a normal state of the object from an image of the object, and generating a pseudo-abnormal image, which is an image of the object in which the region of interest is replaced with a pseudo-abnormal region that represents an abnormality of the object, based on the region of interest and a machine learning model, wherein the machine learning model is a model learned through machine learning of the relationship between the image representing the normal state of the object and the image in which the abnormality of the object appears.

[0095] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference.

[0096] REFERENCE SIGNS LIST 1, 1X Learning device 1Y Image generation device 2 Storage device 11 Processor 12 Memory 13 Interface 21 Abnormal image DB 22 Normal image DB 23 Inference device information 100 Learning system

Claims

1. A learning device comprising: a region of interest extraction means for extracting regions of interest from an abnormal region of an abnormal image of an object having the abnormal region and from a normal region other than the abnormal region; a pair generation means for generating pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region; and a learning means for executing machine learning of an inference device that makes inferences regarding a pseudo-abnormal image in which a pseudo-abnormality of the object is shown from a normal image that represents the normal state of the object, based on the pairs.

2. The learning device of claim 1, further comprising a combining means for extracting a peripheral region of the region of interest that is congruent with the region of interest and combining the region of interest and the peripheral region in a channel direction, wherein the learning means performs machine learning of the inference unit based on the pair of the region of interest whose peripheral region is combined in the channel direction.

3. The learning device according to claim 2, wherein the combining means probabilistically extracts the surrounding region based on the distance to the region of interest.

4. The learning device according to claim 1, wherein said region of interest extraction means extracts the regions of interest of a plurality of sizes and converts the extracted regions of interest so as to normalize the sizes of the extracted regions of interest.

5. The learning device described in claim 1, wherein the abnormal image is associated with mask information indicating the abnormal region, and the region of interest extraction means extracts a region of interest in the abnormal region and a region of interest in the normal region based on the mask information.

6. The learning device according to claim 1, wherein the inference unit is a diffusion model having a denoising process using a neural network to infer noise, and the learning means inputs a noise image of the region of interest extracted from the abnormal region to the neural network, and uses the region of interest extracted from the normal region for conditioning in the denoising process.

7. The learning device described in claim 2, wherein the combining means extracts positions obtained by shifting the region of interest by a predetermined number of pixels in multiple directions as the surrounding regions, and combines the surrounding regions extracted in each of the multiple directions with the region of interest.

8. An image generation device comprising: a region of interest extraction means for extracting a region of interest representing a normal state of an object from an image of the object; and a pseudo-abnormal image generation means for generating a pseudo-abnormal image, which is an image of the object in which the region of interest has been replaced with a pseudo-abnormal region representing an abnormality in the object, based on the region of interest and a machine learning model, wherein the machine learning model is a model obtained by machine learning of the relationship between an image representing the normal state of the object and an image representing an abnormality in the object.

9. The image generating device described in claim 8, wherein the image of the object has a specified range to be replaced with the abnormal region, the region of interest extraction means sets multiple regions of interest so that each pixel within the range is included in at least one of the regions of interest, and the pseudo-abnormal image generating means generates the pseudo-abnormal image in which the range has been replaced with the abnormal region based on the multiple regions of interest and the machine learning model.

10. The image generating device of claim 8, further comprising a combining means for extracting a peripheral region of the region of interest that is congruent with the region of interest and combining the region of interest and the peripheral region in the channel direction, wherein the pseudo-abnormal image generating means generates the pseudo-abnormal image based on the region of interest with the peripheral region combined in the channel direction and the machine learning model.

11. A learning method in which a computer extracts regions of interest from abnormal regions of an object having the abnormal regions and from normal regions other than the abnormal regions, generates pairs of the regions of interest extracted from the abnormal regions and the regions of interest extracted from the normal regions, and performs machine learning of an inference device that makes inferences regarding pseudo-abnormal images in which pseudo-abnormalities of the object are shown from normal images that represent the normal state of the object, based on the pairs.

12. A storage medium storing a program that causes a computer to execute a process of performing machine learning of an inference device that extracts regions of interest from abnormal regions of an abnormal image having an abnormal region of an object and from normal regions other than the abnormal region, generates pairs of the region of interest extracted from the abnormal region and the region of interest extracted from the normal region, and, based on the pairs, makes inferences regarding pseudo-abnormal images that show pseudo-abnormalities in the object from normal images that show the normal state of the object.

Images