Appearance inspection system, appearance inspection device, appearance inspection method, and program
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- MITSUBISHI ELECTRIC CORP
- Filing Date
- 2025-01-30
- Publication Date
- 2026-08-06
Smart Images
Figure JP2025003119_06082026_PF_FP_ABST
Abstract
Description
Visual inspection system, visual inspection device, visual inspection method, and program
[0001] This disclosure relates to a visual inspection system, a visual inspection apparatus, a visual inspection method, and a program.
[0002] To build a visual inspection system for inspecting the appearance of industrial products, it is necessary to train a learning model to recognize the quality of industrial products using multiple images of good and defective products. However, collecting the necessary images of good and defective industrial products to generate a high-quality learning model requires a great deal of time and effort. Therefore, conventional visual inspection systems employ methods to efficiently and automatically generate pseudo-image data from a small number of original image data.
[0003] For example, Patent Document 1 discloses a system that calculates a single boundary surface from the feature distribution of the original image data and generates pseudo-good product images and defective product images that do not affect the approximate position of that boundary surface. As a result, the pseudo-image data can be used as training data that maintains to some extent the bias of the feature distribution of the original image data.
[0004] Japanese Patent Publication No. 2021-125003
[0005] The technology disclosed in Patent Document 1 generates pseudo-images while maintaining a certain degree of bias in the feature distribution. That is, even when it is desired to generate pseudo-images mainly for feature spaces with sparse distribution density, pseudo-images are generated not only for feature spaces with sparse distribution density but also for feature spaces other than those with sparse distribution density. Therefore, it is difficult to efficiently generate pseudo-images corresponding to feature spaces with sparse distribution density using the technology disclosed in Patent Document 1.
[0006] This disclosure is made in view of the circumstances described above, and aims to efficiently generate pseudo-images corresponding to feature spaces with sparse distribution density.
[0007] To achieve the above objectives, the appearance inspection system relating to this disclosure is an appearance inspection system for inspecting the appearance of an object to be inspected, and comprises a feature detection means, a feature distribution information generation means, a pseudo-image generation means, and a pseudo-image information generation means. The feature detection means uses a verification target model that outputs a defect likelihood score, which is an index representing the possibility of a product being defective, for an input appearance image of an object to be inspected, and detects the defect likelihood score of the appearance image as a feature. The feature distribution information generation means generates feature distribution information that shows the distribution of the defect likelihood scores of the appearance image detected by the feature detection means. The pseudo-image generation means generates pseudo-images from the original images that correspond to feature spaces with sparse distribution density in the feature distribution information generated from the original images, which include images of good products and images of defective products, respectively, of the appearance of good products and defective products of the object to be inspected. The pseudo-image information generation means generates pseudo-image information that shows the image processing settings when the pseudo-image generation means generates the pseudo-images. Furthermore, the pseudo-image generation means refers to the pseudo-image information, calculates image processing settings for generating a pseudo-image corresponding to the feature space, and generates a pseudo-image based on the calculated image processing settings.
[0008] According to this disclosure, the visual inspection system refers to pseudo-image information indicating the image processing settings for the generated pseudo-image, calculates image processing settings for generating a pseudo-image corresponding to the feature space, and generates a pseudo-image corresponding to a feature space with a sparse distribution density based on the calculated image processing settings. This makes it possible to efficiently generate pseudo-images corresponding to a feature space with a sparse distribution density.
[0009] Block diagram showing an example configuration of the visual inspection system according to Embodiment 1 of this disclosure Block diagram showing an example hardware configuration of the visual inspection device Figure showing an example of a score region with a sparse distribution density in the feature distribution information from the original image Figure showing an example of class setting in the feature distribution information from the original image Figure showing an example of pseudo-image information Figure showing an example of feature distribution information from the original image and pseudo-image Figure showing another example of feature distribution information from the original image and pseudo-image Flowchart showing the threshold setting process Flowchart showing the pseudo-image generation process Figure for explaining the pseudo-image generation procedure Block diagram showing an example configuration of the visual inspection system according to Embodiment 2 of this disclosure
[0010] The following describes in detail the visual inspection system, visual inspection apparatus, visual inspection method, and program according to the embodiments of this disclosure with reference to the drawings. The visual inspection system, etc. according to the embodiments of this disclosure determine whether the object to be inspected is good or bad based on an appropriate threshold.
[0011] (Embodiment 1) The visual inspection system 1 according to Embodiment 1 of the present disclosure, as shown in Figure 1, comprises at least one visual inspection device 100 for performing visual inspection of industrial products, and at least one operation terminal 200 operated by a user of the visual inspection system 1. The visual inspection device 100 and the operation terminal 200 are connected to communicate via any network. The network is composed of, for example, an internet line network, a LAN (Local Area Network), a WAN (Wide Area Network), a VPN (Virtual Private Network), a dedicated communication network, etc., and may involve various network relay devices such as antennas, gateways, routers, and hubs.
[0012] The visual inspection device 100 is a general-purpose computer device such as a personal computer or server computer, and it determines the quality of the appearance of an industrial product based on an image of the industrial product being inspected. Based on operations performed by the user of the visual inspection system 1 via the operation terminal 200, the visual inspection device 100 sets, for example, the number of pseudo-images to be generated, the threshold used for quality determination, etc. The visual inspection device 100 may be implemented as an IPC (Industrial Personal Computer), PLC (Programmable Logic Controller), etc., which has computing resources capable of executing each process, a storage area for storing multiple image data, etc. Alternatively, the visual inspection device 100 may be logically implemented using cloud computing.
[0013] The operation terminal 200 is a general-purpose computer device, such as a personal computer, tablet, or smartphone, used by the user of the visual inspection system 1. The operation terminal 200 displays various information acquired from the visual inspection device 100, operation images for specifying thresholds used in pass / fail judgment, and so on. The operation terminal 200 also supplies operation signals to the visual inspection device 100 based on user operations.
[0014] (Example of Hardware Configuration of Visual Inspection Device 100) Next, an example of the hardware configuration of the visual inspection device 100 will be described. As shown in Figure 2, the visual inspection device 100 physically comprises a processor 101, memory 102, storage 103, display device 104, input / output interface (Input / Output I / F (InterFace)) 105, and communication interface (Communication I / F) 106. Each of these components is electrically connected to one another via a bus line 107.
[0015] The processor 101 is a computing unit that controls the operation of the entire visual inspection device 100. The processor 101 is a general-purpose processor such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit). Furthermore, the processor 101 is not limited to a general-purpose processor, but may also be a dedicated processor composed of an ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc.
[0016] Memory 102 is the main memory and includes ROM (Read Only Memory), RAM (Random Access Memory), etc. The processor 101 reads programs and various data from ROM or storage 103 onto RAM and executes processing to realize the overall control and functions of the visual inspection device 100.
[0017] Storage 103 is an auxiliary storage device that stores programs and various data necessary for program execution, and includes non-volatile storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives). Storage 103 stores, for example, the OS (Operating System), which is the basic software that controls the entire visual inspection device 100, and applications that run on the OS and provide various functions. It also stores the model to be verified, multiple image data, etc. Some of the programs and various data necessary for program execution may be stored in ROM.
[0018] The display device 104 is an image display device such as an LCD (Liquid Crystal Display), PDP (Plasma Display Panel), or organic EL (Electro-Luminescence) display, and displays various images according to the control of the processor 101. The display device 104 displays, for example, an image related to the pseudo-image generation process on the display screen. The display device 104 may also be used in a configuration connected to the input / output interface 105.
[0019] The input / output interface (input / output I / F) 105 is an interface for connecting to input devices such as keyboards and mice that input operation signals, and output devices such as speakers that output audio data. For example, the input / output interface 105 takes in operation data entered by the user via the input device and outputs data such as moving images and audio to the output device. Note that if user operation is not required, the visual inspection device 100 does not need to be equipped with a display device 104 and an input / output interface 105.
[0020] The communication interface (communication I / F) 106 is an interface that connects to a communication network and allows the visual inspection device 100 to communicate data with the operation terminal 200. The communication interface 106 includes, for example, a network board, a LAN module, etc.
[0021] (Functional Configuration Example of Appearance Inspection Device 100) Next, the functional configuration of the appearance inspection device 100 will be described. As shown in FIG. 1, the appearance inspection device 100 functionally includes a control unit 110, a storage unit 120, a display unit 130, an input / output unit 140, and a communication unit 150.
[0022] The control unit 110 controls the functions of the entire appearance inspection device 100. The control unit 110 is realized, for example, by a process in which the processor 101 shown in FIG. 2 executes a program developed from the storage 103 to the RAM of the memory 102. Note that the control unit 110 may be realized by hardware using a dedicated integrated circuit such as an ASIC or FPGA. The control unit 110 includes a feature amount detection unit 111 that detects feature amounts of an appearance image of an industrial product, a feature amount distribution information generation unit 112 that generates feature amount distribution information indicating the feature amount distribution of the appearance image, a pseudo-image generation unit 113 that generates a pseudo-image based on the original image, a pseudo-image information generation unit 114 that generates pseudo-image information indicating the image processing settings of the pseudo-image, and a determination unit 115 that determines the quality of the appearance of an industrial product.
[0023] The feature amount detection unit 111 detects feature amounts of an appearance image of an industrial product to be inspected. The feature amount detection unit 111 detects the defect-likeness score obtained by inputting the appearance image of the industrial product to be inspected into the verification target model stored in the storage unit 120 as the feature amount of the appearance image of the industrial product. The defect-likeness score is an index representing the possibility of being a defective product, and the higher the value, the higher the prediction score indicating that the inspection target is more likely to be a defective product. In the following description, the defect-likeness score may be simply referred to as the "score". The defect-likeness score is, for example, a scalar value within a certain range such as 1 to 100. Note that the defect-likeness score may be represented as an N-dimensional vector associated with N label candidates given to the input image. N is a natural number. Instead of the defect-likeness score, a "non-defect-likeness score", which is an index representing the possibility of being a non-defective product, may be used to detect the feature amount of the appearance image of the industrial product. The feature amount detection unit 111 is an example of feature amount detection means.
[0024] The feature quantity distribution information generation unit 112 generates feature quantity distribution information indicating the feature quantity distribution of the appearance image of an industrial product. The feature quantity distribution information generation unit 112 generates, for example, feature quantity distribution information indicating the distribution status of the defective-likeness scores obtained by inputting each of the original images, including a non-defective product image obtained by imaging a non-defective industrial product and a defective product image obtained by imaging a defective industrial product, into the verification target model. Further, the feature quantity distribution information generation unit 112 generates feature quantity distribution information that also shows the distribution status of the defective-likeness scores obtained by inputting the pseudo-images generated by the pseudo-image generation unit 113 into the verification target model, in addition to the distribution status of the defective-likeness scores of the original images. The feature quantity distribution information generation unit 112 generates the feature quantity distribution information in the form of a histogram so that, for example, it can be displayed in a form that is easy for the user to visually grasp. The feature quantity distribution information generation unit 112 is an example of the feature quantity distribution information generation means.
[0025] The feature quantity distribution information generation unit 112 generates, for example, feature quantity distribution information indicating the distribution status of the defective-likeness scores obtained by inputting the original images into the verification target model, in the form of the histogram shown in FIG. 3A. In the histogram shown in FIG. 3A, the class of the non-defective product image is hatched with right-up diagonal lines, and the class of the defective product image is hatched with right-down diagonal lines.
[0026] The pseudo-image generation unit 113 generates a pseudo-image based on the original image. Specifically, the pseudo-image generation unit 113 generates a pseudo-image, which is an appearance image corresponding to the feature quantity space with a sparse distribution density, from the original image in the feature quantity distribution information generated by the original images including the non-defective product image and the defective product image in which the appearances of the non-defective and defective industrial products are imaged respectively. When generating a pseudo-image, the pseudo-image generation unit 113, for example, randomly selects one non-defective product image and one defective product image from the original images stored in the storage unit 120, and synthesizes these two images to generate a pseudo-image. The pseudo-image generation unit 113 selects again when generating the next pseudo-image. The pseudo-image generation unit 113 is an example of the pseudo-image generation means.
[0027] The pseudo-image generation unit 113 divides the feature distribution information generated from the original image into M equal parts using an arbitrary natural number M, representing a feature space with a sparse distribution density, for example, a score region in the histogram shown in Figure 3B where the defect likelihood scores of good product images and defective product images are not distributed, and sets up M classes. The pseudo-image generation unit 113 generates pseudo-images that have a defect likelihood score belonging to each of the set classes. That is, the pseudo-image generation unit 113 refers to the pseudo-image information generated by the pseudo-image information generation unit 114, calculates image processing settings for generating pseudo-images corresponding to feature spaces with sparse distribution density, and generates pseudo-images using the calculated image processing settings.
[0028] The pseudo-image information generation unit 114 generates pseudo-image information that indicates the image processing settings for the pseudo-image. The pseudo-image information indicates what kind of image processing the pseudo-image generation unit 113 used to generate the pseudo-image. As shown in Figure 4, the pseudo-image information can be represented in a table format and includes, for example, the following items: "pseudo-image identification information," "original image identification information," "defective product likelihood score," and "processed data." "Pseudo-image identification information" is information for identifying the pseudo-image. "Original image identification information" is information for identifying the good product image and the defective product image used when generating the pseudo-image. "Defective product likelihood score" is the defective product likelihood score obtained by inputting the pseudo-image into the model under verification. The "processing data" indicates the image processing settings used when generating the pseudo-image, and includes, for example, a "paste area" indicating the area where the defective part is pasted in the good product image, a "cropping area" indicating the area where the defective part is cut out in the defective product image, a "magnification" indicating the ratio of enlargement or reduction of the defective part, a "rotation angle" indicating the angle when the defective part is rotated around a certain point, and a "color tone" indicating the hue, brightness, and saturation of the paste area and the defective part. In the pseudo-image information shown in Figure 4, the "paste area" and "cropping area" are represented by the coordinates of the lower left corner and the upper right corner of each area, i.e., two points on the diagonal of each area, with the lower left of the good product image and the lower left of the defective product image as the origin (0,0). Note that the paste area and cropping area are not limited to rectangular areas; a circular area can be specified by a center point and radius, or a polygonal area can be specified by multiple points. The pseudo-image information is updated each time the pseudo-image generation unit 113 generates a pseudo-image and stored in the storage unit 120. The pseudo-image information generation unit 114 is an example of a pseudo-image information generation means.
[0029] Here, a method for the pseudo-image generation unit 113 to refer to the pseudo-image information and calculate an image processing setting for generating a pseudo-image corresponding to a feature amount space with a sparse distribution density will be described. The pseudo-image generation unit 113 calculates the image processing setting using, for example, Bayesian optimization. More specifically, for the j (j is a natural number) parameters of the processed data of the i-th image processing setting in the pseudo-image information including i (i is a natural number) image processing settings, such as the "pasting area", "cutting area", "magnification", "rotation angle", "color tone", etc. shown in FIG. 4, are represented as Z ｉ＋１，２ , ｉ＋１，ｊ , Z ｉ，２ , …, Z ｉ，ｊ ; and the defective-likeness score of the i-th pseudo-image calculated based on the parameters of the processed data of the i-th image processing setting is represented as F ｉ，ｊ (Z ｉ，１ , Z ｉ，２ , …, Z ｉ，ｊ ); the target value of the defective-likeness score corresponding to the feature amount space with a sparse distribution density is represented as S; and the objective function is represented as G ｉ，ｊ (Z ｉ，１ , Z ｉ，２ , …, Z ｉ，ｊ ). The pseudo-image generation unit 113 calculates the parameters Z ｉ＋１，ｊ (Z ｉ＋１，１ , Z ｉ＋１，２ , …, Z ｉ＋１，ｊ ) of the processed data of the image processing setting of the (i + 1)-th pseudo-image using Bayesian optimization such that the defective-likeness score F ｉ＋１，１ (Z ｉ＋１，２ , Z ｉ＋１，ｊ ) of the (i + 1)-th pseudo-image approaches the target value S. That is, the pseudo-image generation unit 113 calculates Z ｉ＋１，ｊ (Z ｉ＋１，１ , Z ｉ＋１，２ , …, Z ｉ＋１，ｊ ) such that G ｉ＋１，１ [Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (X)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)][Equation (1)]<br) - S | ...(1)
[0030] The actual i+1th pseudo-image has a defect-likeness score of F. ｉ＋１，ｊ (Z ｉ＋１，１ , Z ｉ＋１，２ , ..., Z ｉ＋１，ｊ ) is parameter Z of the processing data in the image processing settings for the (i+1)th pseudo-image. ｉ＋１，１ , Z ｉ＋１，２ , ..., Z ｉ＋１，ｊ The parameter Z of the processed data for the calculated image processing settings is calculated. ｉ＋１，１ , Z ｉ＋１，２ , ..., Z ｉ＋１，ｊ After the (i+1)th pseudo-image is generated, the feature detection unit 111 reads the model to be validated stored in the memory unit 120 and inputs the (i+1)th pseudo-image to the model to be validated. Here, for example, when generating a pseudo-image for the first time, if there is no pseudo-image information for the pseudo-image generation unit 113 to refer to, the pseudo-image generation unit 113 can simply calculate the initial image processing settings randomly. Alternatively, the user may set the initial image processing settings in advance, or the user may pre-define the range of parameters for the processing data of the initial image processing settings when the pseudo-image generation unit 113 calculates the initial image processing settings randomly. Even in such cases, as pseudo-image generation is repeated, the amount of referable pseudo-image information increases, so pseudo-images corresponding to feature spaces with sparse distribution density can be generated without any particular problems.
[0031] Furthermore, it is preferable that the pseudo-image generated by the pseudo-image generation unit 113 is a new pseudo-image that does not overlap with any already generated pseudo-images. That is, it is preferable that the pseudo-image generation unit 113 refers to the pseudo-image information generated by the pseudo-image information generation unit 114, calculates new image processing settings to generate a pseudo-image that corresponds to a feature space with a sparse distribution density and does not overlap with any already generated pseudo-images, and generates a new pseudo-image using the calculated image processing settings. In this case, "not overlapping" means that the image processing settings do not overlap with any already generated pseudo-images, and even if a pseudo-image is generated with different image processing settings than any already generated pseudo-image, the resulting defect likelihood score may overlap with that of any already generated pseudo-image. Details of the pseudo-image generation procedure will be described later.
[0032] Returning to Figure 1, the determination unit 115 determines whether the appearance of the industrial product is good or bad. The determination unit 115 sets a threshold for the user-specified feature, i.e., the defective product likelihood score, according to the distribution of the original image and pseudo-image generated by the feature distribution information generation unit 112, and determines whether the appearance of the industrial product being inspected is good or bad based on the set threshold. For example, if the threshold for the defective product likelihood score is set to 60, the defective product likelihood score of the product being inspected is determined to be good if it is between 1 and 59, and defective if it is between 60 and 100. Note that the threshold may be two or more values, not just one. For example, if a defective product likelihood score of 1 to 40 is considered good and 60 to 100 is considered defective, the thresholds may be set to 41 and 59, and 41 to 59 may be classified as "undeterminable". The determination unit 115 is an example of a determination means.
[0033] The storage unit 120 stores control programs executed by the control unit 110, various data, etc. The storage unit 120 is implemented, for example, by the memory 102 and storage 103 shown in Figure 2. The storage unit 120 includes, for example, a verification target model storage unit 121, an original image storage unit 122, a pseudo-image storage unit 123, and a pseudo-image information storage unit 124.
[0034] The validation model storage unit 121 stores the validation model. The validation model is a trained model that outputs a defect probability score, which is an index representing the likelihood that an input image of an industrial product is defective. The validation model is generated using known machine learning algorithms, such as a convolutional neural network (CNN) that extracts features locally.
[0035] The original image storage unit 122 stores original images obtained by capturing images of the appearance of industrial products. The original images include good product images, which are captured images of good products, and defective product images, which are captured images of defective products. The pseudo-image storage unit 123 stores pseudo-images generated by the pseudo-image generation unit. The pseudo-image information storage unit 124 stores pseudo-image information indicating the image processing settings for the pseudo-images generated by the pseudo-image information generation unit 114.
[0036] The display unit 130 displays various information according to the control of the control unit 110. The display unit 130 is implemented, for example, by the display device 104 shown in Figure 2. The input / output unit 140 receives various data from the input device and outputs various data to the output device according to the control of the control unit 110. The input / output unit 140 is implemented, for example, by the input / output interface 105 shown in Figure 2. The communication unit 150 sends and receives various data with the operation terminal 200 according to the control of the control unit 110. The communication unit 150 is implemented, for example, by the communication interface 106 shown in Figure 2. The communication unit 150 transmits feature distribution information to the operation terminal 200 and receives various operation instructions from the operation terminal 200. The communication unit 150 is an example of a threshold acquisition means.
[0037] (Example of Functional Configuration of Operation Terminal 200) Next, an example of the functional configuration of the operation terminal 200 will be described with reference to Figure 1. Note that the hardware configuration example of the operation terminal 200 is the same as the hardware configuration example of the visual inspection device 100 shown in Figure 2, so the explanation will be omitted. The operation terminal 200 includes a control unit 210, a storage unit 220, a display unit 230, an input / output unit 240, and a communication unit 250.
[0038] The control unit 210 is implemented by a processor and memory and controls the overall functions of the operating terminal 200. For example, the control unit 210 performs display control of various information on the display unit 230, input / output control of the input / output unit 240, and communication control of the communication unit 250 with the visual inspection device 100.
[0039] The storage unit 220 stores control programs executed by the control unit 210, various data, etc. The storage unit 220 stores, for example, feature distribution information supplied by the visual inspection device 100. The display unit 230 displays various information according to the control of the control unit 210. The display unit 230 displays, for example, feature distribution information supplied by the visual inspection device 100. The input / output unit 240 receives various data from the input device and outputs various data to the output device according to the control of the control unit 210. The input / output unit 240 accepts, for example, user instructions such as starting the generation of pseudo-images, the number of pseudo-images to generate, and the threshold for determining whether an image is good or bad. The communication unit 250 sends and receives various data with the operation terminal 200 according to the control of the control unit 210. The communication unit 250 receives, for example, feature distribution information from the visual inspection device 100 and transmits various operation instructions to the visual inspection device 100. The input / output unit 240 and the communication unit 250 are examples of threshold acquisition means.
[0040] The display unit 230 displays feature distribution information represented as a histogram, for example, as shown in Figure 5A. When the user hovers the mouse cursor over each class of the dot-hatched pseudo-image, the pseudo-image and defect likelihood score corresponding to that class are displayed in a pop-up window. In other words, the display unit 230 displays the feature distribution information, pseudo-image, and defect likelihood score in association. Note that the display of the pseudo-image and defect likelihood score can be in any format, not just a pop-up window, as long as the pseudo-image and defect likelihood score are displayed in association. By checking the pseudo-image and defect likelihood score in the pop-up window, the user can understand which distribution on the feature distribution the displayed pseudo-image belongs to, and based on the displayed content, can set an appropriate threshold for determining whether the product is good or bad. Note that the display unit 230 is just one example of a display means.
[0041] Furthermore, if the output of the model under test is feature information represented as an N-dimensional vector, the display unit 230 displays feature distribution information with added pull-down displays that allow the content of the pop-up display for each class of the histogram to be switched and displayed for each feature, as shown in Figure 5B.
[0042] Next, the operation of the visual inspection system 1 having the above configuration will be described. The flowchart shown in Figure 6 is an example of the operation of the visual inspection system 1, and is a flowchart relating to the threshold setting process executed by the visual inspection device 100. This operation corresponds to the visual inspection method of the visual inspection device 100 according to this embodiment.
[0043] The control unit 110 of the visual inspection device 100 starts the threshold setting process in response to an operation input from a user on the operation terminal 200, for example.
[0044] When the threshold setting process is started, the control unit 110 first generates feature distribution information based on the original image (step S101). The feature distribution information generation unit 112 of the control unit 110 reads the good product image, the defective product image, and the model to be verified from the storage unit 120. Then, the feature distribution information generation unit 112 inputs the good product image and the defective product image to the model to be verified, respectively, and obtains a defective product likelihood score for each image. Based on the defective product likelihood scores of the acquired good product image and defective product image, the feature distribution information generation unit 112 generates feature distribution information based on the original image. As feature distribution information, the feature distribution information generation unit 112 generates, for example, a histogram that visually represents the distribution of the defective product likelihood scores.
[0045] Next, the control unit 110 identifies score regions in the histogram where the distribution density of defect likelihood scores is sparse and divides those regions (step S102). The feature distribution information generation unit 112 identifies score regions in the histogram showing the distribution of defect likelihood scores in the original image where the distribution density of defect likelihood scores is sparse and divides those regions equally into M score intervals. Hereinafter, M is a natural number.
[0046] Next, the control unit 110 sets the number of pseudo-images to be generated corresponding to score regions with sparse distribution density (step S103). The feature distribution information generation unit 112 sets, for example, the number of pseudo-images to be generated according to the user's setting instructions. This makes it possible to uniformly generate the number of pseudo-images desired by the user for score regions with sparse distribution density. Here, the number of pseudo-images to be generated may be set for each score interval. In this case, it is possible to generate the number of pseudo-images desired by the user for each score interval for score regions with sparse distribution density.
[0047] In step S103, after setting the number of pseudo-images to be generated, the control unit 110 executes the pseudo-image generation process (step S104).
[0048] Now, referring to Figure 7, the pseudo-image generation process (step S104), which is a subroutine of the threshold setting process, will be explained.
[0049] When the control unit 110 starts the pseudo-image generation process, it first selects good product images and defective product images (step S201). The pseudo-image generation unit 113 of the control unit 110 selects one good product image and one defective product image from the original images stored in the original image storage unit 122 of the storage unit 120 to be used as the source for generating pseudo-images.
[0050] Next, the control unit 110 acquires a defective area image from the defective product image (step S202). The pseudo-image generation unit 113 acquires a defective area image DPa by, for example, cropping an image that captures the defective area from the defective product image DIa shown in Figure 8. When cropping the defective area image DPa from the defective product image DIa, the pseudo-image generation unit 113 recognizes the area of the defective area by, for example, edge detection, and obtains a defective area image DPa that captures only the defective area by performing trimming on the recognized area. The pseudo-image generation unit 113 may, if necessary, use a defective area image DPa prepared in advance by the user instead of performing the process in step S202. That is, the user can visually detect the defective area in the defective product image DIa in advance, manually crop it from the defective product image DIa, and perform appropriate processing such as trimming to obtain an image that captures the defective area, which can then be used as the defective area image DPa in subsequent processing.
[0051] Next, the control unit 110 extracts feature quantities from the good product image and the defective area image (step S203). The pseudo-image generation unit 113 extracts, for example, hue, brightness, saturation, and other color elements from the good product image GIa and the defective area image DPa shown in Figure 8.
[0052] Next, the control unit 110 refers to the pseudo-image information (step S204). The pseudo-image generation unit 113 reads and refers to the pseudo-image information stored in the pseudo-image information storage unit 124 of the storage unit 120.
[0053] Next, the control unit 110 calculates the image processing settings (step S205). The pseudo-image generation unit 113 calculates the image processing settings for generating a pseudo-image corresponding to a feature space with a sparse distribution density, based on the feature quantities of the good product image and the defective area image extracted in step S203, and the pseudo-image information read out in step S204. As image processing settings, the pseudo-image generation unit 113 calculates, for example, the paste area for pasting the defective area image on the good product image GIa shown in Figure 8, the magnification of the defective area image DPa, the rotation angle, etc.
[0054] Next, the control unit 110 processes the good product image and the defective area image based on the image processing settings calculated in step S205 (step S206). The pseudo-image generation unit 113 generates a good product image GIb by, for example, extracting the outline of the industrial product within the good product image GIa shown in Figure 8, thereby removing the unnecessary background around the industrial product. The pseudo-image generation unit 113 also generates a defective area image DPb by, for example, rotating and scaling the defective area image DPa, adjusting its color tone, etc., to match the pasting area of the good product image GIb shown in Figure 8.
[0055] Next, the control unit 110 combines the image of the good product and the image of the defective area (step S207). The pseudo-image generation unit 113 combines the image of the good product and the image of the defective area, which were processed in step S206 based on the image processing settings. For example, the pseudo-image generation unit 113 combines the image of the good product GIb and the image of the defective area DPb shown in Figure 8 to generate a pseudo-image PIa. Furthermore, the pseudo-image generation unit 113 generates a more natural pseudo-image PIb by adjusting the transparency, color tone, etc., of the edge portion of the combined area. After executing the process in step S207, the control unit 110 terminates the pseudo-image generation process.
[0056] Returning to Figure 6, after the pseudo-image generation process is performed in step S104, the control unit 110 obtains a defect likelihood score for the new pseudo-image (step S105). The feature detection unit 111 of the control unit 110 reads the model to be verified stored in the storage unit 120, inputs the newly generated pseudo-image to the model to be verified, and obtains its defect likelihood score.
[0057] Next, the control unit 110 updates the pseudo-image information (step S106). The pseudo-image information generation unit 114 of the control unit 110 generates pseudo-image information indicating the image processing settings and the likelihood of a defective product score for the newly generated pseudo-image, and updates the pseudo-image information by adding it to the existing pseudo-image information.
[0058] After executing the process in step S106, the control unit 110 determines whether the set number of pseudo-images to be generated has been reached (step S107). The control unit 110, for example, refers to a counter that counts the number of pseudo-images to be generated and determines whether the number of generated images set in step S103 has been reached. If it is determined that the set number of pseudo-images to be generated has not been reached (step S107: NO), the control unit 110 returns to step S104 and repeats the series of processes from step S104 to step S107.
[0059] On the other hand, if it is determined that the set number of pseudo-images to be generated has been reached (step S107: YES), the control unit 110 generates feature distribution information from the original image and the pseudo-image (step S108). The feature distribution information generation unit 112 of the control unit 110 adds, for example, the distribution of the defect likelihood score of the pseudo-images generated by the pseudo-image generation unit 113 to the feature distribution information from the original image generated in step S101, and generates new feature distribution information from the original image and the pseudo-image.
[0060] Next, the control unit 110 supplies feature distribution information to the operation terminal 200 (step S109). The control unit 110 transmits the feature distribution information, which is generated by the feature distribution information generation unit 112 using the original image and pseudo-image, to the operation terminal 200 via the communication unit 150.
[0061] Next, the control unit 110 sets a threshold for determining whether the product is good or bad (step S110). The control unit 110 sets a threshold for determining whether the product is good or bad based on the threshold setting instruction input by the user via the input / output unit 240 of the operation terminal 200, according to the feature distribution information supplied to the operation terminal 200.
[0062] After executing the process in step S110, the control unit 110 terminates the threshold setting process. The determination unit 115 of the control unit 110 determines whether the industrial product is good or bad by comparing the defect likelihood score obtained by inputting an image of the appearance of the industrial product to be inspected into the model to be verified with the threshold set by the threshold setting process.
[0063] As described above, in the appearance inspection system 1 according to this embodiment, the appearance inspection device 100 uses a verification target model that outputs a defect likelihood score for the appearance image of an input industrial product, detects the defect likelihood scores of pre-prepared good product images and defective product images, respectively, as features, and generates pseudo-images from the original images that correspond to feature spaces with sparse distribution density in the feature distribution information that shows the distribution status of these features. Furthermore, the appearance inspection device 100 generates pseudo-image information that shows the image processing settings when the pseudo-image was generated, calculates image processing settings for generating pseudo-images corresponding to feature spaces by referring to this pseudo-image information, and generates pseudo-images based on the calculated image processing settings. As a result, the appearance inspection device 100 can efficiently and effectively generate pseudo-images that correspond to feature spaces with sparse distribution density. Consequently, the visual inspection device 100 can generate pseudo-images corresponding to feature space with sparse distribution density, thereby naturally acquiring pseudo-images that approximate limit samples, which are product samples indicating the limits of whether an industrial product is a "good product" or a "defective product."
[0064] Furthermore, the visual inspection device 100 displays the feature distribution information, the simulated image, and the defect likelihood score in association with each other, accepts an input operation for a threshold for determining whether the inspected item is good or bad, and determines whether the inspected item is good or bad based on this threshold. This allows the user to easily determine which distribution on the feature distribution the simulated image that approximates the limit sample belongs to, and to appropriately set the threshold for determining whether the item is good or bad. Therefore, the visual inspection system 1 according to this embodiment can improve the accuracy of the visual inspection of the inspected item.
[0065] The visual inspection system 1 according to this embodiment generates multiple pseudo-images uniformly, each containing features of the boundary between good and defective products that were not present in the original image, based on the feature distribution of the original image. Therefore, the generated pseudo-images can be used as validation data without distribution bias. Furthermore, by using the pseudo-images as training information, it is possible to improve the inference performance of the validation model, which is a trained model.
[0066] (Embodiment 2) In Embodiment 1 described above, an appearance inspection system 1 comprising an appearance inspection device 100 and an operation terminal 200 was described. However, the device configuration of the appearance inspection system 1 is not limited to this, and for example, one of the devices may have all of the functions that the appearance inspection device 100 and the operation terminal 200 each have.
[0067] The visual inspection system 1 may, for example, consist only of a visual inspection device 300 that combines the functions of the visual inspection device 100 and the operation terminal 200 described in Embodiment 1, as shown in Figure 9. The visual inspection device 300 is assumed to be a device that adds the functions of the visual inspection device 100 to the operation terminal 200. It is desirable that the visual inspection device 300 has sufficient processing capacity and memory resources equivalent to those of the visual inspection device 100.
[0068] Functionally, the visual inspection device 300 comprises a control unit 310, a storage unit 320, a display unit 330, an input / output unit 340, and a communication unit 350. The control unit 310 includes a feature detection unit 311, a feature distribution information generation unit 312, a pseudo-image generation unit 313, a pseudo-image information generation unit 314, and a determination unit 315. The storage unit 320 includes a verification target model storage unit 321, an original image storage unit 322, a pseudo-image storage unit 323, and a pseudo-image information storage unit 324.
[0069] The control unit 310, storage unit 320, display unit 330, input / output unit 340, and communication unit 350 are the same as the control unit 110, storage unit 120, display unit 130, input / output unit 140, and communication unit 150 in the visual inspection device 100 of Embodiment 1, so a detailed explanation is omitted. The feature quantity detection unit 311, feature quantity distribution information generation unit 312, pseudo-image generation unit 313, pseudo-image information generation unit 314, and determination unit 315 of the control unit 310 are the same as the feature quantity detection unit 111, feature quantity distribution information generation unit 112, pseudo-image generation unit 113, pseudo-image information generation unit 114, and determination unit 115 of Embodiment 1. Furthermore, the verification target model storage unit 321, original image storage unit 322, pseudo-image storage unit 323, and pseudo-image information storage unit 324 of the storage unit 320 are the same as those of the verification target model storage unit 121, original image storage unit 122, pseudo-image storage unit 123, and pseudo-image information storage unit 124 of Embodiment 1.
[0070] With this device configuration, users of the visual inspection system 1 can operate the entire system's functions on the visual inspection device 300 without using the visual inspection device 100. In addition, the visual inspection device 300, like in Embodiment 1, can improve the accuracy of visual inspection of the object being inspected, and can also reduce the number of devices and eliminate communication delays.
[0071] This disclosure is not limited to the embodiments described above, and various modifications and applications are possible without departing from the spirit of this disclosure.
[0072] In the above embodiment, the visual inspection of the industrial product to be inspected was performed using a verification target model that the visual inspection device 100 has held in advance, but this is not limited to this. For example, a trained model that has been machine-learned using training information generated based on inspection information obtained from an inspection information source provided in another visual inspection system may be incorporated as the verification target model of the visual inspection system 1, and the verification target model may be updated by retraining using training information held by the visual inspection device 100 or obtained from an external source.
[0073] In the above embodiment, for example, the control program executed by the processor 101 that realizes the control unit 110 of the visual inspection device 100 was mainly stored in storage 103 beforehand. However, the disclosure is not limited thereto, and the control program for executing the above-mentioned various processes may be implemented in an existing general-purpose computer, framework, workstation, etc., to function as a device equivalent to the visual inspection device 100 according to the above embodiment. The same applies to the control program executed by the processor that realizes the control unit 210 of the operation terminal 200.
[0074] The method of providing such programs is optional. For example, they may be distributed by storing them on a computer-readable storage medium (flexible disk, CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM), or they may be stored on network storage such as the Internet and provided for download.
[0075] Furthermore, when the above processing is performed by a division of labor between the OS and the application program, or by collaboration between the OS and the application program, only the application program may be stored on a recording medium, storage, etc. It is also possible to superimpose the program onto the carrier wave and distribute it over a network. For example, the above program may be posted on a bulletin board system (BBS) on a network and distributed over the network. The program may then be designed to execute the above processing by launching this program and running it under the control of the OS, just like other application programs.
[0076] This disclosure allows for various embodiments and modifications without departing from the broad spirit and scope of this disclosure. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of this disclosure. In other words, the scope of this disclosure is indicated by the claims, not by the embodiments. Various modifications made within the scope of the claims and the equivalent significance of the disclosure are considered to be within the scope of this disclosure.
[0077] 1...Visual inspection system, 100...Visual inspection device, 101...Processor, 102...Memory, 103...Storage, 104...Display device, 105...Input / output interface, 106...Communication interface, 107...Bus line, 110...Control unit, 111...Feature detection unit, 112...Feature distribution information generation unit, 113...Pseudo-image generation unit, 114...Pseudo-image information generation unit, 115...Determination unit, 120...Storage unit, 121...Verification target model storage unit, 122...Original image storage unit, 123...Pseudo-image storage unit, 124...Pseudo-image information storage unit, 130...Display unit, 140...Input / output unit, 150...Communication unit, 200...Operation terminal At the end, 210...control unit, 220...storage unit, 230...display unit, 240...input / output unit, 250...communication unit, 300...visual inspection device, 310...control unit, 311...feature quantity detection unit, 312...feature quantity distribution information generation unit, 313...pseudo-image generation unit, 314...pseudo-image information generation unit, 315...judgment unit, 320...storage unit, 321...verification target model storage unit, 322...original image storage unit, 323...pseudo-image storage unit, 324...pseudo-image information storage unit, 330...display unit, 340...input / output unit, 350...communication unit, DIa...defective product image, DPa, DPb...defective area image, GIa, GIb...good product image, PIa, PIb...pseudo-image.
Claims
1. An appearance inspection system for inspecting the appearance of an object to be inspected, comprising: a feature detection means for detecting the defect likelihood score of an appearance image of an input object to be inspected as a feature, using a verification target model that outputs a defect likelihood score, which is an index indicating the possibility of a defect, for the appearance image of the object to be inspected; a feature distribution information generation means for generating feature distribution information showing the distribution of the defect likelihood scores of the appearance image detected by the feature detection means; a pseudo-image generation means for generating pseudo-images from the original images corresponding to feature spaces with sparse distribution density in the feature distribution information generated from original images including images of good and defective products, respectively, of good and defective products of the object to be inspected; and a pseudo-image information generation means for generating pseudo-image information showing the image processing settings when the pseudo-image generation means generated the pseudo-images, wherein the pseudo-image generation means refers to the pseudo-image information, calculates image processing settings for generating pseudo-images corresponding to the feature spaces, and generates the pseudo-images based on the calculated image processing settings.
2. The appearance inspection system according to claim 1, further comprising: display means for displaying the feature distribution information, the pseudo-image, and the defect likelihood score in association; threshold acquisition means for receiving an input operation for a threshold for determining whether the inspection target is good or bad and acquiring the threshold; and determination means for determining whether the inspection target is good or bad based on the threshold.
3. The appearance inspection system according to claim 1 or 2, wherein the pseudo-image generation means calculates new image processing settings for generating a pseudo-image that does not overlap with a pseudo-image already generated, and generates a new pseudo-image based on the calculated image processing settings.
4. The appearance inspection system according to any one of claims 1 to 3, wherein the feature distribution information generation means generates the feature distribution information as a histogram.
5. The appearance inspection system according to any one of claims 1 to 4, wherein the pseudo-image generation means generates the pseudo-image by adjusting at least one element from the region, color tone, and rotation angle included in the image processing settings and pasting it onto the pasting location of the good product image.
6. An appearance inspection device for inspecting the appearance of an object to be inspected, comprising: a feature detection means for detecting the defect likelihood score of an appearance image of an input object to be inspected as a feature, using a verification target model that outputs a defect likelihood score, which is an index indicating the possibility of a defect, for the appearance image of the object to be inspected; a feature distribution information generation means for generating feature distribution information showing the distribution of the defect likelihood scores of the appearance image detected by the feature detection means; a pseudo-image generation means for generating pseudo-images from the original images corresponding to feature spaces with sparse distribution density in the feature distribution information generated from original images including images of good and defective products, respectively, of good and defective products of the object to be inspected; and a pseudo-image information generation means for generating pseudo-image information showing the image processing settings when the pseudo-image generation means generated the pseudo-images, wherein the pseudo-image generation means refers to the pseudo-image information, calculates image processing settings for generating pseudo-images corresponding to the feature spaces, and generates the pseudo-images based on the calculated image processing settings.
7. An appearance inspection method performed by an appearance inspection system for inspecting the appearance of an object to be inspected, wherein a computer in the appearance inspection system uses a verification target model that outputs a defect likelihood score, which is an index indicating the possibility of a product being defective, for an input appearance image of the object to be inspected, to detect the defect likelihood score of the appearance image as a feature, generates feature distribution information showing the distribution of the defect likelihood scores of the appearance image, generates pseudo-images from the original images corresponding to feature spaces with sparse distribution density in the feature distribution information generated from original images including good product images and defective product images, respectively, of the appearance of good and defective products of the object to be inspected, generates pseudo-image information showing the image processing settings when the pseudo-images were generated, and the computer, when generating pseudo-images corresponding to the feature spaces, refers to the pseudo-image information, calculates image processing settings for generating pseudo-images corresponding to the feature spaces, and generates the pseudo-images based on the calculated image processing settings.
8. A computer functions as a feature detection means that uses a verification target model that outputs a defect likelihood score, which is an index indicating the probability of an input image of an object to be inspected being a defective product, to detect the defect likelihood score of the image of the object as a feature; a feature distribution information generation means that generates feature distribution information showing the distribution of the defect likelihood scores of the image of the object detected by the feature detection means; a pseudo-image generation means that generates a pseudo-image from the original images corresponding to a feature space with a sparse distribution density in the feature distribution information generated from original images including images of good and defective products, respectively, of which the appearances of good and defective products of the object to be inspected have been captured; and a pseudo-image information generation means that generates pseudo-image information showing the image processing settings when the pseudo-image generation means generated the pseudo-image, wherein the pseudo-image generation means refers to the pseudo-image information, calculates the image processing settings for generating a pseudo-image corresponding to the feature space, and generates the pseudo-image based on the calculated image processing settings.